In a major and possibly decisive victory for NBC in the "Born Innocent" case, Superior COURT Judge Robert L. Dossee ruled yesterday that the sole issue in a jury trial will be whether the network deliberately had invited someone to rape a 9-year-old Olivia Niemi with a beer bottle.

The ruling means that the judge intends to try the unprecedented $11 million television violence case on First Amendment grounds, as urged by NBC.

Happy NBC attorneys hugged and kissed each other when Dossee announced the ruling three hours before selection of the jury - from a panel of 120 persons - began.

Marvin E. Lewis, attorney for Olivia, said he was "shocked."

He had argued that the First Amendment should be excluded - that the trial issue was the alleged negligence and recklessness of NBC in including in the two-hour TV drama a vivid depiction of a rape with the wooden handle of a plumber's plunger, in failing to foresee real-life imitation and stimulation of such a rape, and in airing the program in evening hours when large numbers of youngsters were in the audience.

A 15-year-old girl and three companions assaulted Niemi on Sept. 14, 1974, four days after the showing of "Born Innocent," in which four girls assaulted the 15-year-old star of the film in a reformatory shower room.

Whether Niemi's assailants had seen the film is disputed.

Under a 1969 Supreme Court ruling, constitutional protection is denied to speech or expression "directed to inciting or producing imminent lawless action . . . (or) likely to incite or produce such action." Lewis told the judge that it would be "impossible" and "utterly ridiculous" for him to try to persuade a jury that NBC had set out to cause a rape. He emphasized that he never had claimed that NBC had such an intention.

Rather, he contended, NBC's inclusion of the rape scene in "Born Innocent" was negligence and recklessness comparable to that of a person who falsely cries out "Fire" in a crowded theater.

Lewis said Tuesday that if the judge were to rule that he had to prove that NBC intentionally had incited the attack on Niemi, he would not go to trial. Instead, he said, he would appeal such a ruling, and, if necessary, carry the case to the Supreme Court.

After Dossee ruled yesterday, however, Lewis said that he would try the case if the judge was defining incitement to mean stimulation or imitation arising from negligence or recklessness.

Lewis contended that the rape scene was, for children and adolescents, legally "indecent," in that it caused them to model their behavior on that of the adolescents in the film who were not punished for their crime.

Dossee said he would admit only that evidence of negligence that is relevant to incitement. He relied, he said, on an opinion of a state appellate court when it ruled last October that Niemi improperly had been denied a jury trial by a lower-court judge.

The opinion cited the 1969 Supreme Court definition of incitement, as NBC attorney Floyd Abrams repeatedly emphasized.

While holding that Niemi was entitled to a jury trial on the issue of whether the film rape scene constituted incitement unprotected by the First Amendment, the appeals court stressed "the overriding constitutional principle that material communicated by the public media, including fictional material such as the television drama here at issue, is generally to be accorded protection under the First Amendment . . ."

The appeals court ruling was allowed to stand by the California and U.S. Supreme Courts. Both denied NBC's petition for review.

In reply to questions by Lewis, Dossee said he would decide as the trial proceeded whether each item of evidence of negligence offered by Lewis would be admitted as relevant to the incitement issue.

Lewis obtained a recess to prepare for further argument and to decide whether to proceed with the case.

The argument after the recess did not budge Dossee, who ordered jury selection to proceed.

The judge did not respond to a plea by Lewis to say whether he would allow incitement to be inferred from negligent or reckless conduct or to define incitement in a civil sense (speech that arouses, urges, provokes or stimulates) as opposed to a criminal sense (falsely crying "Fire"). He reiterated that he will rule on evidence as it is offered.

The judge did not respond to a plea by Lewis to say whether he would allow incitement to be inferred from negligent or reckless conduct or to define incitement more strictly as speech that arouses, urges, provokes or stimulates - except to reiterate that he will rule on evidence as it is offered.

He ignored an unusual proposal by Lewis to let the lawyer make his opening statement to the jury, after which the judge, on his own initiative, would decide if Lewis had a "nonsuit" and that the trial should end.

Dossee also disregarded a contention that his ruling was in a fatally defective legal form, inviting reversal and a second trial after the one now starting.