The Fairfax County School Board last night ended five years of turbulent debate over the county's high school sex education program by lifting a prohibition on classroom discussions of contraception, abortion, masturbatiion, homosexuality and rape.

By an 8-to-2 vote, the board agreed to liberalize what has been considered the Washington area's most restrictive sex education program over the protests of many county residents, who had argued that the change would undermine the moral character of the county's youth.

The board reiterated, however, that no student will be required to take the revised sex education course, which will be introduced in eight of the county's 23 high schools next fall. Under the plan approved last night, parents will be given the right to choose whether their children will take either the sex education course or one of two other courses designed as alternatives.

In calling for the plan's approval, board members said the school system has a responsibility to give its students the information and judgment to resolve the difficult choices they will face in everyday life. "Ignorance is not bliss, and it will not lead to chastity or anything else," said board member Eltse carter.

Students in county sex education classes now will be permitted to discuss what teachers call the "Big Five" sex topics. The program abolishes a county rule that requires students to submit all questions about sex in writing; teachers have hitherto been forbidden to anwer any questions dealing with the five taboo subjects.

Conservative religious groups and antiabortion activists were vehement in their opposition to the proposal for expanding the Fairfax program, and Elizabeth Burch, a representative of the Northern Virginia Movement to Restore Decency, said last night she was "absolutely horrified" at the board's decision.

"There are no values being taught here. Students won't know what's right or wrong, she said. "It's not right to have premarital sex. It's not right to have an abortion. The Ten Commandments should be their standards."

Burch's concerns were similar to those voice earlier by board member Gary L. Jones, a top Education Department official in the Reagan administration, who had waged an unsuccessful battle to continue the county's ban on in-school discussions of abortion, homosexualty and masturbation.

Board Chairman Ann P. Kahn maintained that county staff had incorporated moral guidelines in the course curriculum.

"I think the most essential value is there in the course," she said. "We're teaching students to begin to assume responsibility for the course of their own lives, and to understand that responsibility goes far further than the pleasures of the moment."

Describing the view of the board majority earlier yesterday, Kahn said, "We don't see it as a liberal-conservative kind of question. I don't think it would be conservative to let an epidemic of teen-age pregnancies continue in this country.

"We're looking for a constructive way of helping parents work with their children on these issues. I don't think that's a liberal view. I think it's a practical view." she said.

The program approved last night gives parents the option of enrolling their 10th-grade students in a four-week unit of an elective biology course that includes a full discussion of the controversial topics, or they may choose a course that contains only a limited discussion of human reproduction. Under a third option, parents could enroll their children in an environmental sciences biology unit.

The county's current "family life curriculum," as it is called, has been consistently criticized by students, who say they could learn more about sex by reading magazines and watching television than they could in county classrooms. The classes are, in fact, so limited that less than 2 percent of eligible high school students even sign up to take them.

A recent school survey found that 75 percent of Fairfax parents favored expansion of the course, while another survey by the county's Parent-Teacher Associations found that an even greater majority of students took that positions.

School board members said yesterday the county's high school dropout rate, which has increased from 964 students in the 1975-6 school year to 1,061 last year, was one of their biggest concerns in approving the program expansion. Most teen-age girls who drop out of school leave as a result of pregnancy, they said.

Opponents of the program had argued that a sex education curriculum would encourage pregnancies by implying that sexual activity is acceptable.

School officials said they plan to expand the new sex education program, which will begin in only eight schools in the fall, to all county high schools next year.

Joining board member Jones in voting against the new program was member Anthony T. Lane. Student board member David McCreight, who does not have a vote, registered as approving of the program.