A Fairfax County citizens task force voted early yesterday to recommend that the School Board close Fort Hunt High School in the southern part of the county, where school enrollments are declining, and covert it to an intermediate school.

The recommendation, which was approved by a vote of 20 to 5 and will be presented to the School Board Oct. 25, comes at a difficult time for the school, where a sophomore was allegedly raped twice in a bathroom during a football game on Oct. 3 while a group of teen-age boys looked on.

The vote follows six months of weekly meetings by the 26-member task force of parents from the Mount Vernon and Lee districts who were asked by Superintendent William J. Burkholder to study how the school system should address the declining enrollment problem at three area high schools.

"The proposal will be a very important ingredient in our final decision," said Joy G. Korologos, an at-large member of the Fairfax School Board.

It is the first time a community group, rather than the school system, has offered such a drastic change as a solution to the enrollment problem.

The recommendation pits the Fort Hunt community against its neighboring school districts, with Fort Hunt parents blaming those of Groveton and Mount Vernon high schools for conspiring to close down their school.

About 250 Fort Hunt parents showed up at a meeting last Wednesday, clutching the school's latest test scores and denouncing the "conspiracy" they said had been mounted by the neighboring school districts of Groveton and Mount Vernon.

The Fort Hunt parents, who have already turned out in the hundreds to protest the impending proposal, have charged that other school districts are motivated by a "jealousy" of the school's high athletic and academic standing.

"Why do they want to close us down?" shouted Henry Gray to the large audience last week. "They envy us. They want our good students."

Some of the accusations have had racial overtones, with Fort Hunt parents suggesting that Groveton and Mount Vernon want the influx of Fort Hunt students to "dilute" their higher concentrations of minorities.

One parent, who asked not to be identified, said the problem was that "Groveton and Mount Vernon have a lot more minority students than we do and that's why their scores are lower and that's why they want our students in our schools to offset that."

Minority students make up 23 percent of Mount Vernon High's population, 28 percent of Groveton's and 16 percent of Fort Hunt's, according to official enrollment figures.

Although the recent rape at the school apparently was not commited by students from Fort Hunt, some parents fear it will nonetheless hurt the high school's bid for political support outside the Mount Vernon area.

"To the extent we are going to raise the debate outside the Fort Hunt school district , it could have an adverse effect," Frank Dunham, a Fort Hunt parent and a representative on the task force, said of the rape's effect on the discussion to close the school.

This is the second time the high school's future has been threatened. In 1979, two Fort Hunt students and one alumnus set fire to the school, causing $4.5 million worth of damage.

Even five years ago, the prospect of smaller enrollments had prompted some school officials to consider not reopening the school after the fire. But the wealthy and cohesive Fort Hunt community mounted what some local officials still recall as one of the county's most effective campaigns to preserve a school.

That strategy hasn't changed, according to Peter H. Brinitzer, who headed up the effort to rebuild the school after the fire and is organizing the latest battle to keep Fort Hunt as a high school. "We will use every means at our disposal to influence the School Board and the Board of Supervisors and others," he said.

In this campaign, the Fort Hunt community is armed with proposals to boost the school's enrollment. One would place the students from the Fort Belvoir military post who now attend Hayfield Secondary school at Fort Hunt.

The homogeneity of the Fort Hunt community explains in part why the school has such fervent support in the community, according to teachers and parents. But they also said that Fort Hunt's location in the middle of a residential area, which allows 70 percent of the students to walk to school, has made it a natural center of the community's life.