A proposal by the Northern Virginia chapter of B'nai B'rith to build a 91-unit, partially subsidized high-rise for the elderly in eastern Fairfax County has encountered a storm of protest, prompting Supervisor T. Farrell Egge to withdraw his support for the plan and placing its future in doubt.

Critics contend that the project, planned for the 1700 block of Collingwood Road south of Alexandria, would add to already critical traffic problems there, would come to within five feet of adjacent properties and could lead to improper zoning changes elsewhere in the county. B'nai B'rith's application requests that the area's single-family zoning be changed to allow more people to live on the site.

Citing the opposition, B'nai B'rith has requested a two-month delay in a scheduled November Planning Commission hearing on the project. It needs a special zoning exception from the Fairfax County Board of Supervisors.

"Jamming a thing like this right next to people's houses would not be an attractive addition to the neighborhood," said Edward Bennett, a Collingwood Road resident who has been one of the most outspoken critics of the proposal.

"I think they're afraid the wrong element will move into the building," countered Leonard Middleberg, president of the Northern Virginia chapter of B'nai B'rith. "After all, it is for low- and middle-income people. The reasons they [the opponents] give me aren't very valid."

"Some people have determined that this is going to be a facility for the poor, the destitute and people on welfare, and there's very little you can do to change their minds," Middleberg said.

Middleberg said his organization had pushed the project largely because of support last April from Egge, who represents the Mount Vernon area. "He gave us a letter of enthusiastic support," Middleberg said. "He said he would like to have the program in his district and that he would do everything he could to help."

In a June 6 letter to Bennett, Egge said he also is seeking the opinion of residents in the area but that he believed the project "could be an attractive addition" to the neighborhood. Egge said he recognized that the proposal required an exception from the county's comprehensive plan.

"I didn't realize by any stretch of the imagination the vehemence of the response," Egge said in an interview. He said his intervention on B'nai B'rith's behalf was merely to give the organization "an opportunity to present their proposal." He said he told them from the beginning that his backing would be contingent upon community support.

Egge and Middleberg differed over the degree of opposition to the project. While the supervisor said the opposition was nearly unanimous, Middleberg said Egge received petitions and telephone calls from numerous Mount Vernon residents who favored B'nai B'rith's plan.

The supervisor said he was concerned about the shortage of housing for the elderly in Fairfax and he wanted to work with B'nai B'rith to find another site for the project.

Middleberg said the organization, which has an option to buy the Collingwood Road property for $275,000, "hasn't been able to find anything we can afford." He also said that losing the present location would cost B'nai B'rith about $22,000 in architects' and lawyers' fees. He didn't disclose the building's costs.

Fairfax County Board Chairman John F. Herrity, who has taken no position on the Mount Vernon project, sponsored an ordinance in 1974 that would permit the construction of housing for the elderly in areas zoned for single-family housing. However, he said that as a general rule he will not support such changes unless they have the support of at least 50 percent of the residents in the area.

"You're never going to get 100 percent [support], but something has to have some support in the community to be viable," Herrity said.

Senior citizens "are one of the fastest growing elements of our population," said Herrity. Projections show the county will have about 90,000 in the year 2000, up from 48,000 in 1980, he said.