How residents in Alexandria's Rosemont neighborhood feel about the current furor at City Hall may depend on which side of the street they live on.

City Council member Donald C. Casey lives on North View Terrace, a street north of the Masonic Temple; City Manager Douglas Harman lives there, too, on the opposite side.

Last week, when a judge named a special grand jury to investigate Casey's allegations that a top Harman appointee may have halted a drug investigation when Alexandria Sheriff Michael E. Norris' name surfaced, most of the neighborhood already had divided into camps: Harman's and Casey's.

The divisions in Rosemont are significant because they illustrate how Alexandria, despite its emergence as a fashionable, close-in suburb with Georgetown-like neighborhoods, remains in many ways a small, southern town. Everyone who is anyone there knows everyone else: their businesses, their families and, often, intimate details about their lives. As one longtime resident put it last week: "I feel like my close friends just got divorced."

The side-taking became clear during the Christmas holiday when Harman, Casey and Alexandria prosecutor John E. Kloch showed up at a neighborhood party six days after the allegations were raised in the Alexandria Port Packet, a community newspaper. Each of the officials found his corner and his supporters.

"It used to be one big clique; now it's just a few smaller ones," said Lewis Fulwiler, a North Terrace View resident for 35 years. Fulwiler said that the only thing that he knows so far that is criminal "is all this yak, yak, yak."

The harsh words between many top officials, including Casey, who most view as either an incurable troubleshooter or troublemaker, and Harman, who has gained wide respect in his nine years as the city's top administrator, have divided not only North View Terrace, but most of Alexandria. As the city approaches its May elections and the final rounds of the prestigious All-America City contest, even Casey said that the timing of the grand jury investigation is "dreadfully poor."

"Old Town has a long-established reputation for being charming and traditional," said Ann Dupre, a travel agent at the Old Town Travel Agency across from City Hall. "I think that's why people are more surprised to hear about something like this."

"I'm shocked. In this town you get the feeling that nothing should go wrong," said newcomer Cathryn Butts. "Everything appears so proper here, you automatically feel that carries over to politics."

Judith Feaver, a former head of the Alexandria School Board and a political activist, said the city's relatively small size may be part of the reason so many "peculiar" stories seem to crop up there. She recalled two similar recent incidents in which strong allegations were raised against public officials. Somehow, after all the talk and investigations quieted down, Feaver said, many residents said they were more confused about what had happened.

Commonwealth's Attorney William L. Cowhig was indicted in 1979 on charges that he accepted bribes from a bingo game operation. He was absolved of the criminal charges, but he resigned after it was disclosed he had accepted a sexual favor from the wife of a drug suspect.

Vice Mayor James P. Moran was found guilty last year of a misdemeanor conflict-of-interest charge and resigned. He was convicted of voting on the City Council to conduct city business with a developer with whom he had personal business deals, but he now is regarded as having a good chance to win one of the six seats on the council in the spring elections.

The current controversy comes as Alexandria has been named one of the 17 finalists in the 1985 All-America City contest. A spokeswoman said last week that a secret "field verification" team soon will be sent to the city to make certain that "what the judges thought in November from the slide presentations was actually true."

Perhaps, some city officials say, the team may decide that nothing is more American than having a political donnybrook at City Hall.