An Alexandria lawyer has alleged that Virginia state troopers routinely spy on enclosed toilet stalls in public highway restrooms in search of illegal homosexual activity.

In a brief filed with the Supreme Court, attorney James M. Lowe charged that the troopers peer through fake ceiling vents over the stalls. He said the vents-ostensibly for heating and air conditioning-were installed for the specific purpose of permitting surveillance and are located in women's and men's restrooms at rest stops throughout the state.

State police officials and a spokesman for the state Department of Highways and Transportation conceded that surveillance has taken place at two Prince William County facilities, but denied the practice was routine or had occurred elsewhere in Virginia.

Lowe disputed their statements. "I can't prove that the troopers have spied on the women's stalls, but human nature being what it is, I think you can safely assume they have," Lowe said.

Lowe has asked the high court to overturn the convictions of three men arrested in October 1977 by state troopers who observed them committing sodomy in toilet stalls in public restrooms along Interstate 66 about 30 miles southwest of Washington. He contends the surveillance violated Fourth Amendment protections against unreasonable searches.

"We responded to specific complaints at these facilities and we took action to learn what the true facts were," said State Police Capt. Robert L. Suthard, commander of the police division that includes Prince William. "We do not routinely stake out the stalls add we don't say. "Today's Thursday, let's go over there and look at the stalls.'"

Albert W. Coates Jr. of the Virginia highway department said holes were cut in ceilings over the men's room stalls at the Prince William facilities "at the request of the state police. It was done in this one instance and that's the only one." Coates said he didn't know if the holes were still there. He denied any holes were cut over women's room stalls.

But Lowe said state troopers admitted at the trials of the three men in February 1978 that the dummy vents were also in place in the wormen's rest rooms and at others of the estimated 30 highway restrooms throughout the state.

Local officials asked the state police for help more than two years ago after residents complained that the Prince William facilities were gathering places for homosexuals.

"It's supposed to be a restroom, not a gay bar," said Assistant Commonwealth's Attorney Stuart Robeson Jr., who estimated that as many as 40 arrests and convictions resulted from the state police surveillance.

Three of those arrested-a hospital orderly, an airplane mechanic and a hairdresser, all living in Northern Virginia-came to Lowe, who has made a reputation for defending gay rights' cases. He filed motions to suppress the troppers' testimony on the grounds that it was obtained illegally. In February 1978, State Circuit Court Judge Arthur W. Sinclair denied the motions, saying:

". . . I do not believe that it can be successfully argued that the defendants, who were charged with engaging in homosexual acts in enclosed toilet stalls in public restrooms, are entitled to have a reasonable expetation of privacy under the circumstances. . . ."

A week later, the men pleaded guilty, waived their right to jury trials, were convicted by Sinclair and received suspended 5-year sentences.

Lowe appealed to the Virginia Supreme Court, which last December upheld the convictions by refusing to hear the case. Lowe's appeal to the Supreme Court was filed March 19. Court spokeswoman Cecily Coleman said the justices most likely will decide within the next three weeks whether to hear the case.

According to Lowe, the Maryland Court of Appeals in 1968 ruled against similar surveillance in the case of a drug addict arrested in a toilet stall, and state supreme courts in Minnesota and California have found toilet stall spying unconstitutional in sodomy cases. Lowe's brief also relies on a 1967 Supreme Court ruling against warrantless wiretapping in an enclosed public telephone booth.

The Prince William commonwealth's attorney's office and the state attorney general have not filed replies to Lowe's brief.