The U.S. Supreme Court yesterday refused to hear the appeal of three Northern Virginia men convicted of sodomy after state troopers spied on them through ceiling vents over toilet stalls in restrooms along Interstate 66 in Prince William County.

The ruling, issued without comment, leaves intact the 1978 convictions of the three men, whose lawyer argued that the police surveillance, undertaken without search warrants, violated Fourth Amendment protections against unreasonable searches.

The men were arrested in separate incidents in October 1977, at a time when state police had staked out two restrooms on I-66 about 30 miles southwest of Washington in response to citizen complaints about homosexual activity in the men's rooms.

All three were convicted and given five-year suspended sentences after Circuit Court Judge Arthur W. Sinclair denied a request to suppress the evidence against them on grounds that it was obtained illegally.

The Virginia Supreme Court refused to hear the defendants' appeal last December.

In another matter, the Supreme Court yesterday refused to hear a Mannassas women's argument that she should not have to pay rent on her daughter's public school textbooks.

Dr. Antoinette Foster challenged a state law requiring rental payments for textbooks on grounds that the right to a free public education ought to inlcude the use of free textbooks. But the court's refusal to hear the case left intact a Virginia Supreme Court ruling that the law was constitutional.