Former representative Robert E. Bauman, who came within 1,071 votes of winning the Republican congressional primary race from which he had withdrawn, said yesterday that he would not challenge the election results in that 1st District contest.

The Maryland Republican said that, despite voting machine problems in Queen Anne's County, he would not press for a new vote there because the results from the entire district were not close enough to make a challenge worthwhile.

Bauman, once a rising star of the GOP right who was defeated in 1980 after admitting to problems of homosexuality and alcoholism, said he made the decision "for a number of reasons, not the least of which is my continued serenity."

The count Thursday of 555 absentee ballots widened the margin of victory for former state senator C.A. Porter Hopkins, who had about 47 percent of the vote compared to Bauman's 43 percent in the three-way race. Hopkins will now face Democratic incumbent Roy Dyson in the general election.

Bauman withdrew from the primary race July 29, accusing Hopkins of running a smear campaign against him, a charge Hopkins vehemently denied. Still, Bauman's name appeared on the ballot because his withdrawal came after the official deadline.

One unofficial vote tally initially had placed Bauman within 273 votes of victory, but when that tally proved inaccurate Bauman said he felt "the math was wrong" for a challenge.

The former congressman left the door open for a reentry into politics, telling reporters: "This is not a Shermanesque or Nixonesque final statement. "If this many people support me under these circumstances there may be a place in the political firmament for me."

Bauman said he had been approached about doing a book about politics and was set to do a preliminary outline.

As to a future political career, Bauman said: "Who knows what happens in politics? . . . Lord knows my life is a textbook example of the fact that no one knows what may happen in politics."