Rep. Robert E. Bauman, the conservative congressman from Maryland's Eastern Shore, ended speculation yesterday that he might challenge Sen. Charles McC. Mathias in the GOP primary this year by announcing that he will seek reelection to the House from the 1st Congressional District.

Bauman revealed his decision at the State House in Annpolis, saying that while many backers agreed with him that"Mac could be defeated in a primary," they also warned him that it was not likely he could beat the Democratic nominee, whoever that may be, in the general election.

"I am not a kamikaze," Bauman said, "I understand the limitations of a conservative Republican in this state . . . The reality of Maryland politics, the conventional wisdom, is that I cannot win statewide."

Perhaps the happiest person to hear of Bauman's decision (aside from Mathias, who was campaigning in Baltimore), was state GOP chairman Allan H. Levey, who praised Bauman's statement as "healthy for the party, which I have spent a year trying to bring together."

Party loyalty, in addition to political reality, also was cited by Bauman, who said giving up his seat in favor of "a bruising primary that I might not win," could result in the loss of his 1st District seat, one of only two in the eight-member Maryland House delegation controlled by Republicans.

Also, Bauman said, Mathias, who is seen as a prohibitive favorite over any Democrat likely to challenge him, could represent the swing vote in the next Congress, which Bauman said could be controlled by conservatives if Republicans make substantial gains in the fall. Twenty-four of the 34 Senate seats up for grabs are now held by Democrats.

Mathias refused to respond to Bauman's announcement, claiming he had nothing to say about a "hypothetical campaign."

Bauman displayed his acerbic humor throughout the announcement and press conference that followed. He said "many Marylanders" had urged him to stay out of the Senate race, "and I mean in addition to State Senators Victor L. Crawford" and Rosalie Abrams and Baltimore City Council President Walter S. Orlinsky, three Democrats considered most likely to challenge Mathias.

If Bauman decided to run, a number of other Democrats reportedly were poised to enter the primary, believing, as Bauman conceded, that he might win the primary but lose the general election.

Even though he will not challenge Mathias in the May 7 primary, Bauman was not ready to endorse the senator for reelection.

He offered tongue-in-cheek congratulations to Mathias, one of the most liberal Republicans in Congress, for sending out a computer printed solicitation for money in which Mathias expressed "a new-found interest in a strong national defense and a balanced budget," and other conservative points of view.

Bauman said that while "I always believe in late conversions, in politics as elsewhere," the test of whether Mathias "is sincere" will be shown in his voting pattern this year. "He's got new music, now I want to listen to him sing," Bauman said.

Although he dodged the question of whether he might support a conservative Democratic nominee against Mathias in the fall, Bauman said "the obvious one" that might fit that description is State Senate President James Clark Jr. of Howard County.

Pressed as to whether he might abandon his party's nominee in favor of someone who shares his political philosophy, Bauman flashed his impish grin and said:

"As Kennedy says, I'll cross that bridge when I come to it."