Maryland congressman Robert E. Bauman, the darling of the Republican right wing, said yesterday he will meet with advisers late this month to help him decide whether he should challenge Sen. Charles McC. Mathias (R-Md.) in the GOP primary this year.

Bauman, president of the 200,000-member American Conservative Union, said mischievously: "Next to taking on Frank Church, there's nothing that would make a conservative's hair stand on edge" more than the excitement of trying to unseat the liberal Mathias.

The four-term representative from Maryland's Eastern Shore said he could raise $1 million "in a matter of weeks" from conservatives across the Mathias beaten.

Bauman whetted the hopes of conservatives this week by having one of his supporters pick up filing forms at the State House in Annapolis for both a Senate and House race, but he did test.

"The deadline is the end of February," Bauman said. He noted that filing fee is $100 for the House and $290 for the Senate, "which shows that not everyone thinks the two houses are equal."

Meanwhile, Mathias is stepping up his fund-raising and hand-shaking activities, in anticipation of the formal announcement, scheduled Jan. 26 in his home town of Frederick, that he will seek a third term in the Senate.

Nearly as nervous as Mathias over the prospects of an ideological blood-letting within the party is state GOP chairman Allan Levey, who said "I don't like a primary fight when we have an incumbent."

State GOP executive director Thomas Buckmaster was more succinct: "We're not blessed with an overwhelming Republican presence in the congressional delegation or the legislature. I'd like to see us preserve our holdings, and build on that."

The 42-year-old Bauman, who came to the House after a special election in 1973, and has been re-elected from the 1st District by increasingly wide margins, siad in a telephone interview form his home in Easton yesterday that "you come to a point in politics where you might as well go up or get out."

Party functionaries pointed to the possibility that Bauman could be elected minority whip in the House this year as one reason he is not likely to risk the security of that chamber for a race against Mathias, who has been the party's most popular vote getter for more than a decade.

"Entirely possible," responded Bauman to the suggestion that he will seek to move up within the GOP hierarchy in the House. "If I am elected to the House," he added, in the kind of teasing response that is frightening party loyalists who value a united front over philosophical debates.

Bauman already has earned a reputation, somewhat begrudgingly from Democrats, of being one of the foremost tacticians in the House. Within the Rules Committee, of which he is a member, and on the floor, where he is one of only a handful of members who nearly is always present, he consistently invokes parliamentary technicalities that prevent the majority Democrats from adopting legislation on voice votes.

An outspoken opponent of big government, Bauman's credo it, "everytime Congress is in session, American is in danger."

He could hardly be further across the political spectrum from his Repulican colleague, Mathias, whose liberal views over the years have produced invitations from Democrats for him to move across the party line. But Mathias, although admitting he has been tempted, contends Bauman and his conservative supporters are a vocal minority within the GOP.

Bauman disputes that, saying a poll taken last fall in the five urban counties of Maryland "showed me that Mathias is the trouble if a strong conservative runs against him in the primary." He would not make public the poll, or disclose its sponsorship other than to say it was conducted for "a leading national conservative organization" other than the American Conservative Union.

Mathias aides said they know of no such poll, and challenge its validity, as did party chairman Levey and executive director Buckmaster.

"I'd be surprised to see numbers that show Mac vulunerable in a primary," Buckmaster said.

Another salvo fired by Bauman during the current congressional recess is that money raised by Mathias "is either from liberal groups out of state, moderate-to-liberal groups in the state, or orginized labor."

A Mathias aide said the $150,000 raised so far has come from "a broadbased coalition." About $60,000 of Mathias'money came from a $250-a-ticket affair last October at the Capitol Hill Club attended by members of political action committees (PACs) representing bankers, labor leaders, engineers, attorneys, physicians and industrialists.

For his part, Bauman freely acknowledges that he has a list of 10,000 persons -- about 5,000 of whom live outside his district who contribute to his campaigns.

One Bauman supporter who believes that Bob wants to run for the Senate more than anything in the world," isn't convinced Bauman will risk his growing national reputation by challenging Mathias.

"The feeling is that if Mathias is weak, (Democrat Sen. Paul S.) Sarbans is weaker, and all this talk keeps Bob's name in the limelight for 1982," when freshman Sarbanes' term expires.

In the meantime, Maryland Democrats still are hoping to find a strong challenger to Mathias, or whoever wins the Republican primary, in the fall. State Sen. Victor L. Crawford of Montgomery County has annouced his candidacy, and Baltimore City Council President Walter S. Orlinsky is considering a run. Neither is believed to have the widespread support party leaders think is neccessary to oust Mathias, who beat then-Baltimore City Council member Barbara Mikulski by 150,000 votes in 1974. Mentioned as more likely to attract statewide support are state senaors James Clark Jr. and Rosalie S. Abrams. Clark is Senate president and Abrams is state Democratic chairperson.