Former congressman Robert Bauman has dropped out of politics twice in as many years but, to hear Eastern Shore politicians tell it, he still is the most imposing presence in Maryland's First Congressional District race.

As Republican C.A. Porter Hopkins trudges through the final days of his uphill fight against incumbent Democrat Rep. Roy Dyson, Hopkins' aides insist that accusations by Bauman have cost the GOP nominee support within the Republican Party.

And many politicians in both parties say that Dyson got to Congress in the first place only because of Bauman's personal problems.

"Anything Bauman does to keep alive the idea that he may run again someday is disastrous for Hopkins . . . and good news for us," said a staff member of the Democratic Congressional Campaign Committee. "We sense that this one is going to be ours. The only question is by how much."

Dyson's hold on the office was strengthened last summer when the Republican primary between Bauman and Hopkins turned into a bitter feud. Bauman pulled out of the race, asserting that Hopkins' aides had made "virulent and scurrilous personal attacks" on his personal life. In 1980, Bauman's admission that he had problems involving homosexuality and alcoholism cost him the seat in Congress that he had held for seven years.

Hopkins dismissed Bauman's allegations as ridiculous, but he could not dismiss their effect on his campaign.

Bauman's name remained on the ballot on primary day, since he had pulled out of the race after the withdrawal deadline. He came within 1,071 votes of beating Hopkins, an accomplishment that Bauman will not let anyone forget.

"The only thing more humiliating for Hopkins would be to be defeated by a dead man," Bauman said when he was weighing a challenge to the vote count. Four days later, Bauman decided not to contest the primary. But by then, according to Hopkins' aides, the damage had been done.

The national GOP, whose various committees can contribute nearly $67,000 in cash and services to a congressional candidate, only gave Hopkins $9,500. Even more damaging, the National Republican Congressional Committee did not include the contest in the list of the 102 "opportunity races" that it distributed to political action committees to help them decide where their money should go.

But Hopkins, a gentleman farmer who has poured $60,000 of his own funds into the race, has traveled from the massive district's northern tip at the Pennsylvania border to its southern reaches on the shore in an aggressive campaign against the incumbent.

He calls Dyson a "wishy-washy person . . . who got to Congress pretty much by luck," pointing to Dyson's victory by the slimmest of margins over Bauman just weeks after the disclosures of Bauman's troubled personal life. To support this, Hopkins' office has served up a steady diet of strongly worded press releases for the small newspapers and radio stations that dot the district.

One week Hopkins charges that his opponent voted for two "vastly different versions" of the balanced budget amendment, as well as for four radically divergent budget proposals. The next, he asserts that Dyson refuses to "defend his record in face-to-face debates."

The incumbent scoffs at the charges, although he was the only congressman to embrace all four budget proposals and he did vote for both the Republican and Democratic versions of the balanced budget amendment. He explained that he favors the balanced budget approach and voted for the other budget proposals, all of which were rejected by the House, because he felt that Congress had an obligation to pass one of them.

To Hopkins' allegation that he has refused debates, Dyson responded: "That's a joke. He goes to reporters and says, 'Dyson won't debate me.' It's a good way to get attention."

Still, the incumbent and the challenger agree on many of the issues that count in the conservative district: Both favor a strong national defense, balanced budgets and the New Right's school prayer amendment. Republican Hopkins agrees with the Reagan administration's economic approach, while Democrat Dyson joined with the Republicans and "boll weevils" to pass most of the major tenets of the Reagan plan.

The difference is more a matter of style. Hopkins, 52, has been teacher, businessman, gentleman farmer and state senator since graduating Phi Beta Kappa from Johns Hopkins University. He is a man equally at home shooting ducks near his Dorchester County farm or sketching them in the pad he carries on his campaign travels. He seeks votes with an air of indifference, shrugging off the idea of a Dyson victory by saying, "If that's what the people want, that's what they're going to get."

Dyson, 33, is a politician first. He left the University of Maryland in 1972 to take a job on Capitol Hill, and dreamed of becoming a congressman. He saw his chance to get into politics in 1974 when a new state legislative district was carved out around his southern Maryland home. Dyson ran and won and in 1976 took on the formidable Bauman, winning a surprising 46 percent of the vote. Fellow officeholders never cease to be amazed that since then Dyson always has seemed to be running and that his presence seems ubiquitous in the massive district.

State Del. William Cox, whose Belair home is three hours from Dyson's, recalls inviting him to a fund-raising breakfast for his own campaign. "It started at 8:30 in the morning, and I got there early," Cox recalled. "But Roy got there before I did."