The Senate yesterday brushed aside four budget alternatives aimed at eliminating or drastically reducing deficits by the end of the decade as a small band of Republican moderates continued to hold up GOP agreement on a low-tax, high-deficit budget.

As the moderates pressed behind the scenes for tax increases to help reduce deficits, the Senate:

Voted, 75 to 23, against a proposal by Orrin G. Hatch (R-Utah), backed by the U.S. Chamber of Commerce and other business groups, that would have preserved President Reagan's tax cuts and reduced deficits, largely by clamping down on domestic spending.

Voted, 82 to 16, to reject an alternative deficit-reducing proposal from Ernest F. Hollings (D-S.C.) to repeal the remainder of Reagan's tax-cut program, partially freeze domestic spending and sharply curtail the military buildup by slashing Reagan's proposed 10 percent after-inflation growth to 3 percent.

Voted, 83 to 13, against a plan sponsored by J. Bennett Johnston (D-La.) and several other moderate-to-conservative Democrats that would have imposed across-the-board limits on spending growth, including caps on cost-of-living increases for benefit entitlement programs. It would also have raised taxes enough to roll back most of Reagan's future tax cuts, including inflation adjustments in tax rates after next year.

By voice vote rejected a plan by Charles E. Grassley (R-Iowa) that would have frozen defense and domestic spending but would have left Reagan's tax cuts in place. Also rejected, 50 to 45, was a proposal by Max Baucus (D-Mont.), endorsed by the Democratic caucus, to add back $400 million for Medicare. That action halves the Budget Committee's proposed cuts in the health care system for the elderly and restricts savings to cost-containment measures that would not drive up costs paid by patients.

As Republicans held back their new budget plan in hopes of gaining at least 51 GOP votes for it, Democrats began to grumble about the delay and suggest that Reagan, in embracing the effort, was reneging on vows of bipartisan cooperation in developing the budget.

The GOP plan, which is expected to be offered as a substitute for a bipartisan budget approved last month by the Senate Budget Committee, would deny Reagan most of the domestic spending cuts he wants but give him most of his defense buildup and preserve his tax cuts.

The committee plan would raise taxes enough to jeopardize the 10 percent income tax cut scheduled for July and indexing of tax rates to inflation in future years.

The Republican moderates, who include Mark O. Hatfield (Ore.), Robert T. Stafford (Vt.), John H. Chafee (R.I.), Lowell P. Weicker Jr. (Conn.) and Charles McC. Mathias Jr. (Md.), are holding out for at least some tax increases to bring down deficits.

Majority Leader Howard H. Baker Jr. (R-Tenn.), joined by White House chief of staff James A. Baker III, met with the moderates late yesterday in an unsuccessful attempt to resolve the dispute.

Meanwhile, David A. Stockman, director of the Office of Management and Budget, took issue yesterday with administration critics who claim that the Reagan program has been unfair to the poor, tilted toward the Pentagon and away from domestic spending.

Testifying before the congressional Joint Economic Committee, Stockman claimed that the "fairness issue" stems from charges that "have been tirelessly repeated by partisan critics and reflexively photocopied by the press."

Stockman sought to refute what he identified as the three basic charges of unfairness against the Reagan program: that poor people have borne the brunt of budget cuts; that the three-year tax cut benefited the rich; and that the administration has tilted the budget toward military spending.

Contrary to what Reagan critics say, the administration has made only "marginal" and "modest" changes in actual spending for domestic social programs over those contemplated by President Carter in his last budget, Stockman said.