Federal Reserve Chairman Paul A. Volcker told Congress yesterday the Fed intends to stick with its tight-money policy to fight inflation, and warned that the only way to get interest rates down was to cut spending further or increase taxes.

Interest rates have stayed at near-record levels, Volcker said, because financial markets "are reacting to the harsh reality of continuing inflationary momentum and heavy current and prospective financing demands" arising from large federal budget deficits.

Volcker, in testimony to the Senate Budget Committee, said that "in voting tax reductions by large majorities, the Congress accepted the challenge of cutting the spending suit to fit the revenue cloth." The prospective revenue cloth is so shrunken, he declared, that "in terms of actions yet to be taken by 1984 you are talking, in round numbers, about $100 billion."

"Amid encouraging signs of progress on inflation, with your strong efforts toward expenditure control, with firm monetary restraint in place, the markets seem to be expressing doubts," he continued. "But after all, Americans have not seen for many years a successful fight on inflation or balanced budgets or so massive a tax reduction. A lot of bets on the future are still being hedged against the possibility that you, and we, will not carry through."

President Reagan and his aides are putting together a package of spending cuts for fiscal 1982 of about $15 billion in an effort to hold the budget deficit at $42.5 billion. Cuts no larger than that more likely would produce a deficit of $60 billion or more, according to many analysts including those of the Congressional Budget Office.

CBO director Alice Rivlin presented numbers to the committee Tuesday generally agreeing that cuts of about $100 billion would be needed to balance the budget in 1984 even if the economy performs well in the meantime.

Volcker said the size of the federal budget deficit is more critical than in the past because total government borrowing needs, including those of off-budget agencies, are absorbing about half the country's savings other than that used to replace worn-out capital goods.

"The effect of a deficit on the economy and capital markets can only be judged in the context of a particular economic situation," Volcker said. "There may be relatively little risk of 'crowding out' in a period of high actual or potential savings, falling investment demands, adequate homebuilding relative to demands and low interest rates. But that surely is not the circumstances of today, in which we have a clash in the market among competing demands."

Volcker also rejected credit controls, which have been suggested recently by some congressional leaders as a way to reduce interest rates. Trying to ration scarce credit and savings by controls would prove "ultimately unenforceable and efforts by borrowers to protect themselves and consequent market disruptions would likely only make the situation worse," he said.

Short-term interest rates have fallen recently from their highs, but long-term rates have moved down only slightly. Several members of the committee expressed concern about the impact high rates are having on constituents. However, none called for the Fed to move to an easier policy stance.

Sen. William L. Armstrong (R-Colo.) said the situation involves "a human tragedy of immense proportions" with families doubling up in housing, children not able to go to college and even "families breaking up." Nevertheless, Armstrong advised Volcker, "Stick to your guns."

Several committee Democrats blamed the Reagan administration for the high rates. Sen. Gary Hart (D-Colo.) accused the administration of "promising the people they can have everything," including large tax cuts, higher defense spending, lower inflation and low interest rates even though neither Reagan nor Congress is "even close" to identifying the unspecified cuts needed to reduce budget deficits in future years.

Hart urged that the personal income tax cuts in last month's bill be postponed so the budget could be balanced within 24 months.

Asked by committee chairman Sen. Pete V. Domenici (R-N.M.) why interest rates have come down so little even though inflation has fallen "substantially," Volcker replied, "I would not use the adjective 'substantially.' "

"I would say we have moved the inflation rate down, oh, 1 to 2 percent," the Fed chairman said. Additional, sustainable improvement in inflation will not come until wage rates begin rising less rapidly, he said. "At the very least, it is far, far too early to claim victory . . . Evidence of sustained decline is yet to come."