Federal Reserve Board Chairman Paul A. Volcker, side-stepping comment on President Carter's criticism of Fed policy, yesterday acknowledged that he, too, is concerned that interest rates have jumped too quickly in the past few days.

Using somewhat different language, Volcker appeared to express agreement with the observation by Treasury Secretary G. William Miller on Thursday that private commerical banks had overreacted by raising the prime interest rate to 14 percent.

"The prime rate is a little more jumpy on the upside than it is on the downside," Volcker said, "and I think this may be one of those cases."

Meanwhile, the nation's basic money supply fell $3.4 billion to a seasonally adjusted average of $382.7 billion, while the broader money supply (M1-B) dropped $3.6 billion to an average of $405.7 billion. In the past several weeks, the money supply had been expanding rapidly, well beyond the targets of the Fed.

The money supply expansion has been one element of uncertainty in the markets, leading to expectations of rising inflation and stimulating the upward surge in interest rates. Volcker said yesterday that efforts to control money supply growth eventually would be successful.

"I think we've learned -- people generally have learned -- that it's not something you control from week to week. I think we have learned what we always knew, but we learned it in spades this year. You don't control the money supply from month to month, either. But if you look at it in broader terms, I don't have any sense that the money supply is out of control."

Talking to reporters at the annual meeting of the World Bank and International Monetary Fund, Volcker said that the tendency of the private credit markets "to jump and anticipate obviously gives us problems, and you wonder whether it hasn't jumped and anticipated too much."

But Volcker refused to say whether he saw a danger that rising interest rates might choke off economic recovery. On Thursday, Carter had sharply attacked Federal Reserve policies as "ill-advised" and laid the blame for the current high interest-rate pattern on Fed policy.

Miller also had said that neither he nor the Fed "have a handle" on what is happening. But Volcker, asked if he had a handle on what is going on, said: "You never know every fact about everything and feel entirely comfortable. But yes, I think we have some sense of what's going on."

Meanwhile, the White House returnedto an attack on the banks for boosting the prime rate. Press Secretary Jody Powell said that "neither the president nor his advisers see the latest increase in the prime as being justified by what is happening to the cost of money."

But another high administration official attempted to soften at least one aspect of the president's comments on Thursday. In campaigning in Landsdowne, Pa., Carter had warned that "Congress and I together would have ultimate authority to override some of the consequences of the Fed's decisions [on focusing attention on money supply targets]."

Stuart E. Eizenstat, head of the White House domestic policy staff, used a briefing on legislative issues to emphasize that Carter was not talking about negating the Fed's actions per se but rather the possible consequences of its actions. For example, Eizenstat indicated that if Fed actions caused a housing crunch, the administration might have to do something to ease that crunch.

In a rejoinder to Carter's initial blast, former Treasury secretary George P. Shultz charged at press conference that, by demanding that the Fed ease up on its attempts to control the money supply, Carter "lays in a certainty of higher inflation and higher interest rates" in the months ahead.

Schultz, a Reagan adviser who frequently is mentioned as a possible secretary of State in a new Republican administration, noted that Carter had appointed all but two of the present seven-member Fed Board of Governors.

Former President Gerald Ford also joined in the fray, saying that Carter had acted in a "cowardly" way in criticizing the Fed. "I find it unbelievable," Ford said during a political appearance in New Jersey. "He appointed them. They're his board."