Federal Reserve Chairman Paul Volcker indicated yesterday the Fed will move as quickly as it can to remove the special credits controls it imposed in March, but it must be careful not to trigger a new round of speculation in financial markets.

The controls, which include a voluntary 6 percent to 9 percent limit on growth in bank lending and investments and some special deposit requirements that make consumer credit more costly for lenders, "inherently can't take care of . . . different conditions at different banks in different regions of the country," Volcker told a group of reporters.

"You don't want to leave it on any longer than it clearly appears to be justified," he declared.

Volcker added, however, that credit markets have been easing with interest rates falling for only six or eight weeks. "Credit demand was proceeding at an excessive rate only a short time ago (and) there are a lot of (loan) commitments out there.

"Attitudes can shift and change very rapidly in this world," he continued. "That's what we have to balance against the desirability of getting (the controls) off."

While stressing that the Federal Reserve would continue to seek "a restrained growth of money" that would be consistent with unwinding inflation, Volcker hinted the direct controls might be dropped one by one.

"We will be looking at this -- you can assume closely -- as time passes. And it's not completely black and white, you know. You don't have to go from where we are now to nothing all in one step," the Fed chairman explained. f

Volcker resisted imposing the controls, particularly the requirement that an amount of equal to 15 percent of any increase in consumer credit extended by larger lenders be deposited in a non-interest-bearing account at the Fed. Carter administration officials, thinking the step would help reduce inflationary expectations among consumers, insisted, and Volcker finally went along.

Yesterday Volcker, who met for lunch last week with President Carter and his top economic advisers, was asked if the administration agreed the controls should be lifted as quickly as possible. "I'll let them speak for themselves," he replied, "But I have no reason to think they would disagree with that general comment."

The controls may be dropped because the economy is dropping into a recession, interest rates are tumbling swiftly and the money supply growth shrank in April instead of growing at about a 5 percent annual rate, the Fed's target for 1980.

At a hearing before a House Banking subcommittee on a bill to "modernize" the Federal Reserve system Rep. Parren Mitchell (D-Md.) was sharply critical of Volcker and the Fed for allowing money supply growth to fall so far below its target and thereby contribute to unemployment.

"We won't achieve economic stability and you won't promote it unless you get M1-B (one members of the money supply) back on target." Mitchell complained.

Volcker maintained that the Fed is continuining to supply cash to the banking system through its open market operations but because of the slump in bank lending -- partly due to the voluntary limits imposed by the Fed -- the availability of those cash reserves has not been translated into an expansion of the money supply.

Fed officials expect the money supply to resume growing soon. As they continue to make reserves available, and interest rates continue to fall, at some point loans will pick up again and so will the growth of money, they believe.

But they certainly are not about to push any panic buttons, especially on the basis of a single month's monetary statistics, they insist.

As Volcker told an audience of savings bankers in Florida earlier this week, "I am at least as suspicious as any of you about interpreting any single month's figures. We know that some technical factors probably helped account for part of the April decline; the data are, in any event, inherently volatile.

"But, with all the qualifications," he continued, "the point remains; money and credit growth have slowed appreciably. . . . My point is that interest rates have not in any sense been 'forced' lower -- nor will they be at the expense of excessive growth in money and credit, at the risk of a resurgence in inflation and inflationary expectations."

That is the key to interpreting any controls. The controls may be dropped, one at a time, but that will signal no change in basic Fed policy. The 6 percent to 9 percent limit on increases in bank lending, for example, were set because they were consistent with the Fed's targets for money growth. w

They still are consistent, Volcker said yesterday, and even if controls are dropped, lenders and borrowers should keep that in mind.