INTEREST RATES are down -- a little. Most of the banks dropped their prime lending rate a quarter of a percentage point last week, from 13 percent to 12 3/4. The reason is that the economy is no longer growing as fast as it did last winter and spring. Inflation is staying relatively low. But, amid such good economic performance, why are the rates still so high?

The explanation is the great surge of borrowing that is now under way. With the country nearly two years into the recovery from the recession, businesses are straining their internal resources and increasingly going to the credit markets. There they have to compete with the voracious credit demands of the U.S. Treasury as it borrows to finance the administration's huge budget deficits.

All this borrowing has to be financed out of a trickle of savings that in this country remains remarkably small. Business, the consumer and the Treasury are all vigorously bidding for access to that trickle of savings, and the bidding keeps the interest rates high. The Treasury just sold an issue of one-year notes at a yield more than 7 percentage points higher than the inflation rate -- three or four times the yield that most investors considered adequate until several years ago.

Mr. Reagan's original economic plan was supposed to increase savings and provide a great surge of capital into business investment and industrial expansion at moderate interest costs. That, of course, has not happened. Of all the failures of the Reagan plan, this one is probably the most damaging in its implications for the country's industrial development.

High as the interest rates are, they would be higher still if it were not for the foreign money coming into this country. That's the link between the exchange rate of the dollar abroad and the interest rates here at home. If the flow of incoming foreign money were to drop suddenly, the exchange rate would fall and interest rates would rise very fast. This possibility now constitutes a substantial danger to the American economy.

A careful and foresighted government would avert that danger by reducing the budget deficit. That would diminish the Treasury's demands for credit and, as interest rates came down, the dependence on foreign capital would decline. One effect would be a gradual decline in the dollar's exchange rate, to the great benefit of all the American industries facing foreign competition.

That is what a careful and foresighted government would do. The present administration does not seem to be moving in that direction. Instead, it is standing pat and congratulating itself for all the good economic news. After all, didn't the prime rate just come down a quarter of a point?