SUPPOSE THAT budget deficits stay high through the rest of this decade and even keep rising. So what? Does it make any difference? The Congressional Budget Office provides a clear and balanced answer in its annual assessment of the outlook for the next five years. President Reagan, in his State of the Union message, once again said, "The best way to reduce deficits is through economic growth." Sorry, but that won't work. The CBO, also once again, warns that growth won't do it. It calculates that, even with rather optimistic economic assumptions, present policy will push deficits from $214 billion this year -- a little less than the administration's current estimate -- to $296 billion by 1990.

The country doesn't seem inclined to do much about it. President Reagan won't hear of tax increases or significant cuts in defense. Without cuts in defense, Congress isn't going to do much cutting anywhere else. Among the voting public at large, people agree with a yawn that deficits are terrible, then also agree that things seem to be going along pretty well in spite of them -- and turn the conversation to more rewarding subjects.

Even very large deficits, the CBO says, are unlikely to generate a sudden economic crisis. It could be a long time, the CBO continues, before even very large deficits began to result in stagnating standards of living, or in reduced defense capabilities. But though it happens slowly, it will happen -- and, as the damage accumulates, it will be increasingly difficult to reverse.

High deficits can lead to low capital formation, since the government is soaking up money that would otherwise go into productive investment. Lower investment means lower productivity and, in turn, lower economic growth. Perhaps the present stream of foreign capital will keep pouring into this country. It doesn't seem likely to continue at the present rate, year after year. If it should dry up, interest rates would shoot up and threaten what the CBO terms an explosion of federal debt -- a vicious circle in which high interest rates began compounding a rapidly rising debt. Along with all of these possibilities there is the reality that, whether the rise in debt is fast or slow, the pressures for inflation will rise with it.

In the politics of the federal deficit, the optimists are the people who hope for a financial crisis -- sharp, but manageable -- this year. Without some sort of crisis, they argue, there will never be a sufficiently strong sense of urgency in the government or in the country to force an end to the present drift. But the CBO is delivering the somber message that there's no reason necessarily to expect a crisis. The real cost may be, instead, a steady erosion of American prosperity, and economic power, that does not become apparent until the 1990s.