THE ACCUSATION is that President Carter is neglecting his social responsiblities in his pursuit of more military power and less inflaton. The accusation comes from within Mr. Carter's own party, from those Democrats who take the greatest pride in the social advances of the 1960s and who rely most heavily on direct federal action to acieve further advances in the 1980s. Mr. Carter has already said that the only social initiative of any magnitude in his new budget will be an effort to reduce unemployment among young people.

Over the span of the Carter administration, it has not been defense that has led the increase in the budget. It has been Social Security, which has consistently grown faster than the military, faster than the total budget and, for that matter, faster than the national economy. Closely following Social Security come the other federal insurance and medical benefits. Just about half of the federal budget is now a gigantic insurance operation that offers Americans protection against a great variety of personal economic disasters. Because this protection is immensely popular and because most of it is indexed to the inflation rate, it is growing rapidly in a time when American's incomes are not.

President Carter is entitled to little of the credit -- or, if you prefer, the blame -- for the steady rise in social insurance benefits.Nearly all of it is happening automatically under legislation passed before he came to Washington. Because it is not his work, it is not accompanied by the bugling and drum-beating by which presidents call attention to their acievements.It is happening silently, but it is real, and it -- not Afghanistan or the MX -- has been the most important factor in shaping the present budget an d all the others in the Carter years.

That rise in insurance payments will continue, at the same time that most of Congress appears to agree with Mr. Carter that military spending must be accelerated. To add new interventionist social programs to the budget, in the style of the 1960s, would mean either higher taxes or higher inflation. But attempting social improvement at a cost in higher inflation is altogether self-defeating, since inflation itself has become an important cause of the social evils that the federal government is struggling to cure. Mr. Carter's youth employment campaign is an example. Inflation contributes to slow economic growth and high unemployment. When unemployment remains high for the labor force as a whole, it is idle to think that more money for training can make much difference for the youngest and most vulnerable workers.

The president's budget and, therefore, his social policies are in a vise. As long as Americans continue to give their unnqualified support to the automatic expansion of insurance funds like Social Security, there is not going to be much money available for other kinds of social endeavor. That is true for the current president, and it will be equally true for whoever is president a year from now.