PRESIDENT REAGAN likes to suggest that the years before he took office were a period of neglect of the military. Now comes an angry Jimmy Carter to say not so. In remarks to The New York Times, the former president accused Mr. Reagan of deliberately twisting the record, of persisting in saying things "he knows are not true and which he personally promised me not to repeat."

Part of the dispute is personal. Mr. Carter was badly beaten in 1980 in an election that was a referendum on his performance in office and his capacity to lead. Mr. Reagan, partly on the strength of his rendering of the comparative record then and since, has easily been the more effective public figure. That has to hurt. "This is the first time I have gone public," Mr. Carter said plaintively the other day, "but some of his statements are almost more than a human being can bear."

But their differences also have policy implications. The supposed decade of neglect that preceded his election was one of the justifications for Mr. Reagan's buildup. The less neglect, the less alarm one presumably needs to feel. Conversely, the earlier the buildup began, the less of it is chargeable to Mr. Reagan. If others had already started the shift of resources from domestic programs to defense, Mr. Reagan cannot be as readily pummeled, by those who would pummel, for also having done so.

Mr. Reagan said in his defense speech last week that "our modernization program -- the MX, the Trident submarine, the B-1 and Stealth bombers -- represents the first significant improvement in America's strategic deterrent in 20 years." Mr. Carter's accurate response is that all these programs originated in the Nixon-Ford-Carter years, when defense was supposedly being written off. The president suggested that defense spending was declining when he took office. In fact, it declined in real terms only in the early 1970s in the pulling-away from Vietnam, and then rose in the Carter term.

By no means did it rise as much as it has under Mr. Reagan. Nor did Mr. Carter assert the primacy of defense in the budget as Mr. Reagan has. But Mr. Carter's next point is the flip side of this: that defense has risen too fast under Mr. Reagan, destroying the "national consensus . . . necessary for sustained . . . improvements in the future."

There is, of course, a lot of second-guessing and positioning for history on both sides of the debate. The point here is not to oversimplify. The defense budget has risen a lot more under Mr. Reagan than under his immediate predecessors. It is also true that under those predecessors defense fared better than is often now recalled.