''Nothing ever gets settled in this town,'' Secretary of State George Shultz complained to a House committee some months ago. ''It's a seething debating society in which the debate never stops, in which people never give up, including me.'' And Caspar Weinberger, he might have added with a certain grim emphasis.

But Shultz didn't need to. As secretary of defense these past seven years, Weinberger has given new meaning to tenacity. Nor has there been any secret about the seething debates that have all too often characterized relations between the two secretaries and their departments the past five years.

That would be reason enough to expect that the impending departure of Weinberger (apparently for personal reasons unrelated to policy) will make a significant difference in the general way the administration conducts national security affairs in its final year -- and in the way it approaches arms control negotiations with the Soviets, in particular.

Arms control, former national security adviser Brent Scowcroft told reporters recently, is ''the most difficult issue for the president to come down on because his heart is with Cap Weinberger and his head is with George Shultz.'' Reminded in a telephone interview of that remark in light of Weinberger's leave-taking, Scowcroft responded: ''Now he's going to lose his heart.''

By this, Scowcroft meant that the president and his longtime associate shared the same deep, emotional distaste for dealing with the ''evil empire''; that Weinberger's easy access regularly reinforced the president's distrust of the Soviets; and that his resignation would remove a badly needed ''brake'' on arms control negotiations that Scowcroft and other critics believe are moving in the wrong direction and too fast. Supporters of the way arms control bargaining is proceeding, on the other hand, will welcome the absence of Weinberger as the removal of a monkey wrench in the policy-making machinery.

Either way, it comes down to the same thing, not just with arms control but more or less across the board. The rancor level will be reduced. Shultz will have a freer hand and better hope of realizing the role he claimed for himself at that same House hearing as the president's ''principal foreign policy adviser.'' More decisions may stay ''settled.''

So it is easy to make the case that the removal of Weinberger (described by the late Theodore White as a ''man obsessed with his mission'') and his replacement by a skillful technocrat like Frank Carlucci (who played important supporting roles in Democratic as well as Republican administrations) will shift the administration's center of gravity from ideological dogmatism to prudent pragmatism. But it is also easy to overestimate the extent of the shift or to misread some of its implications for U.S. policy.

For one thing, the change at Defense is only the latest piece in a pattern: Vernon Walters for Jeane Kirkpatrick at the United Nations; William Webster for William Casey at CIA; Howard Baker for Don Regan and Carlucci for John Poindexter at the White House. For another, Ronald Reagan will still be president, still beset by the same inner conflicts, still responsive to the clamor of his old conservative constituency.

He wants arms control agreements before he leaves office. But Weinberger could not have blocked the first step. A deal on intermediate-range nuclear forces has been all but made. The president also wants to preserve his ''Star Wars,'' with its dream of airtight nuclear defenses, which is the key to a second-step agreement on a 50 percent cut in strategic missiles.

Here Weinberger might have made a difference. He has resolutely resisted a compromise with the Soviets on how to interpret the prohibitions of the Anti-Ballistic Missile Treaty on the testing of defensive systems in space. But even if the State Department's more flexible position on the meaning of ABM (forget the legalisms, negotiate the nuts and bolts) now prevails, it doesn't necessarily mean that the worst fears of the arms control critics -- a ''sloppy deal'' on strategic arms, as Scowcroft puts it -- will be realized.

Time is one factor. The problem of Senate ratification is another. A case can be made that any agreement reached without Weinberger in on the process will be more vulnerable to conservative challenge.

One pictures Weinberger proudly presiding over the largest peacetime military buildup in history. But the Gramm-Rudman-Hollings hatchet would still be hanging over the defense budget even with Weinberger on the job. One remembers that Weinberger has consistently been more hesitant to commit U.S. military forces to combat than Shultz (witness Lebanon). But his sensible reservations reflected those of the uniformed military -- and the Joint Chiefs will still be on the job.

Things will be different without Weinberger, but not to the degree, and not necessarily in the ways, that you might suppose.