IN WARTIME, we're supposed to pull together -- but this shouldn't mean that we forget how to think. Yet as soon as the fighting against Iraq began, much of Washington's opinion-making class let down its critical faculties in a way it would be ashamed of in normal times.

Barely three days after the first strike on Baghdad, talk-shows and op-ed pages rang with the first great "lesson" of the gulf war: "High technology" worked for America's military. From that it followed that the Reagan administration deserved our thanks for increasing military budgets (although the Carter administration could fight for some credit too) and that the "defense reform" movement, which criticized the Pentagon's patterns of spending and procurement, had been proved conclusively wrong.

"This is just an unbelievable validation of what the defense industry has been doing," a Paine Webber analyst named Jack Modzelewski was quoted as saying in the Los Angeles Times. George Will looked upon the Patriot missile and concluded that, since it could shoot down Scuds, it was time to revive SDI. The Scud, of course, has a single warhead, which follows an absolutely predictable path as it falls. Soviet missiles, which SDI is supposed to intercept, have many nuclear warheads, each of which can be maneuvered as it homes in on its target -- and which, unlike Scuds, must be destroyed far, far away from the target if lives are to be saved. Distinctions like these would come automatically to Will or most other people in analyzing other subjects. They should be made about "high tech" and "defense reform" as well.

The Patriots have in fact been effective, which tells us something about "point defense" systems. But it proves almost nothing about the Reagan budgets or high-tech weaponry in general. To turn the logic around, the most "advanced" and expensive single weapon of the Reagan years was the B-1 bomber, which has not been sent on a single sortie over Iraq.

The argument about appropriate weaponry is often presented as a simple high-tech versus low-tech choice, as if it were a matter of preferring swords and muskets to laser-guided bombs and night-vision tanks. We should be interested in any level of technology that works; but we should be empiricists, wanting to be convinced case by case, via thorough testing and performance. The defense reform analysis was never about technology per se. It was, above all else, about military effectiveness, which led to a subsidiary bias against needless complexity in weapons design.

Complexity is not the same thing as high technology: a semiconductor chip is much more technologically advanced than a vacuum tube, yet it is much simpler, more robust and more effective in military or civilian use. By analogy, certain weapons use advanced technology to become more reliable: One example is the A-10 attack plane, which carried out very dangerous low-level missions in the first two weeks of combat without suffering any losses. Jamming devices and anti-radiation missiles, which home in on the enemy's radar, are technologically advanced and yet effective. (That these comparatively low-cost technical edges have not been the main object of the pundits' attention is a tell-tale indication of the true nature of their ardor.)

The problem with complexity is that it introduces more possibilities for failure -- within the machine itself, between man and machine and between the machine and the environment in which it's used. Highly complex weapons cost more to build and are harder to maintain, so for the same amount of money there are fewer of them in action. The appropriate level of complexity, then, depends on how effective they are -- in realistic tests and in combat.

What the empirical evidence from Persian Gulf combat tells us is . . . almost nothing so far. No one yet knows the most crucial fact about this war: how it will turn out. It seems certain that the U.S. side will prevail, since its material advantages are so overwhelming. (Defense Secretary Richard B. Cheney has emphasized that Iraq spent $50 billion over the past decade to build its vast arsenal. The U.S. military has spent $50 billion since Thanksgiving and over the last decade outspent Iraq about 50 to 1.)

But, despite the spectacular videotapes, we do not yet know how effective our weapons have been in the unpredictable circumstances of real war. Two days into the fighting, U.S. military briefers said they believed that all the fixed-site Scud launchers had been destroyed. The Scuds kept coming and, according to sources in the Pentagon, it now appears that most of the "fixed-site launchers" initially hit were in fact decoys. A Soviet expert told the BBC last week he had advised the Iraqis on techniques of deception such as hiding airplanes in apartment buildings and painting "craters" on runways to deter further bombing runs to keep the U.S. technical advantage from being put to full use. With similar tricks the Iraqi command-and-control system remained at least partly operational under intense attack.

Nor do we know how many of our weapons have slithered through windows and down chimneys, compared to how many have landed in empty fields or on civilian targets. The mistakes too are recorded on tape that is for now, and perhaps forever, unseen. After the bombing raid of Libya in 1986, the Pentagon released a video of a direct hit. That turned out to have been one of the few accurate bombing runs. A GAO analysis of the mission concluded that laser-guided bombs were actually less accurate than old-fashioned unguided bombs.

In the 1989 raid on Panama, a bomb from an F-117 Stealth fighter missed its target by over 300 yards, despite the Pentagon's initial claims of "pinpoint accuracy." In an astonishing interview last week on CNN, John Lehman said off-handedly that when he was secretary of the Navy he used to pay settlements "at least once a month" for damage done when laser-guided bombs hit resort towns in California and Nevada, two or three miles from the target area. Lehman later told Fred Kaplan, of the Boston Globe, that laser-guided weapons were hitting targets in Iraq about 60 percent of the time. This, Lehman said, was "consistent with the test performances" -- but quite inconsistent with the ratio suggested by the very few videos the Pentagon has shown.

Similarly, we have been told in briefings that Tomahawk cruise missiles, the only truly new weapon used so far in Iraq, have hit more than 90 percent of their targets. Yet just two days ago CNN ran footage of Tomahawks flying into Baghdad and destroying residential buildings. Cruise missiles may be promising, but we should see more evidence of their performance, especially about their vulnerability even to unguided fire from the ground.

Cost has been the great unmentionable in the "everything works" reaction. Against any enemy except one we outspend 50 to 1, our resources are limited. Choosing to build one weapon means choosing not to build or properly maintain another -- in the case of complex weapons, choosing to build one F-117 or B-2 means choosing not to build scores of A-10s or F-16s. During the Vietnam War, about two-thirds of all sorties flown were to drop ordnance. In the gulf, only about half are -- the rest are for the increased support these airplanes require. Expensive, delicate weapons have to be not just good, but far better than good, because of the options they foreclose.

Indeed, the wave of excitement about the weapons is itself a sign of trouble. If there were more realistic testing within the Pentagon, there would be less doubt about how the weapons would actually perform. But the system that develops the weapons has a bias against realistic testing; it is not by accident that overpriced fiascos like the B-2 bomber have rarely been tested at all, and then usually fraudulently. Both the Tomahawk and the Stealth fighter were proposed for use in Libya in 1986 -- and both were turned down, because the Pentagon was afraid they would fail.

The Pentagon procurement system -- with its bias against realistic testing and cost-cutting, with its potential to convert the officer corps into budget-boosters rather than military leaders -- is at the heart of the "reform" debate. Almost every outside analysis of the system's incentives and performance concludes that things are severely amiss. Paine Webber's Modzelewski gave the game away with his reaction to the first news from the war: "The real beneficiaries are going to be the big companies like General Dynamics, McDonnell Douglas and Lockheed." The weapons that work often emerge in spite of this system, rather than because of it -- for instance, the Air Force tried to wriggle out of the A-10.

Although it's now a cliche' to remark that there is still a long way to go in this war, and that major ground action is increasingly likely, many in Washington (in and out of uniform, with and without TV shows) don't seem to have realized that this prospect alone should dampen their technophoria. The consequences of technical failure look much different when we move from the Tomahawk to the Abrams tank or the TOW anti-tank missile. And as defense consultant Pierre Sprey observes, "When an air-to-ground missile fails, the pilot still comes home. When an infantryman's TOW fails, he does not come home."

The point of these cautions is not to say "nothing works" or to quibble about imperfections. The question we should consider, once we know enough about the gulf war to draw sensible conclusions, is whether our weapons proved effective enough -- enough to forestall grisly land warfare, enough to justify their great cost. We won't always enjoy such a one-sided advantage in spending and tactical realities. We must still think, rather than just enthuse, about the way to build our forces.

James Fallows, author of "National Defense," is Washington editor of the Atlantic Monthly. Scott Shuger, a former naval intelligence officer, is an editor of the Washington Monthly.