The widely denounced "Vietnam syndrome" refers to a supposed American reluctance to get involved in messy Third World security situations that might lead to war. But there is another, less visible "Vietnam syndrome" -- a reluctance to get involved in messy Third World negotiating situations that might lead to peace. American policy in El Salvador shows a touch of it.

Vietnam gave negotiations a bad name. Our quest for peace came to be seen as a sellout of an ally flowing from weakness and from the illusion that we could salvage at the table what we had not gained on the battlefield. Nor, some add, was this simply a matter of a battle-weary popular mood or a detene-blinded political leadership. We tend to project our Anglo-Saxon taste for compromise on parties that scorn that tradition, it is said. We are vulnerable to tactics by which the other side, itself immune to political judo, uses our media and public opinion against a government trying to hang touch.

This heightened skepticism toward negotiations is the flip side of the familiar Reagan judgment that we must be prepared to fight our battles, and to support our friends in fighting theirs.

For some time in El Salvador, various international parties have been trying to set up a negotiating process. So far they have failed, and the question is why. If you could get an honest short answer from either side, it would surely be that each feels it can do better by going another route. The junta feels it can do better by fighting on and consolidating its power, or at least its American connection, in the elections it intends to hold next year. The guerrillas feel they can do better by fighting on and wearing the junta and its American patrons down. There is enough hate and distrust and weaponry around to let both sides test their shared judgment indefinitely.

So much for the short answer. There is also a long answer as to why the current stalemate of arms and of international support has not yielded to negotiation.

In addressing that question to the guerrilla side, one comes quickly to a document, "The Negotiations Maneuver," sent last Feb. 3 to the military command by the civilian opposition coalition or front. The gist of it was that the opposition should fake an interest in negotiating and press the battle. Junta foreing minister Fidel Chavez Mena presented word of it to a group of us at breakfast last Thursday by way of challenging the front's sincerity in calling for negotiations.

As it happened, front leader Ruben Zamora lunched with the same group the same day and, to my suprise, authenticated the document. He added only that it was just an option paper and did not reflect the front's policy now. I thought he bolstered the foreign minister's case.

What strikes me about El Salvador is the special dedication of the 1,000 or 2,000 core guerrillas, whose commitment to armed struggle seems to give them a powerful moral leverage over the civilians, including the many "moderate" middle-class people in the front, such as Ruben Zamora, a decent lawyer.

The long answer about the government's reluctance to negotiate also involves a consideration of the weight of the military as against the weight of the civilians within the junta. It is regularly asked, for instance, whether President Napoleon Durate could survive the launching of an earnest negotiating initiative. That question was freshened the other day when negotiations were rejected, and elections on the army's terms reaffirmed, by Col. Jamie Abdul Gutierrez, who is not only vice president but commander-in-chief. Durate, that is, does not command the army. pHe seems, too, from a distance, a decent lawyer -- actually, he's an engineer.

It is something of an article of faith in the Reagan administration that the guerrillas can take power only by negotiations, not by elections or battle. This is the Gutierrez line. But who can imagine that the army, which is responsible for much of the terror, can run elections that will do more -- if they do that -- than register the favor of its relatively few existing supporters? The administration, having already thrown its weight heavily to the military wing of the junta, has just rejected negotiations and endorsed the Gutierrez election stratgegy. At this point elections seem to me only another name for staying on the current course.

It takes two to tango, and all that. But here is the important thing. The military side of the junta has a connection to some of the terror, but it is fighting a war against guerrillas, and the administration would not want to cut it off. But only the civilian side can reach out to bring in either the mass of currently detached people or the legitimate part of the political opposition. For the United States to stiff-arm a negotiating process, even in part, because of a questionable set of "lessons" left over from Vietnam would be unworthy -- a second victory for the Hanoi way.