The Reagan administration passed up at least three initiatives by the Cuban government during the past 18 months to reduce tensions in Central America, the State Department's former top representative in Havana charges in a Foreign Policy magazine article released today.

The administration also exaggerated the extent of Cuban arms shipments to leftist rebels in the region, writes Wayne S. Smith, chief of the U.S. interests section in Havana until he resigned from the Foreign Service last month.

The U.S. evidence on arms shipments "has never been solid," according to Smith. "While some arms have been sent from Cuba to El Salvador, the quantities are almost certainly far less than alleged. If the guerrillas had received all the arms reported by U.S. intelligence, the Salvadoran army would be outgunned by 20 to 1."

The current U.S. posture is consistent with a record of two decades of militant confrontation toward Cuban leader Fidel Castro that has time and again worked against the best interests of the United States, Smith contends in the fall issue of Foreign Policy.

His sharp public criticism represents a rare case in which an official who has served at the fulcrum of a major foreign policy issue decided to leave the Foreign Service, and the anonymous channels of internal dissent, to speak out. Prior to his 1979-1982 Havana assignment, Smith was director of the State Department's Office of Cuban Affairs from 1977 to 1979.

Such resignations are highly unusual, but Smith is the second senior diplomat to leave in the Reagan administration because of differences over Central American policy.

The former ambassador to El Salvador, Robert E. White, took a similar step after being removed as envoy shortly after President Reagan took office. White also has since been outspoken in his criticism of U.S. policy in the region.

A State Department spokesman said yesterday in response to Smith's article that "successive U.S. administrations from President Kennedy onward have been concerned about Cuba's arms buildup and its persistent efforts to interfere in the domestic affairs of Latin American countries and of countries outside this hemisphere.

"All have sought to dissuade Cuba and its Soviet backers from these efforts. Unfortunately, concerted efforts to export revolution and subversion have long characterized Cuban policy . . . . We have made it clear on numerous occasions that a change in Cuban behavior, not just a change in rhetoric is what is required."

In his article, "Dateline Havana: Myopic Diplomacy," Smith details three initiatives by the Cubans to open talks with the United States on the growing confrontation over El Salvador and Nicaragua: the first shortly after Reagan took office in early 1981, the second in the late spring of last year and the third this past April.

Each time, Smith writes, the Cubans either were rebuffed or met with silence.

Smith says the first initiative came immediately after the Salvadoran army defeated a rebel defensive in January, 1981.

Cuba and Nicaragua had increased their support for the Salvadoran rebels in anticipation of the offensive, he says, but after it was defeated Havana and Managua "reevaluated their positions. Very shortly after Reagan's inauguration, arms shipments declined."

The Cubans then signaled their interest in pursuing a political settlement in El Salvador.

"The U.S. interests section in Havana reported these demarches to Washington, and I urged in several cables that we respond to the Cuban overtures, if only to emphasize the strength of U.S. feelings," Smith writes. "The Department of State never responded in any way."

State Department spokesman Susan Pittman, in answer to Smith's charges that Cuban approaches had been spurned, said yesterday, "We have never closed the door on dialogue and we have in fact maintained a dialogue with Cuba even though as Assistant Secretary Thomas O. Enders said last March, the U.S. finds the record on negotiating with Cuba 'daunting.' "

In the late spring of 1981, Smith says the Cubans, "concerned over rising regional tensions, stated they would favor mutual security guarantees and would be willing to play a positive role in bringing them about.

"The U.S. Embassy in Managua and I strongly recommended to the State Department that the United States pursue this opening . . . . During a visit to Washington a short time later, I was told there was no interest in such a negotiating process."

Instead, Smith says, the United States embarked on a series of steps to "escalate pressure against Cuba and to create uncertainty in Havana about American intentions."

"Why Washington expected Cuba to crumble under tough talk is difficult to understand. A senior policy maker explained in July, 1981, that the new administration was convinced its predecessors had not fully explored the possibilities of exerting pressure on Castro. It was determined that all options should be tried," Smith writes.

"My reply to this, in a letter dated that month, was that Castro had seen it all before. Neither harsh words nor harsh measures had ever succeeded. Hence unless the United States was prepared to carry its threats to their logical denouement, it had best not issue them at all. Bluffs would not work, for Castro was certain to call them."

Smith then says the Cubans, who had claimed in April, 1981, that they were not arming the Salvadoran rebels, told the United States in December that they had halted arms shipment to Nicaragua, which the United States says is the primary conduit to the rebels.

Following this, in April, 1982, he says Havana offered to begin talks, with no preconditions. Washington's response, he writes, was yet more attempts to isolate Castro, coupled with continued claims of arms shipments.

Arguing that contacts with the Cubans by then-secretary of state Alexander M. Haig Jr. and Special Ambassador Vernon A. Walters were stonewalling sessions, Smith says that genuine discussions "might have offered a productive alternative where confrontation failed. With the door closed to negotiations, the situation has continued to deteriorate."

The former diplomat gives an equally detailed accounting of the U.S.-Cuban wrangling over Havana's troops in Angola and Ethiopia, pointing out that a confrontational approach so far has failed to get them removed.

State Department spokesman Pittman responded to this section of Smith's account by saying, "Cuban intervention in those countries, which of course continues, speaks for itself about Cuba's self-assumed role of military collaborator and surrogate of the U.S.S.R."

Smith concludes: "What the United States has tried and what the Reagan administration is attempting all over again -- threats, pressures . . . -- has not worked. It is time to turn to a policy of gradual engagement."

But he cautions that such an approach "will not be easy" because the Cubans are "in a temper" over the recent U.S. rebuffs.

Even a gradual approach is unlikely to produce "miracles," he adds, since "Castro is a convinced revolutionary and many of his objectives are antithetical to U.S. goals."

"It is not impossible to deal with Castro," Smith writes. "Indeed, sooner or later Washington must do so, not because Americans like him or because they wish to be perceived as nice fellows, but in order to advance U.S. interests."