THE REAGAN administration is pursuing its campaign of confrontation against Libyan leader Col. Muammar Qaddafi despite intelligence estimates warning that such tactics for dealing with the erratic Qaddafi could backfire.

The intelligence and diplomatic reports describe Qaddafi as bizarre character who during one recent trip wore makeup, carried a teddy bear and refused to sleep on bedsheets provided by his hotel. A 1982 Central Intelligence Agency psychological profile and recent updates conclude that Qaddafi has a "borderline personality disorder."

The reports confirm President Reagan's judgment that Qaddafi is "flaky." They document the serious threat he poses to U.S. friends in the Middle East. But they don't support the view that he is at war with the U.S. or that he is international public enemy No. 1.

A close reading of these intelligence reports suggests that the administration's policy of confrontation, rather then containing Qaddafi, could bring on the nightmare that frightens everyone -- Qaddafi hit squads on the streets of America and Europe intentionally attacking U.S. citizens and property. By treating Qaddafi as a virtual war-time enemy, we could transform him into just that.

A major CIA analysis, classified "secret," warned less than a year ago how U.S. actions could affect Qaddafi. This report, a Special National Intelligence Estimate of March 1985, titled "Libya's Qaddafi: The Challenge to the United States and Western Interests," said:

"We believe Qaddafi would directly target U.S. personnel or installations if Qaddafi . . . believed the U.S. was engaging in a direct threat to his person or was actively attempting to overthrow his regime."

If left to himself, the estimate said, Qaddafi would probably focus until the fall of 1986 on regional subversion in Chad, Sudan and Tunisia and would exploit other diplomatic and political opportunities worldwide.

The estimate also said that Qaddafi might attack U.S. targets directly if he thought this could be done without a U.S. retaliation. Administration policy may encourage that dangerous confidence if the last month's saber rattling turns out to be only a bluff.

"Essentially Qaddafi is not controllable," said the estimate, which reflected the collective opinion of the U.S. intelligence agencies that diplomatic pressure and economic sanctions would have little or no impact on Qaddafi's behavior. The document noted Qaddafi's diplomatic achievements and notoriety during 1984 and 1985 and added that "these successes have bolstered his self-confidence and spur him to further and potentially more dangerous adventurism."

Despite these cautionary points from the intelligence community, the Administration has chosen an aggressive strategy. President Reagan approved a covert CIA plan last year to undermine the Qaddafi regime. After the Dec. 27 terrorist attacks at the Rome and Vienna airports that killed five American bystanders, the administration increased economic sanctions against Libya, beefed up the CIA covert-action plan, sent a high-level emissary to Egypt for contingency planning against Qaddafi, and moved two carrier battle groups toward Libya for flight operations and exercises.

The administration's rhetoric is a useful reminder of what the erratic Qaddafi is capable of doing to Western interests. But so far, Qaddafi's bite has been far less than his bark. The history of his actions, as recounted in intelligence summaries, repeatedly makes two key points:

*Qaddafi's terrorist attacks in the last 10 years have been aimed narrowly and almost exclusively at his Libyan opponents in exile abroad or at moderate Arab and African states.

*Qaddafi has rarely launched terrorist or other attacks directly at United States interests. The last such attack was in December 1979 when the U.S. embassy in the Libyan capital of Tripoli was sacked and burned.

There's no doubt that Qaddafi is a menace. He is a revolutionary who uses terrorism to undermine pro-western regimes in Egypt, Sudan, Tunisia and other nearby countries.

The question is how best to deal with Qaddafi. Many of the Arab regimes directly threatened by the Libyan leader have urged the U.S. to lower its rhetoric and stop its military posturing. They recognize that Qaddafi's regime lives off crisis and that a campaign of confrontation is likely to feed the fire that the administration wants put out.

The administration's own intelligence reports tell much about the hazards of dealing with Qaddafi. A classified CIA psychological profile of Qaddafi written in 1982 said that Qaddafi "is judged to suffer from a severe personality disturbance -- a 'borderline personality disorder' . . . under severe stress he is subject to episodes of bizarre behavior when his judgment may be faulty."

A "borderline personality disorder" is one of the hot topics in modern psychiatry. Essentially such a person is not psychotic, but is difficult and frustrating to handle because he alternates between crazy and non-crazy behavior.

Updates of the Qaddafi psychological profile describe the same tendencies, sources said. They add that during the last several years Qaddafi has used excessive amounts of sleeping pills and various "uppers."

A bizarre portrait of Qaddafi emerged after a December 1984 trip to Spain and Majorca. Diplomatic and other reports said Qaddafi was wearing high heels and that his aides brought a toy teddy bear to him; he also wore makeup and for some unexplained reason did not trust the bedsheets at the hotel where he was staying and dispatched aides to go out to stores and randomly buy new sheets.

One U.S. ambassador reported that Qaddafi's arrival with his entourage of aides and retainers could only be compared to that of singer Michael Jackson.

Administration rhetoric and actions have treated this eccentric character as a powerful adversary. At his Jan. 7 press conference, President Reagan said, "By providing material support to terrorist groups which attack U.S. citizens, Libya has engaged in armed aggression against the United States . . . ."

The U.S. is threatening to respond in kind to this "armed aggression." Referring to the new economic sanctions he was announcing, Reagan said, "If these steps do not end Qaddafi's terrorism, I promise you that further steps will be taken."

"It was an absolute tragedy," one Administration source said recently of the evolution of U.S. policy. "Libya does provide support to (terrorists) . . . but as we've learned so many times support is not control . . . and to virtually declare war on Qaddafi is way out of proportion to the threat and the problem."

In the course of escalating its anti-Qaddafi campaign, the State Department on January 8 released several papers and documents that are in fact sanitized versions or summaries of highly classified intelligence reports.

Though these public documents were intended to bolster the Administration's confrontation campaign, the papers make clear that Qaddafi is a regional threat, rather than a direct antagonist of the U.S.

Among the documents released by the State Department is a five-page summary entitled: "Chronology of Libyan Support for Terrorism 1980-85." A total of 58 incidents are described. Only two mention the United States. The first is the well-publicized murder of an anti-Qaddafi Libyan student in Colorado during July 1981. The second involves an alleged plan in May 1985 by 17 Libyans to kill anti-Qaddafi dissidents in the United States; the FBI thwarted the plan and no one was killed.

The chronology lists 56 terrorist acts in Europe (10 in Italy alone), the Middle East and Africa that in almost every case were directed at Libyan exiles or dissidents or anti-Qaddafi plotters, officials, diplomats or leaders who oppose Qaddafi.

U.S. intelligence analysts have for years concluded that Qaddafi is obsessed with killing off any possible opposition to his regime, either inside Libya or outside. He will go to absurd extremes in dispatching assassins to murder those opponents.

The State Department papers support this point, saying "The main targets of Libyan terrorist activities have been expatriate Libyan dissidents and leading officials of moderate Arab and African governments." The paper cites Qaddafi's efforts to kill or oust Egyptian President Hosni Mubarak, former president Jaafar Nimeiri of Sudan, President Mobutu Sese Seko of Zaire, Tunisian President Habib Bourguiba, and Chadian President Hissene Habre.

Qaddafi has been more aggressive in recent years and is even giving support to revolutionary movements in Latin America, the Caribbean, Southeast Asia and the Philippines, both the State Department and classified intelligence reports show.

But the Libyan leader appears wary of attacking American targets. Last month's seven-page State Department paper, "Libya Under Qaddafi: A Pattern of Aggression" devotes only two paragraphs under the heading, "Libyan Terrorism Against the United States." The first paragraph merely quotes Qaddafi rhetoric -- his much repeated threat to export his terror to "the heart of America" and to U.S. "streets."

The second paragraph refers to "several instances over the years of Libyan-sponsored attacks against U.S. interests." It cites three examples outside the United States and one more inside the U.S. The first is a plot to plant explosives in stereo speakers in a U.S. embassy club in Sudan; and the second a 1977 plan to assassinate the U.S. ambassador in Cairo. Both these were thwarted. The third incident was in May 1984 when the FBI arrested two Libyans near Philadelphia for attempting to buy silenced handguns. The fourth incident was in December 1979 when Libyans set fire to and sacked the U.S. embassy in Tripoli.

One administration official said that the president and his top foreign policy advisers find Qaddafi's terrorist campaign particularly repugnant because they believe in democracy and Qaddafi is, in the words of this official, "engaged in the most savage and relentless effort to snuff out any alternative."

Because Qaddafi's vendettas spill blood onto the streets of Europe, and even once in the United States, this official argued that it is proper for the United States to be concerned. When asked if this amounted to what the president called "armed aggression against the United States," the official did not answer.

The administration campaign has made Qaddafi an anti-Western celebrity and pushed Arab states that dislike him to hold their noses and -- in the name of Arab solidarity -- voice public support for the Libyan leader.

The unstable Qaddafi may view all the hoopla as an invitation to demonstrate his manhood, power and influence. Ronald Reagan has said Qaddafi is a threat to the United States, and the Libyan may have to prove it.

As the president himself said at the end of his press conference on January 7, "I just think that the man is a zealot. He is pursuing a revolutionary cause that could affect a great many countries." By portraying Qaddafi as the new international bad boy, the president is giving Qaddafi too much leverage over the United States.

Qaddafi will remain a menace. He won't go away if he's ignored. But by turning down the flame, the plans and the rhetoric, the U.S. can put him on the back burner where he belongs.