A rare public outburst by Secretary of State George P. Shultz helped to persuade the White House to back away -- for the third time in three years -- from efforts to expand the use of polygraph examinations among government officials.

Shultz's uncharacteristic public dissent Thursday and the turnabout in administration policy Friday put him at odds with CIA Director William J. Casey, reportedly the prime mover in favor of expanding polygraph tests, and with Defense Secretary Caspar W. Weinberger, also reportedly a strong advocate within the Cabinet of polygraphs.

Casey, Weinberger and the new White House national security affairs adviser, Rear Adm. John M. Poindexter, are described by administration officials as up in arms about losses of U.S. secrets through espionage and about leaks from government inner sanctums to the nation's news media.

Reagan's decision to sign the Nov. 1 order, including expanded polygraph tests, was based on staff advice that it was necessary to stop espionage activities, including those of the Soviet Union and China, according to White House officials.

Lie detectors, as the controversial emotion-measuring devices are known, are seen by their advocates as valuable weapons against both espionage and news leaks. Critics say polygraphs are inaccurate and can be easily evaded by people trained to do so.

In March 1983, Reagan signed National Security Decision Directive 84 requiring government officials with access to classified information to submit to polygraph examinations on request, in an effort aimed primarily at leaks to the news media. After an uproar in Congress and the threat of legislation to stop it, Reagan suspended this part of his order in February 1984, before it took effect.

In September 1983, Reagan ordered polygraph tests of senior administration officials and others who had access to information about a decision to order U.S. air strikes on Syrian positions in Lebanon. This decision was leaked to news organizations within hours after it was made.

Shultz and then-White House chief of staff James A. Baker III are reliably reported to have threatened to resign if required to take polygraph tests, and Reagan quickly dropped that aspect of the probe. An extensive Justice Department investigation of the Lebanon leak, including Federal Bureau of Investigation interviews with a number of officials, failed to identify the source of the disclosures.

The most recent effort to broaden the use of polygraphs beyond the Central Intelligence Agency and highly secret National Security Agency, where they have long been used, apparently arose primarily from concern about leaks to spies -- although a separate set of internal administration meetings is reported to be studying ways to stop unauthorized news disclosures.

A series of espionage arrests this spring and summer, especially the celebrated Walker family spy case, helped to spur a National Security Council-sponsored study of ways to counter other nations' efforts to get U.S. secrets.

An Aug. 7 meeting of Reagan's National Security Planning Group, a highly secret forum for top officials to discuss national-security policy, took up recommendations from a senior interagency group on counterintelligence.

Among the recommendations, which were endorsed as a result of the meeting, were tighter control over the movement of Soviet and Soviet bloc diplomats and other personnel in this country, the replacement of many Soviet and Soviet bloc nationals at U.S. embassies with U.S. citizens, and gradual equalization of the number of Soviet diplomats assigned to this country -- currently about 320 -- with the number of U.S. diplomatic personnel in the Soviet Union -- currently about 270.

There appears to have been little controversy among top decision-makers except over creation of a "national polygraph program," including authorization of tests for "all individuals" in the executive branch and government contractors with access to certain types of secret information. At least 183,000 federal employes and contractors, including 4,550 in the State Department, would have been included.

In the policy shift announced Friday, the White House said polygraph tests would be required government-wide only as a "limited" tool in conjunction with other procedures in espionage cases.

Shultz's opposition to polygraph tests was known from the 1983 White House episode, and at the Aug. 7 meeting Shultz is reported to have made clear his belief that polygraphs often mistake the innocent for the guilty. Shultz's objections, according to associates, are based in his sense of personal integrity and public responsibility.

Shultz, speaking with unusual force and emotion, said as much Thursday in announcing before reporters, "The minute in this government I am told that I'm not trusted is the day that I leave."

Despite Shultz's remarks in the Aug. 7 meeting, the intelligence bureaucracies proceeded in drafting National Security Decision Directive 196, including the order for expanded polygraph tests. Reagan signed it Nov. 1, the day before Shultz left for Moscow to prepare the way for the Geneva summit.

As soon as Shultz received a copy of the Reagan order, he knew "he had a problem," according to an official familiar with his thinking. At this point, however, the order was still a secret, so there was no threat of public controversy. Also, the order set up a task force -- including a State Department representative -- that provided a forum for future battles over how to apply the broad language Reagan had adopted.

Nobody knew what the polygraph sections of the order would mean in practice, according to several officials. Initial meetings of the task force did nothing to settle the question, and it was agreed that policymaking about the polygraph issue would be put off until January.

This changed Dec. 11 with a Los Angeles Times story by Robert C. Toth that disclosed Reagan's Nov. 1 directive and emphasized the stepped-up polygraph tests. Shultz, in Europe when the news broke, refused to answer questions about the polygraph, on grounds he would discuss such a domestic question only at home.

"Shultz had told people in confidence in the weeks before what he thought about the polygraphs, and it was a matter of deep conviction," said an aide. He added that questions from reporters, following disclosure of the order, "forced his hand," leading to Shultz's declaration that, in a contest between his sense of personal integrity and his loyalty to the administration, he would choose to follow his own star.