Even as President Reagan is ordering wider use of lie detector tests to try to stop security leaks within the government, the Pentagon's health director has charged that the polygraph "misclassifies innocent people as liars."

John F. Beary III, a medical doctor who is acting assistant secretary of defense for health affairs, told Defense Secretary Caspar W. Weinberger and other Pentagon executives in a memo obtained by The Washington Post that polygraph tests often can be misleading in determining whether people are telling the truth.

Beary's memo came amid a flap last winter over a draft Pentagon directive authorizing wider use of lie detector tests in leak investigations. That Pentagon directive was a prelude to Reagan's decision last month to make greater use of lie detector tests in all government agencies.

Henry E. Catto Jr., assistant secretary of defense for public affairs, said yesterday that he had received a copy of the Beary memo and supplemented it with a letter of his own to Weinberger warning that wide use of polygraphs within the Pentagon could be "a public relations disaster." Catto said the Beary memo "adds a technical rationale to my own public affairs perspective."

Beary, after having a physiologist on his staff, Jerome Bricker, make a preliminary review of the literature, made these claims in his memo to Weinberger entitled, "Scientific Limitations of the Polygraph:"

"No machine can detect a lie. The machine can only detect stress; however, the stress may result from several emotional causes other than guilt, such as fear, surprise or anger.

"Even setting aside the argument that the theory is flawed, there are accuracy problems. We have only been able to locate two scientifically acceptable studies so far . . . . In one the polygraph accuracy is 62 percent. In the other the accuracy is 72 percent.

"The polygraph misclassifies innocent people as liars. In one study, 49 percent of truthful subjects were scored as deceptive. In another study, 55 percent of the innocent were misclassified . . . ."

Beary closed his memo to Weinberger by stating: "I regret I have no alternatives to propose to solve the serious problem of security leaks, but I feel it is important for you to be informed of the serious scientific concerns about the theory of and the accuracy of the polygraph technique."

An aide to Beary, in confirming the authenticity of the memo, said the health affairs director "felt it was his duty as the senior adviser" in the health field to apprise Weinberger of his own views on polygraphs as well as those of experts in the field. The Beary memo was dated Dec. 16, 1982, when it was a classified document. It was declassified March 15.

Catto said yesterday that Weinberger "hasn't focused on" the polygraph issue yet, and that the Pentagon's general counsel, William H. Taft IV, is overseeing the matter at the moment.

The Reagan directive goes beyond the one now in force at the Pentagon by ordering departments to draft regulations that would enable them to require polygraph tests and punish employes who refused to submit to them.

Taft, when asked his view of lie detector tests, said of the polygraph: "If you use it with its limitations in mind, you can get some benefit from it. It can clear people," he said, and indicate "people behaving badly that you had no suspicion of."

The Pentagon last year launched an investigation in a vain effort to find who disclosed to The Washington Post the finding in an internal Defense Department report that it would cost $750 billion more over five years than Reagan had budgeted to buy the forces the military said it would need to carry out his policies.

The Pentagon notified a Pentagon official who had flunked three polygraph tests during that investigation that he was to be removed from his job, but he eventually was cleared despite the polygraph results.