Lie detectors can be outwitted by subjects who take a common tranquilizer before undergoing the polygraph test in a research laboratory, doctors have reported in Science magazine.

Among 44 student volunteers at the University of Pennsylvania, examiners correctly picked out 77 percent of those who lied on the test. But when students took a small dose of the tranquilizer meprobamate, the examiners were wrong 73 percent of the time in identifying the liars.

The examiners also were asked to judge which of the 44 volunteers had taken the drug. Their choices were 100 percent wrong.

"These tests were done in a laboratory and may not have the same results in the field," said Dr. Martin Orne, leader of the research team. "In the field, professional polygraphers set up a real confrontation between themselves and the client, they create extra stress . . . and are very intrusive." Dr. Orne said the result reported in Science is only preliminary, and the tests will be done in the field soon to see if the laboratory conclusions are borne out.

The lie tests, which are commonly used both by police and businesses, monitor heart rate, breathing and electrical changes in the skin during a period of questioning. The tests are based on the assumption that a person lying to officials on some questions will show a detectably greater stress in his physical reactions on those questions.

Despite the widespread use of the polygraph, the Pennsylvania researchers wrote in their Science article, "the validity and reliability of the polygraph tests have yet to be established and remain a subject of controversy . . . . One question that is important . . . is whether a tranquilizer" can reduce stress enough to destroy any effectiveness of the lie test.

The researchers, led by William Waid and Martin Orne, divided the 44 volunteers up into four groups of 11 persons each. One set of 11 was simply instructed to tell the truth. The other three sets of volunteers memorized half a dozen words that would appear on the lie test. After memorizing them, they were told to lie on the tests, saying they were not familiar with the words.

Of the 33 who lied on the test, one group had no pill, one group had a placebo (a pill containing no drug), and the last group was given the tranquilizer called meprobamate (or Miltown or Equanil, in two of its many trade names).

The professional polygraphers identified all 11 of the innocent volunteers. Among those who lied, and took no pill or a placebo, the polygraphers correctly identified 17 of 22 volunteers, or 77 percent.

Among those who lied but also took a tranquilizer to blunt their stress reactions, the polygraphers correctly identified only three of 11 volunteers.

The correct identification of the 77 percent was accomplished by use of the test of changes in electrical conductivity of the skin. Using only increased rate of heartbeat as a measure, the lie detector was wrong in 20 of 22 cases. Using only breathing as a measure, the lie detector was wrong in 15 of 22 cases.