Special prosecutor Leon Silverman today asked the key witness in the investigation of Secretary of Labor Raymond J. Donovan to take a lie detector test but insisted that it be confined to one allegation.

The witness, Mario Montuoro, former secretary treasurer of Laborers Union Local 29, said he would submit to a polygraph examination only if it covered all of his charges of improper relationships between Local 29 and Donovan's company, Schiavone Construction Co., of Secaucus, N.J.

Silverman refused, demanding that the test be limited to Montuoro's charge that Donovan was present at a luncheon in 1977 when another Schiavone executive handed the president of Local 29 an envelope containing $2,000 in cash.

Silverman wanted Montuoro to take the test Friday, but Montuoro refused.

"I'll take the lie detector test on 42nd Street" only if it includes "everything," Montuoro told Silverman in a telephone conversation this afternoon.

The impasse was surprising since it came after Montuoro testified for 2 1/2 days before a federal grand jury in Brooklyn.

During that testimony, Silverman questioned Montuoro at length about the 1977 luncheon at Prudenti's Restaurant in Long Island City and about several other illegal favors Schiavone Construction allegedly bestowed on the Local 29 leaders.

Montuoro met reporters in his lawyer's Manhattan office this afternoon to express his short-lived satisfaction.

"The guy Silverman said he's going to cover everything and he is, whether it's got anything to do with Donovan or not," Montuoro declared confidently. "He's going into every aspect of Schiavone."

According to Montuoro, Silverman and his aides apparently have compiled evidence that Montuoro was at Prudenti's in mid-1977 as he has claimed. Montuoro told prosecutors that, on the way to the restaurant's restroom, he saw some people he knew.

Montuoro told reporters today that he understands that prosecutors "have credit-card receipts from the accounts of the people I saw there."

Montuoro said he also testified about "no-show" employes on the Schiavone payroll as well as free lumber and equipment that a Schiavone official made available for Local 29 officials.

Montuoro's lawyer, Arthur Schwartz, told reporters he is more skeptical and, to support his view, cited a recent conversation he had with one of Silverman's aides.

"He the aide said he didn't think they had the authority to prosecute anyone else, other than Donovan," Schwartz said.

A few minutes after Schwartz's remark, Silverman phoned to request the lie detector test. Montuoro and Schwartz took the call in another room, but their end of the conversation could be heard clearly throughout the law office.

"He Silverman said they were going to limit it to Prudenti's," Montuoro recounted moments later. "I said, 'Forget about it.' "

Montuoro said Silverman later offered a "maybe" lie detector test about other issues but refused to put the offer in writing. Montuoro said he could not accept that.

"He Silverman said, 'I will interpret my authority as I see fit, and I only want to give him a lie detector test about the restaurant,' " Schwartz recalled.

"He Silverman said to Mario, 'Do you know Donovan knew about the no-shows?' and Mario said, 'No, how can I say that?' Then Silverman said, 'Then I'm only going to ask about the restaurant.' "

Silverman could not be reached for comment.