The Justice Department yesterday refused to provide the Senate Labor Committee with the secretly recorded conversations of a reputed Mafia soldier involving Secretary of Labor Raymond J. Donovan.

Although the tapes were provided last year to the alleged mobster and his co-defendants in two criminal cases, the FBI took the position that the contents "could compromise the identity of a confidential source" and might "result in danger to the source and others close to the source."

By contrast, special prosecutor Leon Silverman, whose staff has already reviewed all of the so-called "Masselli tapes," has said he had no objection to the committee's request.

Administration officials, it was clear, were still miffed by the committee's release last weekend of certain incriminating allegations and other information about Donovan that had remained buried in the FBI's files during last year's stormy Senate confirmation hearings.

Committee Chairman Orrin G. Hatch (R-Utah) and Sen. Edward M. Kennedy (D-Mass.) decided to make the details public and called on the FBI for an explanation of what went wrong.

Justice Department spokesman Tom DeCair indicated that no thought was given to providing the tapes to the committee in return for a pledge to keep them secret.

"There's no reason to believe that anything given to Sen. Hatch can remain confidential," DeCair said. "The last batch of stuff we sent up there wound up coming out in Senate report with a stamp on it saying 'Embargoed for Sunday release.' "

In a letter delivered yesterday to Hatch and Kennedy, Assistant Attorney General Robert A. McConnell said another reason for the denial was that "the material requested also contains allegations of improper conduct, uncorroborated by investigation, concerning government officials" other than Donovan.

"The release of this information will constitute a clearly unwarranted invasion of their personal privacy," McConnell said.

The tapes are the fruits of seven months of FBI eavesdropping and wiretapping in 1979 at the Bronx warehouse of William P. Masselli, whom the Bureau has described as "an alleged self-admitted soldier" in the Genovese crime family.

Masselli also was head of a trucking company that grew into a multimillion-dollar business as a subcontractor on New York City subway projects for Donovan's company, Schiavone Construction of Secaucus, N.J.

Masselli's company, Jopel Construction and Trucking, moved in by taking over the assets of another Schiavone subcontractor, Louis R. Nargi, who reportedly had fallen into debt to Masselli.

In their March 19 Justice request Hatch and Kennedy said that "Some unanswered questions remain from last year's Senate confirmation hearings regarding relationships between the Schiavone Co. and its officials, including the secretary, William Masselli and the Jopel Co."

The questions, the senators continued, "involve allegations of labor racketeering," and the Masselli recordings, they said, were "vital evidence . . . for some of the allegations we are examining."

Accordingly, they asked for those portions of the Masselli tapes that contain:

* References to, or conversations with, "Ray" Donovan or any one of a number of other Schiavone Construction Co. officials, including Ronald Schaivone, Al Mangrini, Richard Callahan, Jerry Liguori, Joseph DeCarolis and Vito Paradise.

* References to business or personal arrangements between Masselli and his associates and Schiavone Construction and its officers.

* References to work done by Masselli's predecessor, Nargi, to the difficulties he faced and to "a takeover of Nargi's business and resulting violence."

The takeover reportedly led to a mob "sitdown" over the jurisdictional niceties involved, and may have been a factor in the subsequent killing in 1978 of an organized crime figure.

FBI spokesman Roger Young insisted yesterday that release of the tapes the committee wanted "together with certain information which may or may not be available to the public" still could endanger the life of a source. He said he could not be more explicit, but reiterated, "It is, I assure you, a real concern."

Silverman, whose work began in December with allegations of a $2,000 union payoff involving Donovan, has said he intends to include the results of his review of the tapes in his final report.

FBI and Justice Department officials, however, said that this will not include the items that have them worried. The Senate request, DeCair said, "goes to the Schiavone Co., and the special prosecutor may not."