The Senate Labor Committee endorsed the nomination of Raymond J. Donovan as secretary of labor yesterday despite Democratic misgivings that seem likely to grow louder on the Senate floor next week.

The vote for Donovan, coming on the heels of an intensive FBI investigation concerning alleged links to organized crime, was 11 to 0. But five Democrats, led by Sen. Edward M. Kennedy (Mass.), voiced their continuing doubts by simply voting "present."

Committee Chairman Orrin Hatch (R-Utah) said he felt Donovan had been badly maligned in the dispute over his nomination. Hatch maintained that Donovan, 50, a New Jersey construction company executive, had weathered the FBI investigation with his reputation for honesty and integrity unscathed.

The FBI investigation more than a dozen allegations linking Donovan or his company to union corruption or organized crime, but was unable to corroborate any of the claims despite more than 100 interviews and lengthy inspection of corporate records. In a letter to the committee yesterday morning, FBI Executive Assistant Director Francis M. Mullen Jr. said that nothing new has come to the bureau's attention that would warrant further investigation.

Yesterday's hearing began in a hectic atmosphere that included the sudden appearance, on NBC-TV's "Today" show, of Donovan's chief accuser, Ralph Picardo, an FBI informer who had reportedly declined to testify before the committee.

Donovan denounced Picardo earlier this week as "murdering slime" and denied any wrongdoing. Apparently stung by Donovan's attack, Picardo went on television to reaffirm his charges, most notably an allegation that he periodically picked up payments for "labor peace" from Donovan at the offices of Donovan's company, Schiavone Construction, in the late 1960s.

Picardo said yesterday that he was sure Donovan was the man named "Ray" he used to deal with. The FBI informer, who has been a key witness at four major trials of organized crime figures, said he turned over the payments to two captains in the Vito Genovese organized crime family, Salvatore Briguglio and Armand Faugno. (Briguglio was killed in a gangland slaying in 1978. Faugno disappeared years ago and is believed to be dead.)

Picardo's televised debut came under swift attack from Sen. Paula Hawkins (R-Fla.). The new committee member criticized NBC for giving "a murderer, a liar, and every other name you would want to use" primetime exposure when he wouldn't even testify before the committee.She called it "a cheap shot."

Sens. Harrison Williams (D-N.J.) and Jennings Randolph (D-W.Va.) voted with the Republican majority in approving Donovan. But the other Democrats said they were still troubled by the fact that so many FBI informants, perhaps five or six in addition to Picardo, came up with allegations about the nominee or his company. Although none of the charges could be substantiated, FBI officials said all of the informants were rated as "reliable" and had submitted accurate information in the past.

Sen. Claiborne Pell (D-R.I.) told his colleagues, "I could not vote against him but the evidence is such that I could not vote for him." Sen. Howard Metzenbaum (D-Ohio) said he felt Cabinet nominees should be held to a higher standard than the one used in criminal courts, where a person is innocent until proven guilty.

After the session, Sen. Thomas Eagleton (D-Mo.) told reporters that he felt some important allegations had not yet been checked out, including one asserting that Donovan's company had supplied thousands of dollars worth of building materials free of charge for the construction of gangland figure Anthony (Tony Pro) Provenzano's New Jersey home in the 1960s.

"Nobody's found a smoking brick yet," Eagleton said, but he doubted that this allegation, which he just heard of Wednesday, had been thoroughly checked out.

Some sources said the allegation actually involved another gangland figure's home and insisted that it had been pursued to the point where the FBI felt confident in dismissing it. Other sourses, however, said the investigation on this source had been altogether inadequate.

Kennedy and the four other Democrats voting "present" promised to make up their minds by Tuesday, when the nomination is expected to come up on the Senate floor. Hatch reluctantly agreed to wait until then so that a formal report could be printed before the vote.