FBI agents visited Labor Secretary-designate Raymond J. Donovan's New Jersey construction firm's office yesterday to check allegations that Donovan periodically delivered $500 checks to a Teamster union courier in the late 1960s to buy labor peace from one of the union's locals, sources said.

The charges were made, sources said, by Ralph Picardo, an FBI informer who was then close to Anthony (Tony Pro) Provenzano, the head of the Union City, NJ., Teamsters local. Picardo testified on a 1979 case that led to Provenzano's conviction on labor racketeering charges, and is considered a credible witness by federal authorities, sources said.

The Senate Labor Committee abruptly postponed a vote on Donovan's confirmation Thursday after FBI Director William H. Webster told committee members of the allegations. Donovan, executive vice president of the Schiavone Construction Co. of Secaucus, N.J., has denounced the allegations as "scurrilous and untrue."

Sources said the FBI is also checking a number of other new allegations against Donovan, but there was no way of immediately assessing whether there was any substance to them. At least one came from an anonymous telephone tipster.

President-elect Ronald Reagan continued to support his nominee yesterday. "I have perfect confidence in him," Reagan told reporters yesterday. He said he thought Donovan would be confirmed despite the controversy started by the allegations.

Edwin Meese III, who will be Reagan's White House counselor, told reporters at a breakfast meeting: "Sen. [Edward M.] Kennedy's staff has been working very hard on this, but I'm sure there's nothing there that will bother the committee when it has the facts.

"Mr. Donovan is in a business where there are many allegations made, but we have complete confidence in his character and record," he said.

Kennedy, the ranking Democrat on the committee, said through an aide that the FBI must pursue the "very serious allegations" thoroughly. He said the committee "must not be bound by any artificial deadline," and that once the investigation is completed the committee's findings will be put on the public record.

Sen. Orrin G. Hatch (R-Utah), the committee chairman, said the investigation is likely to take several days. "I personally think he is an honest man," Hatch said. "If not, he is the best liar in the world."

High Reagan aides made it clear last night that, while they believed Donovan's denials, they wanted the allegations thoroughly checked out by the FBI and the committee. Thus is seemed clear that Donovan's confirmation will be delayed past Reagan's inauguration Tuesday.

One source said yesterday that the FBI learned of Picardo's allegation when an FBI agent asked him if he'd ever heard of Donovan's construction company. Picardo said he had, and then identified Donovan's picture in the paper as the "Ray" he had met to get payoffs for the Teamsters, the souce said.

The $500 checks Picardo said he picked up from Donovan were made out to two phony companies, "OK Trucking" and "XYZ Leasing," in 1967 and 1968, the sources said.FBI agents and committee investigators are trying to check Schiavone records to find the checks, they said. According to Picardo's story, phony invoices were drawn up for shipments the companies never made to Schiavone construction sites.

Donovan, who was Reagan's campaign chairman in New Jersey, adamantly denied payoffs when asked about another alleged incident during his confirmation hearing Monday. "We have never been extorted. We have never given a payoff," he said.

His sharp answer was prompted by questioning about charges that his firm had brought labor peace from another Teamsters local in 1977 by paying the salary of an employe who did no work. Donovan said he had no knowledge of the arrangement until a week ago, though three of his company's employes had testified about it before a grand jury. He said Schiavone had no choice but to hire the man -- the chauffeur of a New York City Teamsters local official -- because of the terms of a union contract.

The Picardo allegations focus on the New Jersey local where Provenzano was president and secretary treasurer. He and three associates were convicted in 1979 of forcing trucking firms to pay thousands of dollars in kickbacks to buy labor peace.

Provenzano has long been a prime suspect in the disappearance of former Teamsters president Jimmy Hoffa. Picardo is now a protected government witness. He reportedly was once convicted of murder, but the conviction was overturned.