Raymond J. Donovan, a political outsider whose 44 months as secretary of labor have been hampered by poor relations with Congress and organized labor, has spent much of his tenure rebutting allegations that he had ties to organized-crime figures.

Despite the series of investigations into his activities as former executive vice president of Schiavone Construction Co. of Secaucus, N.J., he has remained in the job longer than any Republican appointee since the Eisenhower administration. At the same time, as he followed President Reagan's agenda of making deep cuts in Labor Department programs, he has had what organized-labor officials describe as icier relations with them than have any of his predecessors.

Donovan has resisted intense pressure to resign by White House chief of staff James A. Baker III and other administration adversaries, choosing to stay and protest his innocence. In a famous "turkey-blind interview" in January 1983, for instance, Baker told a Texas newspaper, "Ray Donovan shouldn't be in here. What's he thinking about? He's got his good name now . . . . He ought to do what's right for the president."

Only Reagan could have forced Donovan out, according to administration sources, but Reagan, reluctant to believe anything bad about his employes or to fire them, likes Donovan personally and has backed him throughout his service.

The indictment of a Cabinet member a month before a presidential election, however, could have an impact on the Reagan campaign, regardless of whether Donovan remains in office.

Donovan, 53, grew up with 11 brothers and sisters in a working-class area of Bayonne, N.J. His parents died when he was young. He attended Notre Dame Seminary and helped raise his siblings after his parents' death. In 1959, after working as a union electrician and insurance salesman, he joined the Schiavone company as vice president in charge of labor relations and financing. At that time, the firm had assets of less than $20,000.

When he left it in 1981 as executive vice president, its contracts totaled more than $600 million.

During the 1980 presidential campaign, at the request of Reagan aides, Donovan raised more than $600,000 and persuaded Frank Sinatra to appear at a Reagan fund-raiser. He served as chairman of the Reagan-Bush committee in New Jersey for the fall campaign. It was then that he also reportedly charmed the future president.

Donovan's appointment as secretary of labor drew immediate opposition from organized labor, which contended that he was appointed only because he was a major Republican fund-raiser and that he lacked the background to deal fairly with unions.

Murray Seeger, spokesman for the AFL-CIO, refused to comment on the indictment last night, but he repeated complaints that Donovan had the worst relations with organized labor of any modern labor secretary, had cut back on virtually every program supported by unions, and had made many "anti-union" appointments to key jobs.

The only program to get beefed-up funding under Donovan was the inspector general's division that dealt with investigating unions, Seeger said, "and we contended all along that was part of the harassment."

Donovan met rarely with top AFL-CIO officials, who represent 13.5 million of the nation's 20 million union workers. He met only two or three times with AFL-CIO President Lane Kirkland, whose disdain for Donovan reached the point where he called Donovan "the custodian of the [Labor Department] building" and would not use Donovan's name in public comments, calling him "Secretary who?"

Donovan carried out an administration mandate to cut government costs by reducing his department's budget by more than any other department.

Under Donovan, the Labor Department reduced funds for health and safety inspections, mine safety, and various labor standards investigations such as "sweatshop" probes. The Comprehensive Employment and Training Act was eliminated and a revamped Job Training Partnership Act enacted during his tenure.

Donovan has maintained that he was representing the interests of all working people, not just union members and their leaders. He also pointed out that he maintained good working relationships with certain unions, including the construction trades and the Teamsters.

The combative Donovan surprised many of his critics with his staying power, combined with an aggressive campaign to overhaul his image.

Donovan once called his chief accuser, a government informer, "murdering slime" and portrayed himself as a victim of the "New Jersey syndrome," a reference to a popular stereotype of the state as riddled with underworld corruption.

Nearly a year ago, still hounded by rumors that he was on the verge of resigning, Donovan told a reporter that "I paid such a high entrance fee, I'm gonna stay for the double feature" -- indicating his intention to further confound critics by remaining through a second Reagan term.

Just last week, Donovan waived immunity and testified for almost five hours before a Bronx grand jury, telling reporters that the investigation was a "witch hunt."

"I am angry. I am sick of this line of questions. I know you are. I trust the American people are," he said.

"With the hope, the real hope of once and for all ending this witch hunt, I submitted to a polygraph test. I was not surprised I passed it with flying colors."