A federal judge held yesterday that the criminal charges against Labor Secretary Raymond J. Donovan "in no way pertain" to his official duties and rejected Donovan's petition to have his case moved into federal court.

U.S. District Court Judge Lloyd F. MacMahon of New York ruled that the indictment should be prosecuted in the state court in the Bronx where it was brought in September. The indictment accuses Donovan and 11 other defendants of grand larceny, falsifying business records and filing false documents in connection with a six-year-old New York City subway project.

Donovan and his lawyers had sought to sever his case from that of the others and have it tried separately in federal court under a so-called "removal statute" dating to 1815. The law was enacted to protect federal officials from being tried in state courts for acts committed "under color" of their government positions.

MacMahon rejected Donovan's claims in a 14-page opinion. "The charges confronting petitioner in no way pertain to his performance of official duties; nor do his duties as secretary of labor provide any explanation or excuse for the crimes charged," the judge said.

Donovan, who is on a leave of absence, also maintained that he was the victim of "hostility" on the part of Bronx District Attorney Mario Merola and that the indictment was timed to have an adverse impact on President Reagan's reelection campaign.

The judge said such complaints, "if established," may have some relevance as the case progresses, but were "immaterial to the issue of removal."

Donovan, the first sitting Cabinet officer to be indicted, was executive vice president of the Schiavone Construction Co. of Secaucus, N.J., before he joined the Cabinet in 1981. The 137-count indictment accuses him, nine other men and their two companies of padding payments to a subcontractor, Jopel Contracting & Trucking Corp., on a $186 million subway project undertaken by Schiavone.

MacMahon, who conducted a four-day evidentiary hearing in Manhattan last month, noted that 72 of the counts against Donovan accuse him of crimes committed after he became secretary of labor, but the judge said that all of them stem from allegations that he was "a member of a continuing conspiracy which began in 1979."

Donovan, the judge said, "specifically testified that he took no action in his official capacity with regard to the subject matter of the indictment. Since this prosecution is not brought against petitioner 'for any act under color of such office,' we are compelled to conclude that the right of the states to enforce and execute their own laws must be respected."

Had Donovan's petition been granted, Bronx prosecutors would have had to press their case against him in federal court, under federal procedural law and New York substantive law. The other defendants would have faced trial together in Bronx Supreme Court.

The decision came as a surprise to lawyers for both sides in light of MacMahon's sometimes gruff demeanor toward Bronx prosecutors during last month's hearings.

Donovan lawyer William O. Bittman said he was "disappointed" but added there would be no appeal. He said he was still "very confident" that the Bronx courts would dismiss the indictment against Donovan, "principally on the grounds of insufficiency of evidence."

Merola, a spokesman said, saw the ruling as affirmation that "politics do not enter this at all. This is a local crime and will be tried in the local courts."

MacMahon emphasized that he was making no findings about the validity of the indictment and related issues. But he did reject Donovan's complaint that the case is based on substantially the same evidence that special prosecutor Leon Silverman concluded in 1982 was insufficient to warrant Donovan's prosecution for any federal crime.