Refusing to involve itself in the largest Private antitrust case in history, the Supreme Court dealt a sharp setback yesterday to the General Atomic Co. and others with stakes in the so-called international uranium cartel, including the government of Canada.

The action heightened the possibility that GAC, which is half-owned by Gulf Oil Corp., ultimately may have to give up at least $700 million worth of uranium to its adversaries in the case, principally United Nuclear Corp. of Falls Church.

The court rejected challenges by GAC to two separate pre-trial orders issued in recent months by New Mexico Judge Edwin L. Felter in Santa Fe.

In the most critical order, issued March 2, Felter judged GAC in default, and therefore liable on the ground that it had obstructed justice by consistently concealing with "the utmost bad faith" the "true facts" about the cartel and about Gulf Oil's participation in it.

The original but not the amended version of Felter's order criticized GAC's Washington law firm, Howrey and Simon, in connection with a suit in federal court in Pittsburgh. In that action, Westinghouse Electric Corp., a principal supplier of nuclear fuel to utilities, seeks $7.5 billion in antitrust damages from Gulf Oil and 28 other uranium producers.

In a complaint similar to UNC's Westinghouse accuses them of having joined in a conspiracy that increased uranium prices sevenfold between 1972 and 1975.

Last Aug. 10, the trial judge, Daniel J. Snyder Jr., made public 41 documents that Gulf had submitted for inspection and that were, he ruled, outside the attorney-client privilege that Gulf's counsel, Howrey and Simon, could invoke.

Judge Felter said that Snyder's action "had a direct bearing" on the effort in his own court to discover documentary evidence as to whether GAC and Gulf had engaged in antitrust activity in violation of New Mexico laws. His amended order added:

"Despite the fact that the same law firm which represents GAC in this action represented Gulf before Judge Snyder . . . this matter inexplicably was not brought to this court's attention promptly by GAC."

UNC disclosed the existence of the documents in open court last Oct. 7, but that was 19 months after GAC had agreed to produce such papers, Felter said.

"The failure to reveal the existence and indentity of all of Snyder documents in a timely manner . . . was a deliberate attempt to further conceal the existence and indentity of that evidence and avoid turning relevant documents over to parties to this litigation in compliance with lawful discovery demands," Felter charged.

This maze of litigation dates back to 1972 when, at the initiative of Canda, the cartel, called the Uranium Market Research Organization, was formed secretly in Paris. Other participants included Australia, France, South Africa and Gulf Minerals of Canada, Ltd., a wholly owned Gulf Oil subsidiary chartered in Canada.

In 1973, UNC signed a contract to supply 27 million pounds of uranium, at $6 to $10 a pound, to CAC which, in turn was obligated to resell much or all of the fuel to utitlities. But during the next two years, the market price rose sharply - to $40 last October. In late 1975, UNC sought to invalidate the contract by filing, in Santa Fe, the antitrust, suit against GAC. Recently, Felter ruled the contract void.