A federal judge ordered IBM yesterday to comply with a government subpoena that the company says will require it to turn over more than 5 billion documents from 120 countries at a cost of more than $1 billion and 62,000 man-years of work.

In a 35-page opinion, U.S. District Court Judge David N. Edelstein in New York said he found the government request for the information "to be neither unreasonable nor oppressive" under the Federal Rules of Civil Procedure.

The subpoena requires IBM Chairman Frank T. Cary to produce the documents when he appears at a deposition in the trial. The government requested all IBM records about Gary "from age 21 to present."

The government filed suit against IBM Jan. 17, 1969, the last day of the Lyndon Johnson administration, charging the huge conglomerate with monopolizing the computer industry and seeking to break the company into several independent units.

The case went to trial May 19, 1975, and is not close to completion. After the government took three years to present its case, IBM took over April 26, 1978. The current controversy involves the government's cross-examination of Cary, who will appear as a defense witness.

In addition to objecting to the sweeping nature of th subpoena, "calling for everything IBM has," Cary and the company claimed the government subpoena violated the Fourth Amendment prohibition against unreasonable searches and seizures.

Rejecting that argument, Edelstein said, "It strains common sense to conclude that the Fourth Amendment was meant to protect . . . a private litigant in the course of civil litigation."

The government contended that there is reason to believe that Cary's testimony will focus on the period from 1974 to today - a period covered in its document request - by covering such issues as future trends in the electronic data processing industry. Government lawyers pointed out that they previously have sought virtually no documents from IBM relating to Cary for the period since 1974.

Government attorneys have offered to negotiate with IBM over the size of the request and the actual documents needed. Officials say they never expected the final request to be anywhere near as broad as IBM claims it is.

It is expected that government and IBM attorneys will meet to negotiate limiting the subpoena.

But IBM lead counsel Thomas B. Barr already has told the court in chambers that "it will at least be very many months before we could resume the trial, and frankly, your honor, perhaps years."

Barr has said the government subpoena would "have a catastrophic effect on the trial," and has told Judge Edelstein that the "only possible consequence" would be that 'the trial will now, must now, be adjourned for a very long period of time . . . I cannot see any other alternative."

Officially, IBM would say only that it was "studying the ruling and its implications for the future of the trial."

The trial, already the longest antitrust case in history, has filled more than 94,000 pages of transcript and produced 9,000 exhibits in 616 trial days. More than 70 witnesses have been heard.