Top Justice Department officials in the Carter administration considered prosecuting CIA personnel for civil rights violations against renegade CIA officer Philip Agee, according to documents filed in federal court here yesterday.

The records indicated that the idea was rejected primarily because of sensitivity of the disclosures that would have to be made.

Agee's lawyers submitted the heavily sanitized documents to U.S. District Court Judge Gerhard A. Gesell in an effort to win reinstatement of Agee's freedom-of-information lawsuit against the CIA.

They said the documents, obtained as a result of their continuing litigation against the Justice Department, showed "without any possibility of contradiction that the CIA's activities against Mr. Agee were so serious that the Department of Justice conducted an investigation to determine if CIA personnel should be prosecuted."

The lawyers, Melvin L. Wulf and William Schaap, added that "what those activities might have been are left to our imagination," but they said they felt sure "they would not consist of such everyday CIA activities such as physical and electronic surveillance or interception of Mr. Agee's mail."

A CIA spokesman said the agency would have no comment.

One knowledgeable source, however, insisted that the speculation of Agee's lawyers about "activities intended to do physical harm" was exaggerated.

"It was what they would call run-of-the-mill stuff, possible electronic surveillance and consideration of physical searches of Agee's premises overseas . . . yes, black-bag jobs. They thought they were stopping more agents from being exposed."

A former CIA officer who quit the agency in 1968 and is now one of its most outspoken enemies, Agee, who now lives in Hamburg, has made a career of exposing the names of CIA personnel and attacking its methods.

The government has listed numerous occasions between 1971 and mid-1978 when Agee identified more than 160 alleged CIA officers operating in countries from Australia to Switzerland.

He filed his freedom-of-information suit against the CIA and other government agencies in December, 1979, primarily, he has said, to find out details about what he called "a massive violation of my civil and constitutional rights."

In July, however, Gesell dismissed the CIA as a defendant after conducting a random in camera inspection of the agency's records on Agee. In Gesell's ruling he said he could not resolve the question of whether "some kind of illegality may have occurred," but he said he was "entirely satisfied . . . that no [FOIA] exemption is being claimed as a pretext to conceal misconduct."

Lawyers Wulf and Schaap, the latter of whom is co-editor of the anti-CIA publication "Covert Action Information Bulletin," contended that the documents they got from the Justice Department this month indicated that "very serious misconduct" was being concealed and that "the court itself must consider whether it has not been misled by the CIA and the Department of Justice."

One document dated only "8/21," evidently prepared when the Justice Department was still contemplating prosecution of Agee, concluded that "prosecution is impossible without disclosing the illegal acts."

Another, dated Jan. 19, 1977, from the attorney general to Robert L. Keuch, the deputy assistant in charge of the Criminal Division, said:

"My understanding is that the Criminal Division is investigating whether the activities of the CIA involved any criminal violation and is also considering referring the matter to the Civil Rights Division."

The same question, "whether to prosecute CIA officials for civil rights violations against Philip Agee," was mentioned again in a Jan. 16, 1978, memo from Drew S. Days, then assistant attorney general in charge of civil rights, to then acting deputy attorney general Benjamin R. Civiletti.

Still another document, dated April 14, 1978, from the Justice Department's internal security section, reaffirms the earlier decision not to prosecute Agee, even in light of his "latest activities," because of the fact that "Agee could still discover information which the CIA has informed us cannot be revealed . . . ."