Sen. Patrick J. Leahy (Vt.), ranking Democrat on the Senate Select Committee on Intelligence, accused the Central Intelligence Agency yesterday of "yearning to go back to the good old days" when Congress had no oversight of CIA covert operations and the United States had made "some of the most colossal failures, intelligence failures, ever."

"I hear people yearning to go back to the good old days," Leahy said. "Well, the good old days are the Bay of Pigs and Salvador Allende and Patrice Lumumba and a lot of other failures."

Leahy told a news briefing that he was not accusing the CIA of "wanting to pull another Bay of Pigs," the aborted U.S.-backed invasion of Cuba in 1961, but he said that "when you had no congressional oversight" the agency had become embroiled in such adventures as attempts to poison Cuban leader Fidel Castro, the bloody coup against leftist Chilean president Allende in 1973 and the support of murder plots against Lumumba, a leftist Zairian premier assassinated in 1961.

Leahy's comments were the latest salvo in an acerbic exchange this week between Senate intelligence committee leaders and CIA Director William J. Casey. On Wednesday, Sen. David F. Durenberger (R-Minn.) criticized Casey for not providing the CIA with a "sense of direction." Casey, in turn, accused Durenberger on Thursday of conducting intelligence oversight in an "off the cuff" manner that had involved "repeated compromise of sensitive intelligence sources and methods."

The unusual public acrimony reflects a crisis of confidence between the Reagan administration and the Congress over who is to blame for a recent spate of unauthorized intelligence disclosures. It also has raised the thorny issue -- which has surfaced in at least the past three administrations -- of the media's responsibility toward the public and government in reporting on delicate, often divisive intelligence and foreign policy matters in the administration.

Leahy yesterday supported Durenberger's charges that the administration was guilty of "selective leaking." The Vermont Democrat said the Reagan administration was "the worst ever" compared with those of presidents Gerald R. Ford or Jimmy Carter. He added that "there are a whole lot" of U.S. secrets that members of the intelligence committee learned of "first in the press."

The debate seems likely to persist, partly because of increasing CIA activity around the world under the Reagan administration and partly because Congress is sharply divided, though not strictly along party lines, on the issue of its oversight role of intelligence operations and the making of foreign policy.

The public exchanges this week have highlighted the sharp differences of opinion. Durenberger has said he wants to change "the definition of oversight" of intelligence operations and to "open that process up a little bit more so it isn't just their the administration's mistakes that become a problem."

Rep. Lee H. Hamilton (D-Ind.), chairman of the House Permanent Select Committee on Intelligence, said he endorsed Durenberger's idea of a larger public debate on general intelligence policy but was leery of open discussions of operations that risk "damage being done to our interests."

The two most recent examples of the confidence crisis have been reporting on the short-lived defection of the Soviet KGB official, Vitaly Yurchenko, and an administration decision to authorize a CIA plan to seek to undermine the regime of Libyan leader Col. Muammar Qaddafi.

Many administration officials were furious at a Nov. 3 front-page article in The Washington Post about the CIA plan to help Libya's neighbors or opponents topple Qaddafi; President Reagan has ordered an investigation of the disclosure. Hamilton said he regards it "as a very serious leak of a different magnitude than the others."

Several senior U.S. officials have questioned the wisdom of The Post's decision to publish the article, a decision that they say has compromised U.S. diplomacy and seriously embarrassed the opposition to Qaddafi and its Arab backers.

In response to the article, Egypt and Algeria -- two neighboring states at odds with Qaddafi -- have said they will have nothing to do with any CIA "plot" against another Arab leader. The National Front for the Salvation of Libya, the main Libyan group within the badly fragmented Libyan opposition, said in a statement from London that the article was "liable to discredit and undermine the genuine Libyan strength and preempt any national action that might be carried out against Qaddafi."

Leonard Downie Jr., managing editor of The Washington Post, in defending the newspaper's decision to publish the article, said the CIA plan was being "widely and hotly debated" inside the agency and between the CIA and the congressional committees responsible for oversight of such operations.

The debate was "significant," Downie said, and "the whole question of what kinds of covert operations the CIA should engage in is one suitable for public scrutiny."

Critics of the plan, he said, were even questioning whether the operation was "legal" because it might have ended in the assassination of Qaddafi, who has long been accused of supporting international terrorism. A longstanding executive order signed by Reagan forbids the CIA or any other U.S. agency from direct or indirect involvement in any assassination plan.

Downie said The Post article had disclosed no precise details of what the CIA was planning to do, "which we should not and did not do." He also said that the reporter involved, Bob Woodward, interviewed a number of knowledgeable government sources in reporting the article and that neither before nor after publication had any of them called to suggest that disclosure of the plan might endanger national security or U.S. lives.

Qaddafi has used the article to rally renewed support at home and in the Arab world for his embattled regime, picturing himself as a target of "the great American Satan," as one U.S. analyst put it.

The analyst was highly critical of any CIA anti-Qaddafi plan relying on Libyan opposition figures, describing them as "nobodies, klutzes and incompetents" lacking internal support.

In the Yurchenko situation, the defector, who returned to Moscow earlier this month after three months in CIA custody, has said that information leaked to the press about his defection had upset him and some observers have suggested that it may have affected his thinking about remaining in the United States.

Durenberger told a group of reporters Wednesday that he felt the CIA probably should have said less about Yurchenko, although he also acknowledged that the CIA feels the same way about members of his committee.

In discussing the administration's "selective leaking" of secrets, Durenberger added, "All of you know that with regard to Central America in particular they have leaked classified information about arms flow at various times." This was apparently a reference to Soviet and Cuban arms shipments to the Sandinista government in Nicaragua.

Ironically, many of the disclosures about Yurchenko's defection -- the fact that he had defected, his alleged ranking as No. 5 in the KGB, and his alleged role in triggering other defections -- were printed in the Italian press a month or more before they surfaced in the United States.

As early as Aug. 8, the state-run Italian radio reported Yurchenko's disappearance in Rome and probable defection. By Aug. 31, it was a front-page article in Corriere della Sera containing many of the details, assumptions and speculation about who he was that were to appear later in the American press.

Corriere, in its Sept. 1 edition, identified Yurchenko in a front-page article as "the No. 5 five in the KBG," a sensational bit of news that took the U.S. media more than three weeks to report on the basis of "leaked" information here.