CIA Director William J. Casey has, so far, been just about everybody's favorite candidate for scapegoat in the imbroglio of the debate briefing books.

White House chief of staff James A. Baker III obviously seemed to think that people would have no trouble believing anything about Casey. In his reply to the historic letter of inquiry from Rep. Donald J. Albosta (D-Mich.), Baker fingered the intelligence chief as the man who gave him "the large looseleaf bound book" from the Carter campaign.

Even the president seemed prepared to hear that his old friend and campaign chairman had been up to something.

At least at his news conference, the president was exceptionally voluble about how well he understands how Casey might not have "paid any attention" to the hottest item that could have passed over his desk during the frantic autumn of 1980, the Carter blueprint for the October debate.

Casey's fall-guy credentials go way back. Long before he became Reagan's campaign manager in February, 1980, the 70-year old Casey had demonstrated that he is as much the swashbuckler in politics as he was in Wall Street, where he made his mint.

Doing "whatever is necessary," as they say in the agency he now heads, comes naturally to the bluff, gravelly voiced Casey. As chairman of the Securities and Exchange Commission, for instance, he delayed the SEC investigation of financier Robert Vesco, a big giver in the campaign. He insisted, subsequently, that he did not do it at the urging of the Nixon White House.

In another case, the matter of ITT and its largess to the GOP, Casey acted summarily in defense of his president. As congressional committees began to circle ominously, Casey, who had custody of the "politically sensitive" documents, bundled them up and shipped them off to the privileged sanctuary of John N. Mitchell's Justice Department.

Casey is no more a stickler for the ethical niceties than Reagan, who said memorably of Watergate transgressors H.R. (Bob) Haldeman and John D. Ehrlichman that they are not "criminals at heart."

More recently, Casey displayed his cavalier attitude about picky little questions of conflict of interest by continuing to play the stock market without regard to the fact that, as CIA chief, he has access to the world's most delicate secrets.

In a casual gesture towards appearances, a bizarre arrangement was made, whereby CIA Deputy Director John McMahon and general counsel Stanley Sporkin go over Casey's portfolio and let him know if the stocks purchased by his broker show a "conflict of interest" between Casey the spymaster and Casey the investor. Exactly what would happen if they were to blow the whistle we are not told. Whether Casey gives up certain stocks or "recuses" himself from certain CIA operations is not specified.

For these and other reasons, when Casey wrote his letter to Albosta, chairman of the House subcommittee that is investigating the debate affair, saying he had "no recollection" of ever receiving, hearing of or learning in any other way about the Carter debate plans, there was a great deal of skepticism and some horse laughter. Baker, the organized, meticulous technician, whose "best recollection" is of being given them by Casey, was far ahead in the race for credibility.

And even while the two contrary claims were clashing in the air--and President Reagan ws moving from "much ado about nothing" through a "monitoring" by the Justice Department to an FBI investigation, another story about Casey surfaced to confirm everyone's worst suspicions.

He had, it was revealed, launched an "intelligence" operation to counter the specter of a Carter "October surprise" that had obsessed him all through the campaign.

While Carter workers pored over precinct lists, a network of retired military officers scanned the skies for telltale aircraft headed in the general direction of the hostages in Iran. What would the old soldiers have done in the event of a spotting? Surely, they would not shoot it down. Would they have told the public that Carter was planning a coup to rescue the hostages--and himself--at the risk of endangering the lives of those involved?

Casey has said that it really didn't amount to much, but the whole idea feeds a general impression that Republicans think of political campaigns as war--in contrast to Democrats, who see them as sporting events. It was more ammunition for the Baker faction--a glimpse into the mentality that brings moles, heists and other squalors into the electoral process.

At that moment, Casey struck back in dramatic fashion. He marched himself down to the Washington bureau of The New York Times and shucked off the scapegoat mantle. He told three reporters that it would have been "totally uncharacteristic and quite incredible" for him to have trafficked in stolen campaign goods. Jim Baker, he said, was "mistaken."

Clearly, one of them is.