A Bahamian bank that had been the subject of a controversial 10-year investigation by the Internal Revenue Service quietly closed its doors last week after the government there pulled its license to do business.

Ironically, on the very day Castle Bank & Trust Ltd. in Nassau was closed, a federal judge in Cleveland dismissed the evidence gathered by the IRS in its investigation of the bank.

As a result of the Cleveland judge's decision, sources close to the IRS report that the agent who headed the investigation has been suspended from the agency.

Called Project Haven, the IRS probe was aimed at proving that Castle Bank and some other off-shore banks were being used as illegal tax dodges for the wealthy and money laundaries for the underworld.

In late 1972, Norman Caster, an IRS informant, lured Michael Wolstoncroft, Castle Bank's 46-year-old manager, to Miami. While Wolstoncroft was out on a date with Sybil Kennedy, a friend of Casper's, the informant got hold of a briefcase that Wolstoncroft had left at Kennedy's apartment.

The briefcase contained the names of about 300 American depositors in the bank. Casper turned the documents over to Richard Jaffe, the Miami IRS agent responsible for developing Project Haven.

Jaffe photographed the contents, then Casper returned the briefcase to Kennedy's apartment before the couple came back from their night on the town.

U.S. District Judge John M. Manos ruled last Thursday that the evidence had been gathered "illegally."

Manos dismissed the evidence in the government's case against Jack Paynor, a Cleveland businessman who was accused of lying on his 1972 tax return when he claimed he had no foreign bank account. Paynor's name was among those on the list of Castle Trust depositors gotten in the Miami caper.

Dora Sabaruni, the prosecuting attorney in the case, was in Miami yesterday where the Justice Department and the IRS were mapping the next move in the Project Haven prosecutions.

Cono R. Namorato, chief of the criminal section of Justice Department's tax division, said: "We are reviewing the (Cleveland) decision and deciding whether to appeal."

Two other cases growing out of the Castle Trust investigation are awaiting trial in Chicago federal court.

One involves Burton Kanter, a Chicago tax attorney who directed his wealthy clients to Castle Trust Kanter, who government investigators claim controls Castle Trust, has been indicted on a single count of conspiring to impede tax collection.

The other case names Kanter's partner, Roger S. Baskes, as a defendant. The government charges he was part of a complicated commodities trading scheme to hide profits in Castle Trust.

Jaffe, the IRS agent who was reportedly suspended after the Cleveland court decision, could not be reached yesterday.

Norman Casper, the informant in the Castle Bank case who says he is a good friend of Jaffe, said: "He's fighting for his life. He only has two years until retirement."

Casper said the Justice Department meeting in Miami yesterday was to decide what to do with Jaffe.

"The hatchet squad is down here. They're doing a job on him because of the judge's order," he said. "Dick has been taking the blame for a lot of things that happened, but it should be on my shoulder."

In Nassau, the Central Bank blamed the closing of Castle Trust on "adverse publicity . . . which could (harm) the reputation of the Bahamas as an off-shore financial center."

This statement seemed to confirm reports that the U.S. government has applied pressure on the Bahamian government. A Castle Trust branch in Grand Cayman, another tax island tax haven, was also closed down recently. Castle Trust will not be based in Panama.

As it closed the bank, the Bahamian government assured Castle Trust depositors that they will have "full indemnity against any losses or contingencies."