THE SIGN says "Mayknoll" -- a huge, somber Victorian house at the end of a shaded drive on Long Island's opulent north shore, in a town called Roslyn Harbor. It was built in 1855 and seems untouched by this century, except for two black Lincoln Town Cars, one parked under the porte-cochere, the other under a weeping willow and flanked by large watchful men in blazers. Somewhere behind them, toward the bottom of the estate, past the sculpture garden and the gazebo, lies Long Island Sound.

The CIA has come to West Egg, the fictional home of new money in F. Scott Fitzgerald's "The Great Gatsby." If anyone gets shot in this swimming pool, however, it's not likely to be the owner, William Joseph Casey.

He wears a faded golf shirt and plaid trousers, and wanders through a succession of rooms with 12-foot ceilings, intricate moldings, heavy furniture, sconces with real candles, and crystal chandeliers where he has lived for 35 years in all his incarnations, which include lawyer, author, businessman, politician and now director of central intelligence. He encounters another security agent, this one in slacks and blouse, in the library dedicated to his Revolutionary War research.

"Are they going to let me onto that golf course?" she asks.

Casey is golfing today at an exclusively male links, and last time there the fellows made the female agent feel unwelcome. He assures her they'll let her on, and continues on to the main library, an enclosed, paneled porch spanning the back of the house. Half glass, padded with worn oriental rugs, it retains a kind of fusty Rooseveltian charm--that's Teddy Roosevelt, not Franklin. But there are no horned heads on these walls, just photographs of Casey on the covers of Finance and Business Week, and in the company of American Republican presidents.

Someone has written dialogue into one photograph of Richard Nixon and Casey seated in the Oval Office. Nixon is telling Casey, "Bunny, tell me about Bernadette's library," and Casey is saying, "But Mr. President, I came here to talk about the stock market."

Bernadette is his daughter--his only child.

"You can tell a lot about a man from his books," says Casey, 70, who has collected them for years. "A hell of a lot more than you can tell from his bank account."

His books deal with World War II, biography, Christian missions to the undeveloped world, ancient history. The handbooks on law, finance and real estate that Casey produced as a young man, books that made him moderately wealthy, are stacked under the window.

A CIA officer in coat and tie sits nearby, a notebook on his lap. Being surrounded by guards of one sort or another goes with the job of director of central intelligence, all part of what Casey calls "the dues." Reporters are also part of the dues. They ask the questions that Casey finds annoying when he does not find them infuriating.

"The director will blow up," an aide has warned. "Something you ask will trigger it. He'll come up out of his chair. But it quickly passes."

"The way you guys work," Casey says, blinking rapidly behind his Yves St. Laurent spectacles, "the way the media works, they put a label on you. In my case, it's a false label. I've been confirmed by the Senate five times. Every now and then somebody tries to get something on me, and nobody's ever succeeded."

Controversy is also part of the dues. The latest involves the pilfered Carter debate papers, and what may or may not have been Casey's involvement in that episode. Reportedly the FBI would like Casey, as well as James Baker, the White House chief of staff, and some other people to share their memories with a polygraph.

Asked whether or not he would cooperate in such a probe, Casey approaches the point of detonation.

"That's a dumb question," he says. "a hypothetical question. I'm not going to take a position on that." He adds, "The FBI has not asked me to take a polygraph. If they do, I will respond at that time. I'm routinely subject to polygraphing in discharging my responsibilities."

Daumier would have loved him. He has a scholar's slouch, and the eyes of a bail bondsman.

Casey's sentences tend to slide off into the confabulatory murk. The presence in Mayknoll of a secure telephone--two purring metal boxes studded with keys and switches--seems to contradict the familiar joke that Casey is the only director of central intelligence who doesn't need a scrambler. A former law partner solved the problem of Casey's inaudibility by learning to lip-read. "It's real hard to hear him," confides a current associate. "When you're riding in the car with him, and the radio's on, and he's mumbling, it drives you crazy."

"It's a tremendous advantage to Casey to have people underestimate him," suggests a lawyer who has worked with Casey. "They hear him mumble, and ask, 'How bright can this man be?' The next thing they know, Casey's eating their lunch."

That's one version of William Casey. His career includes intelligence work in World War II, and managing Ronald Reagan's 1980 campaign; he wrote an armchair traveler's book about the Revolutionary War, and directed the Securities and Exchange Commission for President Nixon. He wrote campaign speeches for Wendell Wilkie, and plays golf with Secretary of State George Shultz. He's been an editor, an entrepreneur, head of the Export-Import Bank, an undersecretary of state, and a failed candidate for Congress.

Other people have added to the list dancer on the tightrope of business propriety, self-promoter, curmudgeon. He doesn't give a damn, he says, how he's remembered, but if he had to advise a young man today, he would advise him to be a lawyer like himself: "A lawyer has a ringside seat at the human comedy."

His grandfather, an Irish immigrant, worked as a fireman in Queens. His father, a civil servant, died while Casey was still in school. As a boy in Belmore, Long Island, Casey read the adventures of Horatio Alger, Buffalo Bill and Nick Carter instead of going to movies. He worked his way through Fordham University and received a fellowship from the Catholic University School of Social Work. He put himself through St. John's University School of Law, and supported his mother, sister and brother as a home relief investigator. He made money with apparent ease, early, speculative dollars radiating from his corporate persona.

From the beginning he has dealt in words and information, a kind of intelligence. His first real job was to help compile information for books published by the Research Institute of America. Later, while practicing law, he ran a publishing enterprise with Prentice-Hall, on the side, and probably "earned more royalties than Ernest Hemingway."

"He can digest tons of paper," according to Stanley Sporkin, the CIA's general counsel. "He's got a photographic memory, or something."

A little-known version of Casey wanders through bookstores in Washington, New York and foreign capitals, buying books by the armload and reading half a dozen at a sitting. He disposed of the novel "The Spike" in two hours and James Michener's "Caravans" in an evening. "The Seven Pillars of Wisdom," in all of Lawrence of Arabia's extreme prolixity, required a weekend. Leonard Mosley's prodigious "Marshall, Hero for Our Times," was consumed on a commuter hop.

Casey says of his reading methods, "I look at the sources, the table of contents. I usually know a lot about the subject. Maybe this guy's got a couple of new formulations, a couple of new sources . . . I don't read every word." He disposed of a friend's 500-page manuscript in the back seat of his limousine, between Mayknoll and the first tee.

A lawyer named Ted Barraux remembers Casey walking into his office at the Securities and Exchange Commission, picking up a magazine from Barraux's desk, and carrying on a conversation while reading it.

"Weeks later, he would refer to one of the articles in detail," says Barraux.

He was also sued for plagiarism by a writer who claimed two pages of his rejected manuscript showed up in a compendium published by Casey's Institute for Business Planning. Casey says he agreed to pay a settlement, to be rid of a nuisance, but with no acknowledgment of wrongdoing.

(He can be as volatile with lawyers as with reporters. According to court records, he told an attorney taking his deposition, "God damn, if you're not a gentleman, I will kick your ass out of here.")

In different law firms, clients and companies mixed in an increasingly heady brew of politics and money. Over the years some of the froth has blown back. For instance, Casey lobbied successfully for a foreign tax credit for Indonesia without registering as a foreign agent, and was later investigated, and cleared, by the Justice Department. A lyrical sounding firm of which he was a founder, Multiponics, went bankrupt and investors sued Casey, among others. The case has yet to be resolved. Casey denies any impropriety.

"He's not sleazy," says a former senior intelligence official. "He just took fliers, because that's how you get rich. You don't get rich by buying stock in General Electric."

Disputes over facts, information and intelligence pepper his long career as an entrepreneur, providing a frequent source of interest to congressional committees.

Casey's financial statement filed with the Senate Select Committee on Intelligence was, according to that body, "deficient in several respects," including lawsuits, debts, liabilities, directorships, clients and money. Casey had to add to it.

Then as CIA director he at first declined to place his holdings in a blind trust. Public pressure mounted. He was, and is, unrepentant: "That's just the way the press behaves, the way you make a thing out."

The trust, which he finally set up, is unnecessary, according to Casey, who says his investment adviser has full authority to make decisions for him. Casey says he gives him no information. "The only difference between that and a blind trust is a piece of paper."

He toys with a paper clip, controlling himself. "Anybody in government could make money if they wanted to, if they wanted to use the information." He says people at the CIA don't have time to worry about investments. "Theoretically I could make money at the CIA, but the idea that I did is crap. It's hogwash."

Casey's domestic difficulties have also included one Max Hugel, a 1980 Republican campaign aide hired as the pivotal deputy director of operations, head of clandestine operations at the CIA. Hugel resigned after questions were raised about his previous stock and business dealings. Casey still insists that Hugel is "an awfully good man."

The Hugel affair prompted Sen. Barry M. Goldwater (R-Ariz.) to suggest that Casey might like to resign.

Casey stands up, walks out of the library and across the Italian terrazzo tiles in the arching center hallway. He mounts steps leading to the second floor, followed by the CIA officer. They enter a room with a huge metal cross hanging on one wall, bearing the embossed faces of the Apostles. On another wall is a blowup of a photograph showing Ronald and Nancy Reagan and Casey, taken in California after a victory in some primary election. Casey stands between them, smiling, arms outspread, a rare portrait of ebullience. Another photograph shows him 20 years earlier--his campaign poster. He had more hair then, and a sticky grin as he extended a hand toward the camera.

He ducks under the Victorian chandelier, and goes to the table by the window. He opens a scrapbook. It contains a letter from the late Wild Bill Donovan, director of the Office of Strategic Services, the bit of memorabilia of which Casey is proudest.

"You took up one of the heaviest loads," Donovan wrote, "which any of us had to carry at a time when the going was roughest, and you delivered brilliantly."

Casey the novitiate spy arrived in Donovan's Washington office in 1943. He had a commission in the Navy, but "wanted to do something more relevant than help procure landing barges." Using a former law partner as an intermediary, Casey got Donovan to hire him, and then organized Donovan's secretariat. Donovan sent him to London, and soon he was managing the infiltration of Allied agents into Nazi Germany.

He shared his Grosvenor Street digs with Richard Helms. A third future head of the CIA, William Colby, dangled at the end of a parachute, while Casey was a boss. Helms remembers Casey's ability to make unpleasant decisions. Also, Helms says, "He had what the Germans call fingerspitzengefuehl--a feel for the clandestine."

Casey says, "The OSS was exciting, challenging, high-spirited, mysterious."

"In Casey," Joseph Persico wrote in "Piercing the Reich," "OSS had a man with an analytical mind, tenacious will, and a capacity to generate high morale among his staff. He delegated authority easily to trusted subordinates and set a simple standard--results. He had no patience with the well-born elite who had flocked to the OSS, people he dubbed 'the white-shoe boys.' "

Casey is now wearing white shoes. He insists that the Ivy League Waspishness of spookdom is no longer a factor in the CIA, ticking off the names of intelligence brass who did not graduate from Harvard, Yale, Brown or Princeton. He asks the CIA officer with the note pad, "Where did you go to school?"

"Millsaps College," he says. "In Mississippi."

"I didn't have any trouble getting along with what I called the white-shoe boys," Casey goes on. "There was Junius Morgan, a product of the establishment if there ever was one, and a Navy four-striper. Once he called a meeting (in London), and we all came in. Raymond Guest, the lieutenant commander, was there. Guest said, 'Lt. Casey, bring that chair over here.' I said, 'Bring it over yourself.' I was just a lieutenant j.g. Later, Morgan called me in, and said, 'Commander Guest was in to tell me to bring you before the mast, to discipline you. I don't really know how to do that, but do try to be nicer to Raymond in the future.' "

At the end of the war, he says, he "had a pretty heavy responsibility for somebody 31 years old."

He says he was tempted to stay in government then, and was offered several jobs, including several in intelligence. "I thought I should establish my financial independence first. I've always wanted to go back."

Casey maintained his interest in intelligence and national security, and his contacts. Donovan appointed him to a committee studying European intelligence methods, for recommendations for establishing a permanent American unit.

In 1962 Casey founded the National Strategy Information Center, with alleged CIA links, to push for increased military spending. He says he "went along" with the war in Vietnam, though he wrote in his book, "Where and How the War Was Fought, An Armchair Tour of the American Revolution," of "the folly, the humiliation and the cost of carrying on a land war across an ocean against an armed population."

President Ford appointed Casey to the Foreign Intelligence Advisory Board in 1974. Now Casey likes to point out that for half of its 36 years, the CIA has been run by veterans of the OSS. He lists as outstanding CIA directors Beetle Smith, Allen Dulles, John McCone, Helms, and George Bush "if he had stayed longer." Casey, the latest OSS veteran, has increased the frequency of national intelligence estimates, which had dwindled to a dozen a year, and presided over one of the most spectacular covert actions in CIA history, reportedly 10,000 anti-Sandinist "contras" along the Nicaraguan border.

"The remarkable thing about the CIA is the will, the can-do attitude. We get 250,000 applications a year," he says. They hire about 2,000 people from all over the country, after heavy screening. They're trained, and some are sent out into the field. "From then on they're pretty much on their own . . . That's what I like about the job, working with these types of people."

The job of CIA director, of course, is a political appointment.

Since the 1940s Casey had taken part in presidential campaigns, what he calls "a turning point, a time when you try to correct the slide down the slippery slope toward collectivism."

He was interested as a young man in what he calls "distributive justice" and utopian socialism, but the Jesuits at Fordham, and Franklin Roosevelt's attempt to pack the Supreme Court in 1934, turned him.

"I pass the test," he says, "that says a man who isn't a socialist at 20 has no heart, and a man who is a socialist at 40 has no head."

The more practical politics he learned from Leonard Hall, lawyer and Republican national chairman. Casey represented a rare combination of ambition, intelligence, and what a friend calls Casey's "testicularity." In 1966, Hall encouraged Casey to run for the Republican nomination for Congress in Nassau County against one Steven B. Derounian. In the campaign, Casey portrayed Derounian as a reactionary by using a picture of the back of Derounian's bald head. Casey still thinks it was an interesting idea; he lost anyway.

Casey stayed on the periphery of Nassau County politics, a force in the Republican Golf outings who had broader interests. He founded the Citizens Committee for Peace with Security to lobby for Nixon's ABM program. In 1971 he was appointed chairman of the SEC. He brought a decisiveness and a sense of direction to the agency, getting the funds he wanted from Congress, and consolidating enforcement activities. "I don't think Casey has any principles that can't be accommodated to get a good result," says Harvey Pitt, a lawyer who worked in the enforcement division with Casey, and one of many admirers from the old SEC days.

Casey went next to the State Department as undersecretary for economic affairs, then to the Export-Import Bank, then to the law firm of Nixon's secretary of state, William Rogers. "Nobody who has ever known Bill Casey has any doubts about his integrity," says Rogers. "But he's careless about things. We never knew how much money we owed him. We'd say, 'Come on, Bill, we owe you more than that!' "

Along with the money came influence known only in the upper reaches of the political aviary.

In early 1979, John Connally called and asked for his support. "I wanted to help him. I said, 'John, I'll give you money, and wish you luck.' I sent him $1,000. I got a second call from George Bush. I said, 'George, I'll give you money and wish you luck.' I sent him a check for $1,000. Then I got a call from Ronald Reagan."

He told Reagan there would never have been a Jimmy Carter presidency if the Republicans had nominated Reagan four years before. "We had an interesting conversation. When he came to Long Island to make a speech, we had breakfast together. I talked with him one-on-one for about an hour and a half. I asked him questions about the economy, national security." He was impressed with Reagan's responses.

Casey organized the dinner in New York for Reagan's formal announcement of candidacy. Reagan made him vice president of the executive campaign committee. When friction developed between the West Coast operatives and the campaign manager, John Sears, Casey got another call. This time Reagan wanted him to take over.

He sees Reagan as the recrudescence of a conservative political strata running back through Eisenhower, Coolidge and Taft. "Taft was the first president to accept the safety net concept, but he wanted it for people who really needed it. Now it has become a vast redistribution machine that threatens to sink the economy. Reagan's trying to get it under control."

Now, at the CIA, Casey has elevated his position within the White House social hierarchy, an interesting development. He says he gets access whenever he wants it. "I see him (Reagan) a couple of times a week," he says. Usually, it happens at National Security Council and Cabinet meetings, and privately whenever Casey wants. "We have a warm relationship, but I'm not a personal confidant. I stick to my knitting."

He believes Reagan shares his view that the CIA "is something unique in the world. Its depth was created over 25 years, then it went through a time of bad criticism--sensational, inaccurate, unfair and distorted. The government turned its back on intelligence, and the process of gathering it. I want to restore the earlier, good days."

But there have been more dues to pay in the controversy over the debate papers used to prepare Carter for a television confrontation with Reagan. Somehow they found their way into the Reagan camp. James Baker, Casey's neighbor off Foxhall Road in Washington, said publicly that he got the papers from Casey's office.

That, according to Casey's friends, was a mistake. "Casey is not the sort of person you want to be in a fight with," says a former intelligence official. "He could figure out more ways to cut off my ----s than I can imagine."

Casey publicly contradicted Baker. If he saw the debate papers, said Casey, a man known to have memorized an article while discussing some other subject, he had forgotten about them.

It could be a matter of style. A former Reagan campaign official who had no involvement in the presidential debate says that, in general, Casey is "a classic '50s cold warrior. In those days there was a constant need to know. You didn't see your adversary on television all the time, as you do today. Political opponents were always trying to find out things about each other."

"Life has become more politicized than it used to be," Casey says. "The world's a lot more interdependent, and dangerous. Threats from abroad are more serious. When I think back about how we used to worry about the Germans in South America! Now the Soviets are more actively engaged on every continent. They have a much greater preponderance of power in Europe than Germany did before she launched her conquest."

He calls to his wife, Sophia, who is in the next room, "Tootsie, can you throw together a couple of sandwiches?"

While they are being prepared, Casey leads the way back downstairs and out a side door. A security agent in a black toupee watches, one hand resting on the limousine's hood. Casey crosses the patio, beneath a huge linden tree. The swimming pool glistens from a fold in the lawn. At the bottom of the estate a teen-ager can be seen riding a bright red all-terrain vehicle through the timeless summer noon.

Casey points to another house on the property. "We used to raise chickens in there, then we found out we could buy eggs cheaper at the A&P."

A Rolls Royce sits nearby, gently baking.

"Can you guess what I paid for this place?" he asks.

Casey says he paid $50,000, an extraordinarily good price even 35 years ago. But then Casey is not known for making bad deals.

The teen-ager rides up and says, "Hi, Uncle Bill."

Casey appraises the all-terrain vehicle. "Did you pay for it," he asks, "or did your father pay for it?"

"I paid for it."

"Good for you!"

Later, Casey says, "I'm high on the free-market system, instead of the quasi-collective systems. I think a nation should promote its values in the world, protect its security, and set an example among nations not seeking expansion."

Light beer in plastic glasses, and cheese and salami sandwiches are served in the library. Casey sits with his back to the view, his bald head and white hair faintly luminous in the sunlight. Not long ago, a friend asked what Casey would like to be doing, if he had his fondest wish. He says Casey told him, "Dropping spies into Nazi Germany."

Casey asks his wife, "Did you make these with your own little hands?"

Sophia Casey says she did. She has short white hair, and seems very deferential. She sets a solar music box on the window sill, and sits on the very edge of the couch as the frail melody drifts the length of the room.

"Everything my husband does is for the patriotic good," she says, eyes fixed on the copper beeches outside, "whether or not the media agrees. I don't like what's written and said about him. But we always say, 'It only hurts for a day.' "