"I'm going to miss the place," says Clyde Carter in a mellifluous Maine drawl that drops the Rs and makes Jimmy Carter's speech sound like High Diction. "You get attached to a house after a while."

At the peak of his career as the town's premier caretaker, Carter, 42, suddenly finds himself out in the cold a man without a big house. His boss, Nelson Rockfeller, is selling the summer "cottage," a 21-room, horseshoe-shaped castle of granite and glass perched high on 3 acre of brooding Maine coast. The price tag on the parcel is $1 million.

The vagaries of summer gentry affect small towns all along the coast of Maine. After all, the state is a relic of 17th century England, inhabited by woodsmen, watermen and guides serving the pleasure-seeking summer people, as Arnold Toynbee suggested, and Seal Harbor is Maine's most exclusive town of caretakers.

For two months a year, July and August, the bluest of blood pretend to be just plain folks, fleeing the cities for the opportunity to cruise the roads in their "woodies," hear seals yapping for supper and wait in line for a haircut. Docks discreetly court the "discriminating yachtsman" in the Red Book, the quasi-official list of summer residents.

J. P. Morgan used to sail into the exclusive coves of Mount Desert Island on his yacht, the Corsair, and employ was obtained after the fashion of competitive bidding. As a boy, Police Chief Maitland Murphy, now 53, remembers standing on the shore, cheering his father as the proud caretakers raced against each other in rowboats. Several scions took pleasure in awarding their summer accounts to the first man to reach the yacht.

More often, though, the jobs were tenured, passing down from one generation of caretaker to the next, inherited in the manner of property.

CLYDE CARTER was born here. For the last 14 years, he has cleaned Nelson Rockfeller's heated salt-water pool, kept the pipes from freezing in winter, raised enough flowers to fill a castle and fended off the interlopers. He has rousted campers from their sleeping beds in the middle of the night, and shooed gawkers from the grounds.

At night, he usually goes home to sleep, but has come to consider "The Anchorage" just as much as his house as Nelson Rockefeller's.

After the sale, he'll continue tending the boathouse, a converted coal shed where Rockefeller plans to keep an office, Picassos in the sail lockers, an international racing yacht and three other boats.

But it has been decreed: The cottage must go.

Clyde Carter left the island once, to sample New Jersey on a construction job. He scampered home. His father, a carpenter and 30-year Rockefeller veteran, had informed him that Seal Harbor's First Family was hiring.

"I was on my way back anyway," he says. "I didn't like it down there at all." He gazes out across the dark blue water as he talks, waving at lazy seagulls floating by and plump mallards paddling about. His son, Carl, is playing on the dock, an 11-year-old who is carefree and innocent to the ways of big cities.

"How could a person stay away from all this?" asks Carter.

ROCKEFELLER'S father, John D. Jr., or "Junior" as the benefactor is gratefully remembered hereabouts, also fell in love with the Maine coast, specifically the rugged landscape of Mount Desert Island, and brought his wife, Abby, to summer here. In 1908, Nelson was born in the summer cottage they had rented at Seal Harbor, a remote inlet facing the Atlantic.

Two years later, Junior purchased a 99-room mansion, the Eyrie, set high above the harbor, installed an Armenian to care for his priceless oriental rugs and hired more than enough help to landscape the grounds and spill out 15 servants' bedrooms.

John D. took daily hikes about the manor. Once, the story goes, he stopped to chat with the blacksmith, who didn't recognize his lord. It appeared as if Junior wanted a job. "Forget about that," snarled the smithy, "the old SOB ain't hiring."

A crew of piano movers is said to have once tipped a ragged-looking old man 50 cents for lending a shoulder. John D. reportedly pocketed the tip and toddled off.

He never gave out free advice, but his entrepreneurial spirit evidently rubbed off on the help. A group of caretakers crowded themselves "The Big Five" and did rather well doing general contracting for the rich. Two sons of caretakers, Bob Suminsby and Dwight Carter, are now among the area's leading realtors.

"A young fella growing up in this environment ends up with a great deal more sophistication than a typical "Down East Mainer,'" explains Suminsby.

The town was also a summer testing ground for Junior's five boys, who set up a flower stall outside the country club and cornered the market on corsages. One day as they were hitchhiking back to the Eyrie, a summer resident stopped to give them a lift, goes the tale.The driver sniffed surprise that a young Rockefeller didn't have a car. "Who do you think we are, Vanderbilts?" came the reply.

A wall was constructed around the Eyrie, and dappled with sparkling tile taken off the Great Wall of China. Junior proceeded apace to buy up vast tracts of land surrounding his Shangri-La, and, as would become a Rockefeller pattern of environmental noblesse oblige, he presented it to the federal government.

The 5,000-acre Acadia National Park nearby, a playground for 2 million visitors each year, is a spellbinding carpet of lush pine and birch forest that rolls gently across granite mountains and frames crystalline lakes.

Such a grand gesture, of course, was tax-deductible, and created a comfortable green buffer between the rich and the rest. If Junior couldn't make nature a monopoly, at least he would forever insure his market share of it. A taxpayer-subsidized DMZ would stiff-arm the common camper and keep tacko subdivisions from ever tiptoeing upon his heady spaces.

To fend off curiosity-seekers and nurse their properties back from the naughty ravages of winter, the Rockefellers and their landed neighbors hired caretakers, proud ancestors of men like Clyde Carter. These locals - "Maine-iacs," they call themselves - are straight-talking and stubborn, a determined breed rooted umbilically to place and family. Poverty and John D. Rockefeller not withstanding, it is a fair bet they would have remained here anyhow, even if it meant limping along. Which it often did.

SEVEN HUNDRED DOLLARS was all the money Cecil Carter (no relation to Clyde) earned one year in the '30s. "That was tough," he says. "But we got by - smelting fish, gardening, digging clams."

Then he found work as a night watchman at the Eyrie. He says John D. gave strict orders not to feed the night watchmen, never said hello when he came or goodbye when he left. "Meanest man that ever lived," says Cecil. Such are the ways the rich are remembered.

"There was always a feeling you had to look up to 'em because that was your living," he says. "You always tipped your hat and never looked 'em in the eye when you talked."

He is 78 now, a thin, feisty old man soon to retire from his second caretaking job for a wealthy Philadelphia matron. She's selling her modern house on a cliff for $600,000, so Cecil will move his wife from the basement back to a small house in town.

"If someone rich dies or decides to sell, you wonder what's going to happen to the property and what's going to happen to you. It's the same the world over. If you're a caretaker, you're never secure. You come and go with the property . . .

He sits at a dark mahogany table, beneath splashes of modern art, a simple, wizened old man in paint-splattered pants. "I've worked hard for what I've got and I've never wanted to change places," he says. "If I had their money, what would be the satisfaction? I say the only satisfaction you can get from money is if you earn it. Then you get the pleasure of spending it. Otherwise, it's just paper."

During the Depression, Junior funded what, in effect, was a private WPA, and set the natives to building elaborate carriage houses and miles of trails. In 1947, a row of smart mansions burned up in a fire at nearby Bar Harbor, and Junior put everyone to work cleaning up.

"In those days," says Cecil, "you didn't have a backhoe; you had your back and a shovel. If you drove a truck, they didn't allow you a helper to load or unload. Maybe that's why so many of us have lived to be 80, 90 years old."

'THE OLD CARETAKERS were paid to keep their heads down and jump in a hurry," says Bob Suminsby. "There was a great deal of paternalism.

"Today, the social barriers are not as evident as before World War II. There's still a distinction. But it's come around from a landlord-peasant relationship to employe-employer."

Most natives now call the Rockefellers and others by first names, having dropped the once-customary "Mister," as in "Mr. Nelson." A good politician, Nelson Rockefeller has made most everyone feel he loves them.

Six years ago, Hugh Gilley, his father's tree man for 45 years, took his skates and went to New York for the first time, to glide about the ice at Rockefeller Center. "We called upstairs," he beams proudly, "but Mr. Nelson wasn't there. His secretary said she saw us out the window. Mr. Nelson would have come down or invited us up to lunch if he'd been there. He's that kind of person."

In Seal Harbor, rich and poor alike are said to greet each other cheerily in the streets, as if they all belonged to the same club. It is not so stiff as in the old days.

One former gardener praises "Peggy," the wife of banker David Rockefeller, Nelson's brother. As fine a woman as you'd ever want to meet, he says. Every time she sees him, she insists on shaking his hand, mud and all.

"The more money they have, the easier they are to get along with," observes one native. "It's the ones in between who act like they really are somebody."

COFFEE BREAKS in Seal Harbor are taken at the Lighthouse Restaurant, the only cafe in town. The Rockefellers, the Henry Fords and other cottage dwellers buy their papers here, and sometimes stay for breakfast.

At the counter, Larry Wescott, 37, a portly sign painter, was reminiscing over coffee the other morning between jobs. Nelson Rockefeller used to give free sailing lessons to the island children, he was saying, and Wescott once played with the governor's son, Michael, before he was lost in New Guinea, and with his sister, Mary.

"That was before she went big-time and started going to the Yacht Club and the Harbor Club," he says.

Those were the Gatsby days of $100,000 summer weddings on the lawn and whirlwind soirees with orchestras packed in from New York. Everyone had a chauffeur, and long, fat limousines clogged the narrow, winding two-lane blacktop that snakes around the island.

Nowadays, the rich are more understated, taking cocktails quietly at home. The Harbor Club does not serve alcoholic beverages, as well specified by its founding father, John D., a teetotaling Baptist. There, heads turn when a rare burger is pushed through the self-service window and the PA system crackles, "ROCKEFELLER." The rich are curious, too.

Supper is sometimes taken out, at the Jordan Pond House, an institution that Junior turned over to the Forest Service along with the land. Guests can dine on tasty popovers, lobsters and steaks and marvel at the pink sunset over Cadillac Mountain. Waitresses in green pinafores are still talking about conversation reportedly overheard between Nelson Rockefeller and Henry Ford.

"I could build a wall of gold around this place," Rockefeller is said to have boasted.

"Nelson, let me know when you do and I'll buy it," Ford countered.

Larry Wescott never spent much time dining out. His father was a Rockefeller carpenter. As a teenager, Wescott worked for the Fords. He used to play auto roulette in their garage. A roadster was driven onto the turntable and spun around like a top. The game was short-lived, though. "The chauffeur threatened to break my thumbs."

As a child, he had rich friends, but they grew up and drifted away. "I got maybe three friends I could call if I really needed help," he says. "Watching them made me realize I never wanted to grow up like that, too much phoniness. They grew up in a mold. They had to go by the rules.

"Of course, to be a good peasant, you've got to follow certain rules, too," he says dryly.

"I like to say, Hello, how are you,' not, 'Hello, who are you?"

"I don't need a Cadillac or a color TV. All I want is enough money in the bank to have my wife's hair done and take care of my kids.

"We may not have a lot of money, but we're pretty rich to live in a place like this. People work 50 weeks a year to come here for two weeks. But we live here. This is our home."

IF THE IMPACT of Rockefeller wealth and influence on America remains open to debate, up here, as the Rockefellers go, so goes Seal Harbor.

"They depend on us and we depend on them," says Mildred Gilley, a former innkeeper.

Rockefeller money has seeded a number of projects, from local charities to pollution-free industry. All told, wages at Jackson Laboratory, the nation's largest breeder of experimental white mice, may fall short of what locals might earn at, say, a paper mill. But Seal Harbor still smells good.

Village elders shudder at the thought of a honky-tonk invasions, and consistently vote down commercial rezoning notions. Most want to keep the area quaint and quiet, lest the rich pull up stakes for another paradise.

Property taxes on the area's 518 summer cottages contribute three-fourths of town's $1.8 million annual budget, paying for the schools, a four-man police force, a two-truck fire department, an ambulance, a library.

Wealthy cottage dwellers are asked to dig deep for local philanthropies, and a number of trust funds keep up manicured gardens and public docks.

"At the town meetings," grumbles Suminsby, "someone always says, 'Let's get a donation.' But I say, 'Let them pay a decent wage and we'll pay our own way.'"

Donations helped pay for small, modern airport, summer home for a private air force of learjets.

When airport fundraiser Bob Gerrity, 73, a real estate broker, trekked door-to-door some years back, he blushed to one tycoon that he was mightily embarrassed to be hustling the chap for a third handout.

"That's okay," said the patron. "It's my own neck I'm saving."

"Their interests are our interests," says Carlo Ninfi, tax collector for 35 years. "They don't have a vote, so we have to watch out for them."

At a recent town meeting, someone suggested the police charge for answering a rash of false burglar alarms, set off by howling winter winds and flying branches. He was hooted down.

In Seal Harbor, the rich come first.

A BLUE Ford LTD-2 kicks up dirt and gravel as it races down the private driveway past the country club and comes to a halt in front of the boathouse. A door slams and a policeman bends toward the sunburned man in khakis.

"Some guy's coming up from New York to protest Mr. Rockefeller's selling the house," says the officer. "He'll likely be wearing all black. Man's been plaguing poor Mr. Rockefeller all his life."

"Never heard of the jerk," shrugs Clyde Carter.

"Me neither. I just come to warn you, though. He goes by the name 'Dr. Infinity,' and, Clyde, he's all yours. I'm going home to get drunk."

WHEN THE NAME is Rockefeller, and all of a sudden, you decide to sell a summer house everyone knows you could afford to keep, well, it tends to stir up the elements.

Outsiders start asking why, and the natives begin to wonder if the Maine lobster will be the next to go.

But next Saturday, "Nelson" - one-time co-pilot to presidents, ex-governor of New York, art patron, philanthropist, trust fund baby extraordinaire - will be 70 years old. And there comes a time when a Rockefeller must consider heirs and putting one's affairs in order.

"Mr. Rockefeller believes in looking after his estate while he's still around," says family spokesman Hugh Morrow.

Such a concern presumably overshadows utility. Nelson Rockefeller has spent so little time at The Anchorage in recent years, says Morrow, that he cannot justify the expense.

But David Rockefeller, chairman of Chase Manhattan Bank and the island's largest private landowner, isn't going anywhere. He summers across the cove from his brother, in a large house on a hill. And Nelson's first wife, Mary, who met her ex-husband here one college summer long ago, keeps a cottage a few coves away. Rockefeller children and grandchildren come and go.

Still, losing even one Rockefeller can be fretful.

'THERE'S NO NEED to be afraid," soothes George Ballantyne of Sotheby Parke Bernet, which is handling the sale. "The house will be bought by a substantial person who will need someone like Clyde to turn the water on and off. The new owner will employ the same number of people and contribute as much to the island as the governor."

Already, he says, cables of inquiry from Europe and Texas have come to roost at the New York office. Not one nibble from the Middle East, though. "Too cold, probably," says Ballantyne, a patrician, 28-year-old MBA whose family has a summer cottage in Seal Harbor.

Sotheby's feels it has a social responsibility to America's future generations to keep such large estates intact. And to ensure that the "right" kind of person purchases The Anchorage, says, Ballantyne, Sotheby's may even be able to arrange a mortgage. "I don't want to see someone buy the governor's house and put aluminium siding on it."

Whoever buys the contemporary house, considered radical chic in 1939 when it was built with curved, wall-to-ceiling glass overlooking views Rockefeller himself staked out, a cantilevered deck, Art Deco bathrooms and a watchtower, will likely pay dearly for its upkeep. One real estate man frowns that, these days, even the super-rich are shunning maintenance-heavy estates.

Nowadays, the rich are learning to make their own instant coffee.

IT IS 10 A.M. and Harry Fernal, a veteran of 30 years' duty as Nelson Rockefeller's gardener, is as happy as any man has a right to be in retirement.

He has paid off a mortgage on a two-story white Cape Cod house. He owns a sturdy green Chevrolet pickup, two guns to shoot deer, a fine set of fly rods and a garage full of tools. The other day, a visitor offered him a $400 for his pot-bellied stove and he didn't have to sell.

He spends much of the day surrounded by nude women on the wall of this garage. Miss April, a luxuriant creature, is his favorite, though a stranger might not realize this. Fernal has scissored off her head and, in its place, put Henry Kissinger's. "I guess I oughtn't of done it," he grins, firing up a filter tip. "I like Kissinger."

Only God and Freud knows what it all means.

A wiry, gray-haired man of 76 years, with a face as gaunt as the carved American eagle above the door, Fernal rummages through his tools and produces a bottle of bourbon. "Company comes and no matter what I'm doing, I stop and take a drink."

On a Schmidt's beer clock, he has pasted a magazine photograph of Nelson Rockefeller and his wife, Happy. But his heart belongs to Mary. She once gave the gardener a snapshot of herself in a bathing suit, and frequently visits to pick up cuttings from his roses.

When his former wards fell to the care of Clyde Carter, Harry Fernal got homesick. "I missed my plants," he says. "You can't help but wonder how they're doing. You get to know each one like a person."

He still pays them regular visits, but heavy security has begun to make reunions virtually impossible. Fernal needs an appointment to see his flowers. "I haven't gone back in three weeks," he says.

Still he lives comfortably on two social security checks and a $75-a-month pension from a boss who always shook his hand, even when it reeked of lobster bait. "Everyone who worked for Nelson is satified," he says. "Course, there were a few said, 'With all that money, he should just divide it up.'

"But I never wanted Nelson's money. Money gives people heart attacks. Let them have their money. I've got my health and enough to pay the bills. Besides, if it weren't for the rich, who would hire people like me?"