Hugh Carter, worm farmer, antique dealer, souvenir seller, state senator and cousin to the president of the United States, is not that worried about the future of his home town.

"Thirty-five million people like Jimmy Carter," Hugh told the president the other day, "and they'll still be coming to Plains."

"The rest of 'em," Jimmy Carter replied, "are going to enjoy seeing me in retirement."

Four years ago, this town in the land of peanuts, and Georgia pines implanted itself in the national psyche with the storybook rise of jimmy Carter to the presidency. Natives and outsiders alike sought to cash in on its newfound fame as the home town of the 39th U.S. president.

The national media flocked here and called it "the greening of Plains." But it was boom and then bust for the town of 685 residents. Now, for Plains, and for the president, this is a period of passages. As the transition from one national administration to another nears completion in Washington and Pacific Palisades, the transition in Plains is just beginning.

The lone remaining tour bus sits idle most of the time, and the state tourist center outside of town reports a 17 percent decline in business, following a 54 percent drop the year before. Some outsiders pulled up stakes after the November election, selling out to locals happy to reclaim their town. The hope here is for a modest economic recovery as more curiosity-seekers stray off the interstate for glimpses of the former president and his family.

The town's police force -- consisting of one night patrolman during the pre-Carter days -- has risen and fallen along with the town's fortunes. It jumped in size to eight officers plus three dispatchers during Plains' peak, now has two full-time and two part-time police and two dispatchers and may grow again if the need arises.

Mayor Boze Godwin, the 37-year-old town druggist, hopes for one or two "small industries related to agriculture" rather than to Jimmy Carter.

"Tourism will continue," he said, busily preparing prescriptions, "but nothing like the level it was. I think everybody knew it would play out but it was shorter-lived than expected. The national economy had a lot to do with it -- gas prices and the 80-mile round trip to and from the interstate."

Heralding the latest phase of the Plains pageant, the Main Street Cafe switched hands Nov. 1 when a Missouri entrepreneur sold out to Robert Hale, a county correctional officer who "growed up with Jimmy."

The former owner, Hale said, was always an outsider. "The local people wouldn't fool with him no way. He didn't make his million," scoffed Hale whose two sons were helping him consolidate the souvenir section the other night. Hale will still sell Carter kitsch but plans to expand the restaurant.

"See, folks in Plains haven't got anyplace to eat," he said. "Like on Sunday, after church, they have to go to Americus to eat. We're gonna have a Sunday buffet, and a band during the week, to give local people something to do. We don't have anything against tourists, but we're also gonna look out for local people."

The sale of the Plains Monitor was closed Christmas Day. The paper, established in 1976 and purchased a year later by Hustler magazine, publisher Larry Flynt and his wife, Althea, once boasted a circulation of 10,000. The figure later dropped to 3,000 many of them readers far from Plains.

Under the Flynts' ownership, the paper repeatedly attacked Sumter County as racist, adopted an anti-Billy Carter stance and generally alienated local citizens and advertizers.

Wilton Sheffield, the new publisher, has family and roots in Plains and promises a paper more to the town's liking. His staff already includes two Carter kin, including one of Billy's daughters. The former editor, meanwhile, has moved to Oregon.

The 10-mile stretch of country road linking Plains to Americus, the Sumter County seat, has survived the Carter presidency largely unchanged. Only the signs of what might have been testify to the fleeting hopes of disappointed investors. "Cafeteria Coming Soon," says one faded sign up the road from the Plains Country Station souvenir shop and gas station, both now abandoned.

Faye's Barbecue, a landmark to the visiting journalists who crowded her double-wide trailer on the road to Americus, is gone. Like Alice's Restaurant of the Arlo Guthrie song and film, it went out of fashion and out of business. Proprietor Faye West has moved to another town north of here, retiring from public life.

And the ony Washington-related business to sprout near Plains, a French restaurant named Le Normandie and also located on the way to Americus, may soon close. Adele and Olya Fessard moved here in August 1977 from the nation's capital where he worked in various French restaurants for nine years. They soon found, however, that there were few locals looking for the choice pate, chocolate mousse and duck a l'orange they had to offer. Their customers have come instead from the relatively cosmopolitan centers of Macon, Atlanta and Albany, Ga., and the Fessards are reassessing their future.

For out-of-town visitors, there is still no place to spend the night in Plains. A 90-unit motel for which the city council rezoned some agricultural land in 1977 never materialized.

That leaves the Best Western in Americus, where presidential brother Billy Carter can be seen almost every morning eating breakfast. Since he moved from Plains to a new house in Buena Vista, he is seldom seen at his landmark gas station, now run by his son-in-law.

Since the election, dozens of Carter family photographs have disappeared from the walls of the Best Western dining room and lobby. "They've stolen pictures of his house, of he and Rosalynn," sighed owner Jimmy Murray. "Hell, I had an autographed picture of Jimmy Carter and Rosalynn with the Fords and someone went into my office and stole it off the wall."

To compensate in part for the missing pictures, Murray is hoping for some increased business when the Carters return home. But he denied reports he had once planned to add another wing during the boom. "A creation of the press," he said.

In and around Plains, the real estate market has crashed, and it's not merely a result of rising mortgage interest rates. A scheme to sell square inches of Plains property -- the price rose from $5 in 1976 to $11 in 1977 -- ended after two groups of investors peddled the parcels with limited success.

A 2.4-acre tract adjoining the Carters' brick rambler was on the market four years ago for $125,000. It never sold, and the asking price is now $25,000. Realtor James Dalton said he considers the Carters themselves to be potential purchasers.

Behind the Carter home is a 190-acre farm sold to a group of Toronto investors in December 1976 for $325,000. They still own it, but their only income has come from renting the pasture, and the Canadians are interested in "any good offers," said Dalton. "As it turns out," the Americus broker said, "they'll do good to get their original investment back."

Dalton's last home sale in Plains was in May 1977. "Shortly after, the boom died," he said. "We talked about a boom but it never really materialized."

"It just quit, everything quit," said Fred Horn, another realtor with a "Lots for Sale" sign on the road to Americus. Farmland that once brought $3,500 an acre here, he said, is now worth only $800.

Thus, the talk of Plains these days is much more modest than it was during the boom times. The citizens are hoping Carter will reconsider his decision to locate his presidential library in Atlanta but seem sincerely glad to have him back, whatever the economic consequences.

The last Christmas Carter spent here, two years ago, saw American Indians, Taiwanese nationals and protesting farmers in town, all of them angry at the president. This year, there were only friendly faces, hometown folk and a few tourists to greet the lame-duck president.

"We had three years when you couldn't stir in this town, with 10,000 visitors a day," said Cousin Hugh. During that time and since, he said, he has sold 25,000 copies of his book about him and Jimmy, now in its fourth printing, right here in his shop. Six months ago, however, he lowered the price from $12.50 to $7.50.

"We've enjoyed it, and if it does fall off, that's all right," Hugh Carter said.

And someday -- one must at least consider the possibility -- Plains could go the way of West Branch, Iowa, a once-famous prairie town now faded into obscurity. It was, of course, the home of the last elected president defeated in his bid for another term. His name was Herbert Hoover.