In this state, the home of some of the nation's poorest people, the most controversial political issue this fall is the jet-setting life style of John Y. Brown Jr., the Democratic candidate for governor.

Every time someone brings it up, Brown, a handsome multimillionaire, and his wife Phyllis George, a former Miss America, do a slow burn.

"It's a joke," says Brown with more than a little irritation in his voice. "My opponent is trying to make success a fatal mistake. He's trying to create an image of me that simply isn't true."

But Brown's Republican opponent, former governor Louie B. Nunn, persists with what he calls "the lifstyle issue." He has questioned Brown's gambling habits, his hobnobbing with socialites and bookmakers and even has suggested that when Brown and George were married last St. Patrick's Day they should have done it among "foot-washing Baptists in Kentucky" rather than in New York City.

"His life style is important because you have to be able to relate to the people of this state," says Nunn. "I'm not sure if he does."

Put another way, can a high-living multimillionaire and his glamorous wife find happiness in a governor's mansion in the sleepy little town of Frankfort, Ky.? Brown's answer, of course, is an unqualified yes.

But it does point out the liabilities of "celebrity politics," for Brown and his wife are genuine celebrities here.

There's little doubt that the glamor around Brown's campaign was a crucial factor in his defeat of six opponents for the Democratic gubernatorial nomination last May. "Brown won with the ooh-and-aah vote," says Edward Pritchard, a longtime adviser to Democratic governors here. "It was all the glamor around them. That glamor's not hurting them one bit now."

Handled by a subtler politician, the lifestyle issue might have damaged Brown. But Nunn and his brother and campaign adviser, Lee Nunn, a well-known Republican political operative, are alley fighters. They've hacked at Brown throughout the campaign, when a few subtle slices might have served them better.

Their tactics largely have backfired. And Brown, who made a fortune as head of Kentucky Fried Chicken, the franchise empire he sold eight years ago, has turned the mud-slinging to his advantage. Instead of answering questions about his life style, Brown dismisses them as "Nunnsense."

"They're sick. They're perverted. This is the kind of thing that has to be eliminated from Kentucky politics for once and for all," he declares indignantly.

A lot of people agree, Brown, making his first run for public office, is a hevy favorite going into next Tuesday's election.

"The type of campaign that Louie is running and has run in the past appeals to the worst instincts in people, and it's something I can't abide," says John Yarmuth, a young Republican businessman from Louisville who's supporting Brown. "Nunn has been racially bigoted, anti-Catholic and anti-Semitic whenever it's fit his political convenience."

There's no doubt, however, that Nunn's attacks have gotten under Brown's skin and that of his wife of seven months, a television personality. Their reaction is telling.

When Nunn said during a television debate that he couldn't imagine why Brown and George would slip away from campaigning to attend a Broadway musical cast party at Xenon, a New York discotheque, George, who was watching the debate on television, stood up and flipped her hip at the screen.

"But Louie, I'm a Miss America," she said.

When Nunn said he had been campaigning in a place in eastern Kentucky called Mud Creek the night Brown was discoing among Hollywood celebrities, George said, "That's because you weren't invited -- nah-nah-nah-nah-nah."

That Brown would show up in a New York disco during the heat of a campaign baffles politicians in this Bible Belt state. (Nunn didn't let the event go unnoticed. His brother passed out "to a few clergymen" copies of a photo spread in Penthouse magazine showing men and women dancing in various stages of undress at Xenon.)

But if nothing else, the incident illustrates what an unorthodox campaign Brown has waged. In a state where politicians usually are judged on their promises of new roads and ability to stomach a strange stew called burgoo, Brown is breaking almost all the rules.

He isn't promising any new roads, bridges or patronage jobs. He isn't courting local Democratic power bosses like a lovesick suitor. And he's worked out a campaign schedule that allows him to work into the wee hours of the morning and sleep late.

Brown's campaign has caused some uneasiness in the courthouse crowds that ruled Kentucky politics for generations. "There's a danger in selling politics like soap," says Hart County Judge R. R. (Babe) Thomas. "I've always thought candidates should be attentive to voters.I doubt if John Y. needs the Democratic Party. He's bringing a different method of attracting voters to the campaign.And its working."

The contrast between Brown and Nunn could hardly be greater. Nunn is a battle-scarred political pro, a Republican who survived in a Democratic state by being tough and ruthless. He has had to work his way slowly up the ladder.

Success has come easy for Brown in both politics and business. An encyclopedia salesman during his college days, he borrowed money to buy out Colonel Harland Sanders' Kentucky Fried Chicken Co., built it into an international empire and then sold it for a $30 million profit. (Sanders, who's never forgiven Brown for making a fortune on his chicken, has endorsed Nunn).

Brown is silver-haired, smooth and handsome, the ideal television candidate.

He learned the value of television advertising and image-making as a businessman. Asked about the importance of imagery in a 1975 court deposition, he replied: "Image is everything. Life is very phony. It's what people think you are and not necessarily what you are that's important."

Media adviser Robert Squiers designed a television campaign around both of them, and George has proved an invaluable asset. "The high point of the campaign was when his name recognition went above hers," says Squiers.

Nunn, the only Republican elected to the Kentucky governorship during the last three decades, has used every political trick in the book to go after Brown.

In an attempt to build a coalition of Republicans, antiabortinists, rural voters and conservative Democrats, he has accused Brown of being proabortion and of promoting teacher's strikes (Brown has endcorsed collective bargaining).

A political ally of Richard M. Nixon and John N. Mitchell during their years in power, Nunn now talks about the sins of Watergate and implies that Brown is somehow guilty of them for not offering the public a full disclosure of his income. Nunn also has gone from county to county promising millions of dollars of public works projects if he is elected.

But the most outrageous blow came last week in a seven-page letter sent out over Lee Nunn's signature to 6,000 Kentuckians discussing "the advantages of John Y. Brown Jr. over Louis Nunn." One example is enough

In a tongue-in-cheek reference to a man Brown once hired to run his boat and who later was convicted on a marijuana charge, the letter listed areas where Brown has "the potential to really turn this state around."

"If you want to import large volumes of marijuana for your area, Brown could put you in touch with a man of experience. In fact he has one on his staff. Louie Nunn doesn't know any drug smugglers."