The familiar tiny mustache is still there, though now it is flecked with gray. The sad eyes and enigmatic smile are unchanged.

But as Nguyen Cao Ky sits chainsmoking on his couch in the spacious, modern living room of his suburban home here, it is hard to recall the days when the flamboyant aviator in the black jumpsuit, baseball cap and lavender scarf was South Vietnam's most visible political figure. Today he looks at the world with a deep fatalism.

"Nothing can affect me anymore," said Ky, now 55, and still trim in a gray suit and tight gray boots. "I've spent my whole life in war, suffering, and watching the people of Vietnam suffer. The ups and downs . . . I was on top when I was very young. Now I'm in exile. With the Asian philosophy, we accept it as our destiny."

For Ky, who served as South Vietnam's premier and vice president from 1965 to 1971, destiny has not been kind lately. Last year Ky declared bankruptcy and sold the liquor store that was his main source of income for about three years. He is now trying to sell his house to satisfy creditors.

Last October, at hearings in New York, President Reagan's Commission on Organized Crime accused Ky of being the godfather of Vietnamese organized crime here, the boss of 1,000 gang members in 15 states.

Ky called the allegation "complete fantasy."

Few police officials with knowledge of Vietnamese communities take the allegations seriously. Neither do the Vietnamese themselves, who regard their former leader with little affection or respect.

Most Vietnamese say they resent Ky because they believe he failed them as their country slipped away. Others dismiss him as a bon vivant who spends much of his time playing tennis and mah jong at his home.

If Ky's the godfather, he's the poorest one in history, Vietnamese say. "I've seen him wearing the same suit and the same tie for many years," said one friend, Hoang Ha Thanh, publisher of a Vietnamese newspaper in Los Angeles. "He's living a very miserable life, more poor than many Vietnamese."

Despite reports that Ky escaped from Vietnam with a fortune, Ky said that he left with only a tennis bag and pocket change, and that his wife escaped with about $30,000. He has said he receives financial help from friends.

Police officials said one reason Ky is the object of rumors is that he socializes with known Vietnamese crime figures -- an allegation Ky acknowledged. But he added that he is not close to them and tries to persuade them to go straight.

Ky said he had to declare bankruptcy because a boutique operated by his wife, Dang Tuyet Mai, failed.

Mrs. Ky tried to commit suicide in a Manila hotel room in March 1983 while visiting the Philippines First Lady Imelda Marcos. According to Ky, she has had trouble trying to work and make a household. "She is trying very hard. She's suffered the most of the family," Ky said.

Onefamily acquaintance, Yen Do, editor of a Vietnamese newspaper in Orange County, sees "tragedy" in the Kys' lives. "Mr. and Mrs. Ky, they were always under extreme pressure . . . . they could not behave as average people. They are excellent people, but the community needed leaders. They couldn't live up to this."

Ky said he acts as an intermediary between the Vietnamese and the Americans, often appearing on television as a Vietnamese spokesman. He addresses Lions Clubs and other groups and, his friends said, members of his audience frequently line up to have pictures taken with him. Often when he is in a restaurant, they said, former Vietnamese soldiers approach the table, stand at attention and salute.

Ky said that because he commands respect, he is destined to play a historic role again.

"Someday," he said, "the Vietnamese will recognize that in my position I can play that role as their representative. They'll come to me -- maybe."

But some Vietnamese have their doubts.

Thanh recalled a conversation at Ky's liquor store last year. "He said he tried his best but can't make enough money to make his loan," he said. "It made me emotional. He was vice president and premier in Vietnam, and I thought of him when he was a young and famed fighter. And now he looked very weary."