Ted Kennedy's presidential prospects rest heavily on the shoulders of a reclusive political operative whom the Massachusetts senator has described as "the man who loves to say 'No.'"

He's Steve Smith, Kennedy's brother-in-law and the chief of his presidential campaign. He has always maintained a cool, low-key profile amidst the high-powered Washington soirees and political whirl that surround the Kennedy clan. But Smith is emerging as the pivotal figure in family affairs and the man Ted turns to in a crunch.

The husband of Ted's older sister Jean, the Brooklyn-born Smith is the perfect Boston pol. He loves to operate in the smoke-filled back rooms, shunning the spotlight. But he doesn't share his Irish friends' love for the blarney; he's a sharp-witted, hard-nosed political operator who dislikes small talk and red tape. He's impatient to get to the bottom line.

Smith, a shrewd businessman, has long handles the purse strings for the Kennedys, but he also has a reputation as a top-notch politicl organizer, an unobtrusive man behind-the-scenes who's a stickler for detail.

Those close to Kennedy say he has the same close, intimate relationship with Smith as had the senator's brothers, Jack and Robert. Ted trusts Smith's judgment implicitly. "Everybody leans on him -- everybody," confided one long time friend of Smith's.

"He not only has the headaches of the family real estate and financial affairs, but whenever someone gets in trouble or gets an idea, Steve's the one who gets involved," explained another Kennedy insider.

If Smith had been around the night of the Chappaquiddick incident, suggested several Kennedy intimates, there never would have been the eight-hour delay in notifying authorities.

Smith has spent a good part of his adult life organizing one Kennedy campaign or another, often as de facto boss, but usually as a power behind the throne, who liked to have others out front while he ran the show from backstage.

He entered Ted Kennedy's hastily organized run for the Oval Office reluctantly, however. According to friends, Smith wasn't initially enthusiastic about Ted's candidacy. His wife was initially opposed, fearing for her brother's life, sources told by reporters Jack Mitchell and Matt Speiser.

Kennedy finally resolved that delicate question with family members, including his wife and mother. Smith told us he deviced to join the fray after his brother-in-law made a "personal" decision to be a candidate in 1980.

The Kennedy power brokers decided to make Steve campaign manager to keep the final decision-making power inside the family, and also because Smith's well-known reputation for trustworthiness made normally jealous members of the Kennedy clique more willing to take orders from the top.

Smith said, "My job is to coordinate; I have a broad range of experience and have the position I do because of the confidence between me and the candidate."

The Smiths are part of New York City's East Side social whirl and are regularly seen with such Broadway celebrities as Lauren Bacall and Adolph Green. New York City pols once encouraged Smith to run for the governor's mansion. But after thinking it over, he begged off, preferring to throw his weight to other candidates favored by the Kennedy machine.

A few of Smith's close friends, however, claim he has slowed down as a hard-drinking, bar-hopping pol. "Despite his playboy name, he's turned into middle-aged sherry drinker," joked one buddy, who remembered the Steve Smith of the Bobby Kennedy years.

Smith's political acumen is universally respected by political movers and shakers and watchers alike. "He's a born organizer," said one observer, who added that though Smith was taking a bigger role in this campaign than in the past, he would probally tend to business affairs while other Kennedy advisers, like former senators Dick Clark and Carl Wagner, handled the political chores.

The rest of the Kennedy clan considers Smith a harbor in the storm. One journalist close to the family noted that Smith was one of the few family members who could get away with missing one of Rose Kennedy's legendary birthday dinners without causing a fuss.

Ted can also count on Smith without having to think of him as a future secretary of state or White House chief of staff. Almost all Smith's friends have doubts he would accept any political job in a Kennedy administration, but would remain the unobtrisuve man who's always there for the Kennedys when he's needed.