Adam Yarmolinsky sits in a restaurant two blocks from the White House. Robert McNamara's onetime firebrand doesn't look like a whiz kid any more. He is still wearing bow ties, the brows are still beetle-dark, but something overall about him seems quiet, soft even. When he speaks, he sounds a bit nervous, not arrogant. "I must say, nowadays it's little steps for little feet," he says, drumming his fingers lightly on the tablecloth. His smile is wry.

Adam Yarmolinsky is 55 years old. In the nearly dozen years that have elapsed since he left Washington, he has separated from his wife (he is now dating Kurt Vonnegut's ex-wife), visited a psychiatrist, written a book, held university posts, and otherwise tried to exorcise some bitter memories.

Memories of being called a Communist (by right-wing organizations). Memories of being hounded by J. Edgar Hoover (who alarmed that Yarmolinsky was going to replace him as FBI director, was said to have denounced him as an "enemy of my own, if not of the Republic"). Memories of being hung out on the line by Lyndon Johnson) who, after months of malignly neglecting him in the Defense Department, supposedly said, in a moment of his famous anger, "Don't bring up that damn little Yarmolinsky's name to me again").

Now he's back. Several weeks ago, while a couple dozen friends and half-curious press looked on, a fidgety, professional Adam Yarmolinsky stood in the fifth-floor State Department office of Paul Warnke, whom he has known closely for 30 years, put his hand on a Bible, and swore to uphold the Constitution in his new role as Counselor to the U.S. Arms Control and Disarmament Agency. While he works for Warnke, director of ACDA, Yarmolinsky will be on official leave from his professorship at the University of Massachusetts.

If his latest government post isn't a presidential appointment, something Yarmolinsky always wanted and has never quite gotten, it is, he says, "one more opportunity to make a difference." He says it is too early to tell exactly what direction his job will take (some presentations of treaties to Congress and the public probably), but he thinks he has a metaphor for the relationship between the government decision maker and his proverbial policy adviser.

"The decision maker is at the wheel of a fast car. His adviser is sitting next to him on the front seat. 'You have to turn left at the next corner or we'll run into a dead end,' the adviser says, 'I don't have time to slow down,' the drive answers. It's my job as much as anything to convince him he's got time."

Maybe it's just another case of Potomac Fever - that dark, musky attraction to the center of things. Yes, that lure exists, all right, Adam Yarmolinsky admits. This city always has held for him a certain curious texture, a not-altogether-understood emotional richness, and the spectacle of power has probably never been the least of it. But he thinks there's something more than that at work that brought him back to the meanness of politics after all this time.

"First, I've never primarily thought of myself as a scholar. I've never been so much interested in the analytical as the getting of something done, a program passed. I think of Emerson writing about the American Scholar in 1837 - "Man Thinking," still close to the life of action. In that sense, government would always be more congenial to me than books."

He stops, seeming to deliberate on something. A hand passes through the wire-brush hair, gone gray as ash. "There's a quote from the prophet Hillel I've long recognized as relevant. I think I first came on it in a novel about McCarthyism. It goes, 'If I'm not for myself, who will be for me? If I'm for myself alone, what am I? If not now, when'"

Adam Yarmolinsky, it becomes apparent soon enough, can stir you with high-mindedness; also offend you with retaliatory statements. A couple of years ago, snug in academia, he revisited what he termed "the corridors of power." He wrote a piece about it.

"I had forgotten how long and narrow the corridors were," he wrote, "and how the people in the rooms off the corridors didn't bother to look out the windows much. These are not, I thought, the corridors that novelists write about. In fact, they look more like the corridors of impotence, and their perspective, narrowing into dimness, makes the people in them seem smaller, not bigger, than the people outside."

Those two strains - summoning Hillel and making accusations of impotence - might limn what has always seemed the "problem" of Adam Yarmolinsky as an essential Washington man. "Brilliant but abrasive," is what nearly everybody said of the diminutive (5-feet-4), 39-year-old lawyer (Harvard undergrad. Yale Law School) when he first came to the Defense Department in 1961 as Robert McNamara's chief assistant and brain trust. (Within months, people were calling him "The Cardinal Richelieu of the Pentagon.") He had enormous power and could wield it like a club.

Son of upper-middle-class Russian immigrants to New York (his father, deceased, was a noted biographer and linguist, his mother is a poetess), Yarmolinsky had previously clerked for Supreme Court Justice Stanley Reed, been a secretary of the Fund for the Republic, an editor for Doubleday books, a Washington attorney, and a talent scout for President-elect John F. Kennedy in the two months before he took office. People said he could be doctrinaire, witty, arrogant, perceptive - everything but a diplomat.

Almost from his start he ignited controversy. His staff work on racial integration of the military angered southern congressmen and other conservatives. (The Thunderbolt, virulently segregationist paper of the National States' Rights Party later called him a "satanic integrationist.") His aggressively liberal politics irritated colleagues at the Pentagon. His blunt ways and gnarled looks did not endear him socially to the town. (One columnist described him as looking unfortunately like the anarchist bomb thrower in a political cartoon.)

But it wasn't until 1964, when he helped midwife Lyndon Johnson's war on poverty, that his career went into a tailspin. On loan from the Pentagon, he was Sargeant Shriver's de facto deputy, expecting after the bill went through Congress to be named officially. The conservatives got in the way, Adam Yarmolinsky became a consensus embarassment. The North Carolina congressional delegation demanded the administration abandon him or they would vote against the bill. Push came to shove and Adam Yarmolinsky lost.

The next day, at a press conference, Johnson denied that Yarmolinsky had been sacrificed to get southern votes. No promise had ever been made to Yarmolinsky. LBJ said, Yarmolinsky remained silent. He now says he could have made the President out a liar - that presidential aide Bill Moyers had earlier come as a Johnson emissary promising him the job. (For his part, Moyers says he was an emissary for himself, that "the President listened attentively but never conclusively" about Yarmolinsky.)

After that, it was pretty much downhill for Yarmolinsky in Washington. He went back to the Pentagon, where McNamara had filled his old job in the person of a rising young lawyer named Joe Califano.

He hung around for over a year and then Adam Yarmolinsky accepted a job on the Harvard law faculty. Everyone said he was quitting Washington for good . . .

"Of course, I'm bitter," he is saying, studying a cup of coffee. He has talked openly of himself for an hour. He seems tried."But even bitterness can't last forever. I think I've had a pretty good life. And I think I've learned a few things about myself. One of the things I've learned is that if I impressed people as abrasive back then, it must have been because I was abrasive. It took me a good while to see that. For the longest time I thought there must be purely contextual reasons for all my controversies."

Suddenly he is brightening. He has remembered something. "Do you know someone once fixed me up with a steel-work mask to take to a costume party? I guess that sort of sums me up the first time around."

What follows is not exactly a laugh; more a small, appreciative smile. CAPTION: Picture, no caption, by Harris [WORD ILLEGIBLE] - The Washington Post