Arkady Shevchenko often boasted that he was the youngest Soviet diplomat ever to hold the rank of ambassador, that he was merely "parking" at the United Nations for the time being and that he was going to get a top job upon his return home.

But when the Kremlin finally called, Shevchenko refused to go.

"I still can't believe it," a U.N. official said yesterday summing up the bafflement here over the defection last week of Shevchenko who as an U.N. Under Secretary General was Moscow's senior representative in the U.N. leadership.

Shevchenko, most diplomats say, projected the image of a standard Soviet bureaucrat. "He was a doctrinaire hardliner," one European said. Members of his 90-member staff describe him as exceedingly ambitious, overbearing and quick to lose his temper.

"Whenever Shevchenoko was late for a meeting," another diplomat said, "we used to say that we knew the reason: He had to stop off at the Soviet mission to get new instructions."

Shevchenko used to proclaim he was "a good international civil servant," remarked an acquaintaince, adding "That was really a joke," Shevchenko, all agree, was Moscow's man - a channel frequently used by Secretary General Kurt Waldheim to pass urgent communications to the Soviet government.

But even though Shevchenko had been associated with the United Nations for 12 years he clearly remained as elusive and somewhat enigmatic figure.

"He was the last man I'd expect to do something like this" an East European diplomat said.

His decision to defect from his country seemed inexplicable against the background of his private and public actions and pronouncements, according to those who knew him.

It is this sharp discrepancy that has posed numerous questions about his motives.

"He was a doctrinaire hardliner," a European diplomat said. "Once he threatened that if I didn't take a certain line, has section would not cooperate with my mission. And then again he used to joke that he worked for the Soviet government but was actually a good international servant. That really was a joke."

Most diplomats suggest that Shevchenko projected the image of a regular Soviet bureaucrat. There are some who recall that occasionally he poked fun in private at some policies of his government. Yet, they say, nothing in his behavior ever suggested he had deep doubts about the Soviet system that would lead him to defect.

Shevchenko's job as under secretary in charge of political and security affairs has been held by a russian under the U.N. quota system. "These are political appointments," one of his colleagues said. "It is practically impossible to serve in such position without your government's approval."

Communist diplomats have suggested that Shevchenko had marital problems - his wife has returned to Moscow - and that he was drinking heavily.

His personal assistant at the United Nations, Vyacheslav Kuzmin, has been telling people here "I hope the Americans understand that Shevchenko is a sick man. They must send him back to Moscow so he can get the medical care he needs."

Such statements are dismissed by most diplomats, however. It seems quite clear to those who knew him that Shevchenko had carefully considered his moves and prepared them in advance.

One indication is his selection of a New York lawyer - "a very un-Russian move" as one diplomat put it.

Shevchenko chose Ernest Gross, a respected attorney with extensive experience in U.N. matters that included a stint as a member of the U.S. delegation to the United Nation during the Truman administration. Gross also served as assistant secretary of state and has handled legal affairs for the United Nations over the past two decades.

It was at Gross' office that Shevchenko met with two senior Soviet diplomats Sunday night to tell them about his intention not to return to Moscow. Subsequently the Soviets charged that he is being held "under duress" by U.S. intelligence services.

In many ways the problems of the 47-year-old Ukranian lie in his brilliant career. It proceeded smoothly ever since he joined the Soviet Foreign Ministry in 1956, two years after getting a Ph.D. from the State Institute on International Relations in Moscow.

Shevchenko rose quickly in the ranks, became a disarmament specialist, published two books and numerous articles on the subject and eventually became a leading spokesman on disarmament matters both here and in Geneva.

At age 40 he became ambassador-at-large and a special assistant to Foreign Minister Andrei Gromyko. He was sent here in 1973 to fill the highest U.N. post alloted to the Russians and he was the man, as one ranking U.N. official put it, "who knew the Soviet positions on all key matters."

A man intimately familiar with Soviet policy could not simply stay abroad without being regarded by the Soviets as a traitor. And yet, according to his lawyer, Shevchenko would like to do just that without seeking political assylum in the United States.

"This is like trying to square the circle," one diplomat said.