The talk of city hall these days is who is staying and who is going when Mayor Marion Barry begins his second term Jan. 3. Much of it is just that -- talk -- for many of the players refuse to comment on their future. Still, anxiety is running a bit high as the city government prepares for an inevitable change of top leadership without a change of leader.

The key man in all the calculations is Ivanhoe Donaldson, Barry's longtime adviser, political right hand and sometimes governmental trouble-shooter. Donaldson is now out of the government but under contract to the city to help with the transition task force, which is likely to play a key role in the fate of the top crop of city officials. Many say Donaldson will not rejoin the bureaucracy on a full-time basis, but will continue as a contract consultant to Barry while going into private business, maybe running a restaurant.

The third member of Barry's administrative triumvirate, City Administrator Elijah B. Rogers, also appears to be on the way out, by his own choosing. He has a tough local job but does not get the attention that he would in most cities because he is overshadowed by politicians and bureaucrats on both local and federal levels. Moreover, with a salary of $56,000 a year that Congress is reluctant to see raised, Rogers is the lowest paid city manager in the metropolitan area.

Before Barry, Donaldson and the transition task force start choosing a new team, they first will have to decide whether the city should keep its current bureaucratic structure. At the moment, there is a city administrator, Rogers, overseeing the department directors. The alternative being considered is to switch to a system of several deputy mayors who would report to Barry.

That arrangement would provide top jobs for several aspiring administrators instead of one, and have the effect of retiring Rogers' number. But it also would have the drawback of putting Barry in charge of day-to-day city operations at a time when he is seeking a more prominent role in national politics.

Rogers and Donaldson are the two major players who appear headed out of the Barry government. Others have also announced they are leaving.

One is Carolyn L. Smith, head of the Department of Finance and Revenue, who is returning to her old job with an accounting firm. James A. Buford has announced he is leaving as director of the Department of Human Services, the city's largest agency, and Albert J. Beveridge III, the chairman of the Board of Elections and Ethics, also has decided to go.

There is talk of frustration and bureaucratic burnout among some of Barry's top appointees--Corporation Counsel Judith W. Rogers, legislative director Barbara C. Washington and budget director Gladys W. Mack -- though none has said she plans to go. Rogers is said to be frustrated with the administrative tasks of her job and may decide she wants to leave or take another high-ranking post. Similarly, Washington, who has been criticized for problems that city laws and budgets have had in Congress, may opt for a more rewarding post. There is also talk that Mack, who has been under the strain of the city's budget crisis for several years, may resign as well.

Then there is Robert L. Moore, the city's housing director and one of the highest profiles in the Barry team. Moore told reporters a year ago he definitely would be leaving this month to return to Houston, where he headed the public housing department before coming to Washington. Now, sources say, Moore is saying he wants to stay, but the mayor may not keep him. Barry was thumped during his reelection campaign for the poor condition of the city's public housing and there were frequent questions and embarrassing headlines about his floundering showcase, the Bates Street redevelopment project.

One possibility, sources say, is that Moore would keep his title but the job would be dramatically trimmed to cover only public housing. The housing and community development department's functions as an urban redevelopment and economic development agency would be given to a new Economic Development Bank that would offer the Barry administration more power and visibility in controlling local development.

For each of the shooting or falling stars in Barry's administration, there are others on the rise. The man being touted as the top contender to run the new, improved development department is Thomas Downs, the current head of the Department of Transportation and one of the most popular and trusted figures in the government. Also in a position to move up is Audrey Rowe, who is being mentioned as a possible replacement for Buford at human services, even though she says she is hesitant to take the job.

Downs is also mentioned as a possible city administrator, should Rogers leave and his position not be abolished. Another possibility for that post is an old face around city hall and a survivor of several political and bureaucratic wars. At one point he was a controversial human resources director and then a virtually faceless computer systems analyst. In recent months he has reemerged in the minds of many around the mayor as a man on the rise in the Barry administration. His name: Joseph P. Yeldell.