Andrei Sakharov, the speakers emphasized over and over again, doesn't need any more tributes. The work and philosophy of the Nobel physicist and revered Soviet dissident are his legacies. What he needs, urged the guests at a dinner in his honor last night, is continued support.

"I kept hearing in the corridor of the National Academy of Science annual meeting, 'Why should we bother with him? After all, he has only been removed from the western press' . . . . The reason is . . . Sakharov has come to be the symbol of moral power," said Philip Handler, the president of the academy. "But I know if we do not continue to protest, Gorky will be the least of his worries."

In January the Soviet Union expelled Sakharov from Moscow to isolation in Gorky. "Recently two scientists and one official from the government visited him," said his stepdaughter, Tanya Yankelovich. "Our feeling is that happened so the authorities could make an impression that he has certain freedoms. Actually, he does not have much contact with outsiders and he is very unhappy."

Yankelovich, who lives in Newton, Mass., spoke at the annual "Friends of Freedom" dinner of the Coalition for a Democratic Majority. Participating in the program were Ben J. Wattenberg, one of the founders of the Democratic group, which was created during the 1972 McGovern campaign; Albert Shanker, the teachers' union leaders; senators Daniel P. Moynihan (D-N.Y.) and Henry M. Jackson (D-Wash.), and Huber Matos. Matos was honored at last year's dinner when he was still a political prisoner in Cuba. "Now I don't feel completely happy, but I feel content. I don't feel happy because there in communist Cuba I have left many brothers," said Matos, who was released late last year after an imprisonment of 20 years.

All the tributes to Sakharov contained some reference to the Soviet invasion of Afghanistan and to Sakharov's prediction of Soviet Aggression. Moynihan recalled the dissident's prediction after the Helsinki accord: "At that moment, in a tremendous act of heroism," Sakharov called Western journalists to his apartment and said, 'Detente without democratization is very dangerous indeed.'"

The dinner followed a day-long meeting on foreign policy, which attracted former secretary of state Henry Kissinger; Rep. Jack Kemp (R-N.Y.); writer Midge Decter; Richard Allen, Ronald Reagan's foreign policy adviser, and Marshall Bremmon, a member of the National Security Council staff. Though Decter described the dominant attitude toward the Carter administration "as somewhere between impatience and despair," Wattenberg said the group concluded "the world is a dangerous place and you have to be a realist. It's no longer simply talk of a military buildup or the worst of the military-industrial complex. It's reality."