Initial world reaction to President Carter's call yesterday for relocation, cancellaton, postponement or withdrawal of teams from the Summer Olympics was anything but supportive. Only Egypt immediately agreed to join the United States in its intention not to send a team to Moscow unless Soviet troops are withdrawn from Afghanistan.

Saudi Arabia, its tiny neighbor Qatar, and the small African nation of Djibouti, already had announced their intention to boycott the Moscow Olympics. FOREIGN MINISTERS FROM 43 islamic nations will meet next week in Islamadbad, Pakistan, and are expected to consider the issue of a boycott. i

In Moscow, the state-run radio reported Carter's announcement without comment but quoted observers as saying the move was futile.

"Observers assess Mr. Carter's attempt to exert pressure on the Soviet Union as futile and reflecting Washington's current course to undermine the policy of easing tension and reviving the cold war," the broadcast said.

While some governments expressed sympathy with President Carter's reasoning in calling for a boycott if the Games are not moved from Moscow, they deferred a final decision to their national Olympic committees.

Lord Killanin, the president of the International Olympic Committee (IOC), criticized Carter's decision as hasty and said it could have "disastrous" consequences for the cause of sport.

"I implore him (Carter) to use any weapons he feels justified as a politician to use, but not to use the Olympic Games because I feel it could be damaging not only to sportsmen in his country buy to sportsmen throughout the world," Killanin said.

"To have a great country which has contributed much to sport suddenly pulling out would be disastrous," he continued. "But I think nonacceptance through political pressures could have the most detrimental effect on the long-term future of the Olymic movement, of world sport and specially the Games in Los Angeles."

Julian K. Roosevelt, one of two Americans on the 79-member IOC, said, "If the U.S. were the only one to pull out we'd look pretty stupid, frankly."

Lloyd Cutler, presidential counsel, said at a White House briefing yesterday, "There is a groundswell of public opinion in many sections of the world" that the Games should be moved or postponed.

"My personal view is that a number of nations will ask their Olympic committees to support a postponement or moving of the games," Cutler said.

But a groundswell did not appear to develop immediately, even among the U.S. allies, in the wake of Carter's announcement.

"Our minister of sport has made France's attitude clear. France will be at Moscow," said Claude Collard, president of the French National Olympic and Sports Committee. "There is no question of us supporting any boycott. The Games are for the competitors, not the politicians."

British Olympic Association Chairman Sir Denis Follows said he was "100 percent" behind Killanin in his stand against any boycott. "We in this country are of the view that sports and politics should be divorced as far as possible," Follows said.

Margaret Thatcher, the British prime minister, has said she would "prefer to see the Games moved from Moscow," and originally expressed sympathy with an American boycott proposal, but later deferred to the British Olympic Committee after public opinion polls showed that a vast majority of Britons opposed a boycott.

In Bonn, a government spokesman said West Germany "has great understanding" for Carter's position. But Willi Daume, the chairman of the German Olympic Committee, said, "I deeply regret that this serious issue is played up at this moment. Moderation and far-sighted action is now more necessary than ever."

The head of the Israel Olympic Committee, Yitzhik Ofek, said "We will wait to hear what the U.S. Olympic Committee decides" before making a decision on the boycott. "These are decision which should be made by sports institutions and not governments," he said.

A Greek government source said Premier Constantine Caramanlis, who has suggested the Games be returned to Greece on a permanent basis, was "against politization of the Olympic Games. Such a move would threaten an age-old institution.'

President Carter said in his statement yesterday that he advocated returning the Summer Games to Greece on a permanent basis.

Canadian Prime Minister Joe Clark already has announced that his country would welcome the Olympics should they be moved out of Moscow. But Dick Pound, the head of the Canadian Olympic Committee, told the Montreal Express yesterday, "I think it would be impossible to get any other site ready in time for the 1980 Games, and that includes Montreal."

Despite the obvious difficulties of finding a new site by next summer, 15 prominent American athletes, including Dwight Stones, Franklin Jacobs, Francie Larrieu and Evelyn Ashford, ahve signed a petition calling for the Games to be moved rather than boycotted. "By taking them (the Games) away, Russia would be without its most effective public forum -- Sport," the petition says. "This can be accomplished without a boycott by moving the Games to a nonaggressive nation."