President Carter's warning that the United States might withdraw from the Moscow Olympic Games to protest the Soviet Union's invasion of Afghanstan has dismayed American athletes and their backers as well as the private corporations that stand to gain from U.S. participation in the summer games.

Though the president understands that he has no authority to prohibit U.S. participation in the games, he reportedly has assumed that if he decided on an American withdrawal he could count on U.S. athletes and ports-minded tourists to heed the advice of their government and boycott the Moscow Olympiad.

In his televised speech on events in Afghanistan Friday evening, Carter implied that any American boycott of the games would be coordinated with U.S. allies, but so far no western country has warmed to the prospect.

(The Associated Press reported yesterday that NATO Secretary General Joseph Luns said the defense alliance's council has not formally discussed an Olympic boycott. "The subject was philosophized and some ambassadors talked about [a boycott] in a rather positive manner . . . but I don't believe that one of the ambassadors has been instructed," he said.)

Domestic reaction to Carter's Olympics threat was mixed.

Don Miller, executive director of the U.S. Olympic Committee, said the games should not become a political plaything, and was sharply critical of the president's suggestion of a boycott.

"Any rules by the federal government would have to be carefully considered by the United States Olympic Committee," Miller said in an interview with United Press International. "But the USOC is a private organization and not an integral part of the federal government.

"I am very saddened and I regret that our government would find it necessary to consider the use of the Olympic movement, which for 80 years has stood for good, as a vehicle for international politics," he said.

Walter Byers, executive director of the National Collegiate Atheletic Association, said he would support a presidential call for a boycott to protest the Soviet takeover of Afghanistan.

In striking contrast to Miller and other U.S. athletic leaders, Byers said yesterday that "The interest of the United States and its leadership position in the world far surpasses the interests of amateur athletics, even the Olympic Games."

On the commercial front, where millions of dollars are riding on endorsements and broadcasts of the Moscow games, the reaction was tentative.

NBC, which is paying the Soviet Union $87 million for U.S. television rights and spending millions more on facilities and hookups, described itself as "sensitive to the situation" and "watching it very closely."

A television industry source noted that NBC has paid the Soviets about 70 percent of the constract package -- some $60 million, which might or might not be refundable should the shape of the games change.

NBC, however, has insurance with Lloyds of London against the possibility of the games not taking place, a possibility that network officials last week would not discuss.

"If the United States sends no team, my own view -- and it's only my view -- would be that it's not a real Olympics. And that raises all kinds of other questions," an NBC official said.

Ed Williams of New York City, who competed for the United States in the 1968 winter games and now represents athletes on the USOC board, noted that competitors themselves have been promised a role in any decisions on participation in Moscow, should the question arise.

Don Kardong of Spokane, Wash., a distance runner whose life is finely tuned to the Olympic goal, has his own answer. He competed at Montreal in 1976 and he aims now for Moscow.

"It would be a dangerous precedent to boycott the games," he said.

"Where do you draw the line?A few years ago most of the world felt we were on the wrong side in Vietnam. Nobody boycotted because of that."