Cuba's decision to skip the upcoming Seoul Olympics was greeted with regret yesterday by key U.S. athletic federation officials, while debate grew over whether the move will imperil the Caribbean nation's chances of participation in future international sports competition.

Canadian Richard Pound, a member of the Pan American Sports Organization's executive committee, said the action could cost Cuba the right to host the 1991 Pan American Games in Havana, as scheduled.

But Manuel Vazquez Rana, who heads the Pan American organization, said he will continue to back Cuba as the 1991 site for the quadrennial Games, which are held the year before each Summer Olympics, and said no decision will come until the next Pan American Sports Organization meeting in November in Puerto Rico.

Angel Pino, press attache at the Cuban Mission here, said his government has no plans to give up the Games. "We plan to host the Pan American Games and are preparing everything in Havana," Pino said. "We're putting up buildings and are very involved in the preparations. We hope this {Olympic} decision does not affect our right to host them."

At issue is whether Cuban President Fidel Castro vowed to send his athletes to both the Indianapolis Pan American Games last fall and the Seoul Olympics in September in return for the right to hold the 1991 Games in Havana and, if so, whether he will be held to that vow.

In a strongly worded letter Friday to International Olympic Committee President Juan Antonio Samaranch, Castro declined to attend the Seoul Games, claiming his longtime ally North Korea deserved to be a full cohost and citing concern for the safety of Cuban athletes in South Korea.

Cuba thus became the first and only nation so far to officially decline its invitation. North Korea has indicated it won't attend, but has filed no official response. Already, 160 nations, the greatest number ever, have agreed to compete in the Seoul Olympics. Only six invited nations have not replied. The deadline for replies is today.

U.S. Olympic Committee executive board members meeting in Atlanta this weekend expressed regret over the Cuban action. Bob Smith, who heads the U.S. baseball federation, said Cuba's absence in his sport would hurt the Games.

"This past summer, in particular, there was great interest in Cuba-U.S. baseball," said Smith. "They beat us three games out of five in a series in Havana, and then at Indianapolis we beat them the first game and they beat us in the final.

"We were really looking forward to the competition," Smith said, "even though, with Cuba in, it would decrease our chances for a gold medal."

U.S. Volleyball Federation chief Wilbur Peck likewise said Cuba's top-rated entries would be missed. Cuban women won the Pan American Games title easily, he said, and the men's team was runner-up in Indianapolis after a close, five-game match with the defending Olympic-champion U.S. team.

And Col. Don Hull, who heads the U.S. boxing federation, said the Cuban boxing team is considered "the one to beat. They're No. 1 in the world, with the Soviets and us fighting for No. 2. It's a shame they won't be there."

Given Cuba's national love affair with sports, Castro's decision to skip the Games for the second straight time surprised many. But William LeoGrande, a Latin-American affairs specialist who teaches political science at American University, said the action is in character with developments in Cuba.

LeoGrande said Castro is pushing a "moral and ideological revitilization of the Cuban revolution," and "wasn't willing, I think, to just compromise and go to the Olympics in spite of the repression of the South Korean government and the fact North Korea was left out" of a significant role.

He said it was a major sacrifice, because Cuba is proud of the success of its sports program since the 1959 revolution. And he said losing the Pan American Games "would be a terrible blow."