A federal judge was told yesterday that the United States Olympic Committee has no legal authority to boycott this summer's Games in Moscow, a decision that was demanded by the Carter administration following the Soviet Union's invasion of Afghanistan.

The USOC argued at a hearing in U.S. District Court, however, that under International Olympic Committee rules it has the power to either accept or reject an invitation from the IOC to participate in the Games.

"It's a voluntary act. You are not compelled to enter and you have the right not to enter under the IOC rules," USOC lawyer Patrick H. Sullivan told the court yesterday.

A group of amateur athletes has asked Judge John H. Pratt to overturn the boycott decision reached by the USOC House of Delegates in a 2-to-1 vote last April following heavy pressure from the White House.

The deadline for Olympic entries is May 24. Pratt told both sides yesterday that he expects to decide the case by the end of the week.

William H. Allen, representing the athletes, told Pratt yesterday that when Congress enacted the Amateur Sports Act of 1978, it intended to secure an athlete's right to participate in international sports competitions, particularly in the Olympics.

The act authorized the national committee to organize and finance a U.S. Olympic team and to represent its interests with the International Olympic Committee. Allen argued that Congress deliberately withheld from the U.S. Olympic Committee any authority to decide against participation in the Games, except where there is clear evidence that it would be detrimental to sport.

At one point during yesterday's two hour hearing, Pratt commented that it has been argued that the boycott of the Moscow Games "would be in the best interest . . . of sport."

Allen, however, expressed the athletes' concern that sports are being used as a means of conducting national foreign policy, an action that he contends Congress meant to forbid. He noted that the IOC charter warns that national Olympic committees must resist all political pressure.

Sullivan said the USOC and its predecessor organizations have had the authority to decide whether or not to accept an invitation to the Games since 1896, when the modern Olympic era began. The Amateur Sports Act of 1978 was simply intended to give the USOC power over "all international competition and to set down rules for such participation," Sullivan said.

The Justice Department, which joined in the case in support of the USOC boycott decision, agreed yesterday that the act merely expanded the power of the committee.

Department lawyer William G. Cole told Pratt that it was within the government's right to urge the committee to boycott the Games following the Soviet invasion, which Cole described as "a matter of extremely great importance to our national security."