More than 20 Olympic athletes met in Washington yesterday, lobbying for coins they hope will be far more popular than the Susan B. Anthony dollar.

The Olympic Coin Act of 1981, passed unanimously by the Senate last December, would allow the minting and sale of a set of commemorative coins for the 1984 Games. Upon authorization, the Treasury would produce and sell the coins to the Los Angeles Olympic Organizing Committee, which would resell them to the public.

Marketing of the coins would be handled privately as a joint effort of Occidental Petroleum, the investment banking firm Lazard Freres, and the Franklin Mint.

The bill is before the subcommittee on consumer affairs and coinage of the House Banking, Finance and Urban Affairs Committee.

Rep. Frank Annunzio (D-Ill.), chairman of the subcommittee, introduced an Olympic coin bill last June 11, calling for a silver dollar coin to be sold directly by the Treasury, with profits split between the Treasury and United States Olympic Committee. That bill is also before the subcommittee.

The Olympic athletes were in town to express concern over the Coin Act's status.

"If we don't get the go-ahead soon, it will be too late, and the money from coin sales--a projected $30 million with a 17-coin set--just won't be there to defray costs of training athletes for 1984," said Chris Knepp, a Houston lawyer who is chairman of the USOC's Athletes' Advisory Committee.