The number of whales caught annually by the Japanese is 4,500. The figure of 13,000 to 14,000 reported yesterday represents the yearly worldwide catch.

Norway and Japan took steps yesterday that would allow them to ignore the forthcoming worldwide moratorium on commercial whaling, prompting Secretary of State George P. Shultz to warn of possible economic sanctions in retaliation.

In Tokyo, Japanese officials announced plans to lodge a formal complaint against the ban, a move that would allow commercial hunting of whales after the ban takes effect in 1986.

The International Whaling Commission imposed the moratorium last July with strong support from the United States in an effort to stem the worldwide decline in the whale population. The action was hailed as a victory for conservationists.

Hours after Japan's announcement, Norwegian Fisheries Minister Thor Listau said his country will continue to hunt whales in the northeast Atlantic Ocean after 1986.

The moves set off a round of diplomatic negotiations between the United States and officials of the two nations. Japan is one of the world's leading whaling nations.

The formal objections filed by Japan and Norway are the only legal steps required for whaling nations to exempt themselves from the Whaling Commission's ban.

Japan now catches an estimated 13,000 to 14,000 whales each year, including sperm whales, which are on the American endangered species list.

"If Japan ignores the moratorium, it's the end of the whales," said Lew Regenstein of the Fund for Animals, one of the groups that pushed for the international ban.

"It will make it impossible to achieve conservation measures in the future," he said.

U.S. Ambassador to Japan Mike Mansfield hand-delivered a letter yesterday from Shultz to Japanese Foreign Minister Yoshio Sakurauchi, urging the Japanese not to file an objection to the ban, according to administration officials.

The contents of the letter remained classified last night, but sources said it included a warning that the Japanese could lose fishing rights in American waters if they do not abide by the ban.

Norway has a much smaller annual catch than Japan, estimated at 1,985 minke whales for the 1982 season.

Sakurauchi said in Tokyo that the Japanese government believes the ban is legally flawed.

He also said whales are an important food source and a crucial industry.

Japan and the Soviet Union warned at the time of the commission's 25-to-7 vote in London last July that they would consider opposing the ban in order to protect their whaling industries.

There has been strong political pressure in the United States to enforce the ban despite objections from other countries.

Last August, 66 senators called on Commerce Secretary Malcolm Baldridge to impose economic sanctions on any nation that defies the moratorium "in order to avoid any thought that the United States can be faced down on the whaling issue."

Under U.S. law, any country that violates whale conservation measures automatically loses half its fishery allocation in U.S. waters.

A Reagan administration official said this would have a "substantial impact" on Japan, particularly in the rich fishbeds of the Alaskan coast.

Another federal law allows the United States to impose an embargo on fish products imported from any country violating whaling agreements.

Administration officials said Japan and Norway are well aware of these laws, making it unnecessary for U.S. diplomatic officials openly to threaten to impose sanctions.

Sakurauchi stressed, according to Japanese radio reports, that yesterday's action would not automatically continue whaling after 1986.

The formal objection merely reserves a country's right to deviate from the ban, he said.