It's high noon on the high seas, and the United States and Japan are fast approaching a showdown that could have long-term implications for their trade relations.

The dispute is over the endangered sperm whale, protected by a hunting moratorium imposed by the 38-nation International Whaling Commission. The Japanese object to that moratorium, and this week four small whaling boats set out from the harbor of Ayukawa in defiance of the IWC ban.

If their hunt is successful, the United States, by law, must retaliate against the Japanese. Under the provisions of the Packwood-Magnuson amendment to the Fishery Conservation and Management Act, any nation that has "diminished the effectiveness" of the whaling commission must forfeit half of its allocation of fish from U.S. waters.

Nearly 10 percent of Japan's annual fish catch is taken from U.S. waters off the coast of Alaska.

Commerce Department officials have been negotiating with their Japanese counterparts for more than two months, seeking to head off a confrontation that some believe could escalate into a serious trade war between the United States and one of its best trading partners.

Japanese traders reportedly have threatened to boycott U.S. fishermen if the United States reduces the allocation to Japanese fleets.

But the latest negotiating sessions adjourned Tuesday -- the same day the whaling boats lifted anchor -- and a spokesman for the National Oceanic and Atmospheric Administration said no further talks have been scheduled. Similar discussions at the State Department also have been recessed while Japanese diplomats confer with their superiors in Tokyo.

NOAA spokesman A. Joseph LaCovey said any cutback in fish allocations to the Japanese would have little effect this year, because Japan has harvested most of its 1984 quota. But LaCovey said the cutback could affect future quotas.

The Reagan administration has strongly supported international efforts to preserve the whale population, but conservationists are watching the drama in the chilly waters off Japan with apprehension.

The IWC has declared an all-out moratorium on commercial whaling to begin in 1986. The commission has no power to enforce its ruling, however, and environmentalists believe that the effectiveness of the ban depends on U.S. willingness to use the stick it has at hand.

If the Japanese succeed in compromising with the United States, other whaling nations will follow suit "and soon the IWC will be a worthless piece of paper," said Mark Cheater of the environmental group Greenpeace. QUICKER CHANGE . . .

NOAA Administrator John V. Byrne has agreed to vacate his job by Nov. 1, two weeks earlier than originally planned, suggesting that deputy administrator Anthony J. Calio still has the inside track in the race to be Byrne's successor.

Commerce Secretary Malcolm Baldrige asked Byrne to leave early so he could install Calio as acting administrator. The idea, according to Baldrige spokesman B. Jay Cooper, was to "maintain management consistency."

But other agency officials contend that the faster change was engineered by Calio as a way to help him overcome Senate opposition to his formal nomination. Calio reportedly is opposed by every Democrat on the Senate Commerce, Science and Transportation Committee as well as several leading Senate Republicans.

Critics contend that he has inflated his academic credentials and is a "vindictive" manager who threatens to dismiss subordinates who disagree with him. In an interview last week, Calio acknowledged that he uses the title "Dr." although his degree is honorary. "That precedent goes back quite a ways with NASA," his former agency, he said. "I've never tried to sell myself as a hard-nosed researcher."

He also said he does "not specifically" recall threatening to dismiss any of the agency scientists. "I'm usually very frank," he said. "But that's not a threat. That doesn't mean I'm going to do that . . . . If I had my druthers, fine, but none of us has our druthers.

"The real question is whether I support the administration's programs. I try to carry them out to the best of my ability, and that doesn't sit well with even Republican senators."

The NOAA deputy does have support in some Republican quarters, however, most notably in the office of Alaska's senior senator, Ted Stevens.

"The policy direction under Calio seems to be more responsive," a Stevens aide said last week. "There's been a good allocation of resources when there's tight money."

Alaska, deeply involved in marine issues, traditionally takes a keen interest in NOAA's leadership. "It is our agency as far as we're concerned," Stevens' aide said. "This is a motherhood issue as far as we're concerned. For us, it's a Cabinet-level position."