The Carter administration is sailing two different courses as it tries to reach international agreement over use of the seas.

Government officials and lobbyists involved with the Law of the Sea negotiations conceded in interviews yesterday that this looks confusing but predicted all will end well.

The Pentagon is reminding its skippers and warplane commanders that they have every right to sail or fly to within three miles of another nation's shores.

At the same time, President Carter's representative at the United Nations is working on a treaty that would provide a 12-mile limit.

Elliot L. Richardson, the U.S. official at the Law of the Sea negotiations under way in New York, is trying to assure delegates from other nations that the Pentagon's new assertion of its rights does not mean backing away from the 12-mile limit.

"This has nothing to do with the Law of the Sea negotiations," said a spokesman for Richardson.

Other government executives said the Carter administration concluded it should back up its words about recognizing only the the three-mile limit by sailing through and flying over waters beyond that point even where the countries claim 12 miles as territorial water.

Sources said Charles W. Duncan Jr., deputy secretary of defense who has been chosen by Carter to run the Energy Department, spearheaded the drive, to demonstrate through ships and planes that the United States will not recognize limits beyond three miles.

"His argument was that the U.S. case under international law would be weak if we only claimed such rights and did not exercise them," said one official. The National Security Council agreed, resulting in the new policy for the military.

The United States almost went to war in 1968 after North Korea captured the USS Pueblo spy ship on the high seas for allegedly intruding into North Korea's claimed 12-mile limit.

The U.S. government's position then was not that the ship had a right to sail up to within three miles of the North Korean coast, but that it had stayed outside the claimed 12 miles.

Officials ducked questions about whether the Carter administration would allow a spy ship like the Pueblo to sail up to three miles from another country's coast, no matter what the claimed territorial limits. It is an open secret within the U.S. military that American submarines on "ferret" missions go in that close - sometimes closer.

Military sources, however, put the Pueblo and similar missions in a separate category. They said the current emphasis involves overt sailing and flying missions, and no covert missions like the Pueblo and submarine electronic ferreting.

Samuel R. Levering, a lobbyist for the privately funded U.S. Committee for the Oceans who often has criticized government actions on Law of the Sea issues, raised no objection yesterday to the administration's underscoring of the three-mile limit.

"It's a bargaining chip," said Levering, to serve notice on the Law of the Sea delegates that the United States will go its own way until the package of navigation rights it seeks is agreed on. The United States is willing to accept a 12-mile territorial limit, providing the treaty acknowledges such rights as sailing through narrow straits that could be closed under a strict application of a 12-mile limit.

As for spy ships brazenly sailing up to three miles from a nation's shores, when the country claimed 12, Levering said such "obnoxious" activity is generally considered unacceptable any closer than 12 miles from shore. He put hostile broadcasts from ships in this category, too.

Administration officials insisted that no particular countries are targeted for demonstrating that the United States will use any part of the high seas more than three miles from another country.

Besides the three-mile territorial limit and the nine-mile contiguous zone, for a total of 12, another Law of the Sea issue is how far from shore a coastal nation can regulate economic activity - such as fishing, drilling for oil or vacuuming manganese nodules from the sea bottom.

The United States, through an act of Congress, has unilaterally declared its rights to regulate fishing up to 200 miles from its coast.

The administration says it wants international agreement on seabed mining, but, at the same time, it has endorsed congressional legislation to allow companies to proceed without waiting for a Law of the Sea treaty.

The administration's rationale is that such seabed legislation, expected to be passed by 1980, will give Law of the Sea delegates more incentive to establish international rules.