Paul H. Nitze, one of the most enduring hard-liners in dealing with the Soviet Union on arms control and military matters, is likely to become the chief U.S. negotiator on limiting nuclear-tipped missiles based in Europe, according to administration officials.

"It's about 80 percent certain," one official said, referring to the prospects that Nitze, 74, a conservative Democrat who has served Republican and Democratic administrations for more than 30 years in national security jobs, would handle the upcoming talks with Moscow.

Nitze was a member of the Nixon administration team that negotiated the first Strategic Arms Limitation Treaty (SALT) with Moscow in 1972. He later strongly criticized the SALT II agreement negotiated by the Carter administration in 1979 but never ratified by Congress.

Because of his views on Soviet intentions, Nitze's appointment, if it comes, could cause misgivings among Europeans who are anxious for progress in the talks about European-based weapons.

Nitze, however, is widely viewed as among the most knowledgeable officials in the field, and administration officials suggest that it is unlikely he would take the job if he felt the administration was not serious about moving ahead with the talks.

The SALT negotiations dealing with intercontinental-range missiles and bombers are still stalled, but the Reagan administration has pledged to move ahead with talks on shorter-range weapons deployed in Europe by both the United States and the Soviet Union. The United States agreed to the talks in 1979, when NATO agreed to accept new U.S. missiles in Europe to improve defenses there.

Secretary of State Alexander M. Haig Jr. will meet at the United Nations in New York next month with Soviet Foreign Minister Andrei Gromyko to try to lay the groundwork for more formal negotiations, which he said he hoped could start between mid-November and mid-December.

If Nitze takes the job, he will complete a team of hard-liners who will lead the U.S. in both European and SALT negotiations. Retired Lt. Gen. Edward L. Rowny, who along with Nitze opposed SALT II, will be the chief SALT negotiator if those talks resume. Eugene V. Rostow, a Democrat like Nitze, is the head of the Arms Control and Disarmament Agency that oversees both sets of talks.