The United States and the Soviet Union may be able to agree before the end of this year on overall guidelines for reducing their arsenals of long-range nuclear weapons, despite the current chill in their relations, an American negotiator said yesterday.

Ambassador Edward L. Rowny, the chief U.S. negotiator at the Strategic Arms Reduction Talks (START) in Geneva, said that it remains his hope that the two countries can reach such a preliminary agreement, similar to their 1974 Vladivostok accord, "before Christmas . . . , provided that Secretary of State George P. Shultz and Soviet Foreign Minister Andrei A. Gromyko are talking to each other."

In an interview with editors and reporters of The Washington Post, Rowny said that a meeting between Shultz and Gromyko probably would be necessary to complete such an agreement after the next round of detailed START negotiations in Geneva, which are scheduled to begin Oct. 6.

However, Rowny also said that December would be a "watershed" for related negotiations between the United States and the Soviet Union in Geneva on limiting medium-range nuclear weapons in Europe.

If no agreement is reached in those Intermediate Nuclear Force (INF) negotiations before the scheduled December beginning of deployment of U.S. Pershing II and ground-launched crusie missiles in western Europe, Rowny said, it is uncertain about what will happen.

Nevertheless, Rowny said, progress in the last round START negotiations created the possibility of reaching a preliminary agreement with the Soviets on guidelines for reducing long-range nuclear weapons that could include:

Counting warheads rather than missile launchers.

Setting an overall ceiling on the number of warheads on each side.

Agreeing on equality in total missile throw-weight, or lifting power.

Rowny said he believed the Soviets want such an agreement because they already have plans to reduce their big advantage in throw-weight by moving to smaller, mobile, single-warhead missiles.

He also said that Soviet negotiators are eager to limit the number of two new U.S. nuclear weapons: air-launched cruise missiles and Trident II submarine-launched missiles. This aim gives important leverage to the United States, Rowny said.

If U.S. and Soviet negotiators can complete the rest of their spadework during the next round of negotiations in Geneva and if Schultz and Gromyko are then able to agree on basic trade-offs on overall negotiating guidelines, Rowny said, a Vladivostok-like accord could be formalized at a summit meeting next year between President Reagan and Soviet President Yuri V. Andropov.

The Vladivostok agreement between then President Ford and Soviet President Leonid Brezhnev in November, 1974, set the framework for the SALT II nuclear arms-control treaty that the United States has never ratified, but which both sides are generally observing.

Even if such a preliminary agreement were reached in the START negotiations, which Rowny said have been complicated by the shooting down of the South Korean airliner, completing the negotiations would require many months or years.

Rowny, who met with Reagan last Saturday, expects to see him again to discuss instructions for the next round of the START talks, which will be the fifth since they were resumed by the Reagan administration in June, 1982.

Before the Korean Air Lines passenger jet was shot down, Reagan administration officials had expected that Shultz would be able to discuss substantial arms-control questions in a lengthy meeting with Gromyko in Madrid on Sept. 8.

That was to have been followed by two more Shultz-Gromyko meetings at the United Nations in New York during the next two weeks, according to U.S. officials, who said they had hoped these discussions could narrow many of the differences between the two sides in the START talks.

Consideration was being given to inviting Gromyko to meet with Reagan in Washington if the New York talks went reasonably well, the sources added. Such a Reagan-Gromyko meeting could well have established the groundwork and precedent for a Moscow meeting between Shultz and Andropov, leading a few months thereafter to a Reagan-Andropov summit, they said.

All this was derailed by the shooting down of the South Korean airliner and the Soviets' refusal to accept responsibility.

The Shultz-Gromyko meeting in Madrid went ahead in a climate of bitterness, and was limited in duration and scope.

Last Saturday, Gromyko canceled this trip to the United Nations after his plane was forbidden to land at commercial airports in the New York area.

State Department officials said a substantial "cooling-off period" probably will be required before the relationship can be repaired to the point that a Shultz-Gromyko meeting can be productive.Veteran Foreign Service Officer Nominated to START Delegation

The White House yesterday nominated a career Foreign Service officer highly experienced in dealing with the Soviet Union to be vice chairman of the U.S. delegation to the Strategic Arms Reduction Talks (START) with the Soviets.

Sol Polansky, 56, the No. 2 person in the U.S. Embassy in Austria, would become principal deputy to the chief U.S. negotiator, Edward L. Rowny.

Polansky, who speaks Russian and served two tours of duty at the U.S. Embassy in Moscow, is to replace the current START team deputy, James Goodby, whom the White House yesterday nominated to become the chief U.S. delegate to a new Conference on Disarmament in Europe (CDE), expected to begin early next year in Sweden.

Goodby, along with several other members of the START delegation and the U.S. Arms Control and Disarmament Agency, were described unfavorably in a memorandum from aides to Rowny that surfaced early this year. Two of the officials have left the team.

Rowny has apologized for the memo and said it did not contain his thoughts.

Goodby is known to have accepted that apology, to have stayed on the START team by choice and to have been interested in the CDE job for some time.