President Reagan said yesterday that the new arms control proposal by Soviet leader Yuri V. Andropov is "encouraging" because it recognizes "that we should be negotiating warheads and not just missiles."

But Reagan cautioned in a session with reporters that "you won't know until you really sit across the table from them . . . whether this was just propaganda or a proposal."

Reagan also suggested he had little quarrel with a pastoral letter adopted Tuesday by the U.S. Roman Catholic bishops which has been widely interpreted as a condemnation of administration nuclear policies, and which the administration tried earlier to soften.

"We haven't received it yet," he said, but "I have had some information in advance about it, which indicates that it really is a legitimate effort to do exactly what we're doing, and that is to try to find ways toward world peace. And if so, then we're both doing the same thing."

At the State Department, however, spokesman Alan Romberg said the administration continues to disagree with major portions of the letter, and particularly its call for a halt in production and further deployment of nuclear weapons, which he said would undermine the U.S. negotiating position in the arms control talks in Geneva.

In a Moscow speech Tuesday, Andropov said he was prepared to negotiate an agreement on intermediate-range missiles based in Europe that would leave the Soviet Union and NATO countries with equal numbers of warheads and aircraft capable of carrying nuclear bombs.

The administration wants to move from counting missiles to counting warheads for several reasons.

One is to discourage further development of large multi-warhead missiles on both sides.

Defense Secretary Caspar W. Wieinberger also said at a news conference yesterday that Andropov's willingness to count warheads in the Euromissile negotiations was "a good thing." But he said he was bothered because the Soviet leader continued to count British and French as well as U.S. weapons.

If the British and French missiles are included in the balance, there would be no room left for the Pershing II and ground-launched cruise missiles that the Reagan administration plans to start deploying in Europe in December.

Other government officials pointed out yesterday that planned modernization of the British and French missiles, which includes fitting most of them with additional warheads, would also be jeopardized by the Andropov proposal. If Britain and France increased their warheads, the Soviets would presumably be entitled to increase theirs, under Andropov's plan.

Nonetheless, the president said the Andropov proposal would be given "serious consideration."

In a related development, it was also learned yesterday that the United States has put a series of questions to Soviet Ambassador Anatoliy F. Dobrynin about recent flight tests of two new Soviet intercontinental ballistic missiles, according to government sources.

Administration officials have described the tests as apparent violations of the 1979 strategic arms limitation treaty, SALT II, which has never been ratified but both sides have agreed informally to observe.

Earlier this year the president, pushed by conservatives in Congress and inside the administration, was preparing to denounce the Soviets publicly for violating the treaty, citing the tests of the two new missiles as one example.

The administration's use instead of a private diplomatic note to question the two missile tests marked a temporary victory for officials who argued that going public before making use of established diplomatic and treaty channels would jeopardize new nuclear arms agreements and create yet another public dispute with the Soviets