President Carter's national security affair adviser, Zhigniew Brzezinski, said yesterday he does not believe a Soviet military intervention in Poland is "imminent" and asserted that a major crisis in East-West relations can be averted if all parties to the Polish dispute show "restraint, moderation and cooperation."

At a meeting with reporters, Brzezinski deliberately kept stressing these three words as illustrative of the Carter administration's view on how the various parties involved -- the Soviet Union, the United States and its West European allies and the contending forces within Poland -- should behave in attempting to prevent the Polish situation from escalating to the level of international confrontation.

His use of this terminology came after two days of tough talk by senior administration officials that was capped by the White House on Wednesday issuing a written statement in the president's name. In it, Carter cited "the unprecedented buildup of Soviet forces along the Polish border" and warned that Soviet intervention would have "most negative consequencies" for East West relations.

In a move apparently intended to give Carter bipartisan backing, president-elect Ronald Reagan's chief aide, Edwin Meese III, said yesterday that Kremlin leaders should understand Carter "is totally in control of foreign policy and all the apparatus of government" during the transition period and "we are very supportive of him in maintaining that control."

Meese, who will be counselor to the president when Reagan takes office, said this should be "very clear" and added: "I think the Soviets would make a grave miscalculation if they perceive any weakness in the structure in this kind of interregnum period."

However, Brzenzinski, in his comments yesterday, appeared to be going out of his way to emphasize that the United States does not want the Polish situation to reach the stage where it could thrust East-West relations back to the level of the Cold War. Instead, the Polish-born Brzezinski, who is regarded as the administration's leading advocate of a hard line toward Moscow, sought to dispel the impression that Carter's Wednesday statement about an "unprecedented buildup of Soviet forces" implied that Washington fears the Soviets are preparing to strike against Poland.

While conceding that the administration has evidence that the Soviet units have been brought to "a high state of readiness," Brzezinski said Carter's statement referred not to fear of an imminent intervention but to "the factual situation" that there has not been at any point in the postwar period the kind of troop activity and border closings now taking place on Poland's frontiers with the Soviet Union and East Germany.

In speaking out as it has in recent days, he added, the Carter administration has been attempting to convey two messages to Moscow: first that the United States is not seeking to benefit from the Polish unrest by undermining the Soviet Union's internal security or relationships in Eastern Europe and, secondly, that a Soviet use of force would have "long-lasting, far-reaching negative consequences for international relations in general and U.S.-Soviet relations in particular."

While insisting he didn't want to be "a prophet" about the outcome, !brzezinski stressed: "One mustn't on one hand become hysterical and conclude that intervention is inevitable. At the same time, one can't ignore the situation."

If the contesting Polish factions can reach an accommodation that would defuse the threat of Soviet intervention, he added, it might eventually be possible for both the East and West blocs to get together on some form form of joint effort to help Poland with the long range restructuring of its economc system. "I wouldn't under the right conditions, exclude some kind of East-West consulations on the subject," he said.

Brzezinski also denied a report by the Soviet news agency, Tass, that he met secretly with Polish political scientists and tried to agitate them against that country's communist leadership."Nothing like that ever took place," he asserted.

In another development yesterday, Richard V. Allen, Reagan's top foreign policy adviser, said in a CBS television interview, that he didn't think a U.S. military response to a Soviet intervention in Poland "would be likely." Instead, Allen said, any responses by the United States and its allies would be economic and diplomatic in nature.