Nearly everybody in official Washington expects an early showdown between Zbigniew Brzezinski and Edmund S. Muskie, except the two men themselves. The White House national security adviser and the people's choice for secretary of state both seem breezily confident that a destructive clash will be averted.

Brzezinski, in an interview, said "I have the strong gut feeling that the conventional wisdom, which now is that there will be a new cat fight . . . is going to be proven quite wrong." Calling Muskie a "self-confident . . . knowledgable . . . mature" person, Brzeninski declared: "I don't think he is going to let himself be egged on, and I certainly do not intend to let myself be egged on by others into a counterproductive and pointless conflict."

Muskie, starting with the unscheduled but seemingly deliberate question-and-answer session with reporters following the announcement of his selection last Tuesday, has exuded supreme confidence. "The president has left no doubt in my mind . . . that I will be the foreign policy spokesman" for the administration, Muskie said. In Augusta, Maine, a day later, Muskie said, "I took this job not to be second in foreign policy, but to be first." And once again, he claimed President Carter's backing for his position.

The underlying reason for the speculation about Muskie's standing is the widespread belief that his predecessor, Cyrus R. Vance, was checkmated and finally vanquished by Brzezinski in a struggle over the direction of foreign policy.

If so, it would be the second time in recent years that a highflying national security adviser triumphed over a secretary of state. Henry A. Kissinger, who rose to fame and power from the White House West Wing during the Nixon administration, easily outmaneuvered William Rogers.

Experts and veterans in national security policy-making interviewed by The Washington Post said it is highly likely that conflict will arise between a highly visible national security adviser and an assertive secretary of state. The consensus: Muskie, as secretary of state, will have a harder time than he expects.

Philip Odeen, a former National Security Council and Defense Department official who conducted a five-month White House-sponsored study of foreign-policy-making last year, said a White House-State Department clash probably is inevitable -- and that the national security adviser probably will win.

"The idea that the secretary of state is going to be chief policy spokesman isn't going to work" under present conditions, Odeen said. Today major foreign policy issues have "heavy political, economic and national security implications" and the secretary of state "cannot look at them from the White House point of view," he added.

As for Muskie, "I don't know how he can even begin to compete with Brzezinski substantively," said Odeen.

Retired Gen. Brent Scowcrfot, who served as White House national security adviser during the last 15 months of the Gerald Ford administration, said "it depends on the president whether we have this conflict." Scowcroft, who believes the nation is "poorly served" by highly visible competion between the president's national security adviser and his secretary of state, said any competitor inside the White House holds most of the cards.

"The security adviser is right down the hall, and likely to see the president several times a day. The president is likely to trust him because he has no institutional interest, such as a big department to run, and nobody to serve by the president. And he always has the opportunity for the last word -- a cover note on a memo, or personal comment when he takes it into the Oval Office. This is where the power lies," said Scowcroft.

"Muskie is likely, if crossed, to stomp all over Brzezinski," according to Scowcraft. "On the other hand, Muskie doesn't have a deep background in foreign policy, and he'll have to have his own foreign policy strategy in order to compete. It may take most of the rest of the president's term for him to learn."

I. M. Destler, a senior associate at the Carnegie Endowment, is author of a recent Foreign Policy magazine article advocating that the national security adviser's job be abolished to avoid corrosive damage to foreign policy. Destler said the basis of the internecine conflict is a "built-in tension between what we expect the president to do and what the real world is like."

The president and his staff are inclined to strong and dramatic action, in keeping with public expectations, but the secretary of state is the transmission belt for the constraints imposed by foreign governments and by diplomatic experts on the dangers of hold action, Destler said.

In Destler's view "Muskie has a real chance" of becoming the dominate foreign policy operative -- but only if he moves quickly. "His advantages will never be greater than right now when he's a new figure and has the public spotlight. If he waits until he's master of the system and cable traffic before pushing his own substantive involement, he'll have a difficult time," Destler added.

The senator from Mine originally planned to take several weeks to fulfill his commintment to the Senate budget process before settling down to work at the State Department. But late last week, reportedly at the urging of Carter, he gave overwhelming priority to the new job and agreed to being much sooner.

Muskie had his first lengthy meetings on the substance of foreign policy with Carter, Brzezinski, Secretary of Defense Harold Brown and others at Camp David Friday and yesterday. He is scheduled to begin his confirmation hearings before the State Foreign Relations Committee next Wednesday and is expected to be sworn in just in time to fly to Europe as secretary of state to meet allied foreign ministers as well as Soviet Foreign Minister Andrei A. Gromyko the week of May 12-16.

The immediate focus of the Camp Davis talks is believed to have been the Iranian hostage crisis, the issue which was cited by Vance in his resignation as secretary of state. Many observers in and out of government assume that this disagreement was not solely responsible for the resignation, however.

The Vancian world, a rational place where negotiation and common sense abound, seemed to have lost out to the Brzezinski world of Soviet menace and powerplays -- and the Vance policies were seen as losing out to Brzezinski policies in conflicts at the top of the U.S. government.

Brzezinski, while conceding that "the predominant point of view in Washington" is that he triumphed in a contest with Vance, insisted that the assumption is erroneous.

"The secretary of state was, is and will be the president's principal spokesman and counselor on foreign policy. And I don't think his resignation was the product of struggles with me," said the White House official.

"I honestly don't believe there was [a struggle]. I think there is a strong conviction in this town that there has to be one, even a strong desire on the part of some people to manufacture one," Brzezinski added.

Asked to explain the reasons for the widely accepted belief in a Vance conflict, Brzezinski replied, "I'm not an expert in group psychology, and it's hard for me to interpret the motives of people who go out of their way to predict conflicts and to stimulate them. I suspect that in some cases it may be sheer boredom."

Brzezinski was asked about a charge by Leslie H. Gelb, who served as State director of politico-military affairs under Vance for 2 1/2 years, that the national security adviser misrepresented the State Department position in communications to the president on several occasions. Gelb says he and other department officials learned of Brzezinski's misrepresentations from members of the national security adviser's staff.

Calling this "a very serious charge which is not backed by either Vance or (Harold) Brown to my knowledge," Bzrzinski challenged Gelb to produce evidence to support his "scurrilous innuendoes." Brzexinski also said of Gelb, to produce evidence to support his "scurrilous innuendos." Brzeninski also said of Gelb, "I never respond to pygmies chewing on my toenails."

Gelb said his charge of misrepresentation is "not subject to judicial proof" since Brzezinski's communications with Carter are not circulated within the government. He added, "It is a face that several members of his staff told that to us."

According to Gelb, numerous instances of undercutting and crosspurpose between NSC and State contributed to the decline and demoralization of Vance, Gelb said vance consistentsly demanded that his staff refrain from fighting back in public or private because "that would only make it worse.

Brzezinski and gelb both indicated that basic differences in outlook on a rapidly changing international scene were involved in the policy debates of the past 3 1/2 years. Brzezinski suggested that it was a fair competition of ideas, while Gelb suggested that some of the national security adviser's tactics were unfair.

Vance is so far silent on the policy debate, except for carefully restrained expression of dissent from the use of military force in the recent attempt to rescue American hostages in Tehran. This issue was the immediate cause of his resignation. Some of his friends suggested that private citizen Vance will have more to say about the national security adviser's role in coming weeks.