Vice President Mondale said yesterday that Sen. Edward M. Kennedy (D-Mass.) has no legitimate ideological or policy reason to challenge President Carter and by doing so risks a "poisonous" battle that could make the election next year of either man impossible.

Reviving earlier -- but since muted -- White House suggestions that the Kennedy challenge for the Democratic nomination will divide the party deeply and hand the election to the Republicans, Mondale said he has been through a number of party fights and has never seen one that proved helpful.

"The real danger is that it will be so bitter, so poisonous to the Democratic Party that no Democrat can win," he said.

Portraying Kennedy as a candidate who is switching his positions on some issues and has "not thought deeply" about others, such as inflation, Mondale said the Massachusetts senator has yet to find "an issue-based reason for seeking the presidency."

"I believe that Ted is going to have a hard time making the case that there is some fundamental issue that justifies opposing this president," he said.

The vice president made the remarks to a group of reporters on a day that Carter suffered the first prominent defection from his administration to the Kennedy camp.

Dick Clark, the administration's special ambassador for refugee programs, resigned from the position effective Nov. 1 and told the president in a letter that he will join Kennedy's campaign committee. Clark said in the letter that he had hoped to remain neutral in the Carter-Kennedy race, but had later decided that "such a position is untenable and unwise."

According to a senior White House official, Clark met with Mondale early this month and told the vice president that he wanted to stay in the administration but remain "neutral" in the Carter-Kennedy race. On Oct. 15, the official said, Mondale told Clark that Carter, with some reluctance, had agreed to the ambassador's request.

"That was the last time that anyone in the White House heard from him until today," the official said.

Clark offered a slightly different explanation of the course of events, saying that the White House initially had asked him to undertake political work on Carter's behalf while continuing to serve in the State Department.

"The vice president called me into his office," Clark recalled yesterday, "and he said I ought to go to Iowa and start working for Carter. I said I simply couldn't do that, I had great respect for Sen. Kennedy. . . ."

Clark said he offered to stay on the job while remaining neutral in the race. Mondale first warned that Carter would not accept that, Clark said, but the vice president later called to say that neutrality would be an acceptable position.

White House officials downplayed the significance of the defection of Clark, a former U.S. senator from Iowa who was appointed to the administration after he was defeated for reelection in 1978. One official said he "would not be surprised" if there were further defections as Kennedy seeks to line up support for his challenge.

But it was also clear from the comments of the official and a tart note from the president, accepting Clark's resignation, that the White House believes Clark broke a promise.

"Fritz [Mondale] had reported to me your pledge to remain neutral in the political campaign and to devote your efforts to assisting refugees and other suffering people," Carter wrote to Clark. "I consider this work to be of transcendent importance. Therefore, I accept your resignation with regret, and appreciation."

In other political developments yesterday:

Chicago Mayor Jane Byrne telephoned the president in the morning to tell him she was going through with her endorsement of Kennedy later in the day. She told reporters in Chicago that her decision was based in part on Carter's lackluster reception at a fund-raising dinner in Chicago Oct. 15 on a private poll showing the president's vulnerability in Illinois.

The first federal employes union to endorse a presidential candidate the year, the 70,000-member National Treasury Employes Union, came out for Kennedy and gave him a $5,000 contribution, the maximum allowed by law. The union endorsed Carter in 1976.

In his luncheon meeting with reporters, Mondale laid out the outlines of a Carter attack on Kennedy. The vice president suggested that had Kennedy been president for the last three years the state of the economy would not be much different "because of the difficult issues." He also said Kennedy has been shifting his political ground -- by criticizing additional defense spending, for example, then voting for a supplemental defense appropriations measure.

In the meantime, Mondale said, Carter has established a record of identifying major problems such as energy and proposing solutions.

"A lot of these issues are tough and oratory does not get it done,' he said. "We are not conducting an anti-Kennedy campaign but the burden of proof should be on [him] . . . I believe that President Carter will get stronger every day this debate goes on."

Mondale said he hoped that a divisive campaign could be avoided by keeping the debate "as civilized and positive as possible," but added that "my experience in every one of these disputes is that it doesn't work out that way."

Asked how the president's earlier comment that he would "whip his [Kennedy's] ass" lived up to Mondale's criteria of civilized and positive campaigning, the vice president replied."That could have used some editing."