President Carter said yesterday that Israel's legalization of three settlements on the west bank of the Jordan River poses obstacles to peace, and said he had strongly cautioned Prime Minister Menahem Begin that establishment would be a matter of "deep concern."

In a nationally televised news conference, Carter also said that Palestinians should be represented at a Geneva peace conference and that the United States will hold talks with Palestinian leaders when they abandon their commitment to the destruction of Israel.

Carter's comments about Middle East developments are certain to be scrutinized for nusance and direction in Israel and the Arab countries, especially on the eve of Secretary of State Cyrus R. Vance's trip to the troubled region. Vance is to depart Sunday on a 12-day journey aimed at laying the foundation for a Geneva conference this fall.

The President made no attempt to conceal the substantive gap between the United States and Israel regarding settlements on the west bank, which Washington considers to be under Israeli military occupation but which is claimed by Begin as "our land."

At the same time, Carter said the differences should not be exaggerated. He went out of his way to praise Begin as a man of "great strength" within Israel who is trying to reconcile his campaign commitments with the interest of peace.

The President's remarks about Palestinian representation - which he described as "the major stumbling block" to a new Geneva conference - departed somewhat from the language and emphasis of most previous official statements on the subject.

In September, 1975, then-Secretary of State Henry A. Kissinger promised the Israeli government in a secret memorandum - which later leaked out - that the United States will not recognize or negotiate with the Palestine Liberation Organization (PLO) as long as it does not recognize Israel's "right to exist" and does not accept U.N. Security Council Resolutions 242 and 338. Vance and other officials have said the Carter administration will abide by the Kissinger commitment.

Referring to "the Palestinians" without specifically mentioning th PLO, Carter said explicitly what the Kissinger memorandum only implied - that the United States will deal with the Palestinian movement when and if it abandons its commitment to the destruction of Israel.

Carter further defined what Palestinians must do: "adopt the proposition that Israel is a nation, that it will be a nation permanently, that it has a right to live in peace."

PLO Chairman Yasser Arafat, in statements to a recent delegation of two U.S. House members, came close to meeting Carter's criteria. According to Rep. Lee H. Hamilton (D-Ind.), chairman of the Middle East Subcommittee of the House International Recations Committee, Arafat indicated in a variety of ways that he is prepared to accept the permanent existence of Israel in the context of a peace agreement in which "even a very small" Palestinian state is created. However, Hamilton said Arafat's statements were "circutous" rather than explicit.

U.S. officials acknowledged that Carter's statement about the Palestinians represented a shade of difference from what high officials had said before. But it was unclear whether the difference was significant or merely semantic.

"I have tried to take a balanced position" in the Arab-Israel disprte to enhance the trust of all parties and thus nudge them in the direction of peace, Carter said. He said it would destroy U.S. effectiveness to ever take "a biased position" between Israel and the Arabs.

"I think that we have a good chance to go to Geneva," said Carter. He said his optimism, which startled many observes last week in the aftermath of Begin's visit here, arises primarily from the belief that "all national leaders with whom I have talked genuinely want to go to Geneva to try to work out permanent peace."

On the touchy issue of the Israeli settlements on the west bank, Carter drew a distinction between Israel's recent action in granting legal recognition to existing settlements and a potententially more serious future action to permit the establishment of additional Jewish enclaves.

Carter reported that in his discussions with Begin he did not think of raising the subject of granting legal status to existing settlements, and that Begin gave him no prior notice that such action was to be taken. While saying it is an obstacle to peace - following the language of a State Department declaration Tuesday - Carter added that in his view the obstacle can be overcome.

Speaking of new Israeli settlements, Carter used stronger language. "I let [Begin] know very strongly that this would be a matter that would cause our own government deep concern," he said.

At another point Carter called new settlements "a very difficult thing for public opinion to accept, both here and in the Arab countries."

Carter said he had told Begin it would be easier to accept an increase in the population of existing settlements than the establishment of new ones.

Begin made no commitment in the White House talks, and by all accounts argued that Israel has a right to settle Jews in the Wests Bank as it wishes.

The Los Angeles Times reported earlier this month that 16 new Jewish settlements on the occupied West Bank are in the planning stages.

Former Under Secretary of State Joseph Sisco, recently returned from the Middle East, said yesterday he will not be surprised if new Israeli settlements are established.