Foreign Minister Yitzhak Shamir, who immigrated to what was then Palestine almost 50 years ago, early today was chosen by Israel's Herut Party as its candidate to succeed retiring Prime Minister Menachem Begin.

Shamir defeated Deputy Prime Minister David Levy in a secret ballot by members of the Herut Party central committee. The vote was 436 to 302, a larger margin than had been expected.

The party endorsement put Shamir over the largest hurdle in the struggle to succeed Begin, who announced on Tuesday his decision to resign. However, Shamir must still win the backing of smaller political parties that, together with Herut, form Israel's coalition government.

Shamir is expected to begin that task later today in a series of meetings with leaders of the other parties.

In a speech to the exhausted central committee members after the result of the voting was announced, Shamir repeatedly invoked Begin's name and extolled the record of the Begin government, but he spoke only in generalities about his own plans for Israel's future.

"Time is short and there is much work to be done," he said of the task of holding together the coalition. "In the morning we must sit in meetings to reassemble the government that must resign with the resignation of Prime Minister Begin. . . . This government must be renewed in the nearest future."

The victory of Shamir, 68, signaled the continued if weakened ascendancy in Israeli politics of the old-line European Zionists who have run the country since its birth. Levy, 45, is a native of North Africa and a hero of the Oriental Jewish community that is now a majority of Israel's population.

Intense lobbying preceded the balloting, which took place in a sweltering auditorium in Tel Aviv beginning last night and stretching into the early morning hours because of the slow counting of ballots. It was the first time in its history that the Herut Party had met to decide such a crucial issue without Begin as its leader. The outgoing prime minister did not attend the meeting but was told of the results by telephone before they were announced publicly.

Begin has still not submitted his official letter of resignation to President Chaim Herzog that will bring the automatic resignation of his government. Begin agreed to delay taking that final step to give Herut time to choose his designated successor. It was not known whether he will wait longer to give Shamir time also to line up a parliamentary majority.

Begin's party has 46 seats in the 120-member Israeli parliament, four fewer than the opposition Labor Party, but several smaller parties have voted with it to provide the ruling margin.

Before the Herut voting, both Shamir and Levy pledged cooperation with each other and stressed the importance of party unity. A split in Herut was not anticipated, but Shamir may still face difficulty in holding together the government coalition in the post-Begin era.

The problems the next Israeli prime minister will inherit include the Army's continued presence in Lebanon and a deteriorating domestic economy.

The contest between Shamir and Levy was a classic confrontation between generations, between the remnants of the old guard of European Jews who helped to found the country and the younger, increasingly impatient generation of Sephardim, or Oriental Jews, who came to Israel in the post-statehood waves of immigration in the early 1950s.

Shamir, who like Begin was born in Poland, arrived in what was then Palestine in 1935, two years before Levy was born in Rabat, Morocco. By the time Levy, then 20, and his family reached Israel in 1957, Shamir was a member of the country's intelligence service, Mossad.

Like Begin, Shamir went through the agonies that accompanied Israel's birth as an independent state in 1948, fighting in an underground force against the British authorities in Palestine. He was imprisoned several times and often lived in hiding because of his role in the Stern gang, which was involved in assassinations and other terrorism.

Levy's experience is one shared by many of his generation of immigrant Jews from Middle Eastern and North African countries. It included life in an immigration absorption center, being shunted off to one of Israel's barren "development towns," and experiencing unemployment and poverty.

Former defense minister Ariel Sharon publicly backed Shamir and asked his own supporters in the party to vote for the foreign minister. This prompted the afternoon newspaper Maariv, in a strong editorial yesterday endorsing Levy, to suggest that in return for Sharon's support Shamir was prepared to reinstate him in a position of power.

The paper noted that Sharon was forced to leave the Defense Ministry by the findings of the Israeli commission that investigated last year's massacre of Palestinian refugees in Beirut. It said it would be "inconceivable" for Begin's successor to return Sharon to an active role in government.

Shamir denied making a political deal with Sharon, but said on television tonight that the former defense minister should be given a Cabinet post more important than the powerless job of "minister without portfolio" that he now holds.

Throughout last summer's war in Lebanon, Shamir appeared as a largely ineffective junior partner to the architects of the Israeli invasion, Begin and Sharon.

He was damaged by the massacre commission's report. It criticized him for ignoring information relayed to him by another Cabinet minister on the second day of the massacre that a "slaughter" was taking place.

Shamir testified that he recalled only being told of the "unruliness" of the Phalangist militiamen in the camps and that the information did not particularly alarm him.