Israel's Labor Party opened its convention here today facing the bitterest divisions in the party's history.

For the first time, a ruling prime minister is being openly challenged for the party's leadership. Wednesday, convention delegates will be asked to choose between the incumbent, Yitzhak Rabin, and his challenger, Defense Minister Shimon Peres.

The vote will be a cliff hanger. So close are the candidates in numbers of delegates that no responsible political analyst here will even venture a guess as to the outcome.

The convention meets at a time of unprecedented financial scandal, accusations of impropriety against leading Labor officials, widespread labor unrest and a faltering economy. Perhaps never before has the party that has ruled Israel since independence in 1948 seemed so discredited in the eyes of the public.

Yet the winner of Wednesday's convention vote will probably be Israel's next prime minister. For although Labor is expected to lose ground in the general elections scheduled for May, the party will probably emerge wit a diminished plurality that will enable it to form a government in coalition with other parties.

Both Rabin and Peres have been frantically trying to round up delegate support and both claim to be going into the convention with a majority. Neither can really be accused of exaggerating or bluffing. Many prudent and cautions delegates have solemnly and privately promised both candidates their vote. Since the balloting will be secret, none need be the wiser as to how they really voted.

Both camps believe that a couple of hundred delegates may not yet have made up their minds, and this small number may make the difference.

Both Rabin and Peres are scheduled to address the delegates Wednesday evening before the voting begins. There is a big dispute as to who will speak first and who will speak second. Both men want to speak second, in order to rebut the other. The latest word from party headquarters is that Rabin has refused to settle the matter with a toss of the coin and the presidium will have to decide Wednesday.

Labor Party delegates have never before been asked to choose their leader. Succession questions have usually been decided either by the party's 600-man central committee or by party big shots. The decision to throw the choice to the convention delegates reflects a new mood in Israel to open the smoke-filled rooms and make the selection of candidates more democratic.

The Workers Party, parent of the dominant Labor Party, is older than the state of Israel. In the old days, party loyalties were second only to Zionist ideals.

The power and dominance of the political parties is breaking up and the challenge of Shimon Peres is only one manifestation of the trend. Peres made a bid for the leadership in 1973 after Golda Meir retired but the central committee chose Rabin instead.

The exact number of delegates is still in question because of a Supreme Court decision that held that the 570 outgoing central committeemen could not automatically join the 2,300 elected delegates in voting. The matter is likely to be settled in favor of the old central committee, however.

Rabin is the choice of the party's old guard - symbolized by the still-long and-powerful shadow of Golda Meir, who has bitterly condemned Peres' challenge to party authority.

"Under no circumstances can I absolve someone who has tried to oust the party leader and premier the way Shimon Peres has," Meir said. "I deeply regret that Shimon has chosen this course."

Peres said yesterday that Meir's attitude toward him had never been "very objective."

Meir, despite begin in retirement, continues to give the party the benefit of her guidance or, as some would call it, interference.

Another of Rabin's powerful backers is Finance Minister and former mayor of Tel Aviv Yehoshua Rabinowitz - the closest thing Israel has to an old-style, Tammany Hall godfather. There was a time when such powerful support would have been enough to clinch Rabin's convention victory, but no more. The party machinery is now being questioned and recent scandals have hastened the process of deterioration.

Today, Asher yadlin, former secretary general of the Labor Party Federation's holding company and former general manager of the Labor Federation's health services, was sentenced to five years in jail and a $28,000 fine for receiving bribes and kickbacks.

In his confession Yadlin said that he had used part of the bribes to finance the Labor Party had accused party bosses, among them Rabinowitz, of being implicated in improper campaign financing that milked the Labor Federation-owned companies to feed the party.

The Labor Federation owns companies and industries that amount for about a quarter of Israel's entire enconomy and the health services, which Yadlin managed, cover almost half of Israel's population.

Ironically, Rabin, who was a fighting general and then ambassador to Washington until 1973, is not personally implicated and was not even an important figure in the party when these scandals were said to have taken place.

Peres, by contrast, has been a Labor Party insider for the last 10 years. Yet Peres is able to present himself to the convention as the candidate of change while Rabin is stuck, for better or worse, in the embrace of the party bosses. One of Peres' major bids to the convention delegates is the claim that he would win more seats for Labor in the general election than could Rabin.

Rabin started the campaign as the liberal dove to Peres' more conservative hawk image. Rabin ran as the statesman who could guide Israel through the hazardous waters of American diplomatic pressure as the Arabs press for a return of their occupied lands in future peace negotiations. Rabin's image as the best man to handle the Americans has not been enhanced by the Carter administration's decisions not to allow Israel to sell Kfir jets to Equador and not to sell Israel the concusssion bombs that the Ford administration promised.

In any case the campaign has centered more on domestic considerations, which is shaky ground as far as Rabin is concerned.

The dove-hawk question has been greatly muted by the fact that both candidates have a full spectrum of hawks and doves in their camps.Peres, the alleged hawk, has Abba Eban the dove, for example, while Rabin has Golda Meir the hawk. The urbane Eban, former foreign minister who speaks six languages fluently, dropped his own bid for the party leadership to throw his weight behind Peres in what Eban calls a "balanced ticket." Eban would expect to emerge as Peres' deputy prime minister and foreign minister if Peres wins Wednesday.

The Labor Party is committed to surrending some territory for pece but no return to the 1967 borders and today nobody believes that either a Peres or a Rabin victory will make much difference to Israel's negotiating posture. It is widely believed here, however, that former Ambassador Malcolm Toon advised the Americans to tilt toward Rabin.

To most delegates it will probably boil down to a choice between personalities. Rabin comes across as a statesman with an ability to analyze problems but with an inability to forge a team that could carry the party or the country along with him, Rabin is a poor public speaker and is, at times, brutish and rough in his personal dealings.

Rabin's supporters hope, however, that the prime minister has convinced enough delegates that he, more than Peres, is a man of stature and that he too favors change. Since he was not part of the Labor Party machinery, the hope is that he will not be personally blamed for the current atmostphere of corruption and scandal.

Peres, on the other hand, is a consummate bureaucrat and a facile public speaker and politician. His detractors say that Peres is too ambitious and neither as thoughtful nor as intelligent as Rabin. They also say he lacks Rabin's moral fiber. Peres' supporters say that it is more important to have a prime minister who can work smoothly with others and form a team. They say a new face is needed if the Labor Party is not to become the lightning rod for all of Israel's discontent.

During the last round of negotiations over the Sinai interim agreement Henry A. Kissinger, as quoted by Edward Sheehan in Foreign Policy, found Rabin to be "not strong . . . clumsy and indecisive . . . it was Peres who dominated this negotiation. Ambitious, dogmatic, rather superficial, (Peres) was the strong man ready to risk the disasters Kissinger conjured up," Sheehan wrote.

Many prominent Eruopean Socialists were in town today for the ceremonies opening the convention. They included West Germany's Willy Brandt, Austria's Bruno Kreisky, France's Francois Mitterrand and Sweden's Olaf Palme.