Israel is stepping up its pressure on Lebanon's president-elect, Bashir Gemayel, to agree soon after he takes office later this month to a formal peace treaty with Israel.

The clearest indication of this so far came in a speech last night by Israeli Defense Minister Ariel Sharon, who suggested that Lebanon faces a choice of formally making peace with Israel or accepting continued Israeli domination if not occupation of the southern portion of the country.

"If a future Lebanese government signs a peace treaty with Israel, Lebanon will certainly be united territorially," Sharon said in the northern Israeli town of Qiryat Shemona.

"But if there is a government in Lebanon that for any reason does not sign a treaty," he added, "then there will certainly be a special status in southern Lebanon along the 40-kilometer 25-mile security strip that Israel needs."

Israeli television reported that a rally organized by Gemayel's Phalangists in Sidon was called off by Israeli authorities Monday after the Lebanese refused to display posters calling for a peace treaty with Israel at their mass meeting. Reuter quoted the report as saying the rally had been called to celebrate the evacuation of the PLO guerrillas.

Senior Israeli officials said today that they remain committed to both a peace treaty and a strong Lebanese government and believe both can be achieved depending on "political developments in Lebanon."

An official said that even with a peace treaty "there will always be a need for a special arrangement for southern Lebanon," from which the PLO launched periodic attacks on Israeli communities such as Qiryat Shemona, where Sharon spoke last night. But just how this is to be accomplished, and to what extent it would involve a continuing Israeli role in southern Lebanon, remained vague.

The opposition Labor Party today criticized Sharon's statement about a "special status" for southern Lebanon, asserting that such an implied threat of continued Israeli domination of the area will be harmful in the long run to this country's security interests.

Sharon's speech came just a few hours after Foreign Minister Yitzhak Shamir raised the same topic during a meeting with ambassadors from the 10 member nations of the European Community.

Shamir spoke in terms of devising "security arrangements" for southern Lebanon and said this would best be accomplished through "cooperation between the armies of Israel and Lebanon" in the area.

Neither Sharon nor Shamir specified what he meant by a "special status" or "security arrangements" in southern Lebanon, but the statements indicated that the Israeli government intends to use its military presence in southern Lebanon as leverage to extract a peace treaty from the new government.

Gemayel, 34, the leader of the Lebanese Christian militia, has long been allied with Israel against Lebanon's leftist Moslem factions and the Palestine Liberation Organization. At a parliamentary election last month in which he was the only declared candidate, Gemayel was chosen as the country's next president and is to take office Sept. 23.

But Gemayel will inherit a fragile political situation in a faction-ridden country where the Israeli pressure for a peace treaty could easily backfire. Gemayel has vowed to try to unify the country behind his leadership, a goal that will require him to reach out to a number of Moslem leaders who strongly oppose any peace agreement with Israel.

At a secretly arranged meeting with Israeli Prime Minister Menachem Begin in northern Israel last week, Gemayel reportedly asked for Israeli patience and promised to seek peace after he is firmly in control of the Lebanese government. According to accounts in the Israeli press, Begin asked for the meeting to complain about some of Gemayel's recent critical statements regarding Israel.

The episode illustrated some of the potential conflicts in Israel's goals in Lebanon. On the one hand, the Begin government has announced that it seeks a strong, unified Lebanese central government, which is likely to require compromise between Gemayel and the Moslem leftist factions. On the other, Israel is clearly pressing Gemayel to agree to a peace treaty that could widen political splits in Lebanon and weaken Gemayel's authority.

Meanwhile, Israeli government officials continued their assault on President Reagan's Palestinian initiative. Foreign Minister Shamir told parliament's Foreign Affairs and Defense Committee that the Reagan plan represents a "substantial difference" in the attitude of the U.S. government toward the Camp David autonomy talks.

Previously, Shamir said, the United States had played the role of "honest broker" in the talks. But by coming forward with its own ideas on some of the key issues, he said, the Reagan administration has "gone from being a mediator to an interested party" in the negotiations and thus set back the hopes for reviving the stalled negotiations.