The first contingent of Syrian Army officers arrived here today to join a newly formed Lebanese security team that will supervise a peace plan aimed at pacifying Beirut and its airport.

The five Syrians are the vanguard of a group of 40 military observers and five intelligence agents expected here as part of the new effort by Damascus, the chief outside power in the Beirut area now, to stabilize Lebanon, beginning with the chaos-ridden Moslem half of the capital.

The Syrians' arrival came as shelling intensified between Christian and Moslem sections of Beirut and Lebanon's prosecutor general ordered a criminal investigation into the hijacking last month of a Trans World Airlines passenger plane by Shiite Moslem radicals.

Meanwhile in southern Lebanon, a suicide car bomb exploded near the Israeli-declared security zone, killing 10 persons, including two members of the Israeli-backed South Lebanon Army.

In Cairo, Egyptian Prime Minister Kamal Hassan Aly, who left later for talks in Washington, was quoted as saying that "there is a chance for the release of the seven American hostages" kidnaped over the past 17 months by Shiite radicals in Lebanon "in exchange for a break of the American boycott of Beirut airport," The Associated Press reported. Aly, quoted by the Middle East News Agency, gave no details.

In Washington, a U.S. official said the State Department had no information suggesting such a release and that there was no "linkage" between the kidnapings and the Beirut airport sanctions ordered by President Reagan as a result of the TWA hijacking.

The Syrians in Beirut attended the first full session of a security coordination committee under Lebanese Prime Minister Rashid Karami, who ordered militiamen off the streets and warned that a police force, backed by 300 Lebanese Army troops, would "bring every excess under control" and "strike" at any violators not heeding a 6 a.m. Tuesday deadline.

Lebanese Defense Minister Adel Osseirane said after the meeting that "the Syrians have come with full momentum" behind the security plan. Observers here are skeptical, however, about any miraculous recovery or lasting show of respect for a government-sponsored police and Army force.

The security measures, worked out in talks in Damascus last week among 13 Lebanese Moslem leaders and Syrian officials under the chairmanship of Syrian Vice President Abdel Halim Khaddam, call for a withdrawal of armed militiamen from west Beirut neighborhoods and the closing of many party offices.

The immediate goal is reducing friction between Beirut's two most powerful militias -- Amal, the Shiite Moslem force headed by Nabih Berri, and Walid Jumblatt's Druze force -- as well as bridging the gap between Berri and the Sunni Moslems.

The eventual Syrian aims are to pacify Beirut, provide for the safety of Palestinian refugee camps, rid the Beirut International Airport and roads leading to it of warring militia groups and restore authority to Lebanese police and Army.

But Syria's delicate management of Lebanon's mosaic of religious groups has so far excluded the Christians, who are said to be divided over the peace plan, with former president Camille Chamoun having expressed serious reservations about it. In the absence of a central spokesman for the Christians -- because of differences between the mainstream Phalange Party, which supports President Amin Gemayel, and the new leadership of the more militant Lebanese Forces militia -- Damascus is expected to have problems dealing with them.

Syria has made it plain it will not venture into direct military involvement here before securing political cover from all the factions. The Christian militia rejects any Syrian military role in its midst, but supporters of Gemayel differ with this position.

Despite Karami's stern statement today, barrages continued from both sides of the line that separates Beirut's Christian east and Moslem west. Shells crashed into residential areas in both sectors, keeping intercity access roads closed and spreading general pessimism.

The flareup followed nationally televised remarks yesterday in which Druze leader Walid Jumblatt, speaking to Druze militiamen, ruled out any reconciliation with the Christians as long as the Phalange Party and Gemayel are in power.

"There will be no solution in the presence of that tyrant and his party in power," Jumblatt said. "We shall not go back to the case of no-victor, no-vanquished. Either they kill us or we kill them."

Communications Minister Joseph Hashem, a Phalange Party member, said that the attitude of Jumblatt, who is also a member of the unity government Cabinet, showed that attempts at Christian-Moslem reconciliation were a waste of time.

The car bomb that exploded five miles north of the Israeli border, near Kfar Tibnit in southern Lebanon, killed at least seven Lebanese civilians, two members of the Israeli-supported South Lebanon Army and the car's driver, Israeli officials in Tel Aviv reported. A statement by the National Lebanese Resistance Front had said it had killed six Israeli soldiers and 30 members of the allied militia.

A few hours later, Lebanon's Moslem-controlled branch of state television broadcast a recorded statement by the suicide driver, Hisham Abbas, 22. Abbas said he had many comrades waiting for their turn and that his hope was to kill as many of the Israeli "enemy" as possible. He said he was a Lebanese citizen and a member of the pro-Syrian Baath Party and paid tribute to Syrian President Hafez Assad.

Today's attack and two last week in the vicinity of Israel's security strip in southern Lebanon that killed 19 people, in the view of analysts here, carry the stamp of Syrian secularism rather than the religious zeal of Lebanon's Shiite militants. This is seen as an indication that there is mounting Syrian impatience with Israel's failure to evacuate the border strip. The choice of non-Shiite suicide bombers, according to analysts, also reflects Syrian concern over the rising influence of the radical Hezbollah movement in southern Lebanon.

Lebanon's Prosecutor General Maurice Khawam, meanwhile, instructed Magistrate Sabah Haidar to open a judicial inquiry and prosecute the TWA hijackers on charges of murder, air piracy and the smuggling of weapons aboard a plane. The charges were made against "unknown persons" because, Khawam said, he did not know the names of the perpetrators and those being circulated were "probably false."

Khawam said he has ordered authorities to redouble efforts to identify the hijackers and arrest them. But the hijackers of at least 12 other planes in the past seven years remain at large.