Syria has decided to use its air force to challenge Israeli air strikes in Lebanon and "impose a limit" on their attacks on Palestinian guerrillas, a senior Syrian official said.

The Syrian move sharply raises the danger that more Israeli-Syrian dogfights over Lebanon could lead though miscalculation to broader hoatilities, although neither side is believed to be seeking such a development.

"Syria's decision is clear," Information Minister Ahmed Iskander said in an interview."It is to prevent Israel from continuing the operations of annihilation against Pallestinians and Lebanese. And if we are unable to prevent the Israelis completely - because of the military power imbalance between Israel and Syria - what is required to impose a limit to these operations."

Diplomats here said that the new Syrian policy raises the probability of another Israeli-Syrian dogfight over Lebanon such as that last week in which Syria lost five mig fighters.

Israel and syria both have been sending regular fighter patrols over Lebanon but the Syrians had previously avoided contact with the Israelis to preclude the risk of unforessn and unwanted escalation.

Any intense fighting or even a rise in Arab-Israeli tensions, diplomatic observers here say, is likely to sour the atmosphere of the Egyptian-Israeli talks on Palestinian autonomy and create new obstacles to attracting Palestinians and other Arabs to join them.

The Syrian government apparently has concluded that its standing with the Palestinians, the Lebnanese, the Arab world and its people at home demands a response to the Israeli air raids.

President Hafez Assad has been coming under increasing pressure from his Palestinian proteges to do something about the raids.

But perhaps more important, the presence of more than 20,000 Syrian troops in Lebanon - some within shooting distance of Israel's Palestinian targets - had made the Israeli raids into a challenge that could go unanswered only so long without a loss of face.

This became particularly true after Israel switched its policy from retaliation for specific Palestinian terror attacks in Israel to generalized attacks on Palestinian camps at any time. the change in policy was accompanied by statments from Israeli Prime Minister Menachem Begin seen by Syria as deliberately provocative.

So, since April, Syrian fighters had been flying over Lebanon in a warning that the Israelis could not go unchallenged. In the last several weeks, the Syrian Migs had responded directly to Israeli raids and flown to meet the attacking Israelis.

According to Syrian officials and foreign diplomats here, however, the Israeli pilots pulled away for the first several times declining the challenge.

Then last week, they say, the Israelis organized the fighter pilots' version of a draw play.

The attacking Israeli Skyhawks, Kfirs and F4 Phantoms pulled away as they had before but as the Syrian pilots watched them recede, Israeli F15s that had been lurking high overhead opened up with missiles and downed the outmatched and unsuspecting Syrian Mig21s.

Military analysts here say Assad's use of the aging Soviet-buit Mig21s reflects his desire to avoid broader hostilities - particulary anything that would mean attacks on Syria - as well as Israel's previous refusal to stick around and fight once challenged by the Syrians.

The Syrian air force boasts better planes than the Mig21s if Asad wanted to use them - 50 Mig23s and a dozen Mig27s that would be more of a match for Israel's ultrasophisticated U.S. made F15.

But there is no indication these aircraft are bing used in the continuing patrols over Lebanon, Diplomats say. Nor, they add, is there any sign that Syrian forces are deploying SAM6 missile batteries in Lebanon, regarded as a key indicator of any Syrian willingness to engage the Israelis in more intense fighting.

In the estimation of observers here, Assad is walking a thin line between showing Israel that Syria refuses to be trampled on in Lebanon, and making it clear he is not interested in war.

At the same time, the use of Syrian Fighters over Lebanon, despite the risks, signals to the United States that Israel's raids in Lebanon must be controlled because they could lead to a disaster capable of throwing the Camp David peace process off the track.track.

Finally, the diplomatic observers say, the dogfight and the danger of more clashes serve as a red flag to wave at the Soviet Union, from which Assad has been trying to secure more and better arms to compensate for Egypt's exit form the Arab military alliance against Israel.

The test of Syria's policy will come if and when Israel launches another major attack on Palestinian targets in lebanon. There has been none since last week's dogfight, although Israeli and Syrian air patrols continue. CAPTION: Picture, HAFEZ ASSAD. . . a shift in policy