With the highly publicized execution of 15 members of the extremist Moselem Brotherhood today, Syrian President Hafez Assad displayed an iron hand of state authority in the face of dissent at home and an embarrassing military setback in the skies over Lebanon.

The swift, brutal justice and its immediate announcement by the official Syrian radio seemed intended to demonstrate that despite his recent troubles Assad remains in firm control and determined to crush any challenge to his rule.

The executions followed a series of setbacks that, according to reports by journalists returning from Damascus, have clearly upset Syria's Arab Baath Socialist Party government but have not grown into a threat to Assad's nine-year hold on power.

Those executed were convicted of a variety of offenses ranging from the assassination of a senior Syrian security officer two years ago to membership in an illegal armed group. As far as was known, they had nothing to do with the killing June 16 of an estimated 50 cadets in an artillery academy at the northern Syrian city of Aleppo.

Syrian authorities, however, have blamed that attack on the Moslem Brotherhood, and today's executions dramatized Assad's resolve to stop any drive against him by the brotherhood before it has time to become organized.

Arab observers in Beirut pointed out that Assad had before him two recent examples of religious-related collapses of state authority: Iran and Lebanon. They emphasized, however, that Syria's strong state apparatus and apparently united military command make chances of long-term chaos there along Lebanese or Iranian lines remote. Nevertheless, diplomats in Damascus were quoted as saying that Assad does face a sectarian problem because of resentment at the dominance of his fellow members of the Alawite Moslem minority in the government and the Baath Party bureaucracy.

For that reason, there was some speculation here and in Israel that Assad may have ordered his Migs to seek out yesterday's dogfight over Lebanon to distract attention from his domestic problems.

Syria had done nothing, observers pointed out, during earlier Israeli raids on Palestinian bases and refugee camps in Lebanon - and even stood back when Israel invaded and occupied a swath of southern Lebanon with ground forces in March 1978.

But U.S. diplomats for some time have singled out Lebanon as a potential trouble spot because of the volatility of the armed factions here, and the close contract between Israel just to the south and Palestinian guerrillas alongside more than 20,000 Syrian peacekeeping troops in Lebanon.

Based on their past conduct and recent statements, however, Syrian authorities seemed unlikely to do anything that would risk broader hostilities. A Syrian official warned in a recent interview in the An Nahar Arab International (ANAI) that Israel would try to provoke Syria into a "trap," but that Assad would resist it.

Whatever Syria's future actions, the dogfight seemed to be a clear setback. Damascus acknowledged losing four Mig-21s; Israel claimed its pilots shot down five. Although both Syria and the Palestinian guerrilla movement reported shooting down two Israeli jets near the southern Lebanese town of Jezzine, no Israeli wreckage was found.

The setback coincides with a slow down in the pace of Syria's much-heralded unifying efforts with Iraq. A summit conference between Assad and Iraq's President Ahmed Hassan Bakr last week - billed beforehand as the occasion of a unity announcement - ended with little more than a mutual pledge to continue work on the project.

Taken together, Assad's troubles could have the effect of restricting his role of the Arab campaign to isolate President Anwar Sadat of Egypt. For example, the Syrian leader was forced to postpone a trip to the Soviet Union this week that had been seen in the context of the anti-Sadat drive.