Saudi Arabia's Islamic religious leadership urged severe punishment today for the band of Moslem fanatics who occupied Mecca's Great Mosque for five days before being overcome late yesterday by Saudi security forces.

"The prophet, peace be on him, said that he who comes to you to disunite you and to sow dissension among you, you must behead him," said a decree by the kingdom's ulema, or Islamic wise men.

Their call, issued last night and published today, seemed to foreshadow public executions for those of the gunmen captured by security forces. Past Saudi executions, including that of the assassin of King Faisal in 1975, have been by beheading with the stroke of a heavy sword.

The ulema issued their decree to back King Khalid's order yesterday to attack the remaining gunmen holding out in the Great Mosque. The king's order marked a reversal of earlier Saudi tactics, which had been to avoid fighting in the holy shrine to prevent loss of life and damage to the building. f

"The ulema unanimously agree that fighting inside the mosque has become permissibile," the decree said. "If they fight you, then you must kill them because this is the punishment of the nonbelievers."

So authorized Saudi National Guardsmen opened five in a largescale attack late yesterday and routed the gunmen reportedly with considerable casualities on both sides and some damage to the building, diplomatic sources said.

Witnesses reported seeing flames and smoke rising from the two-story mosque, they said. Saudi guardsmen were seen today walking casually around the roof, indicating that they no longer feared the sniper fire that had plagued them since gunmen occupied the mosque during Tuesday's dawn prayers.

But Sheik Ahmed Zaki Yamani, the Saudi petroleum and natural resources minister, told foreign reporters that authorities feared some of the gunmen were still hiding in underground passageways. Other unconfirmed reports said some of the gang had escaped into the city.

These comments suggested that Saudi security forces had not captured all the surviving gunmen. There was no word on the fate of their leader, said to be a 27-year old dropout Islamic law student previously agitator.

The Saudi government, as it has from the beginning, would not say how many casualties were suffered in the attack or during the five day siege. Officials also maintained their refusal to disclose whether any of the thousands of Moslems present at the mosque when it was occupied were killed or wounded.