The Saudi government said its security forces regained complete control of the Great Mosque of Mecca today after four days of cautious fighting with a band of Moslem fanatics occupying the sacred shrine.

But it was clear from a government communique and reports from other sources that a small group of gunmen were still holding out with a few hostages on the second floor of the giant mosque.

The two-story building contains a maze of pillored galleries, tiny rooms and porticoes in which the occupiers apparently have found a small redoubt that security forces are unable to attack without endangering hostages' lives, the sources said.

The reluctance of Saudi officials to order an all-out assault on the holdouts reflected the royal family's effort to avoid spilling blood unnecessarily or damaging the mosque, considered by Moslems around the world as their faith's holiest site.

It also indicated Saudi determination to take the Islamic extremists alive, reportedly to conduct a thorough investigation. The results are to be presented to the Moslem world as proof that the attackers were, as repeatedly stated by the government, "acting outside Islamic faith."

The government has displayed considerable anguish that the band, occupying the mosque since Tuesday, mounted its unprecedented operation proclaiming Islamic principles. It was learned today that some senior government ministers had at first considered trying to keep the world from learning of the occupation, and that they relented only when younger officials convinced them that the news would get out anyway.

Communications were cut with the rest of the world for 15 hours Tuesday, and since then the government has released only sketchy reports on efforts to retake the mosque. Travel to Mecca is banned for non-Moslems even in ordinary times, and troops have been screening travelers on roads to the holy city since the attack.

There has been no authoritative word on the fate of the hostages, reported to number several hundred when the seige began during early morning prayers. Most were believed to be foreign pilgrims lingering after the annual Moslem hajj, or pilgrimage, which ended last week.

Reliable sources said only a handful of hostages remained in the hands of the gunmen and those liberated by Saudi security forces were believed to be on their way home. The Saudi government asked Moslem nations to speed the return of their pilgrims.

Security police at Jeddah airport, the main pilgrim transit point, were seen herding pilgrims inside the terminal and the pace of takeoffs and landings picked up noticeably.

Moslem shiekhs offering Friday sermons in several Saudi mosques condemned the takeover and called for "the most severe" punishment for the gunmen. The Saudi state radio broadcast protest messeges from Moslem nations underlining the government's eagerness to portray the attackers as renegades from the Moslem faith. A number of kings and presidents personally called King Khalid to express outrage, the royal palace announced.

The gunmen are believed to come from a Saudi group of Mahdists, reformist Moslems who believe in purifying the faith of modern influences and who await an "enlightened one" who will come as a Moslem savior before the end of the world.

The group has no known link to the Iranian Shiite revolution, although its doctrine resembles Shiite belief in the return of the missing leader, according to an Islamic scholar here. Several of its leaders have been under police supervision because of previous disturbances, reliable sources reported. p

Since the mosque takeover, security has tightened at government buildings around Riyadh, the capital, and diplomats predicted increased security measures for next year's pilgrims to Mecca as well.

The Saudi government recently installed new security equipment, including television monitors, and reduced the number of entrances into the mosque courtyard in an effort to exercise better control over the masses of Moslems who pour in every year. According to official estimates, more than two million foreigners attended this year's hajj.