The annual security nightmare has begun in earnest after the warning last night by the powerful minister of the interior, Prince Nayef, against political activity by Moslem pilgrims, especially those from Iraq and Iran.

In a special televised seminar with top security officials, Prince Nayef warned of firm action, noting that one group has already been found holding meetings and carrying posters and pamphlets of a "purely political nature."

Over half a million foreign pilgrims have already flooded through Jeddah on the pilgrimage to Mecca. Last year, poster-carrying Iranian pilgrims demonstrated and distributed leaflets praising Ayatollah Ruhollah Khomeini, but their action was barely noticed amid the 2 million pilgrims. They then were completely overshadowed by the bizarre two-week armed siege of Mecca's Great Mosque by several hundred revolutionary zealots, mostly Saudis.

Sixty-three of the Sunni fundamentalists finally overcome inside the mosque were later beheaded. Since Nayef called on "brother" pilgrims from Iraq and Iran to leave the three-week-old fighting at home -- and despite the closure of Tehran and Baghdad city airports -- tens of thousands of Iraqis and Iranians have arrived or plan to come for the annual pilgrimage. Islam enjoins all Moslems to attempt at least one such pilgrimage in his lifetime.

Diplomats said up to 100,000 Iraqis are expected, many trucking down through Jordan. Both countries are trying to treat the pilgrimage as normal. tNayef no doubt is relieved that Khomeini apparently has decided not to make the pilgrimage. A diplomat said houses lining both sides of a street had been reserved in Mecca for the ayatollah and entourage of several hundred. hBut the Saudi landlords now expect their houses to go unused, he said.

The revolutionary Shiite ayatollah could clearly stir not just Iraqi and Iranian pilgrims but all others especially the Shiites, as compared with the other main Moslem sect, the Sunnis.

Many among Saudi Arabia's traditionally suppressed 250,000 Shiite minority erupted last year in two days of rioting at their Persian Gulf coast homes at Qatif and Saihat. Authorities had tried to prevent traditional Shiite self-flagellation to mourn the death during a seventh century battle in what is now Iraq of the prophet Mohammed's son-in-law and Islam's third caliph, Ali.

The Islamic mourning date falls on Nov. 18 this year, when Saudi authorities, particularly the Bedouin national guard, are expected to exercise restraint. After another day's rioting by youths in Qatif earlier this year, the Saudi development machine has loudly launched a $120 million crash program there for the next five years.

Despite continuing revolutionary calls from Tehran radio, Saudi Shiites have since remained peaceful and have yet to react to the gulf war. Many apparently were interned during the earlier turmoil.

The Saudi kingdom, unusually peaceful by regional standards, is extraordinarily security conscious, with a battery of security forces and intense scrutiny of imported labor and goods. all incoming ship containers, for example, are virtually emptied during port customs checks for alcohol, pornography and arms.

But the human problem is greatest. The annual pilgrimage is traditonally the time for thousands of poorer Moslems to enter -- as is their religious right -- and then remain illegally in search of relatively lucrative work in the development boom. Only North Yemenis are allowed here without Saudi sponsors, who have to extract residence permits from the Ministry of the Interior for imported workers.

For two years, such permits have been increasingly controlled, and in recent weeks police have made successful sweeps for illegal immigrants, according to official announcements. Legal residents have been warned not to harbor illegal pilgrims without registering them officially.

Under the $2.4 billion five-year plan launched this May, the kingdom proposes to stem the inflow of foreigners and encourage the estimated 6 million indigenous Saudis to work. New regulations this summer will, if applied, cut down on families accompanying an estimated 1.5 million foreign workers.

There is a dilemma between the fears of foreign influence and the desire for rapid development -- fueled by ever-increasing oil income that must be seen to go for local benefit.

The kingdom could double its labor force by encouraging women to work outside medicine, social work and education. The government has edged this way but faces opposition from the powerful religious establishment.