Most of the French Legionnaires involved in last week's dramatic rescue of whites trapped in Kolwezi, southern Zaire's big mining center, pulled out yesterday leaving behind only a small force to help the Zairian Army protect the town against possible future rebel attacks.

All but about 150 of the 700 French troops reportedly left overnight by road for the Shaba Province capital of Lubumbashi where they are expected to remain a few more days before being flown back to their base in Corsica.

Before leaving, the Legionnaires destroyed the vast quantities of rebel arms they captured or found in the city.

The departure of the Legionnaires has been partly compensated for by the arrival of more Zairian troops, but it is doubted here that many of the 2,500 Europeans who once lived in Kolwezi will return without some additional foreign troops being stationed there.

Zaire President Mobutu Sese Seko, who is now in Paris, is scheduled to visit Morocco today for talks with King Hassan on the possibility of his sending troops to provide the core of a pan-African force stationed in Shaba Province.

Morocco has sent a military team of about 30 non-combat officers.

The Zairian News Agency reported yesterday that so far Senegal, Mali, Gabon and Morocco have responded favorably to Zaire's call for military assistance. None, however, has sent combat troops yet.

Belguim has kept a battalion of paratroopers in Kamina, 140 miles north of Kolwezi, and some of them have been dispersed to Lubumbashi and several other Shaba Province towns where there are Europeans. It was the Belgains who carried out the evacustion of most of the Europeans from Kolwezi and who have since been criticized for turning the mining center into a ghost town.

The entire white population has in fact, abandoned the mining center, the world's biggest. There is still no electric power, water, fuel or food available there following the occupation of the town by Angolan-backed rebels and the death of about 100 Europeans and several hundred Zairians.

The International Committee of the Red Cross has sent four train cars loaded with food for the 35,000 to 50,000 Zairians still in Kolwezi but a Zairian Red Cross official said food remains the town's major problem now that the area has been cleared of rebels by the French Legionnaires.

Distribution of Red Cross supplies is to begin today but a crowd of townspeople were reported Saturday to have attempted to break into one of the cars in the Kolwezi trainyard.

Meanwhile, the security of Zaire's mineral-rich Shaba Province remains bands of rebels roving the countyside around Kolwezi and the small rail-road town of Mutshatsha 60 miles to the west apparently still in their hands.

Zairian troops reportedly still hold the other towns along the railroad between Mutshatsha and Dilolo on the Angolan border. The rebel strategy at this point remains obscure but some reports reaching here suggest they may now try to attack some of these other towns along the railroad west of Mutshatsha.

Nevertheless, at least 1,000 rebels from the original force of several thousand involved in the assault on Kolwezi have been spotted traveling through Zambian territory apparently on their way back to camps inside Angola.

The exact location of all rebel bases remains something of a mystery. One of six badly wounded rebels left behind in the hospital in Kolwezi told Western correspondents who visited them Thursday that he had been trained a site located in the countryside "near the borders of Angola and Zaire" but not in either of them.

This left reporters wondering whether one or more of the rebel camps may have been located in the largely unpoliced strip of land in the extreme northwest corner of Zambia jutting into Zaire. The wounded rebel was interviewed without any Zairian official present and did not appear to be trying to hide the truth.

One of the six rebels spoke English and no French and said he came from Zambia. The Lunda people, who are known to provide many recruits for the rebel army under the Congo National Liberation Front, stretch across the tri-border area.

A French-speaking rebel interviewed intensively by journalists said he had left the Kolwezi area after the 80-day Shaba war of last year, had been impressed into the Front's army and trained solely by what he called "Katangans."

The Katangan gendarmes were exiles from Shaba Province, formerly known as Katanga, who fled after their abortive secessionist movement there in the early 1960's. They have been seeking revenge ever since.

The French-speaking rebel also said he had never seen a Cuban all the time he was in training. "Only Katangans," he kept saying.