A week ago, Kim Sem welcomed visitors to the unofficial capital of Cambodia's Khmer Rouge guerrilla group a couple of miles south of here across the Thai-Cambodian border. Like other officials of the Khmer Rouge "foreign ministry," he wore a freshly laundered and pressed white shirt and dark pants for the occasion: a visit by Prince Norodom Sihanouk, four ambassadors, their retinues and two busloads of journalists.

Today Kim Sem walked around this dusty evacuation site for about 17,000 refugees in a straw hat, a worn and rumpled shirt, shorts and black sandals cut from old tires. Scarcely distinguishable from the thousands of Cambodian peasants gathered at the site, he was helping with the distribution of U.N. relief supplies and trying to obtain first-aid equipment for wounded guerrillas.

The senior Khmer Rouge leaders who welcomed Sihanouk, the head of a Cambodian resistance coalition government, to the model village of Phum Thmey in western Cambodia last Saturday were nowhere to be seen today. Nor, for that matter, were the armed Khmer Rouge men and women guerrillas who formed an honor guard then, or the prim schoolgirls in blue and white uniforms who turned out to meet the former Cambodian monarch in front of the village's ceremonial pavilions and guest bungalows.

Instead, the scene that greets the eye here is acres of low, makeshift shelters made of sticks, palm fronds and blue U.N.-supplied plastic sheeting. Under them are crammed thousands of families with whatever possessions, including chickens and pigs, that they could bring along, all seeking some respite from the merciless sun beating down on this open scrub land.

These are the latest refugees to flee the dry season offensive being waged by Vietnamese troops against Cambodian resistance settlements on the Thai-Cambodian border. But unlike other refugees who have had to flee Vietnamese onslaughts several times before, these Cambodians are not used to evacuations.

Until now, their strongholds in the hilly, jungle-covered Phnom Malai area of western Cambodia to the south of here have been practically impregnable. Yesterday, however, the Vietnamese succeeded in overrunning the base area that has been a symbol of the Khmer Rouge resistance for most of the six-year-old war in Cambodia.

Sporadic fighting continued today, and Khmer Rouge officials here said their guerrillas were still holding positions in the 150-square-mile Phnom Malai area. But it was clear that the Vietnamese had realized their apparent objective to drive out the civilian population and deny the Khmer Rouge title to what they called a liberated zone.

It also looked as if the civilian followers of the notorious Khmer Rouge, who ruled Cambodia brutally from 1975 until they were ousted by the Vietnamese invasion in 1979, might have to move even farther away from their once-formidable redoubts.

Vietnamese shells landed today about 500 yards from this evacuation site near the Thai village of Ban Nong Pru, sending some refugees scrambling for cover and forcing international relief workers to evacuate temporarily.

At another evacuation site to the south about three miles inside Thailand opposite the overrun Khmer Rouge settlement of Khao Din, about 25,000 Cambodians were threatened by artillery fire, Thai military officials said. One said the refugees were going to be moved to a safer area, but no other details were available.

Thai military sources and western diplomats confirmed today that the Vietnamese had captured Phum Thmey and controlled all of the Phnom Malai's strategic hills.

However, the Thai supreme commander, Gen. Arthit Kamlang-ek, told reporters in Bangkok that the Khmer Rouge were still active in the area and that losing territory did not mean defeat for them in guerrilla warfare.

Here, Kim Sem expressed similar thoughts as he seemed to shrug off the difficulties of the refugees' predicament.

"If we evacuate the population, that doesn't mean we lose the war," he said. "On the contrary, the Vietnamese will lose the war because the Khmer Cambodian population is not with them."

Although at first Kim Sem insisted that Khmer Rouge guerrillas were still holding Phnom Malai's settlements, he later indicated that they have in fact abandoned the villages and split up into small units.

"Our fighters are in the interior, behind the Vietnamese lines," he said. "Our forces are dispersed everywhere, and we are waging guerrilla war against the Vietnamese."

He added, emphatically, "I tell you, the Vietnamese cannot stay in Phnom Malai."

Another Khmer Rouge administrator, who gave his name as Chhe, said, "I think they cannot stay more than two months."