Mayor Marion Barry yesterday refused to see about 30 welfare recipients and community service activists who had gathered outside his fifth-floor office at the District Building to protest his proposed cutbacks in emergency assistance and medical care for thousands of the city's poor.

Instead, the mayor dispatched two top aides, Gladys W. Mack and Audrey Rowe, to meet with the protesters in a first-floor conference room and listen to their grievances.

"They can't just barge in here," Annette Samuels, the mayor's press secretary, said of the protesters. "We understand that they need information, and we're going to try to provide it."

But the protesters, many of whom are members of an umbrella group known as the Coalition on Financial Accountability, saw it differently. "We don't need to have an appointment--we voted for him," said Flora Holt, a coalition member and representative of a senior citizens' group.

The demonstration in front of the mayor's office helped throw the normally quiet District Building into commotion, as the welfare-rights activists--many of them elderly and disabled persons--mingled with hundreds of educators and parents who had turned out for a hearing in the City Council chambers on the mayor's equally controversial 1984 spending proposal for the public schools.

Barry has proposed eliminating the General Public Assistance program, which costs $14 million a year and provides an average of $189 a month in aid to abandoned children and to persons out of work because of disabilities. More than half the 5,000 recipients would receive reduced benefits or no benefits if Barry's plan is adopted, and coalition members said they fear this would cause widespread suffering.

Coalition members, representing more than 30 social service agencies, also fear that Barry's proposal will result in the closing of one of the city's four neighborhood health clinics (although officials say that is far from a certainty) and elimination of free medical, eye and podiatric care for the poor.

"Many people here today have been cut back already by the federal government," said coalition spokeswoman Mary Scott. "Their food stamps have been cut and their rents have gone up. Poor people now need more help from the city, and they are being given less."

Betsy Finley, a former program director for the now-defunct Washington Free Clinic, said that poor people who would be required for the first time to pay fees for some medical services under Barry's plan will delay seeking treatment until their problems become more acute and more expensive to treat. The city's policies "are not cost-effective," she said.

Mack, the mayor's director of policy and program evaluation, and Rowe, the commissioner of social services, said they tried to clear up some of the protesters' misunderstandings about Barry's proposal during the meeting yesterday.

"I don't think they had a clear picture of what the social service cuts will mean," Rowe said. "This group had a lot of misinformation."