In a strong challenge to Mayor Marion Barry, the union that represents half the District government's employes said yesterday that the workers "will not under any circumstances accept anything less" than the 9.1 percent pay increase that federal white-collar workers are to receive Oct. 1.

The statement setsup a confrontation between the mayor, who says the deficit-ridden city government has little money for pay increases this year and none next year, and the employes, squeezed by double-digit inflation, who have come to expect salary parity with their federal counterparts.

Barry notified city workers last night that they would receive cost-of-living salarey increases of 5 percent as of Oct. 1. That is the first time he has committed himself to any increase this year, but it represents an increase far less than the rate of inflation and less than the unions are demanding.

Barry urged the workers to settle for a "bird in the hand," but the union statement, read at a press conference by Geraldine Boykin, executive director of Council 20, American Federation of State, County and Municipal Employees, said that the 16,000 city workers represented by the union would "no long tolerate being the scapegoats of a severely mismanaged city government" and would "have to take more direct action" if their pay demands were not satisfied.

City Council Chairman Arrington Dixon, in an apparent attempt to put distance between himself and the mayor on the politically sensitive salary issue, last night introduced emergency legislation to give city workers a 9.1 percent pay hike if Congress made the funds available. But the bill was soundly defeated on an 11 to 2 vote after two hours of heated and often emotional debate. Only Dixon and council member Wilhelmina J. Rolark (D-Ward 8) voted in favor of the bill.

Dixon, sometimes shouting in defense of his proposal, declared before a council chamber audience including about 400 city workers that "We don't have the money. But let's a least stand behind our employes . . ." The rest of his words were drowned out by the applause and cheers.

Other council members who objected to Dixon's bill said they did so because Congress would be unlikely to come up with the $28 million needed to fund the pay increase and the city alone cannot afford it without more layoffs.

Council member John Wilson (D-Ward 2) told the city workers at the meeting "Look around you. Every third person (here) will be looking for a job," if the council approved the 9.1 percent pay raise.

Boykin said at the union press conference that no strike was contemplated because a strike would be illegal,and that the union rejected such tactics as work slowdowns or sick-ins as "the old way of doing things." Pressed to explain what the union might do to enforce its demands, she said only that "the employes have very creative mind and they have an impact on every living thing in the city."

The union represents trash collectors, water supply technicians, cafeteria workers and clerks who, as she said, "affect the quality of life in our city." Last week another union, the American Federation of Government Employees, which represents 6,000 workers in the corrections, housing, transportation and recreation departments, also rejected the mayor's salary proposals as "unacceptable."

The mayor's proposed budget for the 1981 fiscal year beginning Oct. 1, which is still before Congress, includes $35 million to give city workers and school teachers the cost-of-living salary increase of 5 percent. For 1982, Barry has said his budget will include no funds at all for salary increases.

The unions, however, have now made clear that they will challenge the mayor on several fronts, including the courts if necessary. They argue that the city's home rule charter and its Merit Personnel Act of 1978 actually guarantee city workers the same compensation as workers in the federal civil service system of which they were formerly a part.

The vote defeating Dixon's bill came just hours after the mayor had sent to the council a package of amendments to the peronnel act that would, in effect, authorize the mayor to adjust city worker's pay based on economic conditions and budgetary constraints such as limited revenue or a lower than expected federal government payment to the city.

The amendments, introduced by Councilman William R. Spaulding (D-Ward 5), were passed on a voice vote.

The act requires, among other things, that they mayor engage in collective bargaining over pay with city employes, and Dixon had blamed Barry for delaying action on setting up procedural criteria for such negotiations. Dixon said that they mayor's delay had forced him to introduce his bill last night.