The D.C. City Council, challenging both national trends and Mayor Marion Barry, refused yesterday to grant further exceptions to the residency requirements for newly hired city employes.

In the final meeting of its third two-year session since home rule, the council soundly rejected proposals by the mayor that would have exempted from the law data processing specialists and former employes who have been rehired.

Barry had also proposed that new employes, who are automatically allowed up to six months to become city residents, be allowed extensions of up to six more months. But the council voted yesterday to allow extensions of no more than 90 days.

The residency law, which became effective Jan. 1, has hindered the city government's recruiting of new employes, largely because Washington has the highest housing prices in the area. Of the 528 persons hired since Jan. 1, 100 doctors, nurses and other medical personnel have been exempted from the law and 10 persons have left their jobs because of the requirement that they be city residents within six months of being hired.

Many cities and city officials throughout the country have found residency laws more of a hindrance than a help. Still, the council decided yesterday to draw the line at the current exemptions for fear of making the residency law meaningless.

"We're gutting the bill, gradually, one step at a time," Councilmember Nadine P. Winter (D-Ward 6) told her colleagues. Councilmember Betty Ann Kance (D-At-Large), a staunch supporter of residency, said Barry's proposed exemptions "set a precedent for [exempting] other specialized areas [and] I think we have to be very careful about chipping away at the law."

The council's action prompted immediate criticism from Barry aides concerned about finding computer specialists for the city's automated but troubled financial management system, and from key union leaders.

William H. Simons, president of the Washington Teachers Union, said his organization would seek a court challenge to the application of residency requirements to laid-off school teachers who are being rehired.

"We do plan to pursue the matter through litigation," Simon said. "It's not like someone being hired for the first time. There is no way in the world you can get employes to come back and work for the pittance they're being paid and get them to find available, affordable housing."

Neither Simons nor city personnel officials could give precise figures on how many persons had been laid off and rehired and are now subject to the residency law. Both the city and the teachers union agreed that practically all of the employes in that situation would be school teachers.

Simons said that of the about 250 laid-off teachers who were rehired, so far about 10 have notified the union that they live outside the city.

Barry administration officials, meanwhile, said that without the exemption for computer specialists, the city will have a difficult time filling the positions needed for adequate operation of the financial management system. "This is going to exacerbate out situation," said Robert Story, staff director in the city personnel office."

"There's no question about it," said Alan F. Grip, the mayor's press secretary. "It's going to hurt our attempts to hire computer people. Having the residency requirement apply to them is going to make it twice as difficult."

Story said that several city departments are in desperate need of data processing specialists -- Human Services, Environmental Services, Transportaion, police and finance and revenue and financial management.

"All of these agencies are in dire need of data processing people and haven't been able to find them," Story said.

But the council wouldn't buy that argument. Councilmember Kane said, "I am not convinced it is the residency requirement that is causing difficulty in filling these jobs."

Councilman John Ray (D-At-Large) agreed. "I am not convinced that residency is the culprit in the system," he said.

Councilmember Willie J. Hardy (D-Ward 7), appearing at her last council meeting before voluntarily giving up her seat at the end of this term, said, "The poor people of this city can no longer afford to support suburbanites. Washington, D.C., is for Washingtonians, and if you don't live here you can't work here."

The council votes on residency came in a crammed session in which the members rammed through emergency bill after emergency bill to keep the government operating until the next council convenes Jan. 2, 1981. Yesterday, the council:

Gave final apporval to a new rent control law that sets a 10 percent annual ceiling on rent increases for most Washington apartments, allowing a second 10 percent increase on apartments that are vacated and then rerented.

Approved a 5 percent pay increase for District of Columbia teachers, retroactive to Oct. 1 of this year.

Created a new Office of Energy to consolidate all of the city's various agencies and programs concerned wtih energy and energy conservation.

Repealed the District's out-dated death penalty law, which was effectively invalidated by a 1972 Supreme Court decision.

Confirmed the appointment of Cora Masters Wilds, a sociolology professor at the University of the District of Columbia, to serve on the city's three-member Boxing and Wrestling Commission. Wilds is the first woman to serve on the panel.