In an expression of concern over the effectiveness of aging committee chairmen, restive House Democrats yesterday ousted two senior panel heads, the first deposed in nearly six years.

Democrats voted 152 to 100 in secret balloting to remove Rep. Glenn M. Anderson (Calif.) as chairman of the Public Works and Transportation Committee, a panel that oversees billions of dollars worth of federal highway, water and airport projects.

On a 127 to 125 vote, House Democrats also turned out Administration Committee Chairman Frank Annunzio (Ill.). After the balloting, Annunzio said he would fight to retain his post, but later withdrew when it was apparent he would not prevail.

Last night, lawmakers picked Rep. Robert A. Roe (N.J.), next in seniority on the committee behind Anderson, to head the Public Works panel. Roe, a hard-working legislator known for his attention to detail, defeated Rep. Norman Y. Mineta (Calif.), the panel's No. 3 Democrat who had initially launched the challenge to Anderson, on a 121 to 107 vote.

The lawmakers reached to the House Administration Committee's No. 3 Democrat, Rep. Charles Rose (N.C), to replace Annunzio, soundly rebuffing Rep. Joseph M. Gaydos (Pa.), the second in seniority, on a 158 to 64 tally. Rose, who had openly challenged Annunzio, was instrumental in modernizing the House computer system and televising floor proceedings.

While individual factors weighed heavily in the balloting against Anderson, 77, and Annunzio, 76, the House Democrats' votes signaled a restlessness with senior committee chairmen.

Rep. Henry B. Gonzalez (Tex.), 74, chairman of the Banking Committee, survived a last-minute challenge on a 163 to 89 vote. And significant numbers of negative votes were cast against two other senior committee chairmen who have been in failing health -- Interior Committee Chairman Morris K. Udall (Ariz.), 68, and Merchant Marine and Fisheries Committee Chairman Walter B. Jones (N.C.), 77 -- even though they faced no organized opposition.

"It's a reflection of the members' desire to see change," said Rep. Dave McCurdy (Okla.).

"The message is: If you're a committee chairman, don't limp," said Rep. Pat Williams (Mont.).

House Democrats last deposed a sitting committee chairman in 1985, when they ousted the 80-year-old and frail Rep. Mel Price (Ill.) as head of the Armed Services Committee and replaced him with Rep. Les Aspin (Wis.). In 1987, House Democrats voted against reconfirming Aspin, only to restore him three weeks later.

The other incumbent chairmen were reelected yesterday with few dissenting votes. In addition, Rep. William D. Ford (Mich.) was elected chairman of the Education and Labor Committee, succeeding Rep. Augustus F. Hawkins (Calif.), who retired. Rep. William "Bill" Clay (Mo.) was chosen to replace Ford as chairman of the Post Office and Civil Service Committee.

Last night, House Democrats also elected Rep. George E. Brown Jr. (Calif.), a strong opponent of the military use of outer space, to replace Roe as chairman of the Science, Space and Technology Committee.

Anderson and Annunzio fell victim to concerns about their performances inside and outside Congress. "These are important committees," McCurdy said. "Members are quite concerned that there be effective leadership."

"People were talking about effectiveness," said Rep. Gary L. Ackerman (N.Y.).

Anderson assumed the chairmanship of the Public Works and Transportation Committee in 1988 following the death of Rep. James J. Howard (N.J.). Anderson, often described as withdrawn and quiet, has been criticized as being more concerned about his Long Beach, Calif., district than about national issues and too reliant on his staff. Many lawmakers worried that he was not up to guiding the committee in the next Congress, when lawmakers are to draft major highway and mass transit legislation.

Complaints about constituent service contributed to the downfall of Annunzio, a rough-hewn machine pol from Chicago. The Administration Committee serves lawmakers and their staffs, overseeing House offices, facilities and expenses. Lawmakers complained that the panel's staff was difficult to deal with and often unresponsive to their requests and complaints.

Annunzio is also chairman of the House Banking subcommittee on financial institutions supervision, regulation and insurance. Whether he retains that post will be determined next month when the panel meets to organize.

Gonzalez will remain chairman of Banking, having overcome a last-minute challenge from Rep. Bruce F. Vento (Minn.), the seventh-ranking Democrat on the panel who announced his intentions only yesterday.

Some Democrats have complained that Gonzalez has not been partisan enough in his pursuit of the savings and loan issue. That charge was heightened when he defended Rep. Chalmers P. Wylie (Ohio), the Banking Committee's ranking Republican, during a campaign earlier this year.

The fact that 89 Democratic colleagues would vote against Gonzalez sends the Texan a strong message. "Any chairman who receives a large number of negative votes should spend some time next year talking with the caucus and individual members if he wants to remain chairman," McCurdy said.

Forty-seven Democrats voted against Udall, a venerated lawmaker who has been slowed physically by the effects of Parkinson's disease. There was little appetite to formally challenge Udall, though, because of his popularity and his plans to retire at the end of 1992.

Jones, who makes his way through the the Capitol in a wheelchair, received 46 negative votes.

Also, Rep. Edolphus Towns (D-N.Y.) was elected chairman of the Congressional Black Caucus, succeeding Rep. Ronald V. Dellums (D-Calif.).

Staff writer Susan Schmidt contributed to this report.