It took a fire to get the House District Committee to call its first meeting of the year.

When its metropolitan affairs subcommittee met two weeks ago to investigate the causes of a fire in a Metro tunnel April 16, it was the first public action in 1980 by the congressional committee charged with oversight of affairs of the District government.

Since then, another subcommittee held two days of hearings on the District's infant mortality problem. A committee aide said yesterday that no other hearings are planned "in the immediate future." The full committee, which under House rules normally would meet monthly, has not yet met this year.

Critics of the committee cited the belated start as a sign that its 40 or so full-time employes, who have an annual budget of nearly $1 million, have little to do, or aren't doing what they should.

Defenders, including Chairman Ronald V. Dellums (D-Calif.) say it just proves that the committee "isn't wasting time" by calling unnecessary meetings.

The House District Committee is the smallest of the 21 standing (permanent) committees of the House. But the next smallest committee, Veterans Affairs, already has met 22 times in the first four months of 1980. Two medium-sized committees, Small Business, and Science and Technology, held 40 and 120 public hearings in the same period since the start of the second session of the 96th Congress in mid-January. The larger committees meet almost daily.

One employee of the District Committee said many staffers are "just sitting around doing nothing. And a lot of them are good, industrious people, who are frustrated for something to do."

Edward C. Sylvester, Jr., the committee's staff director, said the work load "certainly goes up and down," and added that "there are a lot of frustrated people at Chrysler, too."

The volatile Dellums responded angrily to questions about the committee's activity, saying it was "absurd" to judge its effectiveness by the number of meetings held.

Rep. Michael D. Barnes (D-Md.). a member of the committee, said he suspects the lack of activity "is an indication that we don't need a full committee on D.C. I've felt for a long time the federal role could be accomplished by a subcommittee, perhaps of the Government Operations Committee."

Dellums said the committee faces "serious issues, not the least of which is getting it out of existence." When Dellums took over in 1979, following the conviction of its former chairman, Rep. Charles C. Diggs Jr. (D-Mich.), he pledged as one of his major objectives phasing out the committee, which in pre-home-rule days often was accused of serving as a colonial overlord of District residents.

Rep. Millicent Fenwick (R-N.J.) complained that as a minority member of the committee she didn't find out about the funding request for the committee until after it had been made.

"I got a notice on March 10, dated March 6, that said the request would be made on March 7. I called Mr. Dellums and complained," said Fenwick.

Last year, Fenwick and Barnes questioned the awarding of a $25,000 contract by the committee to an Arlington consulting firm to study housing trends in the city. Dellums said he awarded the contract without a bid because he knew the firm was best suited for the study since its owner previously had worked with him.

"There was no suggestion of wrong-doing," Fenwick said, "but it's a funny way to do business."

A disgruntled committee employee said the work "should have been done by the city, and if not, by our staff."

"The less the committee does, the better off the city is," said a congressional aide who is sympathetic to the city. "There is a real fear that the budget fiasco at the District Building is endangering home rule, so we don't want to go to the floor [of the House] with anything that can be amended."

Bob Brauer, Dellums' chief aide, said the budget crisis in the administration of Mayor Marion Barry makes it "harder to move" any legislation through Congress. "We're moving a lot slower than we anticipated, because we don't want to blow it."

The major issues confronting the committee are establishing a formula for the annual federal payment to the city, and budget autonomy for the city.Once those questions are settled, Dellums plans to ask that the committee be eliminated in favor of a subcommittee.

The House Committee on Committees considered taking that action when it reviewed committee functions last year. But it decided, according to a staffer, "that it would be move politic" to postpone that action, even though the chairman of another full committee described the District Committee as "an obscenity."

"In retrospect," the staffer said, "if we had to do it over, we'd probably take them on."