They were a somber lot, the visiting mayors, dressed in charcoal gray, dark blues and blacks. Their moods resembled their clothes.

"I don't want to make an Orphan Annie speech," San Francisco Mayor Dianne Feinstein, clothed all in black, said. Then she did, eloquently.

Chicago Mayor Harold Washington said the best words were "devastating" and "chaotic." He talked about "the guts of local government" being "torn apart" and "the future of our cities" being at stake.

Boston Mayor Raymond Flynn said it amounted to nothing less than "urban terrorism."

The nation's big-city mayors were in Washington, and the subject was federal budget cuts and the impact of the Gramm-Rudman-Hollings budget-balancing act.

The mayors were taking part in what has become one of the great American political rituals, the winter pilgrimage to Washington. Other pilgrims, teachers and bankers, governors and state legislators, big shots and little shots, make similar visits en masse each year, to shout at government and seek its favors.

The U.S. Conference of Mayors ushered in the new season yesterday, just as it does every year. And as in other years, the mayors probably set the tone for those who will follow.

Few groups get a better hearing. Mayors, after all, are barons in their own cities. They are far closer to everyday problems than are representatives, senators or governors. Other people can pass laws, but only mayors can make sure garbage gets picked up and potholes filled. And in Washington, they are political powers to be dealt with.

The tone the mayors set yesterday was angry.

The first stop was Room 210 of the Cannon House Office Building, the home of the House Budget Committee. It is a sterile room with gray walls and not a single picture -- fittingly, the kind of place an accountant might work.

Each mayor brought figures. Mayor Washington said his city would lose as much as $155 million in federal funds in 1986, about 8 percent of its budget, as a result of Gramm-Rudman-Hollings and other budget cuts.

This, he said, "would be devastating, particularly when combined with cutbacks we have undergone in the past several years." He said the first things to go would be "the creative, life-giving, forward-looking" parts of city government, like an anti-gang program in Chicago that helped reduce gang related murders by 20 percent last year.

Kansas City Mayor Richard Berkley, a Republican, said that even if all programs that the mayors are concerned about were eliminated, it "would hardly make a dent in the federal deficit but. . . would wreak havoc at the local level."

"City programs have been cut dramatically over the last few years, while federal deficits have continued to climb," he said. "Clearly, other villains have fueled the escalating federal deficits. Cities have already contributed more than their fair share."

Cleveland Mayor George Voinovich, a Republican, said Congress and the Reagan administration are threatening to "end the historical relationship that American has had with its cities and set millions of Americans adrift."

"We've already cut all the fat. We're down to the muscle and bone," said Mayor Tery McKane of Lansing, Mich.

The rhetoric remained equally harsh in conference meetings at a Capitol Hill hotel. Boston Mayor Flynn said "we've heard a great deal of talk from Congress about fighting international terrorism," and suggested that Congress should "focus some attention on the urban terrorism that Gramm-Rudman will be inflicting on the cities of our country by the elimination of vital lifeline programs."

The only lighthearted notes of the day came over the Super Bowl matchup Sunday between the Chicago Bears and the New England Patriots, with Mayors Washington and Flynn kidding each other about their teams.