Abandoning much of the rhetoric and confrontation politics of the activist movements that helped to get them elected the nation's black mayors have chosen a quieter course of pragmatic politics to obtain the millions of dollars in federal funds they believe their cities cannot do without.

There is no longer talk of cutting the massive Defense Department budget to get more money for social programs. Some black mayors now say that whatever the Pentagon budget is, they just want their communities to get a fair share of the contracts.

When more than 100 black mayors gathered here this week for the three-day meeting of the National Conference of Black Mayors, there was no march to the Capitol, not even a bus trip. Instead, the mayors talked with federal bureaucrats who administer various urban aid programs, learned the how-to's of writing proposals for federal grants, and then made plans to open a Washington office to ride herd on their assistance requests.

"Ten years ago, when I was marching in the civil rights movement, we were fighting these agencies," said 35-year-old Johnny Ford, who in 1972 became the first black mayor of Tuskegee, Ala. "Now, we're compromising, we're negotiating, we're sitting down at the table and bargaining. This whole business of government is political. Discretionary funds come to the cities who work for them."

The conference, which it represents all 161 black mayors in the country, began in Tuskegee in 1973 with only eight, who called themselves the Southern Conference of Black Mayors.

Since then, their ranks have been swelled by elections throughout the country as a result of stronger federal voting rights legislation, increased black political awareness and shifts in population. Six months ago, the Southern black mayors were joined by some 60 others from non-Southern areas. This week's convention, held at the Sheraton Park Hotel, was the first meeting of the nationwide group.

The better-known black mayors - Thomas Bradley of Los Angeles, Coleman Young of Detroit, Kenneth Gibson of Newark and Richard G. Hatcher of Gary, Ind. - were either late arrivals or represented by standins.

Most visible were mayors like A.J. Cooper of Prichard, Ala., the first president of this organization; Maynard Jackson of Atlanta, also highly regarded by many of the group's members; and Walter E. Washington of the District, who spent considerable time and money as the conference's host and wound up being elected to its executive board.

The absence of the big-name mayors emphasized the fact that most of the black mayors are from smaller communities, usually towns of 3,000 or less. Despite a meeting goal of a "rural and urban partnership," the group still appears to be Southern dominated and small-town oriented.

The mayors hope to minimize conflicts that could develop because of the differing needs of different sized cities.

"What we have to consider as a national organization is not what is good for my city or your city, but what is good for all cities," Cooper said. "If Prichard's giving up some money (through a change in federal allocation formulae) would assist some of the cities that are in bad shape, and some of the cities that are in good shape with budget surpluses would give up some money too. I would be willing to recommend that we go ahead and give up the money."

In some respects, the emergence of the national black mayors group is evidence of the absence of a black presidence in Washington that might, to some, be represented in the Congressional Black Caucus.

Neither the mayors nor the caucus chairman, Rep. Parren J. Mitchell (D-Md.), speak publicly of rivalry between the two groups. Instead, there is talk mainly of a difference in functions, and the observation that most of the 17 caucus members come from Northern, Midwestern and Western states and just about all of them represent urbanized constituencies.

"The caucus has a different agenda. The caucus has a full-time job helping to see that we get legislation through," said Ford. "We are dealing with administration agencies, and we have to be powerful in our own right."

The caucus has been sharply critical of the performances of the Carter administration, accusing it at times of not being sensitive enough to the needs of blacks. But the mayors' group, with its high dependence on administrative agencies, was noticeably more muffled when asked to assess how the White House is doing.

"Carter is going to be a great President and will do a lot for the black community," Cooper said. "We have to insist that he make the urban and domestic problems the No. 1 priority on his agenda."

Housing and Urban Development Secretary Patricia R. Harris spoke to the mayors and more than 500 guess at a luncheon yesterday, but voiced little more than a possible timetable for release of a long-awaited administration urban policy plan.

She also said any weakening of the Carter administration's embattled energy plan would severely hurt poor people, like those many of the black mayors represent.