Black leaders, sensing that Jimmy Carter was a Southerner who cared about their problems, rallied behind him in 1976. The got out the black vote, which was crucial to his White House victory.

But these days, many black leaders feel that Carter is paying off his political debt to them in Confederate money. Their public criticism is still restrained, but their private comments are scathing. The believe he has put higher defense outlays and business-tax cuts ahead of programs to help the poor and the minorities.

Benjamin Hooks, for example, is executive director of the moderate National Association for the Advancement of Colored People. But his private references to Carter are anything but moderate. "The president wants us to save, save, save - and that means starve, starve, starve," Hooks said. Speaking for blacks and other minorities, he declared simply: "We cannot afford it."

A black congressional source told us ruefully: "We're alwasy a bridesmaid, never a bride." Among the black leaders we interviewed, the frustrated consensus was that Jimmy Carter has been little improvement over Gerald Ford. Carter's efforts in behalf of the blacks has been as half-hearted as that of their Republican predecessors, the leaders complained.

Carter's economists are quick to boast that the national unemployment rate is dropping. But a black congressman noted: "Carter said unemployment is down, but it has only been reduced in white America. We can't let him get away with that." In dejected tones, the legislator told our associate Jack Mitchell: "Carter is setting up an explosive situation for the warm weather. He's certainly not fitting into the Hubert Humphrey mold."

Even Carter's fellow Georgian, State Senator Julian Bond, a charismatic black leader, is outspoken in his disappointment with Carter's budget-balancing neglect of such promised priorities as housing and jobs for the poor. "Carter remembered the words to the hymns," Bond remarked, "but we didn't get a paycheck."

During Carter's first year in the White House, the black leadership was forbearing in criticizing him. They were willing to give him a chance to show that he meant what he said about improving the lot of the blacks. But their patience is waning.

"On my domestic report card, President Carter is flunking," grumped a prominent black congressman. "I don't know if he realizes the depth of the problems. The black community expected more."

Apparently, the president muffed a chance to restore black confidence in his administration during his State of the Union message. Members of the Black Caucus tended to agree with Sen. Barry Goldwater (R-Ariz.) Said one: "Goldwater said the president gave the same speech as the Republican presidential candidate [Goldwater] in 1964 and lost an election on it. That's disturbing to us."

If the State of the Union address broought dismay, Carter's budget and economic messages exacerbated black disappointment. "The budget is deficient," said one veteran black House member. "It's not a constructive budget. It emphasizes defense as opposed to human needs. Economically, Jimmy Carter is following in the path of Gerald Ford.

Dr. Andrew Brimmer, the first black ever to serve on the Federal Reserve Board, offered this studied dissent: "I have a low expectation of the Carter administration. The administration has contradictory economic goals. I think they'll fall far short on all of them."

Carter's tax-cut proposals, for example, may appeal to businessmen and middle-income Americans. But low-income blacks expect to lose more than they will gain. "I oppose the $25-billion tax cut," said Urban League Director Vernon Jordan. "We could better use the money for the urban poor."

Even the president's belated endorsement of the Humphrey-Hawkins ful-employment bill was too little and too late in the opinion of black leaders. It's true the legislation woul help jobless blacks, but one prominent black said the administration had watered it down. "We're only getting token support from Carter on this important legislation anyway," he groused.

At the grass-roots level, the same disenchantment prevails. "We've had no call-to-arms for the cities," a prominent black mayor said. "The nation's mayors were really let down by the State of the Union address." He faulted the president for failing to fight hard enough with Congress over urban programs. "We have to convince the president we're ready to go to the mat if necessary on these social programs."

Blacks in the Republican camp are even more hostile in their comments. One who had top-level status in the Ford administration declared: "I'd give Carter less than zero overall as far as what he has done for minorities." He cited a loss of federal judgeships held by blacks.

The president still has his black defenders among the leaders. Detroit Mayor Coleman Young credits Carter with doing a "remarkable job" in stabilizing conditions in the cities. "He has kept his word so far on what he has promised," Young said of Carter's moves to funnel federal funds to crime-infested areas. "Detroit led the nation in reducing the crime rate. I attribute this to Carter's urban recommendations."

Even his critics among the black leaders give the president high marks for accessibility. "The lines of communication are open to us," one said. All are generally hopeful that in his forthcoming statement on urban policy Carter will justify their political faith in him. That message, a big-city mayor said, "may be the whole ball game."