Black conservatives Lige Richardson and Pete Gresham felt they "took it on the chin" last year when they campaigned for a white conservative, former John Birch Society member in the heavily black 6th Congressional District of Birmingham, Ala.

But the abuse they took from some leaders of the local black establishment seemed worthwhile last November when their man, Albert Lee Smith, won and put them on his staff. Then, before the congressman and his maverick supporters could savor the fruits of their campaign labors, the situation went sour.

The congressman, saying the two aides often ignored the chain of command and went off on "tangents" of their own, fired the duo. And Richardson faces criminal charges for punching Smith in the mouth in a federal court building in Birmingham.

"We really look like fools," Richardson said yesterday in an interview here. "We feel a little like fools, too."

To hear Richardson and Gresham tell it, they were duped by a conservative white politician who took advantage of their "desire to do something for black folks, especially in the area of jobs, without going the old welfare, social services route."

"That's why we campaigned for him . . . We liked his self-help philosophy. The people who called us 'Uncle Toms' for backing Smith didn't bother us," said Richardson, a former banking executive.

The two blacks said yesterday that they got into trouble with Smith's staff by pushing for a job program called Positive Action for Youth (PAY), which they said would have been aimed at ending high unemployment among young blacks in Birmingham.

"Smith knew all along that our game plan was jobs," said Gresham, a 31-year-old Birmingham businessman and "a Methodist who believes that the Moral Majority could be used to end racism in America, because racism is a moral problem, not a civil problem."

"Smith was all for us during the campaign. But the day after the campaign, that all changed," Gresham said.

The two were fired Jan. 31 in a Birmingham meeting with Smith and his staff. Richardson admits to decking the congressman after that stormy session.

Smith said yesterday that he remembers getting hit: "He blindsided me."

But the Republican congressman disagrees with the pair's portrayal of the events that led to the punch-out.

"When we hired them on the staff, we told them that we didn't want people going off on tangents," Smith said. But Gresham and Richardson often ignored the chain of command, the congressman said.

One of the most embarrassing moments came shortly after the Nov. 4 election, when Richardson and Gresham went on a Birmingham radio talk show and castigated local black leaders who stuck with the Democrats, Smith said.

"They didn't clear that one with me. But people understandably were upset with me because of it," Smith said. The congressman said he warned the aides on three occasions to check with his office before action.

"But they didn't do it. At the fourth meeting we had with them, we told them we would have to let them go because they weren't team players," Smith said.

The U.S. attorney's office in Birmingham has filed charges against Richardson for striking a member of Congress and committing an assault on federal property. If convicted on the first count, the ex-aide will face up to a year in jail and a $5,000 fine.

Smith expressed regrets yesterday that things have turned out this way. But, of the U.S. attorney's action, he said: "I'm certainly not going to stop him from doing his job."