D.C. Delegate Walter E. Fauntroy said yesterday that the District of Columbia would suffer froma "tainted image" that would jeopardize approval of the District voting rights amendment if gambling were legalized in the city.

"I have been approached by numerous colleagues in the House who tell me that their constituents would be very concerned if it appeared that the nation's capital was becoming a gambling center," Fauntroy said. "I think that image would be used against us by those who do not wish for us to be represented in the House and Senate."

Fauntroy added that he would oppose a referendum on legalized gambling because "a lottery and legalized gambling impose a tax upon the poor and people who can least afford to lose money."

Fauntroy's comments came after he was informed that there appeared to be sufficient signatures on a citywide petition to place a gambling referendum on the November ballot.

Fauntroy, a Baptist minister, said he would "issue a call to other churchmen who agree with me" in his efforts to defeat the proposed referendum.

Proponents of legalized gambling in the city claim that it would generate nearly $40 million in additional revenue. Fauntroy maintains that the money would be subtracted from the federal payment to the city.

According to a Washington Post poll in June 1978, about 57 percent of the city's Democratic voters favored legalized gambling.