B. Sam Hart, controversial nominee to the U.S. Civil Rights Commission, owes $4,400 in back taxes in Pennsylvania, and did not register to vote there until after the Reagan administration decided to pick him for a federal job, according to local records.

WYIS, a religious radio station owned by Hart, owes $4,400 in back taxes on income, property and business, and rent payments on property his radio station rents from the borough of Phoenixville, Pa., borough council President Joseph Fabian said yesterday.

Fabian said the council probably will take legal action against Hart if the payments, some of which date to 1980, are not made by this weekend.

Voting records indicate that Hart, a radio evangelist, became a registered voter in Philadelphia last Nov. 4, two weeks after the White House notified Congress that he was being considered for the civil rights post. The records indicate that the last time he registered to vote was in 1966.

Hart could not be reached for comment. His son, Bradley Hart, business manager of WYIS, told United Press International that his father was not "completely aware" of the delinquent taxes.

Bradley Hart said many of the payments were not yet a year overdue. "It's not something that's been hanging over our heads a long time," he added.

Hart's nomination was severely criticized by women's groups, gay activists and Pennsylvania's two Republican senators, John Heinz and Arlen Specter, after Hart told a news conference that he opposes the Equal Rights Amendment, busing as a means of integrating schools and the concept of civil rights for homosexuals.

Fabian said Hart's radio station has been consistently delinquent in rent and tax payments to the borough since it rented municipal property for two radio towers three years ago.

"This is not a political thing with us. It doesn't have anything to do with Ronald Reagan," Fabian said in an interview. "We just want Mr. Hart to pay his taxes like everyone else."