B. Sam Hart, the controversial nominee to the U.S. Civil Rights Commission, has asked President Reagan to withdraw his nomination.

Reagan is expected to accept the request today, according to a high White House official who said Hart told the White House that "he wants to spare the president any further embarrassment."

The move came after it was learned that a broadcast company owned by Hart defaulted last fall on a $100,000 loan guaranteed by the Small Business Administration, and is $23,541 behind on repayments to a minority loan program in Pennsylvania.

Hart Broadcasting Inc. was 16 payments behind on the SBA-guaranteed loan with Continental Bank of Philadelphia when the White House first informed Congress that Hart, a black radio evangelist, would be appointed to the commission, sources said.

The firm also is a year behind in payments on $200,000 in loans from the Pennsylvania Minority Business Development Authority, according to sources in Harrisburg. The two loans were taken out in 1977 and 1978 to open radio station WYIS in Phoenixville, Pa.

The loans were the latest in a series of problems for the Hart nomination. Earlier this week it was learned that Hart owes $4,400 in back taxes in Pennsylvania and did not register to vote there for about 20 years.

The SBA took over the delinquent loan Nov. 7, two weeks after Hart had been picked for the civil rights job, sources said. At that time, the broadcasting company, which took out the loan May 22, 1978 for operating expenses, owed $97,067 on the principal and $12,278 in interest.

Hart was then put on a new schedule on which he was to pay $900 a month until this May, when payments are to increase to $1,700 a month, federal sources said.

The firm has made about $6,000 in interest payments during the last five years on its state loans, but is now a year behind in payments totaling $23,541, according to a source in the Pennsylvania Commerce Department.

The SBA confirmed that Hart Broadcasting had defaulted on the loan, but spokesman Joseph Zellner said the agency is prohibited by law from discussing some specifics.

"I'm not going to comment on any connection with any job or political involvement," Zellner said. "The agency looks at this as a business transaction. The loan was well collateralized, and there is no fear of losing any money on it. This is not an unusual arrangement. In a period of economic decline you run into a lot of delinquencies."

The SBA last year had to buy back $180 million in bank loans it had guaranteed because of defaults. Last April President Reagan ordered a crackdown on delinquent government loans and back taxes.

"We must make it clear that debts owed to the federal government must be repaid," the president said. "It is not right that responsible, honest citizens should suffer because of those who do not honor their obligations."

The White House, however, withstood pressure to withdraw the Hart nomination for days. Hart called Reagan Thursday night and asked that his name be withdrawn, according to a White House official.

The White House returned the call late yesterday to inform him the request would be accepted. A high official said the White House had not been aware of Hart's financial problems until this week.

The Hart nomination had become a symbolic contest between moderate and liberal forces and the religious and political right. He was picked for the job because of his close ties to conservative religious broadcasters, who have supported Reagan.

But Hart raised a chorus of criticism from women's rights, civil rights and homosexual rights groups as well as Pennsylvania's two Republican senators, Arlen Specter and John Heinz, when he told a news conference that he opposes the Equal Rights Amendment, busing to integrate schools and the concept of civil rights for homosexuals.

Later it was learned that Hart had been a non-voter since the early 1960s, registering as a Republican last Nov. 4, three days before the SBA-guaranteed loan was declared in default. Specter said Hart told him in a meeting Wednesday that he quit voting because "he was disillusioned with the political process."

WYIS, a religious radio station owned by Hart, owes $4,400 in back taxes on income, property and business taxes, and rent on property it leases from Phoenixville, Pa., according to officials there.

Hart has repeatedly refused comment since his news conference, and didn't respond to telephone requests for an interview yesterday. In his only recent public appearance, Hart told a Philadelphia television audience Thursday that a "hostile" press had distorted his views.

"I wouldn't want the bigot described in the press in this [Civil Rights Commission] position," he said.