Ben Carson is not really qualified for his job. But that’s the least of HUD’s problems. (Matt Rourke/AP)

Housing and Urban Development Secretary Ben Carson, whose biggest headline to date came from getting stuck in an elevator in Miami, is an easy target for progressives who are angry about Trump administration officials who lack qualifications. Carson’s leadership has been messy at best, and his callousness shows in his support for budget cuts and belief that poverty is “a state of mind.”

But while it’s easy blame HUD’s inadequacies on incompetent administrators, the agency’s problems can’t be solved by simply changing personnel. Its failures have persisted for decades regardless of who is in charge — ever since its predecessor, the Federal Housing Administration, began issuing mortgage insurance. That’s because racism was built into the FHA’s lending programs, guiding who had access to home-buying credit. When HUD was established in 1965, it continued that legacy, but rather than using credit policy to advance segregation, it simply left minority homeowners subject to financial predation.

From the beginning, U.S. housing agencies have failed to provide equal opportunities for affordable housing. The New Deal’s housing revolution, which used federal dollars to open up homeownership to a generation of Americans, largely excluded nonwhite citizens. The National Housing Act of 1937 offered public housing on a segregated basis — the price of getting the legislation through a Southern-dominated Congress — and the FHA, which was created to insure mortgage loans at a time when banks were nervous to lend money, made poorer areas with higher concentrations of African Americans ineligible for FHA-insured mortgages — a practice known as “redlining.”

For the next several decades, funding for public housing and the FHA mortgage program failed to address the needs of the urban and largely nonwhite poor. President Lyndon B. Johnson attempted to open the housing market to racial minorities with the creation of HUD and, three years later, the 1968 Fair Housing Act. But President Richard M. Nixon reversed these gains and diminished the federal government’s role in the housing market. By 1972, no HUD office was insuring FHA mortgages in central city areas.

As government-backed loans dried up, predatory lending filled the void. Speculators took advantage of the influx of new buyers entering the market after Section 223(e) of Johnson’s 1968 Housing and Urban Development Act extended mortgage insurance to low-income urban areas, thereby including racial minorities previously excluded from homeownership. They preyed on these buyers in such volume that by the 1970s their efforts had become a full-fledged scandal.

In 1983, Philadelphia Inquirer reporter Tim Weiner uncovered one of the most egregious predatory lending scandals of that decade in Camden, N.J. While wandering the city one day, Weiner came upon a notice for a public auction of hundreds of foreclosed FHA-insured homes. A few months later, he saw that the houses sold in the first auction had been foreclosed upon and were up for sale again. More troubling were the prices for which HUD sold them — $200 or $300 ($491 or $737 in 2017 dollars) — and how the FHA was valuing them afterward, at upward of $30,000 ($73,730 in 2017 dollars), reflecting in some cases outright bribery in exchange for inflated appraisals, but mostly HUD’s tendency to look the other way.

At a time of heightened support for free market ideology bolstered by Ronald Reagan, HUD had little interest in monitoring the true value of the homes, the buyers’ ability to pay for their mortgages or how often these properties went into default.

Weiner’s investigation brought him to Tanya Combs, an unemployed, 24-year-old black mother of four. Combs was more or less the typical home buyer targeted by predatory lenders. In 1982, she had been evicted from a housing project. Because of that, the state of New Jersey had taken away her children. She would only be able to regain custody if she found a new home — a nearly impossible task given her monthly welfare income of just $468.

After seven months without a home, Combs met Bill Luciani, the vice president of Bonafide Investment Corp., a local real estate company. She had learned through word-of-mouth that Luciani could get a house even for those on welfare. Combs, who had never filled out a mortgage application, did what Luciani told her to do: She signed her daughter Michelle’s name and used Michelle’s Social Security number. HUD approved her application, and just like that, Combs owned a $19,300 house with a $270 monthly mortgage — and she could see her kids again.

Weiner discovered that speculators like Luciani followed the same pattern with each sale. After the government paid off a foreclosed FHA mortgage, a real estate company would purchase the house from the FHA and make — or, more often, lie about making — repairs to it. The company, using falsified documents, would then sell the house to a buyer who would later prove unable to pay the monthly mortgage payments, leading again to foreclosure, and the cycle would repeat.

It wasn’t long after the Inquirer published its findings that the FBI and the U.S. attorney’s office conducted its own investigation, eventually obtaining more than  30 convictions, including Luciani’s.

Yet the scandal did little to spur a cleanup at HUD. Selling properties in bulk helped fulfill Reagan’s goal of shrinking bureaucracy and deregulating the housing market. So even when an audit of six cities, including Camden, found that HUD had repeatedly violated its own procedures, and that the estimated cost to HUD and taxpayers for Camden’s scandal alone amounted to roughly $3 million, HUD made no changes.

Nor did an unrelated 1989 influence-peddling scandal involving HUD Secretary Samuel Pierce yield lasting change. For the first time, Congress investigated HUD in earnest, but focused on the agency’s top officials, not its institutional failures.

With the mortgage crisis of the last decade, which has dramatically tightened lending, the housing market has come full circle. For much of the 20th century, redlining prevented African Americans from becoming homeowners. During the housing boom of the late 20th century, those communities were actively targeted for predatory mortgages. And now they are, in effect, being redlined again.

Today, lending standards have tightened to the exclusion of vulnerable buyers. “When we faced the recession and there was retrenchment of credit, some of the older practices and the suspicions of minority creditworthiness became more pronounced again,” explained Bryan Greene, the HUD general deputy assistant secretary for fair housing. “People started to blame the very persons who were victimized by the discriminatory practices. In fact, some of these families were steered into irresponsible mortgage products by unscrupulous lenders and brokers who targeted them because of their race.”

Julia Gordon, executive director of the National Community Stabilization Trust, a nonprofit that promotes the reuse of foreclosed or abandoned property, put it more bluntly: “Now we’re redlining based on the credit box.” She added that while the standards are ostensibly race neutral, “they have a disparate impact.”

There are regulations in place to address redlining and other discrimination. But while HUD typically investigates complaints brought to it, it is more often reactive than proactive. That’s a problem, since buyers may not know they are facing discrimination and therefore aren’t able to bring it to the agency’s attention.

This is not to say that HUD hasn’t seen successes. In 2015, the agency reached a $200 million settlement with a bank accused of redlining in the Midwest. But while regulation in the form of lawsuits or prosecution is important, it is not as effective as preemptive action. The villains of Camden’s scandal went to jail, but the buyers still lost their homes.

As we saw in the bailouts of the late 2000s, when the government intervenes, it does not save the buyer. Government-backed mortgages insure the institution loaning money — not the homeowner. Those who do seek to protect vulnerable buyers are stuck playing a game of whack-a-mole: Every time they hit their target, more offenders come out of the woodwork. In 1998, with Luciani and his cohort behind bars, the Inquirer uncovered another major housing repair scandal in Camden.

Today’s Consumer Financial Protection Bureau, the regulatory body created after the housing crisis, “is a game-changer,” according to Gordon. But the CFPB cannot single-handedly address a century’s worth of institutional failures. And it, too, is subject to political whims — the Trump administration and a Republican Congress will, in all likelihood, curb much of the agency’s power, especially now that Obama appointed director Richard Cordray has resigned.

None of which is to suggest that Ben Carson’s poor leadership isn’t a problem. Low-quality Cabinet picks should inspire outrage. And there is no question that the HUD secretary can do tremendous damage. But there are also larger forces at play, including a legacy of racism and greed. Until the federal government puts in place programs and reforms to undo those legacies, HUD will continue to fall short of its mission, regardless of who’s in charge.