The government’s recent sale of stock in insurance giant American International Group is a welcome milestone in the unwinding of the massive federal investments made in private companies during the financial crisis. The bailout of AIG turned into a gain for the government, TARP investments in the biggest banks have been repaid at a profit, and the Treasury Department is selling off stakes in smaller institutions at an admirable pace. Taxpayers are being compensated for stabilizing the economy at a critical time.

Four years after the worst phase of the financial crisis, however, it is time to end the most costly bailout of all: the government takeover of Fannie Mae and Freddie Mac. Keeping the two housing-finance firms alive has been expensive. Treasury has invested $187 billion in the companies and has received $46 billionin dividends, for a net cost of $141 billion so far.

This support has allowed the two companies to continue to service the $4.5 trillion in guarantees against mortgage default and $900 billion in debt that they had racked up before the crisis, and to underwrite trillions of dollars in new mortgage credit. As a result, Americans have been able to get mortgages to buy homes and (especially) to refinance at lower interest rates.

But the firms shoulder an immense amount of risk, both by guaranteeing home loans against default and by owning them outright. This puts taxpayers on the hook for further losses and short-circuits the normal role of private investors in shaping housing and capital markets. Yet neither the administration nor Congress has a viable plan to end the government control of and exposure to Fannie and Freddie.

There is a way out that would protect the nascent housing recovery, revive the private mortgage marketplace and recover taxpayers’ investments in the firms. And the time to start is now.

Home prices have finally stabilized and are rising in many areas, and Fannie and Freddie are profitable again in their main line of business: bundling mortgages into securities and guaranteeing those securities against losses. This is the good news.

The bad news is that, four years after the takeover of Fannie and Freddie, the government now backstops 90 percent of all new mortgages and has no plan to reduce its market share, no plan to protect taxpayers against future losses on the trillions of dollars of mortgage credit underwritten since the firms were placed under government control and no plan to recover the taxpayer money invested in Fannie and Freddie. With the Troubled Assets Relief Program, the government worked with recipients to replace public investments with private capital as quickly as possible. Not so with Fannie and Freddie. Worse yet, two recent government actions threaten to make the companies permanent wards of the state.

First, Treasury decided to take all future earnings from Fannie and Freddie as a dividend on its outstanding investment. At first glance, that “cash sweep” seems laudable: Taxpayers should recover their investments in the companies before other shareholders recover a dime. The problem is that, if all earnings go to Treasury, the firms cannot use that money to build capital and reserves to protect taxpayers against potential losses. Private insurance companies hold capital against the risks they underwrite and build reserves to make good on future claims. If Fannie and Freddie don’t do so, it means that taxpayers will be on the hook for the firms’ losses for the remaining life of the 30-year mortgages they have written on the government’s watch.

Second, the firms’ regulator, the Federal Housing Finance Agency, just mandated that Fannie and Freddie increase the fees they charge to guarantee mortgages. Again, this at first seems like a sensible and long-overdue move; the companies had grossly underpriced the insurance they provided on mortgages before the crisis, putting taxpayers at risk for the bailout that inevitably came and making it difficult for other private companies to compete with them.

Yet, even after the increases, Fannie and Freddie’s fees will be significantly below what a private company would charge, meaning that private competition will remain muted. Further, while the higher fees are a start toward more rational risk-based pricing, the money will go directly to the government — because Treasury is taking all of the firms’ future earnings — so Fannie and Freddie won’t be able to use it to build capital and reserves.

Taken together, the cash sweep and the new guarantee fees will further entrench Fannie and Freddie within the government. Congressional budget rules allow the higher fees and increased Treasury dividends to pay for spending outside of housing. With Washington hungry for revenue, there will be inexorable pressure to milk Fannie and Freddie’s guarantee fees to support other government spending. The losers will be potential homeowners, as mortgage availability will be determined by government regulators rather than by private firms competing for their business.

To prevent this creeping nationalization of the mortgage market from becoming a fait accompli, Congress needs to pass a bill that does two fundamental things.

First, it should transform the federal backstop of Fannie and Freddie to allow the government’s role to recede and private capital’s role to increase. Today, Treasury guarantees the firms so they in turn can make good on their guarantees of qualified mortgage-backed securities. The government is effectively first in line for losses. In the new system, private mortgage insurers would take the first loss on any bad mortgage before the government reinsurance would kick in. The amount of private capital at risk ahead of the government guarantee should grow over time so that private investors bear the bulk, and perhaps one day all, of the credit risk on qualified mortgages.

It may be counterintuitive that the first step in reducing the government’s role in housing finance is to turn the guarantee of Fannie and Freddie’s solvency into an explicit reinsurance program. This is a political hurdle for conservatives, who are understandably reluctant to establish yet another government credit-support program. But the guarantee of the mortgage market through Fannie and Freddie cannot shrink until it is acknowledged for what it is.

The government already guarantees mortgages. The change in structure we propose would eventually allow the taxpayer exposure to shrink. By contrast, each day the current system remains in place increases the likelihood that the cash sweep becomes embedded in the federal budget, and that Treasury’s backstop for Fannie and Freddie and the government’s seizure of the firms become permanent, leaving private capital on the sidelines indefinitely.

The second fundamental step is to restructure, recapitalize and privatize Fannie and Freddie’s mortgage-guarantee businesses. To end the bailout, the two firms must be transformed into private “first loss” insurers, with their own capital standing in front of the government’s new reinsurance. The highly leveraged investment portfolios each firm ran before the crisis — in effect, government-sponsored hedge funds — should continue to be wound down under federal supervision. The mortgage-guarantee businesses should then be reincorporated and stripped of all special government privileges, such as the line of credit from the Treasury Department.

Without special privileges, the firms should be forced to compete with other private companies willing to pay the government for its reinsurance, with strict regulation to ensure that community banks can originate and securitize mortgages on an even playing field with the giant banks that have come to dominate the business in the past four years. Such competition will spur innovation and reduce systemic risk in mortgage securitization. As the system spurs new entrants in mortgage finance, any one of them can fail — including the new Fannie and Freddie — without the threat of a housing market collapse.

To build the capital required to protect taxpayers on their new reinsurance, Fannie and Freddie should increase their fees to market levels. Further, Treasury should suspend the cash sweep so that Fannie and Freddie can retain those fees and build up capital. Taxpayers effectively own the firms today. Rebuilding their capital is the first step in getting them ready to be sold back into private hands.

Privatizing the newly recapitalized and reincorporated mortgage-guarantee businesses would help Treasury recover its substantial investment in the companies and begin moving toward a safer housing-finance system driven by market incentives, with private capital first in line for losses. Taxpayers deserve both outcomes. Once the companies have enough capital, Treasury should convert its preferred stock into a sufficient percentage of common stock to ensure that taxpayers’ investments can be repaid in full. The firms could then be released from government control and Treasury’s equity in the restructured entities sold to private investors over time. This is exactly what Treasury did with its 92 percent stake in AIG, with great success.

It is time to end the four-year bailout of Fannie and Freddie. Congress and the new administration should prevent the companies from becoming permanent wards of the state and hasten the day when private markets are again the primary source of credit for American homeowners.

Jim Millstein served as chief restructuring officer at the Treasury Department from 2009 to 2011 and led the turnaround of AIG. He is the chairman and chief executive of Millstein & Co., a financial advisory and investment firm. Phillip Swagel served as assistant Treasury secretary for economic policy from 2006 to 2009 and worked on implementing the Troubled Assets Relief Program. He is a professor at the University of Maryland School of Public Policy.

Read more from Outlook, friend us on Facebook, and follow us on Twitter.