Main Street taxpayers have bailed out Wall Street. Now it’s time for Wall Street to return the favor by footing the bill to help millions of honorable Main Street borrowers pay lower interest rates on their mortgages, something that should have happened years ago. Wall Street giving back to Main Street — imagine that!

We’re not talking about anything risky to taxpayers, or a magic fix for the housing market. We’re talking about picking some low-hanging fruit by doing one simple thing: helping borrowers who are current on their payments refinance high-interest mortgages on which taxpayers are already at risk. That would help taxpayers as well as borrowers, because lower monthly payments would stimulate the economy, support housing prices and reduce future defaults.

I’m talking about providing a cheap, streamlined and simple way to refinance fixed-rate mortgages backed by Fannie Mae and Freddie Mac, which own about half the nation’s mortgages and are now effectively owned by the federal government. Fannie and Freddie creditors were bailed out in 2008 when Uncle Sam put the firms into conservatorship to avoid their having to file for bankruptcy; as we’ll soon see, those creditors, consisting primarily of big financial institutions, would bear the cost of helping homeowners.

Mass Fannie and Freddie mortgage refis could provide billions of dollars of economic stimulus and support the prices of homes, many Americans’ biggest single asset. All while costing taxpayers nothing.

This is entirely different from using taxpayer money to subsidize people who have defaulted on their mortgages, a proposal that stirred considerable resentment (including on my part) and gave birth to the tea party.

Rather, I’m talking about helping 13 million or so families who have played by the rules and kept up their house payments but haven’t been able to refinance their loans, often because their houses have fallen sharply in value. Remember, taxpayers are already on the hook for those loans; thus, refinancing them adds no risk, and arguably reduces it by shaving the monthly payment burden and giving homeowners less incentive to default.

Mass refis would produce interest savings averaging $3,200 a year per family, based on an average drop of two percentage points for mortgages averaging $160,000, according to statistics from a leading refi proposal. That’s $40 billion a year of economic stimulus, at no taxpayer cost. (In fact, it might increase federal tax revenues a bit by reducing homeowners’ interest deductions.)

No, this proposal didn’t emanate from some left-wing redistributionist think tank. It comes from Glenn Hubbard, an economic conservative who led George W. Bush’s Council of Economic Advisers and is currently dean of the Columbia Business School. His partners include Chris Mayer, who teaches housing at Columbia, and Alan Boyce, a mortgage market maven. Their plan, which you can find at, isn’t perfect, but it’s a well-thought-out starting point.

The mass-refi idea is so intriguing that the nation’s leading bond investor, Pimco’s Bill Gross, supports it, although it would cost his investors money. His reasoning: It’s good for the country. “I put on my public hat, as opposed to my Pimco hat,” he told me. “There’s a big part of me that supports the 99 instead of the one.”

Despite Congress being bitterly divided, Sen. Johnny Isakson (R-Ga.), a staunch conservative, has joined liberal Sen. Barbara Boxer (D-Calif.) in pushing for simplified, cheaper refis. “It’s a win-win,” Isakson says. “A no-brainer,” Boxer says.

The Federal Reserve, which had been promoting mass refis quietly behind the scenes, is now doing it openly and loudly. Bill Dudley, head of the New York Fed, says the lack of refinancing is hurting the Fed’s efforts to stimulate the economy. (Dudley also is pushing other, more aggressive ideas, but I’m not going there — I’m sticking with Fannie-Freddie mass refis.)

If mass refis are so obvious and wonderful, why haven’t they happened? After spending a lot of time studying the situation and talking with various players, I’ve concluded that the problems are inertia, fear, the complicated paperwork required — and a whispering campaign from institutions that would be big losers from mass refis.

Homeowners, as I’ve said, are the winners. The losers would be holders of mortgage-backed securities that hold the mortgages that would be refinanced. Because the mortgages pay above-market interest rates, these securities trade at a premium price: Call it 107 percent of face value.

However, if the mortgages that back a security refinance and pay off their loans at face value, that premium disappears. We’re talking about doing $2.1 trillion of refis: 13 million at an average of $160,000. That’s about $150 billion of lost market value to holders of the securities.

To which I say: too bad. Absent the taxpayer bailout of Fannie and Freddie, those securities would be trading in the toilet. Wall Street owns the vast majority of this high-interest paper, if you include the Fed, the biggest single holder, as part of Wall Street.

The Fed doesn’t care about losing the premium on its securities. Fannie and Freddie, which own a lot of them, would be made whole by fees from borrowers, and big holders such as foreign governments and commercial banks have already been bailed out by U.S. taxpayers, so there’s no reason to let them continue to gain from homeowners’ pain.

Some small investors would be hurt, and I feel for them. But they — or the managers of REITs or mutual funds in which they invested — knew or should have known about prepayment risk. There’s no reason to put these investors’ interests above those of homeowners.

That brings us to the man who oversees Fannie and Freddie: Ed DeMarco, acting director of the Federal Housing Finance Agency, who as the companies’ conservator is charged with protecting the government’s interests. Some people are trying to demonize DeMarco for not facilitating mass refis. I think DeMarco (who wouldn’t talk to me, alas) isn’t being an obstructionist — he’s just trying to avoid taking any risks he doesn’t have to take.

The Congressional Budget Office and many other analysts think mass refis would enhance Fannie and Freddie’s finances by reducing foreclosures, and I find that argument compelling. But I can respect where DeMarco is coming from.

Some people who oppose mass refis argue that if a mass-refi program takes place, investors would never buy mortgage-backed securities again. But Bill Gross — and many other market experts — think that’s nonsense. The new Fannie and Freddie securities into which the refinanced mortgages would be bundled would yield considerably more than equivalent Treasury securities. They’ll be snapped up.

What can break the logjam? President Obama could order mass refis, a natural for him given his recent populist bent and the obvious (at least to me) “Wall Street pays back Main Street” trope. If DeMarco refused, Obama could, some people think, legally replace him via a recess appointment.

Now, to Mitt Romney, the presumptive Republican presidential nominee. Given the uproar over his Bain Capital career, Romney might welcome a “Wall Street helps Main Street” proposal that costs taxpayers nothing. Romney wouldn’t have to look far for expertise: Hubbard, the big mass-refi proponent mentioned above, is one of his economic advisers.

Neither the White House nor Romney’s campaign — or even the Treasury — would tell me anything of substance about possible plans for mass refis.

But you never know. Good ideas have a way of bubbling to the surface, and having Wall Street bail out Main Street is as good as ideas get.

Sloan is Fortune magazine’s senior editor at large.