Middle-class families are fed up.

President Obama’s attempt to end a tax benefit of college savings accounts exposed a nerve: the frustration felt by families over how hard it is to cover rising tuition costs.

The White House argued that parents relying on 529s were wealthy enough that they didn’t need as much help as lower-income families, for whom the administration wanted to offer more assistance.

But the truth is that everyone, except for the super wealthy, is having a hard time paying for college. Tuition has risen faster than the rate of inflation. Wages certainly have not kept pace with the costs of college, neither has any form of financial aid.

Federal dollars are divvied up in a way that tends to shut out families making more than $50,000 and less than $100,000. Pell grants generally wind up in the hands of students whose families earn less than the median household income of $53,000.

On the other hand, education tax credits mostly go to families making at least $100,000, according to the Tax Policy Center. Tax credits are arguably a poor way of directing aid to families because the money is only available after students have already paid tuition, room and board.

“Let’s be more realistic about what need really means,” said Sandy Baum, a senior fellow at the nonprofit Urban Institute. “People making $80,000 a year still have financial need. If we’re asking them to pay more tuition, fees and room and board at a four-year college, giving them more help seems reasonable.”

She added: “And of course if we took the tax credit money and put it into grant aid, people would feel better about their prospects.”

Increasingly, the government has promoted student loans as a viable way for middle-income families to pay for college. Federal loans carry lower interest rates than those offered by banks, and in some cases the government will even pay the interest while the student is in school. But it’s still debt that has to be repaid. And graduating with $30,000 of student debt, the average these days, is not ideal, no matter how generous the terms of repayment.

It’s also important to remember, though, that colleges and states are setting aside more money for students whose families earn as much as $150,000, if not more in some cases. Schools have been using a portion of the money they once earmarked for the neediest kids to attract students in the “donut hole” of financial aid.

As states cut funding for higher education during the recession, public universities began using more of their own money to entice out-of-state students–who pay double or triple in tuition–to fill seats in their lecture halls. There are also schools that simply want higher income students who are often better prepared for college because of the resources afforded them.

“Public and private institutions are trying to get the best enrollment pool they can get to keep their graduation rates up, keep alumni happy because that serves their long-term financial interests,” said David A. Bergeron, a higher education analyst at the Center for American Progress.

There’s not that much of a difference anymore between the amount of so-called institutional grant aid that state schools award students from households taking in as little as $30,000 or as much as $99,000, according to the latest data from the Department of Education’s National Postsecondary Student Aid Study.

At a four-year public university, about a quarter of students from families that make between $65,000 and $99,000 get institutional aid, compared to roughly 30 percent of those whose families earn up to $99,000 a year. Even 20 percent of students whose parents make $100,000 to $250,000 get aid. And most students who receive money from public schools get less than $9,000, regardless of how much their parents make.

Even grants provided directly by the state are increasingly going to students with higher incomes. Researchers at the College Board found that the percentage of all state grant aid given to undergraduates without regard for financial need has increased from 9 percent in 1993 to 25 percent in 2013. That’s not exactly a sea change, but it shows a gradual shift in priorities.

“There needs to be a re-engagement between the state and the federal government in terms of supporting our public colleges in a way that allows them to keep prices down so that everyone can afford to attend,” said Bergeron, who served more than 30 years in the Education Department under Obama and several other presidents.

But however the aid gets doled out, it’s clear that it’s still a patchwork, very dependent on where students apply to school. And that middle-class families still feel like it’s not enough—something the White House clearly learned this week.