The concern is obvious in this chart, which shows House exit poll data broken out by race and education. (We use education as a proxy for class because it correlates with income levels and isn't subject to shifts from inflation.)
The higher up the line on this chart, the better Republican House candidates did with the voting population. That light red line that's cruising along above the dark red one represents the shift toward the Republicans by white non-college graduates. The gap between college-educated whites and non-college graduates is the widest it's been -- and to the GOP's benefit.
In that first graph, you'll notice that no similar gap has emerged among non-white voters. In fact, the distance between non-college-educated, non-white people and college-educated ones over the last few cycles has narrowed.
There's one bit of upside to that data for the left. As a percentage of all voters, non-college-educated whites have declined over time -- as more white voters have college educations and as non-white voters overall have become a larger part of the electorate. (On the graph below, the size of the bubble correlates to how much of the electorate the group comprised.)
The number of non-white voters is still much lower than the number of white voters.
It's worth noting that the composition of that white vote breaks down in another way -- by age. In an excellent autopsy of the 2014 election, Real Clear Politics' Sean Trende argues that lower turnout (it was substantially lower than 2012) wasn't the only problem Democrats had, reinforcing the point of the hand-wringers.
For example, here's how the electorate has broken down over time by age and race.
Notice that sudden dip among the youngest white voters, who broke from the rest of the pack in 2006. (An 18-year-old in 2006 was born in 1988, a Millennial through-and-through.) Trende's point is illustrated here:
Even at 2012 levels of turnout, young white voters wouldn't have shifted the election for the Democrats, because their 2012 turnout was relatively low as well.
That last graph shows that it's the oldest voters -- just as it's the least college-educated voters -- who are most partisan in their voting. The problem for Democrats is in part that those groups turn out to vote for Republicans much more heavily, which multiplies the effects of a shift away from their party. (Note, by the way, that no white age group has ever been more Democratic than any non-white age group.)
One last note, visible particularly in that last graph. If you cover up the red dots in 2006 and 2008, there's a clear trend from about 1994 to 2014, slowly up to the right. But 2006 and 2008 exist -- years in which whites were still a dominant part of the electorate but they handed power to Democrats. It wasn't turnout, it was politics. So much of the political conversation has been about the increasing size of those blue bubbles. That's happening, slowly, and (back to the first graph) it's also happening as the wide margin of Democratic support fades slightly. But Democrats know that moving the light red lines is possible -- and that they need to, fast.