By Bruce Baker
The idea for this post first came about when I read the following quote in an article in The Economist. This has to be among the most utterly stupid statements I think I’ve ever read in my life:
…no Wall Street financier has done as much damage to American social mobility as the teachers’ unions have.
This quote is ridiculous at many levels. Most notably, the quote is stupid simply because one could never possible contrive a reasonable quantifiable comparison of the supposed negative effects of either the individual hedge fund manager or the supposed monolithic “teachers union.” It’s the empirical equivalent of arguing whether Superman can beat up Hulk.
On the heels of this quote came the Thomas B. Fordham Institute report rating the strength of teachers unions – or unionization more generally – across states. Perhaps the most useful aspect of this report is that it provides us with insights regarding the heterogeneity of unionization across American states. Unions and unionization are not monolithic.
As recognized by the Fordham report, we really don’t have an American education system. We have 51 systems. They are all somewhat different, with different standards, different funding systems, different union rules and protections and different student outcomes. The existing variations across our state systems of education alone render The Economist statement misguided. Those variations also provide for some fun opportunities to explore the relationship between Fordham’s characterization of teachers’ union strength across states and other features of state education systems.
In this post, I use data from several reports that attempt to characterize state education systems to probe two main questions. Is there any association between general indicators of education quality across states and union strength? Is there any association between indicators of educational equality across states and union strength?
How is union strength related to funding levels and funding fairness?
Along with colleagues at the Education Law Center of New Jersey, I have been preparing for the past few years annual reports on education funding fairness. In the Funding Fairness report, we use a statistical model on three years of national data on all school districts to project the cost adjusted per pupil state and local revenues for all districts and state averages nationally, and we characterize the overall fairness – progressiveness or regressiveness of state school finance systems. Below, I evaluate the relationship between “union strength rank” from the Fordham report and funding “levels” (an indicator of adequacy) and funding “fairness” (whether higher poverty districts receive systematically more, or less funding per pupil than lower poverty districts in that state).
An important caveat heref is that I really should not be making scatterplots where the x-axis variable is a “rank” measure. Rank is not an interval measure. But this is purely for illustrative purposes, so please forgive my misuse of rank data in this way! [or at least if you slam me for it, acknowledge that I pointed this out!]
In Figure 1 we can see that states with stronger teachers unions [left hand end] tend to have more adequate overall funding levels. It is however more clearly the case that states with weak teachers unions (ranked 45 to 50th) tend to have particularly low adjusted funding levels. This is certainly not to suggest any direction of causation. That’s the whole trick here. Most of this is probably quite circular – endogenous. [The union cynic might argue that this merely shows that teachers’ unions have extorted funds from the taxpayer.] That states which tend to be more educated and progressive happen to both have stronger teachers unions and to spend more on education – except for those states, like California, that by historical artifact referendum have systematically deprived their education systems for decades.
Perhaps more to the point of The Economist assertion, we see that states with weaker teachers unions also tend to have less-fair funding distributions – or are systems where it is more likely that high-poverty districts have systematically fewer resources per pupil than lower poverty ones. Again, this result is likely a function of the endogenous relationships mentioned previously.
How is union strength related to competitiveness of teacher pay?
Here, I look at the relationship between union strength and the relative wage of teachers compared to non-teachers in the same state. This is a particularly important comparison for two reasons. First of all, the relative competitiveness of teacher wages likely has significant effects on the quality of individuals who choose to enter the teacher workforce versus other employment opportunities (selecting from high school into college). Overall wage competitiveness can have long run effects on overall teacher workforce quality. Further, this is the one comparison I make in this post where we might hypothesize a direct, easily interpreted relationship. That is, we might expect stronger unions to lead to more competitive wages. Here, I compare the weekly wage % (teacher percent of non-teacher) from the Economic Policy Institute with the Fordham Institute’s union strength rank.
Somewhat to my own surprise, this relationship is actually quite strong … with states having stronger teachers unions also having generally more competitive teacher wages.
Is union strength associated with National Assessment of Educational Progress achievement levels?
The usual retort to teacher union bashing is to point out that states such as New Jersey and Massachusetts have strong unions and also have high NAEP scores, and states like Alabama and Mississippi have weak unions and low NAEP scores. Clearly there’s a lot going on there that has little or nothing to do with unions, but let’s indulge this premise a little further with some additional graphs just to see the patterns.
In these first few figures, I present the relationship between NAEP scores for children in families above the 185 percent income level for poverty (not on free or reduced lunch) and union strength. Note that the patterns are similar for scores for children qualified for reduced price/free lunch, but I’ve not included them here (because there are already enough graphs in this post). In general, what we see in Figure 4 and Figure 5 is that NAEP scores for non-low income kids tend to be slightly lower – with little clear pattern – in weak union states.
Figure 6, however, clarifies that NAEP scores tend to be higher for non-low income children in states where incomes are higher for non-low income children.
Figure 6 (but income dictates NAEP)
We can use the information in Figure 6 to adjust the NAEP scores (are they higher or lower than would be expected, given the income levels) for household income differences. When we make that adjustment, we get Figures 7 and 8.
Figure 7 (income adjusted NAEP)
Figure 8 (income adjusted NAEP)
Still we see that adjusted NAEP scores are somewhat though hardly systematically lower in states with weaker unions. What we certainly do not see here is that NAEP scores are systematically lower in states with stronger unions. That is, unions certainly aren’t driving NAEP scores into the ground!
But, while the second set of graphs is more appropriate than the first, both are dreadfully oversimplified characterizations of complex relationships.
Is union strength associated with NAEP achievement gaps?
This question is perhaps most on target with The Economist claim. Following The Economist’s logic, one might assert that teachers unions likely lead to larger achievement gaps, thus limiting social mobility. Measuring poverty related income gaps and comparing them across states is tricky, as I’ve discussed in numerous previous posts. Specifically, the size of the achievement gap between kids not qualified for free or reduced lunch and those qualified for either free or reduced lunch tends to be highly related to the size of the income gap between the two groups – as shown in Figure 9! That is, we can’t just do straight up achievement gap comparisons; we must adjust for the income gap.
Figure 9 (Income Gaps and NAEP Gaps)
Figure 10 and Figure 11 present the income gap-adjusted achievement gaps in relation to union strength rank. What we see is little or no relationship between union strength and achievement gaps. While this does not illustrate that stronger unions lead to smaller achievement gaps, it also does not by any stretch illustrate that stronger unions lead to larger achievement gaps, an expectation that might reasonably be derived from the claim made in The Economist.
Then again… these are still cursory… descriptive analyses – using only two variables at a time to characterize education systems that are far more complex than can be legitimately characterized with only two variables at a time. It’s exploratory. It’s a start… and there’s certainly more to be explored here… but likely questions that can never be satisfactorily untangled with available data.
Is union strength associated with NAEP achievement growth?
Finally, I suspect that some curmudgeonly reactors to this post will attempt to argue that weak union states have seen more growth in NAEP achievement over time. Well, Figure 12 kind of thwarts that notion as well. Not much relationship there either, but certainly the only one in this post at all that shows even the slightest upward tilt.
But alas, even that tiny upward tilt is a function of the fact that states that saw the greatest growth on NAEP were simply the states that had and still have the lowest overall performance levels – as shown in Figure 13. And states with lower average performance levels – now and then – tend to have weaker unions.
For a more thorough discussion on this point, see: http://schoolfinance101.wordpress.com/2012/07/27/learning-from-really-bad-graphs-ill-informed-conclusions-thoughts-on-the-new-pepg-catching-up-report/
So what does this all mean then? Are unions good or are they bad? Do they increase inequality and lower quality? It’s certainly difficult given the data provided above to swallow the bold assertion in The Economist that teachers’ unions are the scourge of the nation and primary cause of declining social mobility.
But then what can we learn here? It is perhaps important that there appears to be at least some likely indirect and certainly endogenous relationship between unionization and funding fairness and funding levels. As I’ve discussed in related research funding fairness and funding levels – and school finance reforms that improve equity and adequacy do matter! To summarize:
Do state school finance reforms matter? Yes. Sustained improvements to the level and distribution of funding across local public school districts can lead to improvements in the level and distribution of student outcomes. While money alone may not be the answer, more equitable and adequate allocation of financial inputs to schooling provide a necessary underlying condition for improving the equity and adequacy of outcomes. The available evidence suggests that appropriate combinations of more adequate funding with more accountability for its use may be most promising.
Further, the potentially more direct relationship between unionization and relative competitiveness of teacher wages compared to other labor market opportunities may be important in the long run. In a related policy brief from last winter, I noted:
To summarize, despite all the uproar about paying teachers based on experience and education, and its misinterpretations in the context of the “Does money matter?” debate, this line of argument misses the point. To whatever degree teacher pay matters in attracting good people into the profession and keeping them around, it’s less about how they are paid than how much. Furthermore, the average salaries of the teaching profession, with respect to other labor market opportunities, can substantively affect the quality of entrants to the teaching profession, applicants to preparation programs, and student outcomes. Diminishing resources for schools can constrain salaries and reduce the quality of the labor supply. Further, salary differentials between schools and districts might help to recruit or retain teachers in high need settings. In other words, resources used for teacher quality matter.
So, while nothing in this post puts to rest the big – unanswerable – questions of the overall equity and quality effects of teachers unions on our supposed monolithic American public education system, these analyses do at least raise serious questions about the notion that teachers unions are the scourge of the nation and the cause of all of the supposed – also unfounded – ills of American public schooling.