Over the weekend, I will put up a concluding post with links to the entire series and a pdf file with all contributions.
Bias in citations is a serious issue. Citations are the “currency” of academic stature, and even though we would all like to think it is one of the many criteria by which scholars are judged, there does seem to be evidence that citations are becoming more and more important: in comparing candidates for promotion, for special prizes and commendations, and for that coveted state: tenure.
The International Organization article by Daniel Maliniak, Ryan Powers and Barbara Walter (henceforth: MPW) is therefore really important work. And research not likely to greatly improve their job prospects in the future! After all, these are serious scholars of international relations who have taken time away from their intellectual interests to expose the possibility of biases within their own intellectual community. In fact, I would guess while this article’s citations zoom, it ironically will be one of the few citations that will not help any of these scholars’ careers in a big way. So I think those of us concerned with integrity in our profession owe them a big thanks for their work.
The MPW article instantly instilled fear and depression in the young women who heard of its findings at the APSA meeting this past September. Not very encouraging news overall, I must say. But if I might cull just a couple of positive bits from this article, I would point to two things. First: For some strange reason, there never seems to have been a “median” citations gap. The table below is drawn from MPW, and it clearly shows that the median number of citations for a female authored article has equaled or exceeded that of male authors for the past three decades. By the 2000s, the median citation count for a female authored article was 15 and for a male authored article it was only 13.
|Median All Male:||5||14||13|
|Median All Female:||6||14||15|
A second positive finding is that the mean citations gap, while it still seems to exist, has declined over time. The mean gap was 7.17 citations in the 1980s, 5.4 citations in the 1990s, and is down to 2.33 in the 2000s, according to the MPW data.
Of course, this doesn’t mean that there is no need to pay attention to disparities between men and women when it comes to citations or to the potential for subtle barriers to true equality in the academic profession more generally. If these numbers have improved, it is because over the past decade, women, often with the help of tenacious and encouraging advisors of all genders, have worked very hard to break into the top ranks of researchers in our field. And yet if we focus on mean citations, the gap remains, is annoying and is potentially serious for careers. It seems especially serious for young careers, because apparently the gap is more serious for untenured women than for senior women. (Strangely, tenured women have, if I read the data correctly, a citations coefficient about 10 times bigger than that for tenured men. In fact, it appears that while tenured women have strong positive “authority” coefficients, these are strongly negative for tenured men. Maybe there is a strong selection effect; women who manage to jump the hurdle and get tenured do very well. Or maybe we should be asking, what is it that guys do after tenure?) The whole gap issue is doubly troublesome, because many of the “obvious” explanations for lower citations – subject matter, methodology used and theoretical orientation – don’t seem to make the mean gap go away.
The blogs have raised the issue – what about controlling for “citability”? Are women unduly under-cited when controlling for the quality of the work? It is an awkward question, to say the least, but an important one. I suppose we will never convince the skeptics that there may be something objectionable to the citations gap, unless and until someone comes up with a study analogous to one of those cool experiments that shows the exact same resume gets much more attention from employers if it has a male name attached rather than a female name. Until someone figures out how to do that with academic citations, MPW’s research findings are the state of the art and serious food for thought.
Let’s think a little further on this quality issue. Is it plausible that a simple “sex” indicator is picking up quality differences in research? I am going to go out on a limb and say, maybe. But before I’m met with gasps of indignation, let’s consider what this means. If we want to say it does have something to do with quality, then the good news is that the quality of women’s research has increased noticeably over the past three decades. This wouldn’t surprise me. Women are increasingly getting the top training they deserve. But what about that stubborn (mean) gap? In my department, graduate students just concluded a survey of student concerns and we found out that our female and male grad students have about the same assessment to make of the program, except that the women feel they are not getting as good advising as the male students feel they are getting. Are faculty really doing all we can do to help make our female students’ research as strong as it can be? Maybe not. I would challenge other departments to take a close look at student perceptions in this regard, and try to do something about a possible “advising” gap. Closing any possibility of a gap as early as we can seems like a good first step.
I have been known to make far more radical suggestions for addressing the citations gap than “let’s do more to give female grad students as good a start as we give our males.” Inspired by a finding of some research that I have done with Judith Kelley about the importance of “performance indicators” for inducing behavioral change (which you can find here), I suggested in the APSA panel that it might be a good idea to develop an Index of Gender Bias in Citations, and rate all of the political science/international relations journals for their performance. And yes, assign them a grade. Maybe editors could do just a bit to remind their authors to be sure they have done a reasonably balanced job citing the relevant literature.