There are none so blind...
A tiny little box labeled "correction" in the latest issue of Nature alerted me to a re-examination of data supposedly showing discrimination against female authors by manuscript reviewers. This claim, echoed in a Nature editorial (now retracted), was based on data showing that the fraction of papers with female first authors increased in one ecology journal when that journal started withholding the names of authors from reviewers, a procedure known as double-blind review. It turns out that other journals in the same field showed a statistically indistinguishable trend (more female authors over time), even though they still provide author names to reviewers. Thomas Webb and coauthors of this re-examination, published in Trends in Ecology and Evolution, note that a larger study published in American Economic Review reinforced their conclusions: double-blind review does not increase relative acceptance rates for papers with female authors.
These results seem to show that one possible form of bias against women is relatively rare, at least in the disciplines studied. Knowing that authors are female apparently has no effect in either direction.
It would be really interesting to see whether double-blind review would have any affect on success in getting grants. Differences in success rates for male versus female scientists could have many causes. For example, female scientists may be younger, on average, giving them shorter publication lists (but perhaps also fresher ideas?). Or there could be sex discrimination (in either direction) by reviewers. A statistically significant improvement in grant programs trying double-blind review, relative to control programs not adopting this approach, would provide evidence for such discrimination. Note that if such discrimination were found, reforms stronger than double-blind review might be needed, as some reviewers would still be able to identify the source of a proposal even if, for example, publication lists were reduced to numbers of papers in each journal.
In my own grant proposals, I usually reveal that my star Ph.D. student, Toby Kiers, was (and is) female, which isn't obvious from her name. Could that be why my two recent grant proposals were rejected? Maybe I should try my own double-blind experiment!
I wish more people would apply the same intellectual rigor to identifying road-blocks to the success of women and minorities in science that we do to our other scientific pursuits. The fact that there are more male than female full professors is certainly evidence of past differences in hiring or promotion (presumably some combination of discrimination and other factors), but if we want to know whether there is discrimination in hiring or promotion today, we need to look at hiring and promotion today. Is an associate professor with a given publication record less likely to be promoted if she is female? Do we put too much emphasis on publications? And so on.
I am reminded of Donald Campbell's 1969 paper, "Reforms as Experiments" (American Psychologist 24:409). Among the insights in this classic paper is the following. In response to some problem (crime, for example), politicians change laws, funding levels, etc. If the situation improves, they credit their "reforms." But crime rates etc. vary somewhat from year to year anyway. Now here's the key point: reforms are most likely to happen in a year when the problem is worse than average. Therefore, simple random variation is likely to result in some "improvement" the next year, even if the "reform" had no effect at all. So, as a minimum, you need to compare changes over years in states that did versus did not institute the "reform." If the editors of Nature had read this paper, substituting "journals" for "states", they might have been less likely to jump to conclusions.