I came across Koenraad in the copy room yesterday photocopying a dozen or so copies of Christine WennerÂs and Agnes Wold’s dissection of the review process for Swedish post. docs (here’s a freely available copy for those without insitutional access), where they very convincingly demonstrate that to get the same score in the evaluations, female candidates had to have 2.5 times as many publications as men. I’d heard about the study while listening to Virginia Vallan’s talk Why So Slow? The Advancement of Women (highly recommended!) but hadn’t actually read it before. After reading, I’m shocked that the results aren’t more widely known and discussed. This paper is almost ten years old!

So in brief: WennerÂs and Wold noticed that while there is an almost equal number of male and female applicants for post docs in medical sciences in Sweden, twice as many men as women actually receive a fellowship. They wanted to know whether this was due to the female applicants simply not being as good, or due to a gender bias. Conveniently, there was central and consistant evaluation of these candidates, but the evaluations were not public. So WennerÂs and Wold went to court to gain access – and they gained it, as the Swedish consitution makes state documents that are not a threat to national security open to the public. Once they had the data, they analysed it in a number of ways. First they found that only the very best women were given a score as high as the average man, and that women with equal numbers of publications as men got lower scores than those men. Women would in fact have to publish 2.5 times as much to get the same score as men. Oh, having an affiliation with a committee member helped a lot, too, even though the person you were affiliated with was not allowed to be directly involved in assessing you. There’s lots more: you should read the paper, it’s only three pages long.

One thing I found interesting was that the quantitative methods WennerÂs and Wold use appear to cut through the gender bias inherent to more qualitative evaluation methods. In the talk I mentioned above, Vallan very engagingly describes dozens and dozens of studies showing that we all, men and women alike, have inherent prejudices and expectations to what men and women are like, and that without being extremely conscious of this, we will always overestimate men’s height, intelligence and leadership capabilities and underestimate those of women.

In the last years, many universities and countries have moved towards more quantitative measures of research production. Here in Norway there are now lists of approved journals sorted into level 1 (good) and level 2 (better). You’re assigned one point for each article you publish in a level 1 journal and five points for a publication in a level 2 journal. 0.7 for a chapter in a book with a level 1 publisher and one point for a chapter in a book with a level 2 publisher. Five points for a whole book with a level 1 publisher and seven for a book with a level 2 publisher. This is all logged electronically, reports are generated where articles in journals not on the list don’t show up at all, departments get extra money according to how many points they’ve earned and so on and so forth.

There’s been a lot of complaining about this. I hate seeing how some of my publications simply become invisible in this system – and it seems unfair when I know that some of my invisible publications are actually the ones being cited most often.

But listen: such a system makes the rules of the game absolutely explicit. I know which journals I should publish in for my university to approve of me, and to make sure I show up as productive in those reports. And other studies of bias in peer review show that women have less publications in high impact (level 2) journals. As Young Female Scientist points out, that might be because women, perhaps, read more widely, and therefore don’t care as much about whether a journal is high impact or not. I know I haven’t cared much about it. In terms of getting ahead career-wise, I guess I should.

Maybe quantitative measuring of research production is actually a way of fighting gender bias. It makes the game rules extremely clear in advance, so women (and men of course) are completely aware of what the criteria we’ll be measured by will be. It also skips the prejudiced evaluations we all make when we assess each other. Presumably, this would help avoid other biases, as well, like those based on ethnicity, age or citizenship. Of course, there’ll still be peer-review in the journals we try to get our papers accepted to, but at that level, perhaps the double blind review is more possible than in assessing the publications and other achievements of an individual. Until they placed a screen between the musician and the evaluations in auditions for orchestra positions, far more men than women were deemed good enough. With the screen in place and the anonymity it granted, women and men were accepted in equal numbers.

WennerÂs and Wold don’t suggest quantitative measure as the solution. They argue instead that evaluations should be public, and that this openness will force fairness. That might be, if checks are run on those evaluations often enough that our consciousness of these isues changes. Pharyngula, in a piece on this discussing the exact same tendency in European research grants (about 10% of male applicants and 5% of female applicants got a grant) notes that anonymous review masks systemic abuse and bias, which can be true. In the long run, though, I think that raising our awareness of our prejudices is the only real way of changing bias – and it’s going to be very, very hard. Perhaps quantitative evaluation and open review processes can help. No doubt, both have other problems.

Our university is striving to become the nation’s best university in terms of gender equality. A draft of the new strategy plan for equality (download as a word doc) has been sent out to all the departments for comments, and on Thursday there’s a conference to discuss it before it’s finalised. I haven’t finished reading the strategy plan yet, but will be at the conference and certainly plan to comment.

Hardly anyone’s signed up for the equality conference, though. So easy to just ignore.