My individual paper for IR15, the Association of Internet Researchers conference this year, was rejected. I’ll still be going, because I’m keen to join in all the selfie discussions we’re bound to have, because there’ll be lots of interesting papers, and (this is important for my getting funding to go to Bangkok!) because our workshop was accepted. Hooray! It’s going to be amazing, bunch of fantastic people working together on selfies with some really fun ideas – read the description here. I’ll post more about how to sign up for the online portion soon!

So I’m not too upset about the rejection, but seeing all the posts on Facebook from people complaining about random reviews from people not in one’s field I was thinking it would be useful if the review process was a lot more open. So I’m posting my proposal and the reviews I received. Maybe others will do so too, whether they were accepted or rejected – I’d love more openness about reviewing in general.

Re-reading my proposal, I must say it’s horrifically short and really doesn’t show much of the work or analysis I was trying to cram into it. I can see how it might receive poor reviews (although I think at least my middle reviewer probably recommended it be accepted – I’m going with that optimistic theory, anyway 😉 You’re supposed to submit 1200 words, which is far too long for an abstract but far too short for a paper. I wish they would simply ask for full papers instead.

I would really like to see the scores my reviewers gave me. Here, for instance, are the scores I assigned to one of the papers I reviewed – why not share these with the author?

The IR15 conferences are very cross-disciplinary, which is a strength, but also a challenge, especially for reviewing. I know the current committee has put a lot of thought into reviewing, and it must be an enormously difficult task to manage, with (I assume?) several hundred proposals to assess. Last year there was a lot of discussion on the AoIR mailing list about reviewing that is able to see different kinds of disciplines. It’s not simple to achieve.

I signed up as a reviewer myself this year, and was disappointed that the two papers I was assigned really weren’t in the humanities or on topics or using methodologies I am very familiar with. I think the problem was that I ticked the “Digital Humanities” box when I selected topics and methodologies I was able to review, and the papers I received were both tagged as digital humanities, but really weren’t. One did some image analysis (I felt confident reviewing that) and the other statistics. Of course I added notes to the committee on my (lack of) expertise in the field, and there’s also a number from 1-10 you can assign to your own familiarity with the topic. Perhaps familiarity with the method would be even more relevant.

I would love to hear from the conference organizers what the average number was for reviewers'”familiarity with topic”. That would be an interesting measure of success in assigning reviewers to papers they were competent to review. Another option would be tracks in the conference.

There are journals that have open peer review. Is there any reason a conference couldn’t do this? A problem might be that people would review familiar names more than the actual research, but you could alleviate this, at least to some extent, by have a quota for PhD students and even a quota for early-career researchers beyond the PhD. Or maybe this all gets too complicated – organizing a conference is a lot of work, and standard, blind peer review can easily be run using existing platforms.

Here’s my proposal: “Cultivate a good life and record it”: Self-Improvement Narratives in Selfies, Scrapbooks and Domestic Blogs. It is not an ethnographic piece, though two of the reviewers want more information about the women who took the course. I’m interested in the texts – primarily emails sent out to subscribers to the courses. And because I know that the AoIR has a reputation for excluding humanities approaches, I tried to explain that – but obviously not well enough. I wrote:

Here are the reviews:

Review 1
========

Comments for the authors
————————
This is a really interesting topic but I would have liked more explanation of the authors’ own approach and methodology – did they sign up for these courses themselves?  Did they approach any users of the services?  Did they speak to anyone behind them?  How popular are the two sites discussed?  Do they have many users?  Why were they chosen?  

The author makes statements such as ‘For many women these courses’ clear breaks with the conventions of only presenting the perfect life rather than self-portraits or images of housework is no doubt important’ but this seems like speculation – I don’t see evidence they have actually spoken to the women involved within this abstract.  I would also like to know why the emphasis is on ‘mothers’ specifically, and if we are dealing with a particular group of women (e.g. nationality/age/sexual orientation/ethnicity) – either in terms of actual user base or in terms of who the sites are hoping to attract.

Although the examples mentioned could be potentially interesting, I would have liked more discussion of why they are significant and how this paper would differ from other literature on blogging, particularly the growing body of work (mainly US-centric) on ‘mommy blogs’ and similar practices.

-=-=-=-=-=-=-=-=-=-=-


Review 2
========

Comments for the authors
————————
This is a nicely conceptualized study that considers the hot topic of “selfies” in relation to online self-improvement courses specifically aimed at (privileged white U.S.?) women.  As my parenthetical comment notes, the author needs to locate the subjects of this study in relation to privilege, ethnicity, and nationhood even if in this case the author is particularly interested in the ways in which subjects utilize these self-improvement courses to overcome a sense of their own powerlessness and invisibility.  

Ethnicity, race, nationality and economics are present here even if (or perhaps especially because of) their absence in the observed courses and self-presentations.  Such critiques will be especially important in the context of Bangkok.  This promises to be an interesting study and one that will garner attention due to the fact that it relates to a “hot topic.” 

-=-=-=-=-=-=-=-=-=-=-


Review 3
========

Comments for the authors
————————
The cases are interesting, but would have liked to have seen the author weave together a more coherent theoretical grounding on which to base a study of these two courses – at the moment, this proposal lacks a deep clarity of argument that would mark a strong proposal.

-=-=-=-=-=-=-=-=-=-=-