My individual paper for IR15, the Association of Internet Researchers conference this year, was rejected. I’ll still be going, because I’m keen to join in all the selfie discussions we’re bound to have, because there’ll be lots of interesting papers, and (this is important for my getting funding to go to Bangkok!) because our workshop was accepted. Hooray! It’s going to be amazing, bunch of fantastic people working together on selfies with some really fun ideas – read the description here. I’ll post more about how to sign up for the online portion soon!

So I’m not too upset about the rejection, but seeing all the posts on Facebook from people complaining about random reviews from people not in one’s field I was thinking it would be useful if the review process was a lot more open. So I’m posting my proposal and the reviews I received. Maybe others will do so too, whether they were accepted or rejected – I’d love more openness about reviewing in general.

Re-reading my proposal, I must say it’s horrifically short and really doesn’t show much of the work or analysis I was trying to cram into it. I can see how it might receive poor reviews (although I think at least my middle reviewer probably recommended it be accepted – I’m going with that optimistic theory, anyway 😉 You’re supposed to submit 1200 words, which is far too long for an abstract but far too short for a paper. I wish they would simply ask for full papers instead.

I would really like to see the scores my reviewers gave me. Here, for instance, are the scores I assigned to one of the papers I reviewed – why not share these with the author?

IR15-review-scores

The IR15 conferences are very cross-disciplinary, which is a strength, but also a challenge, especially for reviewing. I know the current committee has put a lot of thought into reviewing, and it must be an enormously difficult task to manage, with (I assume?) several hundred proposals to assess. Last year there was a lot of discussion on the AoIR mailing list about reviewing that is able to see different kinds of disciplines. It’s not simple to achieve.

I signed up as a reviewer myself this year, and was disappointed that the two papers I was assigned really weren’t in the humanities or on topics or using methodologies I am very familiar with. I think the problem was that I ticked the “Digital Humanities” box when I selected topics and methodologies I was able to review, and the papers I received were both tagged as digital humanities, but really weren’t. One did some image analysis (I felt confident reviewing that) and the other statistics. Of course I added notes to the committee on my (lack of) expertise in the field, and there’s also a number from 1-10 you can assign to your own familiarity with the topic. Perhaps familiarity with the method would be even more relevant.

I would love to hear from the conference organizers what the average number was for reviewers'”familiarity with topic”. That would be an interesting measure of success in assigning reviewers to papers they were competent to review. Another option would be tracks in the conference.

There are journals that have open peer review. Is there any reason a conference couldn’t do this? A problem might be that people would review familiar names more than the actual research, but you could alleviate this, at least to some extent, by have a quota for PhD students and even a quota for early-career researchers beyond the PhD. Or maybe this all gets too complicated – organizing a conference is a lot of work, and standard, blind peer review can easily be run using existing platforms.

Here’s my proposal: “Cultivate a good life and record it”: Self-Improvement Narratives in Selfies, Scrapbooks and Domestic Blogs. It is not an ethnographic piece, though two of the reviewers want more information about the women who took the course. I’m interested in the texts – primarily emails sent out to subscribers to the courses. And because I know that the AoIR has a reputation for excluding humanities approaches, I tried to explain that – but obviously not well enough. I wrote:

This paper uses two online courses to examine ways in which the domestic blogosphere shapes ways in which mothers strive to care for themselves, whether with or against the mainstream. I take a humanities approach, using literary, visual and rhetorical analysis of the course material and of texts and images shared in blogs and social media, as well as drawing on critical theory and historical context to make my argument.

Next time I submit a paper to AoIR I’ll be even more explicit – I didn’t actually say that I wasn’t doing ethnography, and I probably should have. Maybe next time I’ll even have the opportunity to submit a real, full paper instead of 1200 words!

Here are the reviews:

Review 1
========

Comments for the authors
————————
This is a really interesting topic but I would have liked more explanation of the authors’ own approach and methodology – did they sign up for these courses themselves?  Did they approach any users of the services?  Did they speak to anyone behind them?  How popular are the two sites discussed?  Do they have many users?  Why were they chosen?  

The author makes statements such as ‘For many women these courses’ clear breaks with the conventions of only presenting the perfect life rather than self-portraits or images of housework is no doubt important’ but this seems like speculation – I don’t see evidence they have actually spoken to the women involved within this abstract.  I would also like to know why the emphasis is on ‘mothers’ specifically, and if we are dealing with a particular group of women (e.g. nationality/age/sexual orientation/ethnicity) – either in terms of actual user base or in terms of who the sites are hoping to attract.

Although the examples mentioned could be potentially interesting, I would have liked more discussion of why they are significant and how this paper would differ from other literature on blogging, particularly the growing body of work (mainly US-centric) on ‘mommy blogs’ and similar practices.

-=-=-=-=-=-=-=-=-=-=-


Review 2
========

Comments for the authors
————————
This is a nicely conceptualized study that considers the hot topic of “selfies” in relation to online self-improvement courses specifically aimed at (privileged white U.S.?) women.  As my parenthetical comment notes, the author needs to locate the subjects of this study in relation to privilege, ethnicity, and nationhood even if in this case the author is particularly interested in the ways in which subjects utilize these self-improvement courses to overcome a sense of their own powerlessness and invisibility.  

Ethnicity, race, nationality and economics are present here even if (or perhaps especially because of) their absence in the observed courses and self-presentations.  Such critiques will be especially important in the context of Bangkok.  This promises to be an interesting study and one that will garner attention due to the fact that it relates to a “hot topic.” 

-=-=-=-=-=-=-=-=-=-=-


Review 3
========

Comments for the authors
————————
The cases are interesting, but would have liked to have seen the author weave together a more coherent theoretical grounding on which to base a study of these two courses – at the moment, this proposal lacks a deep clarity of argument that would mark a strong proposal.

-=-=-=-=-=-=-=-=-=-=-

2 thoughts on “Proposals and Reviews for IR15, the AoIR conference this year

  1. Ben

    I really enjoyed reading the paper. It is a shame that the paper was not accepted. AOIR needs to realize that this 1200 word thing plays right into social science papers who can often give you the bare bones much easier than humanities papers. I decided not to submit this year because the reviews I got the previous year were all from social scientists who wanted a formulaic paper that conformed to their views of appropriate scholarship.

    Thank you for posting this and being so honest about it. Looking forward to the book project–AOIR is missing out on rich scholarship like yours!!

  2. Jill

    Thanks for responding, Ben. I received a couple of emails from other people who had pretty much the same experience as you – rejection with reviewers who wanted a quantitative or ethnographic paper instead of a humanities paper, and who have not submitted again. That’s a pity – if many people have that experience there’ll be fewer and fewer humanists each year.

Leave A Comment

Recommended Posts

Triple book talk: Watch James Dobson, Jussi Parikka and me discuss our 2023 books

Thanks to everyone who came to the triple book talk of three recent books on machine vision by James Dobson, Jussi Parikka and me, and thanks for excellent questions. Several people have emailed to asked if we recorded it, and yes we did! Here you go! James and Jussi’s books […]

Image on a black background of a human hand holding a graphic showing the word AI with a blue circuit board pattern inside surrounded by blurred blue and yellow dots and a concentric circular blue design.
AI and algorithmic culture Machine Vision

Four visual registers for imaginaries of machine vision

I’m thrilled to announce another publication from our European Research Council (ERC)-funded research project on Machine Vision: Gabriele de Setaand Anya Shchetvina‘s paper analysing how Chinese AI companies visually present machine vision technologies. They find that the Chinese machine vision imaginary is global, blue and competitive.  De Seta, Gabriele, and Anya Shchetvina. “Imagining Machine […]

Do people flock to talks about ChatGPT because they are scared?

Whenever I give talks about ChatGPT and LLMs, whether to ninth graders, businesses or journalists, I meet people who are hungry for information, who really want to understand this new technology. I’ve interpreted this as interest and a need to understand – but yesterday, Eirik Solheim said that every time […]