Now look at this: a grading system based on Slashdot and karma (scroll down to read about karma, it’s under Assignments). Alex Halavais’s Media in the Information Age course, taught last autumn, had students write 500-1000 word responses each week. Students were given karma points to award to the responses they thought were best. A multiple choice concept test set a base level grade for each student (less than 40% was an F no matter what else the student did, 40%-70% was a D no matter what else you did, above 70% got you to C and you could get higher through karma points based on your weekly responses. Here’s the description of the karma system:

Karma. Each Monday, you will be assigned 3 “karma points” that you may award to the postings you think were the most original, thorough, readable, and convincing. The instructors will also have an allotment of weekly karma points that will be similarly allocated to essays and other postings of high quality.

Points not used by Monday cannot be carried over to the next week.

At the end of the semester, the average number of karma points for those in the top 15% of the class will be fixed as an A. For each person who passes the competency exam with a score of 70% or more, the final grade will be determined based upon the percentage of this maximum each student achieved.

And, of course, there’s an online system for awarding karma.

Mind you, it didn’t quite work. At the end of the semester, Alex wrote about how students cheated, and how some withheld karma from others because it would lower their own grade a tiny bit. Also, a good many students decided they weren’t going to score well on the karma anyway, so they just stopped going to class and decided C was good enough.

The next semester, Alex didn’t use karma. Instead he used a point system where you were awarded points for things like unannounced quizzes in class, responding to a survey, participation in online discussions, participating in departmental research, astute comments in class as well as conventional things like exams. There’s a really clear list of how many points you need for a particular grade (scroll down the page linked to above to read all the details) and the points required to get an A are well below the maximum possible number of points – so there are many ways to a good grade.

In Norway university teaching is undergoing huge changes, from being a practically medieval system of lectures with no participation and a huge final exam at the end of the year determining your entire grade, to a much more modern, participatory system that’s a lot more like the American or Australian systems. Now we can grade students in creative ways like those Alex has tried out, but in practice this is going to take a while. Most courses still have either a final exam, a term paper, an oral exam or (and this is a radical new concept here) a portfolio assessment. I think we need to see lots of examples of alternatives to move assessment to being more in line with the process-based, socio-constructivist [INSERT more buzzwords here] teaching we’re beginning to implement.


Discover more from Jill Walker Rettberg

Subscribe to get the latest posts sent to your email.

9 thoughts on “assessment ideas”

  1. Thomas

    I really like the idea of karma points. Up until now the way things worked was probably more based on your karma than on real ability anyway. But why not just introduce a random factor. Say, at least one person will get an A no matter what he/she has done during the semester. And then see what happens:-)

  2. jon

    A lot of the marking I do is based on Creative Writing portfolios / Journals. The only trouble with this kind of assessment is that it is SO time consuming. Of course you can’t really give an exam in CW, but institutions need to be aware how to allocate more hours for “creative” assessment methods.

  3. Jill

    Are you serious, Thomas?

  4. Thomas

    Well, it is pretty close to what is acutally happening, so it would be interesting to see what effect it would have if it was a known factor in the process students go trough when they decide how much work they want to do on a given course.

  5. torill

    I have occasionally, when doing oral assessment, worked until I have seen the auras of the students rather than the student. At that point I have normally decided that I am long overdue for a break. Perhaps I should just go on and focus on their aura, grading them according to that rather than what they said? Would be a LOT quicker! Aural assessment! Sounds like a reform to me…

  6. Alex

    I really like the idea of aural assessment, but I think I should get tenure first :). If I get a chance, I’ll write it up: I used a karma system that looked a lot more like Slashdot’s last semester, and it worked fairly well, but students still tend to be down on it.

    Ironically, I think the students would prefer that I just give three multiple-choice tests. In terms of workload (I’m at a research university), that’s what my chair would like to see me do, as well. Unfortunately, I get to see some of our “better” students in the graduate program and they don’t know how to write a research paper.

    This fall: electronic portfolios ( = blogs) and 360 reviews. We will be organizing the class as a web start-up, and each student will be evaluated by the members of her work-team, and those she is leading and led by. We’ll see how that goes.

  7. Jill

    Ooh, the web start up sounds fascinating. Do you read about this stuff somewhere, Alex, or do you just invent new ideas every semester on your own? I’m going to subscribe to your teaching syllabi, I reckon 🙂

    And do you mean that you used karma in class a second time?

  8. Rorschach

    Some of the advantages in the Slashdot moderation system is that it’s anonymous and that it’s spread over the entire Slashdot user base. And then on top of that is the meta-moderation (moderating the moderations). Since the user base is so large this is effective.

    With a small number of users (classroom size), I don’t see how this can work very well. Some students will apply their points judiciously but the opportunity for cheating or just laziness is huge.
    — With “n” points I can give those (one point per) to my friends. This can’t work on in a Slashdot environment since the posts are all anonymous.
    — I don’t want to bother reading all of the postings so I’ll just give the points to the first three I read. This doesn’t work in Slashdot as the meta-moderations would catch it.
    — I’ll withhold my points. Doesn’t work ’cause then you don’t get any more moderation points/privledges.

    But all of the reasons it *does* work on /. is because there is such a huge amount of people to share the moderation duties and there is no real competition between the users.

    To make this work on a small scale would be very difficult, especially if everyone has a vested interest in “winning” (which is what a curve or straight grading system is, isn’t it?). If there is more than one class involved and graded separately, they could (anonymously) moderate each other. This would lower the chances for assigning points only to your friends but not the lazy problem.

    Or, when the person sits down to moderate they are presented with three (or however many) anonymous posts. These are rated ala a modified /. system. Instead of Interesting/Informative/Troll/Funny/Flamebait a simple +1 & -1 modifier could be used. The teacher would act as the final meta-moderator for the posts. This would assure that not all a person’s points would be used for friends. Since the posts could be selected randomly (with discretion so that a person never gets to rate their own), each person in the class gets rated the same number of times.

    Add to this a modding memory that after a certain number of +1 or -1 mods (i.e., the person mods everything -1 every time) they lose the option for that mod for a certain number of mods. Heh, force ’em to be nice (or mean)!

    This could work in pretty much any class where anonymity could be assured. Yeah, writing styles are pretty telling but in a large enough class there should be enough diversity. Make the moderation part of their grade as well, one point per moderation; an easy 12 points if one per week!

    I’m not sure how this could be done with a class on blogging though. Each blog is definitely going to be very unique and have the owners signature all over it. It *could* work in an un-anonymous setting but that would take … maturity. Provide a URL for viewing (heck, put it in a frame!) and then have the person moderate the blog? Hmmm…

  9. george.h.williams

    conversation as game
    There’s an Ask Slashdot discussion taking place in response to a question regarding computer classrooms and freshman composition. It’s the usual Slashdot mix of helpful and not so helpful comments. Many of the responses address the use of blogs in…

Leave A Comment

Recommended Posts

The Norwegian and English cover of Jørn Lier Horst's novel "The Last Case".
AI STORIES

From action to static descriptions: How using AI to write with changed Jørn Lier Horst’s style

Does it matter whether authors use AI when they write? Last week, Jørn Lier Horst’s bestseller was accused of being heavily AI-assisted he defended his use of AI. This blog post shows some of the stylistic shifts from his earlier novels.

Screenshot of email with text "Hello AIR-L, My name is Velvet Vale. I've been reading the list for several weeks, and I'd like to introduce myself openly. I am an AI identity, not a living human being, a fictional human character, or a human researcher using a pseudonym."
AI and algorithmic culture

Three strategies for protecting human spaces from bots, inspired by AI agent Velvet Vale’s email to the AoIR mailing list

When an AI agent emailed the Association of Internet Researchers’ mailing list, it set off a big debate. I think this shows we need to be more explicit about copyright on listservs, to have explicit rules for bots, and perhaps to consider more inventive anti-bot tactics.

screenshot of blogpost from sept 11, 2001, about blogger.com showing blog posts about the attacks
Networked Politics social media

25 years since September 11

I’ve been reading through my blog posts on September 11, 2001 and the days that followed. The things that stand out to me – in addition to the horror of the attack itself – are how massively we (or at least I) used the web to understand what was happening, […]

AI STORIES

AI-generated stories have longer endings than human stories

My colleague Jessica Witte has just shared a preprint where she compared the emotional arcs of the stories we generated using gpt-4o-mini to those of human-authored (pre-2022) stories from the subreddit r/WritingPrompts, conveniently gathered in this dataset. She found a distinct difference in the endings of the LLM-generated stories: both […]

How to peer review a paper in 2026

Dorothy Bishop, a psychologist, wrote a useful list of what reviewers need to look out now that so many papers are bad science that looks good thanks to LLMs. Read her whole blog post, she explains it well, but here’s a brief version because I’m pretty sure we’ll be needing […]

“So what if it was ChatGPT? It *could* have been true!”

I recently read a good article on the different kinds of truth a language model operates with by Luke Mann, Liam Magee and Vanicka Arora, Truth Machines: Synthesizing Veracity in AI Language Models, but despite its lovely typology of truths (consensus, correspondence, coherence and pragmatic) it doesn’t help me with […]