A few weeks ago Meta released Galactica, a language model that generates scientific papers based on a prompt you type in. They put it online and invited people to try it out, but had to remove it after just three days after people generated convincing but utterly false “papers”. You can still download the code though. According to the paper they published about the model, even the biggest version of the model can run on an NVIDIA A100, a processor that costs 132,000 NOK, so not as utterly out of reach as foundation models were thought to be by academia.

I tested it out while it was online. The tweet I saw announcing was from Yann LeCun, Chief AI Scientist at Meta, who wrote that it “won’t write papers automatically for you, but it will greatly reduce your cognitive load while you write them.”

Trying it out on the kinds of research questions I tend to wonder about does not make me think it’ll be helpful in my writing process. It comes up with falsehoods (“feminist hypertext critics have mostly focused on plot”), provides no references, and is full of clichés. After I read the paper, this makes sense since it is not trained on humanities research at all. It tried to answer though.

The lack of references is startling, although I see from Yann LeCun’s tweet he does claim there are references, so perhaps it is just missing from the demo site. Or perhaps it just couldn’t do references for humanities research. Without references, it is not a research paper, no matter how many papers it has been trained on.

The clichéd structure and writing style is interesting. I tried the prompt “Summarise research on machine vision from feminist posthumanist perspective”, and while it didn’t come up with anything useful, it did replicate the sort of general statement about how important this would be that I and many others have made to the point that they are clichéd.

We will identify relevant literature and analyse it thematically. We will use these themes to develop a set of recommendations for how to make machine vision more equitable, transparent and just. We will then write a paper to summarise our findings.

It suggests a systematic literature review, which is not a methodology I have ever seen used by feminist posthumanists. But yeah, those recommendations for “equitable, transparent and just” AI? I’ve promised them, too. They are simultaneously a dime a dozen and the holy grail of AI.

The “research paper” also gets a bit repetitive.

We will conduct a systematic literature review using the PRISMA guidelines.  We will search academic databases for relevant literature and analyse the literature thematically. We will use these themes to develop a set of recommendations for how to make machine vision more equitable, transparent and just. We will then write a paper to summarise our findings.

The “Potential Challenges” section is particularly helpless.

Potential Challenges
We expect to find a large amount of literature on the harms of machine vision. However, it may be challenging to find literature on how these harms relate to feminist and posthumanist theories. We will need to be sensitive to this potential challenge.

I suppose this isn’t wrong, it’s just that this is not how a human researcher would address the challenge. Well, perhaps an undergrad would, and this is the sort of thing a new MA student might write, but research degrees train you to develop connections and knowledge that isn’t already described by others.

Unsurprisingly, a lot of people criticised Galactica quite fast. Michael Black is the director of Max Planck Institute for Intelligent Systems, and he was not impressed:

And yet, I don’t think this is without promise. Yes, it’s dangerous the way it hallucinates facts and makes up references. So do all the large language models. Their glib command of cliches and writing structures makes them seem all the more convincing, if rather bland. You get so bored you glaze over and might well not check whether it’s actually true.

But if the point of Galactica is to summarise existing research papers that’s could be really helpful, although useless without references. Sites like Elicit already do this, though not well for the humanities. They also present it as a way of finding research, not generating it.

AI has been reading our papers for a good while – I wrote a blog post about how to write for machine readers and I was only half joking. We do need to be aware of how our words will be read, interpreted, processed. At the same time, once your words are out, they’re no longer in your control. If they were ever in your control. If they were ever your words.

After all, we’re all large language models in a sense, trained on all the words and sentences we’ve ever heard or read. AI just skips the subjectivity, anxiety and self-doubt.

(I generated the top image using DALL-E.)