Synthetic media is a current popular term for AI-generated videos, texts and images. I think the first use was only a few years ago in 2018, but I couldn’t find an overview of its use so thought I’d cobble one together here, mostly because I like Elena Pilipets and her colleagues’ term synthetic imaginaries (2024), and my colleague Gabriele de Seta and his colleagues’ term synthetic ethnography (2024), and I want to know the background before I use these terms.

It was at this week’s (wonderful!) workshop for the AI STORIES project I mentioned synthetic imaginaries, a concept I had come across in this report from a DMI summer school last year.

The term “synthetic imaginaries” combines the recent use of synthetic to refer to things generated by AI with the notion of imaginaries, and it refers to the ways generative AI “imagines” things. Elena Pilipets, Marloes Geboers and Riccardo Ventura write about the language model synthesising output.

The concept obviously references the more general idea of imaginaries that has become increasingly popular in the last years, and that describes collectively imagined societies and technologies – so things that could be but might not exist, like sentient AI or a dystopian surveillance state. This use of imaginary originated with the STS concept sociotechnical imaginaries (Jasanoff & Kim 2009) and with many extensions, like surveillance imaginaries (Lyon 2017), Taina Bucher’s algorithmic imaginaries (2017) and so on.

While surveillance imaginaries are how humans collectively imagine surveillance to work, synthetic imaginaries describes how a language model imagines all sorts of things. At least that is how I want to use the term.

So when I mentioned the term “synthetic imaginaries” at the workshop I explained that synthetic meant “artificial” since the AI had generated it.

“No, no,” Elena explained. Luckily I had invited her to the workshop. “By synthetic we meant that it was constructed from several different things. Put together. Altered. It is not only about the AI, but about the synthesis between humans and machines.”

I may have slightly misquoted her (sorry Elena) but I appreciated this point that synthetic does not simply mean artificial, and that for something to be AI-generated you require a lot of human input as well.

The Oxford English Dictionary explains the word to synthesise:

Synthesize: To make a synthesis of; to put together or combine into a complex whole; to make up by combination of parts or elements. Also absol. (Opposed to analyse n.)

(Oxford English Dictionary)

Perhaps a machine learning expert would not agree that a language model synthesizes parts into something new. But it is an interesting use of the word, and if we are talking about AI-generated texts or images or videos, there is an interesting precedent cited by the Oxford English Dictionary:

That Homer had no predecessors,..no well-digested body of myths to synthesize, is an absurd hypothesis.

J. A. Symonds, Studies of Greek Poets i. 9 (1873)

This idea of a poet “synthesising” a “well-digested body of myths” is similar to 20th century theories of intertextuality and of Malraux’s Musée imaginaire, all the artworks in the world that inspire each new artwork. (I write a bit about this in my recent paper “Repeating Ourselves with Generative AI“)

Since I’d like to start using the term synthetic imaginaries I thought I should check to see how synthetic media is used too.

From the 1940s on the term “synthetic media” was used for something completely unrelated to generative AI: it referred to a medium used for growing cell cultures in biological or medical research. By the 2010s online references to the word are mostly ads for synthetic dust filters. Then in 2018 “synthetic media” starts being used to refer to AI-generated content and especially deepfakes, though the dust filters are still around.

I haven’t found an earlier example of “synthetic media” been used about AI-generated content than a June 2018 workshop bringing together technologists, journalists, and researchers in machine learning, “synthetic media” and human rights to discuss deepfakes. The gathering produced a report called Mal-uses of AI-generated Synthetic Media and Deepfakes: Pragmatic Solutions Discovery Convening, June 2018: Summary of Discussions and Next Step Recommendations. The OED’s first source for the term deepfake is also from 2018, although the word deepfakes originates from a Reddit username in 2017. The words are often used together, and though I haven’t done any thorough analysis here, my sense is that the negative associations of deepfakes also adhere to the term synthetic media. They are a threat. Often when “synthetic media” is used it turns out to really refer mostly to deepfakes, although deepfakes are also defined as a subcategory of the broader (and perhaps more serious-sounding?) category synthetic media, as in this scoping review.

Since then many articles have been published using the term, but usually without much interrogation. For example, a 2023 paper in a law journal titled “A Practical Introduction to Generative AI, Synthetic Media, and the Messages Found in the Latest Medium” doesn’t mention the term more than once in the actual article, and only in passing.

A scoping review of deepfakes and synthetic media used the following search terms to find the 182 peer-reviewed articles in their corpus: ‘deepfake’, ‘GAN, ‘, synthetic media’, ‘fake media’, and ‘AI generated media’ with the co-operators ‘education’,
‘higher education’ and ‘tertiary education’. These scholars are interested in cheating but not in AI-generated art. Of course, figuring out which keywords to use is a difficult task in an environment where people are constantly coming up with new words for the same thing: AI art, generative art, LLMs, transformer models, semantic space, vector space, latent space, text generation, generative AI, genAI – what will we call it next year?

In the last few years I’ve seen several useful extensions of the term synthetic media. There is Gabriele de Seta, Matti Pohjonen and Aleksi Knuutila’s synthetic ethnography, which they define as a “qualitative research methodology applied to the study of the social and cultural contexts developing around generative artificial intelligence” and that repurposes AI models “as research tools in their own right”. (See the paper Synthetic Ethnography: Field Devices for the Qualitative Study of Generative Models)

Are there other important uses of synthetic media or synthetic as related to generative AI that I should be aware of? Please let me know!