Operational Hoaxes

It’s easy to discard fakes and hoaxes as misinformation. But that might be too abrupt an ending to a discussion about the aesthetics of non-fact and the role of synthetic images in our visual landscape. In this episode, we talk to Martyna Marciniak and Jussi Parikka about synthetic images and their relationship to the material realities that (re-)/produce.

Resources:

The Eye of the Master by Matteo Pasquinelli
Doppelganger by Naomi Klein
Material Witness by Susan Schuppli
Into the White by Chris P. Heuer
Operational Images by Jussi Parikka

Host & Producer: Ana-Maria Carabelea
Editing: Ana-Maria Carabelea
Music: Karl Julian Schmidinger

The Digital Deal Podcast is part of European Digital Deal, a project co-funded by Creative Europe and the Austrian Federal Ministry for Arts, Culture, the Civil Service and Sport. Views and opinions expressed in this podcast are those of the host and guests only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor the European Education and Culture Executive Agency (EACEA) can be held responsible for them.

Transcript

Ana-Maria Carabelea: Welcome to The Digital Deal Podcast, the series where we talk about how new technologies reshape our democracies and how artists and critical thinkers can help us make sense of these changes. My name is Ana Carabelea and today I'm joined by Martyna Marciniak and Jussi Parikka. 
Martyna Marciniak is a Polish, Berlin-based artist and researcher. Her work explores spatial storytelling, speculative fictions, and 3D reconstruction to investigate cases of systemic violence and human rights abuses and question the role of technology in perpetuating or undoing existing biases and misconceptions. 
She has worked with media outlets including CNN and the BBC, as well as NGO’s including Forensic Architecture, Amnesty International, and Human Rights Watch. The research group Border Emergency Collective, which she co-established, investigated and documented stories of migrating people at the Polish-Belarusian border. Her artworks were exhibited at the Warsaw Biennale, Kinema Icon in Bucharest, Haus Gropius in Dessau, and deTour Festival in Hong Kong, among others. 
Martyna is also currently an artist in residence with Ars Electronica in the European Digital Deal project. 
Dr. Jussi Parikka is Professor in Digital Aesthetics and Culture at Aarhus University in Denmark where he leads the Digital Aesthetics Research Centre (DARC). He is also a visiting professor at Winchester School of Art and at FAMU at the Academy of Performing Arts in Prague. In 2021 he was elected as member of Academia Europaea. His published books include Insect Media (2010), Digital Contagions (2007/2016), A Geology of Media (2015), and A Slow, Contemporary Violence (2016). Recently, he co-edited Photography Off the Scale (2021) and is the co-author of The Lab Book: Situated Practices in Media Studies (2022). His most recent book Operational Images was published in May 2023. He has also worked as a curator, including as part of the curatorial team of transmediale 2023, as well as the Helsinki Biennial and Motores del Clima at Laboral, in the same year. Welcome to both of you. It’s a pleasure to have you.   

Martyna Marciniak: Thank you.  

Jussi Parikka: It really is. 

Ana-Maria Carabelea: I was hoping we could talk about visual culture in general, and, more specifically, about the different kinds of images that we're more and more confronted with these days and that make up visual culture. I think you're both working with very different types of images and can bring in very different viewpoints that can kind of converge to create a fuller picture of the relationship between images, cognition, modes of perception, as well as maybe, if it's not too ambitious for a very short episode, the operations behind the images, so the mechanisms that make them possible. And I'd like to start by asking you to map today's visual culture, and also go a bit into how that has developed over the past few decades. 

Martyna Marciniak: Right. So, I can start by introducing what is my access point to this topic. My focus is definitely on the aesthetic of evidence. I always start from the same anchor point, let's say, the period around 2010-2011 - the kind of early days of the so-called open-source visual investigation, which was enabled by the introduction of smartphone photography and social media sharing. This point in recent history is something that I always relate to in my research into synthetic images – AI-generated images, CGI images, and so forth. But I also look at this period in relation to histories of media, to inheritances of military aesthetics, forensic aesthetics, and so forth. In this quite recent framing, there has already been quite a large transformation in the last ten years. I think the investigation's role has reconditioned our relation to images that otherwise would be seen as content. In the position of investigator, you have the privilege to interact with the image at a much slower pace than you do as a general audience on social media, or audiences of establishment media. In that relationship to the image [the investigator’s], the image becomes a source of data. But also, the relationship is very active. It's a relationship that is really shaped by scaling, measuring, translating, copying, and reconstructing. And what we've seen is that as much as ten years ago, the problem was finding the right image, and now the problem is the over-saturation of images. So, the role of the investigator is now also largely sifting through large volumes of images and prioritizing them. And then as audiences, we are faced with constant choices between the images we see, a constant evaluation task of whether something is trustworthy or not, and we don't have this privilege of time that is allowed for the investigative eye. So, the focus for me is trying to step between these two positions of an investigator and a member of the audience. And within that, I'm also really interested in how we are framing narratives around the problem of misinformation, how establishment media is framing these problems, how actually there is a lot of misconceptions that are made even more problematic by the seeming visual advancements. While really, as audiences and as members of this visual culture, we still haven't really reconciled the biases and problems of the photographic image. So, this is sort of the general point of interest for me. 

Jussi Parikka: Great. There are so many things that I could pick up on. Martyna, you really nailed so many points that I had in mind as well. I think you nailed so many discussions that have been used to describe transformations of visual culture. So, I want to echo you a bit. On the one hand, it's so much about the speed of images, the temporality of images, that doesn't always come intuitively to people who think that images look like something. They [images] also have a speed at which they regulate the ways in which we look at them or the ways in which we have time to look at them, regulate what we consider images. And then the sheer quantity, which is a theme that pops up in all sorts of contexts - from art history and the demands that art history needs to become computational because there's so many images that we need to go through, to the sheer quantity and what it does as part of discussions of the spectacle. The sort of parameters around the image have become, in a way, defining of what the image - if there is such a thing as the image - images are, what images do, and how images operate across multiple contexts, much beyond entertainment, and much beyond the usual way in which we consume images. I just basically jumped directly into what you already catered for because it was so nicely put. My own interest and my own trajectory to images is not really through images. I wasn't trained as somebody who's a specialist in images or visual culture. I slip into it, whether accidentally or quasi-accidentally. I worked my way through media theoretical questions and methods and all kinds of questions around the history of technology. Approximately five years ago, I ended up working with images, leading a project at FAMU in Prague on operational images. And it's one of those [things], that when you are in a project and leading a project on operational images, you start to see operational images everywhere. In other words, it basically becomes the way in which I describe most of digital aesthetics. It is because of my interest having been filtered through the notion of operationality and the ways in which a particular kind of image - which is not really a genre of images it's almost like usability of images in digital or data cultures - comes about, namely, through operations and instruments. That is basically how I've been looking at a lot of transformations in imaging practices. I know that this resonates with your [Martyna’s] practice as well. Not that this would summarize everything you do, far from it, but still, there's a resonance. The idea that, like many others have said at the back of Farun Farocki, Hito Steyerl, Trevor Paglen, and many, many others as well, that there's an operationality that is almost prior to the idea that images look like anything, that there's something about non-representational uses of images. That's a very short summary of very big things. 

Ana-Maria Carabelea: Thank you so much. That's really interesting. And I think it also leaves us on a very good note to bring in Martyna's most recent work that's been exhibited at this year's Ars Electronica Festival. The first chapter of it has already been exhibited, and we're very much looking forward to the next two chapters. And I wanted to ask you, Martyna, to tell us a bit about the images that you're working with. 

Martyna Marciniak: Sure. So, the whole project, Anatomy of Non-fact, is an ongoing project and it focuses on three cases of misinformation with synthetic images. The first one of those is the AI-generated, or supposedly AI-generated image of the Balenciaga Pope. The second one is the fake explosion at the Pentagon that circulated on Twitter in May of 2023. And the third case is a grouping of images/screenshots from the war simulation game Arma 3, which have been used to misinform about different armed conflicts, including in Ukraine and Palestine. The structure of the project is based around this very large-scale mapping, where each of these core case studies is expanded with additional images. These images either contextualize the [main] image within the history of media, or talk about the provenance of the image, or get deeper into analysis in some cases. And then the project has this layer of artistic responses, as I call them, one artistic response per case. The first one, as you mentioned, has been shown this year at the Ars Electronica Festival and it focuses on the Balenciaga Pope image. It's titled AI Hyperrealism. It is a bit tongue in cheek, there is absolutely no AI employed in the work because it is very intentional. The work talks about this particular image in the context of the history of photography. My approach was to reconstruct the whole image, including the very bizarrely constructed, glitchy, full of errors code, and allow it to be animated by a pope impersonator. In the video, we have the Balenciaga Pope, who delivers a monologue about the history of photographic image, and the history of the portrayal of The Pope as a figure of authority of truth. There is also an aspect referring to the history of symbolism of the Catholic Church and its influence on the Western aesthetics of evidence, of fact. Throughout the video, I also weave in a short lexicon that summarizes terminology that I thought should be introduced to the topic of misinformation and interaction with pseudo-evidentiary images. This lexicon also attempts to slightly undo the nomenclature we have attached to AI-generated images and the aesthetics that emerge from AI-generated images. The other two projects will be, as you said, continued in the coming months, and they're quite exciting, but very different in character. The image of the explosion of the Pentagon has led me to a more investigative way of working, and it's proving quite exciting and interesting, but very, very different. 

Ana-Maria Carabelea: That's great. I think I want to pick up on one of the points, which is that you treat these images as visual objects and you kind of deconstruct them, and you create the artifacts. I want to kind of bring Jussi in here to give his two cents. Jussi, do you think that these images can sort of expand your definition of operational images? Can these synthetic images also operate? And if so, in what ways? How do you see that? 

Jussi Parikka: Shall I jump in here? I mean, the description we just got is really great. I had a peek of the project as well, so I'm a lucky one. I think there are so many ways in which if we unfold from what you said you’ll be investigating/you're looking at - especially now, in terms of the Pentagon image – the material effects, or what I would call operations [material effects] that images might have. Fake or not. And that there's something that cuts through the usual categories of fake and not fake or truth and not truth by way of operations: that they have a material impact, whether they're fake or not. There's an increasing way in which we don't only program images (computational images) but images program worlds around them, by way of accidental or intentional or other kinds of uses. There's something interesting that you tease out, and there are many interesting things that you tease out in your work through these topics. I think it's - pun intended - well embodied in the Pope project, there are - I don't know, do people still talk about post-digital things? Well, I do - post-digital ways in which the image itself becomes embodied in material worlds, becomes performed, and itself as a piece really beautifully problematizes the idea of fakery that becomes materially embodied. There's sort of ways in which the fluidity of these images and "real worlds” is what defines them as well. I want to say that there are two things. I'm intrigued by how well these projects also tell the story of synthetic images and misinformation being particular kinds of triggers in the world - trigger word intended as part of the ways in which perhaps weaponization works by way of information. Then, it would be that the image itself is perhaps less interesting than the ways in which we ended up in the image, the production of it by way of computational or non-computational processes, and what happens in relation to the circulation of such images and how those are guided in the world. This is not meant as sort of an explanation. It's more like riffing with the themes that your work caters. Do you want to pick up on this, Martyna?  

Martyna Marciniak: Yeah, I think especially with this fake Pentagon image, you're absolutely spot on with how it conditions or creates realities around it. When I was reading your book, Operational Images, you mentioned the category of an inconsequential image. I think that there is something in the aesthetic of inconsequential image that is very much present in this fake Pentagon explosion. It can be undetected and go under the radar, it's a very unassuming aesthetic, which is something we've been absolutely conditioned to write. So, we've been conditioned through it as audiences and also as generators of content. I mean, personally, on my phone, I just have images that are just there as data, just in case. I think we also kind of populate our social media with these inconsequential images. That's a really fascinating aspect for me. It really unlocked something for me when looking at this particular image. This paired with how much impact this image had: it's supposed to have moved the stock exchange market by billions of dollars within hours, most likely because of how this image was read by trading algorithms, not by human eyes. So, there is something that becomes obscured here, where we like to say, or the establishment media really likes to repeat, that it is the technical capacity of reproduction of reality of the image that is the threat to our reality and to the kind of future of truth and future of evidence. But actually, we're seeing that a very, very poor-quality image, a very mundane image, had capacity to speak to the truth as perceived by a trading algorithm. So, yes, if we sort of start to also look at every image as an operational one, then the realities that can be constructed can be vastly different from what we expect. 

Jussi Parikka: Yeah, that's true. Can I just pick up on that as well? Because it goes both ways as well. It's not just that now data can be visualized, and images are made of data, but any image always could have been data. Especially seen through the current ways in which machine readability of images comes into play, starts to trigger these weird cascading loops. I'm fascinated by the ways in which images can do such things. Still not great always, but at least impactful. 

Ana-Maria Carabelea: That's very interesting. My next question was going to be about the relationship images have to truth. And I think that from what you both said now, this relationship [seems to be] going in two directions: the image either has a relationship to a material reality that it represents, but it now also seems to have a relationship to a material reality that it produces, as the example of the Pentagon image. I'd be interested to hear from you, Martyna because you went from investigating the aesthetic of evidence and looking for evidence in images to looking at images that have evidence of fake in them. I'd be interested to see what connects the two because I think there is a connection. It's not just accidental or, you didn't go against what you were doing before? 

Martyna Marciniak: No. I mean, part of the reason to start investigating this aesthetic of fact or non-fact - which are basically two sides of the same coin - was that I didn't want to focus on images that were real, that have real-life consequences. I thought it would be, at minimum, disrespectful and, in some cases, immoral to do that, to dissect something that is connected to lived realities just in terms of aesthetics. But also what is really fascinating is that hoaxes in themselves really reveal in an almost heightened and exaggerated way the aspects that we have learned to read as evidentiary. In some way, the hoaxes bring elements from the background of an image, or ones that are very hard to put your finger on or articulate [...], to the foreground. And in that way, they are the perfect material to work with. They are an expression of not only individual perception but very much something that is rooted in a wider visual culture. That's why I started this process of excavating images behind the image. And that led me to real-life examples, of course, and also to pop culture examples that really do shape our expectation of both reality and depictions of reality. In some way, I really love the process. And I think that it's important to treat the fake with the same seriousness as we treat the "serious evidence”. It offers a little bit more of a positive approach. Usually, the discussion ends when we decide if something's fake or real but analyzing how it's fake and how it is connected to the real starts offering more of an insight and more of a way forward, maybe also can offer a way towards what we like to call visual literacy. Perhaps we're just always worried about the wrong image. And lastly, not to drag on too much, but I think that the process of analyzing the fake also unmasks the biases and lapses in the established evidentiary medium of photography. You can see this also becoming a practice that people employ now with conspiracies. Hundreds of people analyzing archival photographs and question the depictions of photography and trying to interpret them as something synthetic or trying to relate them to a synthetic image. I find it really fascinating, and I think maybe something interesting is happening in the visual culture as well. 

Jussi Parikka: I mean, it's a version of data apophenia as well. Like many others, have also pointed out that over-investment in a particular way in which an image might reveal something is a persistent feature, but also overly present in certain cases as well. All kinds of things that are not just about the image indeed any more, it's about belief structures and all kinds of various ins and outs of why we look at a particular image in terms of its significance. Whether the data apophenia is really apophenia in terms of data – where we see all kinds of patterns - or whether it's just this sort of over-investment in the idea that there's always something symbolic, there's always something to be discovered, there's always something to be interpreted. At the end of the day, there's a multitude of images that just don't have any meaning. That's my big take on the conspiracy theories, that there's just a massive amount of images with absolutely zero meaning in them as well. It doesn't mean that we don't project meaning into them, but it's curious. 

Ana-Maria Carabelea: I was going to end with the discussion about perception and how the different visual regimes form or reform/reshape our habitual modes of perception, but also how - speaking about data apophenia - modes of perception also colour the way that we look at certain images. I was wondering if you could comment on that. 

Jussi Parikka: True. I mean, there is already a way in which that reads beyond a representational account, at least in symbolic terms. It’s not just that we are training images or models or perception of images. People are also one form of a cognitive mechanism based on training that we crystallize in the brain. Not that the brain is only located inside our heads, but this expanded repertoire through which we perceive patterns, we see particular kinds of meaningful and non-meaningful elements as well. And there's something interesting in terms of when the switch from pattern perception becomes a look at all kinds of hidden meanings, whereas everything might be already on the surface as such as well. I'm very interested in this because basically - already in terms of brain size and cognition and such - it already hints at the massively biggest infrastructure or scaffolding at the back of any kind of pattern perception as you put it in your question as well. I'm curious exactly [about] taking the step back from looking at images as images to looking at what it is and how we come about to see a particular kind of a pattern. Whether the “we” in question is a particular human collective or whether it's a particular kind of a machinic constellation that is being trained to see in a particular way - that relation between both human cognition and non-human cognition and the ways in which these might be dancing together or stumbling together is fascinating. Over to you, Martyna. 

Martyna Marciniak: When I was researching for the project, I got slightly obsessed with a topic I was introduced to before - the history of theories of vision. I went down this rabbit hole into how these theories of vision shape how we depict and how we see, but also something that I find quite beautiful, but it's still quite under-articulated. Essentially, when I was thinking about these synthetic ways of capturing or generating images, if you were to explain how they work, they lean themselves very easily to the emission theory of vision in the case of computer-generated images, or neural radiance fields in some way describe the atomist theory of vision. And I was really interested in these non-photographic or pre-optical theories of vision and how they speak to our modes of perception. Because of course, the apparatus of seeing is the same across hundreds of years, but [while] there is a similarity in the framework and there's also these vast differences. And somehow the digital medium or the synthetic image can bring about in some bizarre, twisted way theories of vision that we have actually discounted. 

Jussi Parikka: I like the dip into the history of science that you just did there as well. We might be running tight on time, otherwise we would go on a long excursion into the history of science. That would be lovely. 

Ana-Maria Carabelea: I think that would be amazing and really interesting. And I'm really sad I have to put an end to it. But let's leave our listeners with some kind of homework. I usually do this at the end, I invite my guests to give listeners a recommendation of a book, a research piece, or something else that they can look into in case they want to dig deeper into this topic. Who wants to go first? 

Martyna Marciniak: When researching for this work, I have really enjoyed reading The Eye of the Master by Matteo Pasquinelli because it undoes the generalizations we hold around AI-generated images in a really straightforward way, I find. But I also really enjoyed reading Naomi Klein's Doppelganger because again, she does a very good job of presenting truth and fake as two sides of something that needs to come together to form a whole. And she also brings - in a really skilled way -examples from art, politics, and media. It was a really enjoyable read. In terms of artists who work on this topic, I really still enjoy artists who engage in more of an event art and engage with parody; those would be The Yes Men, of course, and then I highly recommend the work of Spelling Mistakes Cost Lives. 

Ana-Maria Carabelea: Amazing. Thank you so much. Jussi? 

Jussi Parikka: I was about to say that academics provide this long book list, but you also did that. So, I'm allowed to say more than one? 

Ana-Maria Carabelea: Yes, please. 

Jussi Parikka: Although I'm very stuck now. I mean there’s... 

Ana-Maria Carabelea: There's too much. 

Jussi Parikka: Yeah, there's too much stuff that is amazing. I want to say, because it nicely links to our conversation with Martyna as well, that [...] the ways in which reading images through forensic practices works like a charm. For instance, Susan Schupli’s work is always something I love going back to. And obviously, Susan's film work is great. And then there's a really quirky one. This is so super weird, but it's good to mention this, just because of that. I teach and we talk, and we research, and you do art with digital images. But even with my interest in digital images, I'm constantly going back to these quirky moments in history due to my training as a historian as well. I also just finished a project with Abelardo Gil-Fournier - not the book that came out, but a new video piece which is about snow and light, so part of that research was looking into really interesting cultural history and art history of images like Chris Heuer's Into the White: The Renaissance Arctic and the End of the Image. Again, radically pre-digital images, but the question of whiteness - not just colonial whiteness, but perhaps a bit of that as well - in terms of the image problem that a lot of Arctic expeditions faced when going into the Arctic, where there was an abundance of light and you don't anymore recognize what is an image because you've been trained in a particular way. Obviously, this was only an issue for the ones who traveled there, not for any indigenous populations, but they didn't know how to deal with this. It's a fascinating book, and this moment in Renaissance scientific expeditions could be useful to think about our images and not really knowing anymore what an image is and at what point we have lost track. Do we know what images are? They do things to us, for sure. 

Ana-Maria Carabelea: That's an absolutely amazing ending. It was such a pleasure to have you both. Thank you so much for joining me today. 

Martyna Marciniak: Thank you. 

Jussi Parikka: Thank you so much. 

Ana-Maria Carabelea: That's it for today. Thank you so much for tuning in, and I hope you enjoyed it. The Digital Deal Podcast is part of the European Digital Deal, a three-year project co-funded by Creative Europe. If you want to find out more about the project, check out our website, www.ars.electronica.art/eudigitaldeal.


Other Resources:

Operational Images. From the Visual to the Invisual by Jussi Parikka

Photography Off the Scale, Edited by Tomáš Dvořák, Jussi Parikka

Living Surfaces by Abelardo Gil-Fournier and Jussi Parikka

Seed, Image, Ground - Video by Abelardo Gil-Fournier in collaboration with Jussi Parikka

Anatomy of Non-Fact - Artwork by Martyna Marciniak 

Anatomy of Non-Fact - Video Essay by Martyna Marciniak