A conversation about augmented reality, magic and the consequences of global warming in Stavanger
Stavanger, March 2017
Tanya Toft Ag: For some, Augmented Reality is an unfamiliar concept; for others, it is Pokémon Go. But I would like us to expand on this, and dig into some of the more sophisticated aesthetic potentials of Augmented Reality that we find in artistic practices like that of John Cleater.
First of all, how did you come to develop an interest in Augmented Reality?
John Cleater: Augmented Reality excited me when I was first introduced to it around 2010. My background is in architecture, beginning in the late 1980s and early 1990s – the era of deconstructivism. My architectural work was very experimental. The closest thing we did to a real building site was in something published in a fancy Italian architectural magazine. A lot of what we were making was not possible to build. For many years, I was an architect in New York at an architecture office called Asymptote. We did things at the Venice Biennale, Documenta11, and various interactive installations around the world, and I was involved with designing the Guggenheim Virtual Museum in 1999. It never went public, but it was a fully interactive museum that you could change as you went through it. So I’ve been involved with visualizing impossible kinds of architectural spaces for a pretty long time.
Architecture was an experimental place from which I could experiment with space. When I was introduced to Augmented Reality, I found that it actually offered a more “possible” site to locate some of these unbuildable visions of space and different architectural experiments that I had been doing twenty years earlier.
Sometimes I refer to myself as a spatial engineer, and I have been working in different mediums, including scenic design for the stage in New York with the multi-media theatre company The Builders Association, since 1994. In fact our first piece was Henrik Ibsen’s The Master Builder for which I built Gordon Matta-Clark’s Splitting house, which falls apart throughout the performance. I think of stage sets as a kind of temporary architecture.
For me, Augmented Reality was a new kind of medium I could play with that deals with the in-between space – the one in between virtual and physical space. I have become very interested in this over the years, and more than just in an abstract way.
TTA: You mentioned your early fascination with the impossible. What is it that interests you in this concept? I guess Augmented Reality is kind of impossible – it is not tangible; it is something our perception registers. What fascinates you about that?
JC: I like illusion and magic. Not unlike George Méliès’ use of film to further his magic, I use new technologies to further mine. George Méliès was a magician from France who was alive at the turn of the twentieth century – around the time when film was invented. I have always been interested in new media that comes about, and how I can use it to create illusions and magic and maybe reveal some kind of impossible idea about how you might occupy space, what a space might be, and also how one can blur their physical relationship to the world.
TTA: The exploration of the concept of space is characteristic to your work and I know that you have some reflections about how new media interferes with this concept. Yesterday, you and I had a conversation about Augmented Reality vs. Virtual Reality, which are two different things. Augmented Reality is an overlay on reality that you see through a screen, so you have the real world present here and now, whereas Virtual Reality locks you up in the glasses and transports you to another environment – or, at least your perception. You work, however, only in Augmented Reality.
JC: Yes, I am not that interested in Virtual Reality. I am not a gamer. I feel depressed when I go inside VR or even when staring at the computer for too long. I make artworks that are in between. I’m not taking you away from your phone, but I am using it.
Augmented Reality is a way of addressing the human condition that you see when people are experiencing the world, walking around like this: [holding up his mobile phone and gazing through it]. I play with new ways of seeing the world – ways where you actually still “see” and are in the world.
Let me tell you a little bit about different kinds of Augmented Reality. There is a kind of Augmented Reality that is in relation to where the camera is and you can see the AR anywhere you load the app. This is the case with the visuals that you can see behind me on the projection screen, which is one of my Augmented Reality apps that you can download from the App Store.
Another more popular type of AR uses image recognition and triggers virtual images overlaying the physical target. For example, an app that recognizes your face and overlays a virtual mask.
Then, there is a geo-locative type of Augmented Reality, which is probably the most interesting type and something I will most likely be working with here in Stavanger. Some people might be familiar with this type of Augmented Reality, which we find in the Pokémon Go that became a craze for about a month last summer. I have previously used this technology for shows in specific locations – for example, in the middle of the woods – where the audience had to go there to experience the AR environment. The geo-locative Augmented Reality requires that you must go to Stavanger to see the artwork; you can’t just watch it on your computer at home. It is site specific.
The site specificity of Augmented Reality is interesting to me. While I am here, I will probably pick up several points of view over the harbor, and I have been speaking with the concert hall about installing a viewer there – similar to what you would put a quarter in and look at the Eiffel Tower. I might install a device there that is specific for that point of view. I will cover the whole harbor and fill it with different things.
TTA: What is the opportunity you find in the aesthetic expression of Augmented Reality? What can you do with this that you could not do if you built a physical structure or made a painting, for example?
JC: I have been involved in this particle mist system… I am interested in creating something that blends into the real world, smoothly rather than inserting something like a cut-out cartoon…
TTA: Why do you prefer the smooth integration of Augmented Reality into “real” physical space?
JC: I guess I have always been interested in that kind of blurring where you can’t really tell if it is real or not. Then it becomes this kind of magic, which is what I am interested in doing.
TTA: So, in a way, manipulating our perception of reality? I see that you do that in your work by working with elements of nature, the phenomenal – for example, the tornado you demonstrated behind us. Where does this impulse to activate or reconstruct natural phenomena come from?
JC: At some level, it reflects the global warming condition. It is a combination of the visual expression and how it is an effective visual tool that allows this kind of subtle blending – a blurring of reality – as well as references heavy weather conditions, which I will probably explore here in Stavanger with reference to the oil industry.
TTA: What I am very curious about in your work, and also in Augmented Reality at large, is how it ties into our human perception; into how our human perception is changing today. Studies show that because we have become used to scanning things more quickly, the way in which our eyes relate to our brains is changing, and that is affected by the media aesthetics of our everyday lives and devices. I am interested in artworks that interfere with that evolution, so that they do not become Pokémon Go, and that teach us how Augmented Reality can become part of our world – in this case as entertainment. I think it is very important that art and artists explore these new media-perceptual innovations, because you can show new routes. I think it is interesting that what you are actually working with is our perception of our media aesthetic reality. You are playing with something quite serious here: How we experience our environment.
And on that note, the theme of the Screen City Biennial is quite serious. I am curious – how do you relate to it?
JC: I think at this point, my approach is the medium itself. There is a dark way of looking at how we prepare for the trans-migration of our physical soul and its digital manifestation. The medium is built into a migration of the virtual world into our perception of the physical world. The migration of that – between the physical and the virtual world – is something I have been working with in different kinds of ways over the years through spatial and interactive installations.
These days I am just starting to figure out what I am doing. I will be using this technology of Augmented Realit,y and I have a certain kind of aesthetic, and an image of a mist filling the harbor. I will be digging from the different migration themes and probably deal with obvious ones we are all familiar with: dealing with the sinking rafts in the Mediterranean, the tragedy of migration, the oil industry, and the specific history of Stavanger. I have previously done work with a composer and also created my own kind of collages of real life interviews and some fictional stories, and for this new work I would like to incorporate specific stories. I am not sure how it is going to come together but it will probably be more of an audio layer than a visual layer. We will see.
John Cleater’s work blurs the threshold between physical and virtual space, from speculative architecture to multi-media performance productions and Augmented Reality installations. He experiments with spatial experiences, most recently working with Augmented Reality using mobile devices as lenses through which one views and interacts with virtual objects in real space via GPS and camera recognition technologies. He holds a Master’s Degree in Architecture from Columbia University in New York City.
See the artwork Consequences (2017) commmissioned by Screen City Biennial 2017.