How do we account for the tension between technology’s infinite, unrestricted promise and the impermanence of being human? Rachel Rossin interrogates this slippage. Floating between painting, VR worlds, holograms, and more, the Brooklyn-based artist carries with her the essence of what it means to be alive. Rossin’s work meditates on and pushes the boundaries of human perception, the tenderness, and the vulnerability of empirical experience.
On May 20th, Rachel spoke with Killscreen founder, Jamin Warren, on her childhood underwater and the illusory nature of immersive technology. We also spoke about Rachel’s approach to her process, looking at her recent traveling shows with Rhizome and at Magenta Plains in New York.
Check out the recording of the chat and the transcript below!
The H an Dimension of Virtual Reality with Rachel Rossin
Thu, 5/20 6 43 PM • 54 10
S MARY KEYWORDS
piece, virtual reality, moving, video games, vr, space, games, photogrammetry, feel, facsimile, world, question, paintings, objects, people, technology, part, rachel, bit, thinking
We have the pleasure of having Rachel Rossini with us today. She’s both a painter and visual artist. In one of her pieces, she’s explored what’s lost in translation between physical and virtual spaces, scanning bits of her paintings and photos she’s taken in her studio and apartment and making them into a short video experience to Oculus Rift and hanging in a gallery with her work. We’ll be talking about that piece. She’s shown in solo exhibitions in York and in museums in Basel, Istanbul, and Helsinki. She’s also a fellow at New Inc, founded by the New Museum in New York City. For research, she might hack action adventure video games, like GTA or Call of Duty to better understand how big budget special effects are executed, but also by workings, traditional art making techniques and practices with new technologies. She examines the boundaries between the hyperreal and imaginary between perceptual and embodied space to distill some of the larger themes around human experience. Alright, so welcome, Rachel.
I’d love to hear about how you got started. As a creator and as an artist, we have a photo from your yearbook, which you were pleasant enough to share with us. So how did you get started in art making art making practice?
I started programming using the command line when I was very young, six, and then I started doing light programming when I was eight. But my first experiments using technology, digital technology, were on a dot matrix printer. And I was just making ASCII drawings, and then drawing on top of them. So I started working with hardware and software at a pretty young age. And that space became, which is still something that I mind, or in my work, is something I still mind in my work is this, the way that that virtual space became very therapeutic, I grew up in a pretty intense environment for a bunch of different reasons, but and that became a way to, yeah, find, find the rest of the world outside of the place that I grew up in, find, like, safe escape, safe levels of escapism, and then it would sometimes border into on unhealthy levels of escapism, which I think is pretty relevant to like, what killed Koreans talking about rights, like, the ways that we engage with virtual realities, so I don’t know, started young and it still feels like my mother tongue.
I mean, at the age of eight, like, like, what types of things were you doing with technology?
It’s also probably helpful for any parents out there who are where that interest might go. I think it’s we who are there that kind of like the way that we approach this. This is like the way that we interact.
The technology today is that we kind of have this infinite regress approach of black boxes, like for the most part, like most people don’t understand how these infrastructures work. And what right starting that young helped me understand is that the back end was like, because I started at hardware that then moved to the command line, which is like, for the most part, that’s, one turtle back, and then like, the hardware is, like, a few more turtles back, right. So and I think tinkering on that level is so because I was introduced to the command line, which is the back end of an operating system, like a graphic, a GUI. And so then I was able to open up the axes, which were easy to unzip. And so I just started opening them up and experimenting with them, swapping textures, and that kind of thing, like opening up video games. And yeah, then from there, I could start, learn HTML, of course, and I was like, super, super young, which was, and then that moved into like, opera action, like, like object oriented programming. And that’s like, once you kind of get object oriented programming, that’s pretty like, you kind of start figuring out anything.
What video games were you taking apart?
Back then. It would have been whatever was? Yeah, I mean, I said, Yeah, I would have been whatever was on when I was older. It was like first person shooter games like, Counter Strike, or I don’t think I ever missed. But I feel like that’s impossible because of how much I played it. But there are things that I kind of had careless disregard for like I could break them and knew that nobody would miss them. Like, I remember breaking Minesweeper, but I can’t, I don’t think I switched name the textures out or like some things I would just that would just be broken, ski free. I messed with a lot, but I always couldn’t, I couldn’t figure out how to manipulate it. These are, these are, these are all video games that came loaded on to Windows 95. That I remember being able to really dig deeply into the screensavers, which was really fascinating. Like being able to swap that stuff out. So basically, it’s just like doing really lazy modding. I didn’t know what I was doing. I just was, just doing that. Yeah. So I don’t know if like, yeah, the programming more is like comes into when I just got to, like, move things around. And like, kind of look at the code.
Do you think there’s something special about Windows 95? Do you feel like that was a jumping off point for other artists, maybe who are looking kind of like tinker roughly around the same time?
Maybe this is a stupid response. But I think it was just that it was open, I don’t think there’s anything special. It was just that that was built on that Linux based architectures. It’s a little more closed. And I think for like, someone was eight years old, it was just more open. And just, it’s more porous. You’re just still True. True today, like, so
I could really break things. And that was helpful. Yeah, I mean, it’s interesting. It sounds like you also were doing this in a context like you weren’t around other people who were doing the same thing. Is that right? It sounds like it was pretty self directed, or, oh, yeah, no, there was no yeah, there was no my, my family that we like, so that and the reason that it was even possible to do this is because my great grandfather, who was a German immigrant, but he just a high school dropout, he was a mechanic like a mechanic who was like missing fingers from working in Burroughs adding machines, which were like typewriters that then turned into those, like larger computers that are all vacu based. And then when binary was switched, yeah, exactly. I love that. Exactly. Hilarious. And so then, when the computers were like, the sizes of I didn’t know this until like, a few years ago, and I was like, that’s where those computers came from. It’s like, the and they were just discarded, once he passed away, and I didn’t really know what to do with all this hardware, just kind of sitting there.
From when he had a job as a mechanic, he just would go work on these machines. And then it turned into IBM. Towards the end of his life. Yeah. Burroughs adding machines was bought by IBM, which is, yeah, my father worked at IBM. So on the sales side, he brought a PC home for us. Yeah.
Do you out of curiosity, do you still own any of these?
Oh, I wish No, it was all it was never full machines. The dot matrix I think came intact. And then however, we had the windows 95 machine, but it was just all pieces of stuff. So I was making d b like, I was like, kind of like Said from Toy Story like putting bigger parts together and like, probably suddenly like cackling at them. Yeah, those were the like, they’re just parts that were around.
Well, let’s get started just kind of talking about, like, some of your recent work. I’d like to start with, let’s start with lossy like pull this up here. Can you tell me a little bit about how you got started in virtual reality? And how do you see it intersecting with some of the other bits of your existing practice? Because I think it’s been such an interesting piece in the way you’re working with physical forms, but then also trying to find ways to weave them with technology. But yeah, tell me a little about your first interactions in terms of working with virtual virtual reality, like as an art form?
Well, because I had, so the way that I’ve been also from when I was an adolescent, I started making video games. And so working in that programming language, I never made anything that was successful or even that good, but I just enjoyed tinkering. And so I’ve been like, I downloaded a torrent like Rhino, for example. And was like working in virtual spaces and had like, heard about virtual reality and probably watched like a doc entry or whatever I can find in the early aughts, and the Yeah, and so I had some idea of, had like an affection for it, but I wasn’t introduced. And there was no way to make things for virtual reality until 2014. And I was doing a residency in Miami at a film and art residency called borscht. And they had a headset that they kindly, like, lent London to me. And so then I could use these experiments, I could finally, like, translate a lot of these games I’ve been making into virtual reality. So that’s how I first started making works. The first work that I made was, I had to do with video game death. And it was using the kind of the kind of, I want to call synecdoche, even though that’s, that’s not the right word for it, but the syntax of this like this motif of like, dying and regenerating. And so the way that you would explore the world is it would be sort of taken from you. So dealing a lot with love, just the feeling of loss. And so the world would be exploring a world and it would be taken from you like, he would just sort of experience video game death and then be regenerated. And that was my first VR piece. And there are multiple levels, you kind of fall, regenerate and like a new world and then fall again. So that was an infinite simulation. And you’d be kicked back up to like the origin world. But that was the first virtual reality piece in 2014. And then, I showed the signal gallery, and then the second piece was this piece. And this piece is called I Came and Went as a Ghost Hand, and the show’s title is Lossy. And it’s named after an algorithm that’s in JPEGs, and MP for us. And it shorthand, it just means an entropic, it’s an entropic algorithm, because it’s saving space. So it’s just like cutting data in order to save space.
And it’s necessary, because if we were all like only saving tips, it would be a nightmare. So that’s like the most common, like, what am I going to have are most commonly used algorithms. And so lossy. And so the, the virtual reality piece itself is this, the way that it works functionally is like each each, like there’s a ray cast, which means like, where the viewer is looking is, is there’s a death, I’m trying to say this in like a debt, there’s a decimation script, like, like, entropy scripts, basically. So each viewer’s gaze is eating away parts of the pieces of you as you’re moving through. And there are photogrammetry models, but they’re also models that are fake photogrammetry models. So things that I made from memory that I pressed through a photogrammetry algorithm in order to trick the computer to kind of give it that photogrammetry feel. And photogrammetry is like a really well known and used architecture tool that basically you can just scan a room and turn into a 3D model. It’s very simple. It’s really a really great tool. So anyways, the paintings came in because I was using them. So I had that as sort of the approach, right? This is like the framework the world that I’m using and the way that I approach virtual reality is either project based or I’m exposing more of a psychological space that is something that I feel is important to the work. In this case, it was a bit of both. Does that make sense? Am I explaining it a little too? Seriously?
No, I think that does make sense. I did want to ask you, like, from a like, like the idea of concept of like death in video games or like loss and video games. Is that based on, like an experience that you had had playing games?
I feel like it’s one of the more universal pieces, but we don’t dwell on it too much. Because it happens so often. So often. I think that the facsimile of violence in I mean, often there’s, there’s a tone of that in my work, there’s a, like playing with the facsimile of violence or the way that this there’s this uncanniness and something that is out like, because in order, there’s it’s almost like there’s this something that is so deeply experienced by everyone that is so universal, and, and then there’s this interesting play. And by using it as a therapeutic tool. And I can remember when I worked on
The man mask piece, which I used skipping forward a few years, but I had played Call of Duty, and I was asking myself, why is it that I was so comfortable with this? And it’s because I was exposed to a lot of violence when I was younger, but I remember speaking to a friend who was in Afghanistan, who I went to school with, and I was like, What games are you guys playing?
Like, that’s all that they were playing Call of Duty. And I thought that I was like, Dude, why do you think that is, and he was like, something about being, like being able to have some sort of control over this facsimile was a somewhat of a therapeutic outlet. Now, I don’t think that that’s universally true at all. I think that and also like, you can see how the influence of that has had really awful moral repercussions
in the way that like the government funds, Call of Duty, there’s like millet that’s really upsetting, and so there’s this kind of like, handling the facsimile of violence was, I think, because of my own background was something that I was, that’s a reason why I was drawn to that. Like, are we trivializing this? Or is this also somewhat helpful? And I think if you’re young, and still developing, it can be somewhat helpful. But then like, after a certain point, if your heart becomes too hard, and it’s going to be Yeah, I think that’s one of the things that is difficult for folks to understand about like, like video games, like violence, like your context matters tremendously. Like who you are, and what your intentions are. Yeah, what your intentions are, how you interface with that, they often use my understanding is that they use it to use video games, violent video games like PTSD to help. Exactly. Soldiers create a sense of control over their environment to relive memories in a safe space. But yeah, there’s certainly a flip side where it’s like when not interrogated, both from a production standpoint, in terms of how this gives you created, but also when not interrogated. They obviously can , it’s not helpful. It’s not it’s not an absolute, like, it doesn’t like something like Call of Duty doesn’t exist exactly in that way for all people at all times. Well, that’s what’s tricky. And it’s like, why I wish that those rating systems sometimes worked, but it is interesting. It’s like, there’s a certain where, yeah, exposure then lends itself to being desensitized, and like, the coming deh anized. And I think that’s part of the risk there. That’s what we’re talking about. And so that’s the, that’s the material of what I was playing with, with that. This after going through a pretty large loss there was I was thinking about the facsimile of this kind of with the said, Yeah, I guess I’ll use the word again, like synnex through sort of, like, the symbol of death in video games death and the the hope that you haven’t run out, run out of lives.
From a process standpoint, or the last show, but I guess more specifically for Ghost Hand, how do you structure your time between like working with these different because you mean, you have to pay for that show and we’ll talk about some of the other work in just a bit but yeah, how do you structure your time like as a as a creative in terms of focusing on the oil painting part of your practice versus like the digital the digital portion the practice, particularly when, when Joan, they sort of might live in two different two different places.
It’s project based if I have an idea that if I have an idea that’s based in virtual reality or installation, and I think of the within a virtual reality where some more like, like I approached them were like programmed installations, or like, yeah, I mean, that’s more how I approach them is I think of them between there between video games and installation. So that’s the way that those larger productions work, much like installation is you have an idea, and then you have to execute it. So when I’m in that place, it’s devoted that I’m making drawings while I’m always making drawings, but there’s not much of my practice, that’s like really painting unless I really, therapeutically need it, and I need to get on the screen. For this, and so that’s mostly How it works is it’s project based. And so it’s pretty much half and half. The last year has not been half and half, I’ve really just only wanted to make paintings. And I’ve kept, which is interesting, because it’s paired with me taking the screens that really just become filters on top that’s integrated into the paintings. So there’s, it’s such a light touch compared to the way that I’ve approached a lot of the other work. So that and that’s really because I’ve, I’ve just don’t, when all of my social interactions became tethered. I had to lean so heavily on digital technology. I just got tired for the first time in my life, really, because it’s always been.
But I only used VR last year, maybe? I don’t know, probably 10 times. I was using VR chat, but it’s making me sad. I just missed seeing my friends in real life. So that’s interesting. Yeah, I mean, it’s interesting, because, like, some of the initial enthusiasm around VR was specifically around it, replacing a physical self. And, we got an experiment where, over the past year, where so many of us were indoors, the only way that we could interact was through digital, and here’s something that’s common there, which is like, it’s great to have it as an option. But it’s the only option. There’s a sense in which it feels like you meet your needs. It’s a supplement, not a replacement. to like, physically. Yeah, I mean, that’s how I feel. I don’t know if other people I mean, when I was younger, I could have I lived on the computer and, and lived in digital spaces, but like, and so I don’t know if that’ll happen again, for me, but I feel it’s just that for my practice at least, and that I just wanted to be making paintings. And even though I couldn’t really see people in person, I was happier doing that. And so that’s what I was, just want to just, I just made what I wanted to make, especially in a time that is just so awful, you know? Yeah. Yeah, absolutely. Well, well worth it I did want to ask you about was like the sense of like, embodiment, it’s interesting thing, just thinking through like, what some of the initial promise of VR has been, it’s weird, because it seems like it’s both seems like a long time ago, when the first Oculus Rift Development Kit went out, but also it’s like, not that not that long ago, at the same time.
So yeah, I was curious about you, like, how do you think about how you think about like, like, embodiment? And how maybe has that changed in particular over the past years? You mentioned, when, things like VR chat might have been the only replacements that people had for physically interacting with other people?
How is your thinking around like, embodiment? Like in your work, like the physical bodies in VR space? How has that changed since you started working in VR? versus kind of maybe how you’re thinking about it today?
I don’t know other than my own personal feelings about it, not so much. I mean, I’ve always used the way that I’ve approached my VR, VR works have always been around, or like tethered to states of embodiment. So either it’s by using and the viewer tends to be the viewer tends to be the moderator of the piece or the arbiter of the piece. So for I Came and Went as a Ghost Hand the piece that was in lawsuit would be the edge that you’re the each viewer carved residual like that would their gaze carve the residue of where they were, right. So it’s like with their attention, like they left marks that were felt by the people that would put the headset on after them. And so there was like this kind of there’s in that piece for instance, like so much of Yeah, the, the absence of with also the Yeah, the the presence in is. What’s guiding the peace or the peace?
The Sky is a Gap that I debuted at Sundance in 2017, is using the first developed at the new muse incubator that is completely tethered. So I turned the viewer into a cursor and the cursor moves time forwards and backwards. So basically, like instead of like, where we understand moving time, on a 2d, like scrubbing a timeline, I did that just with your body. So based is just using room scale virtual reality. So if you’ve moved forwards, you are consenting to the destruction of these kinds of larger cataclysmic events. And they’re sort of almost like Russian nesting dolls. So as you move forward, the piece would, for example, explode, and then if you move backwards, it would shrink into itself. And it was linked to sound so that you could hear the movement going forwards and then going backwards. And so I end the last piece. The last virtual reality piece is about the Proteus effect, which is, what is called skin suits that was commissioned by the Akron Art Museum and Skinsuits is about the burnoose effect. And that is this phenomenon where our virtual selves start to seep into our cycle, like our personality and like what we present to the world. So if you believe that you are a part of the projection and the virtual world, it does start to affect your psychology. And so that is, let’s see, that was 2019. I think. So that’s the last large virtual reality piece I made, or was it 20? No, it’s 2019. But I made a RPS recently. But the piece, I don’t know, we don’t have to talk about that one. But that was the new muse commission. That has to do with the body also. So I don’t know. I mean, I guess in short, like, I don’t feel like it’s changed so much. I just like, I do like, the tools have gotten better. I use the Oculus medium all the time.
Yeah, I guess it is more than 10 times, actually, when I think about how much I use, it just feels like such a separate thing like entertainment. And then using it as a tool just feels almost like the web, right? It’s like when really yeah, on the web, and the way that they almost feel like two different things. And so like when I use, I use virtual reality, like all the time as a tool, but not when I don’t really play games, I don’t, I wasn’t socially, I wasn’t socially active, like I was pre pandemic, whereas I used to, like, just go say hi to people in the chat all the time. Yeah, I think you’ve also identified fundamentally what some of the issues have been with virtual reality. And it’s like a wonderful tool that can be used in a variety of professional and commercial contexts. Like a platform and the medium, right? Yeah. Yeah. Yeah. But if you’re looking for stuff to like, do on a day to day basis, like there are there are some limitations there. Yeah, sadly.
With The Sky is a Gap, tell me a little bit about like, the creative process behind that in terms of like, how do you go about developing a piece like this? You mentioned that you start everything, many things with sketches. And tell me a little about the creative process behind developing this particular work.
This is funny, because sometimes ideas come like ideas come to you, there’s kind of like a flash, and you’re like, I’m gonna make that. And then other times, you’re like, oh, man, I have to like, I know, there’s something here. And it’s like a huge struggle and the like, the fish is like flopping around where you’re trying to hold it. And you’re like, please just tell it like, I don’t know. But this was a really interesting idea. Because I’ve been working. I’ve been developing.
Let’s see what was making the product. I mean, of course, I was working in unity. But I was trying to remember what the, I think was one of the things was like, I have the idea, like, period period. And then I was like, Okay, I’m going to build a little prototype. From a game perspective, not even in virtual reality, just to see if this can even work. It’s like, Can I scrub time for the backwards and then like, there’s a bunch of tools built in the unity that would that if I’d made it now would have made it a lot easier. But back then it was like, there’s no timeline. There’s no like, I was like, how like, so I had to figure out how to set Wait, I don’t know, it’s really boring, but it’s like setting waypoints in between.
I was talking to another developer friend. He’s like, that’s never gonna work. And I was like, well, you just you wait. So then the first thing was just really simple with some rigid body dynamics that were just Yeah, really simple closed FB x, that I was just moving forwards and backwards between the waypoints. So I was like, why wouldn’t this work? And then I put that in virtual reality. And I was like, Oh, yeah, this is great. So and then from there. I was like, Well, how does this begin? Like, how could they? How can I begin to build this out? Especially because what I wanted originally was I was inspired by the end of Zabriskie Point, which is this perspective, the moral perspective of that, of that feels almost nihilistic in a way because in order like, like, in order for him to finish, like what he originally wanted to do was he wanted the plains of Scotland to Sky right. Fuck you, America. And so that was originally the intent of like, what the film was, which was this kind of nihilistic. Like, it’s almost was like talking about, like, the materialistic parts of, of that movement in America, and I don’t—
For folks who haven’t seen haven’t seen the film, do you want to just tell us a little bit, just a little bit about it? It’s just like the Yeah, the sequence that you pulled from? Because I do want to ask you a little bit more about that sequence as well.
The giant explosion that’s made from eight different camera angles. Yeah. It’s like, he, he just, I mean, he blew up a house, we have a clip of it. Unfortunately, I don’t have a clip. Okay. So it’s very, it’s beautiful. And it’s also very sad. Because there are these, like, personal effects that you can feel as a memory. And so but the way that this film, this film ends is also this like, and it was in a really.
And I was just, I was thinking again, about like the facsimile of a violence. I don’t know, I don’t need to go back to like, why, but like, just politically, what was happening then. felt just really complicated. And how helpless I felt was expressed in this, I really felt complicated, and, and, yeah, help us. And so the, and so there’s so much about my own experience, when I was developing the pieces I was, as I was developing the piece, that was the overwhelming feeling I had was the sort of like, the idea of, of giving myself control. And there’s no such thing as futility, but the sort of idea of consenting to control and in this in this world that I had created, and so that’s why I tied in the risky point ending is the, the sort of, how do we say like, yeah, world weariness, kind of like the Yeah, they’ll Schmitz this, like that. That kind of double as, it’s like, I’m a part of this, but it’s also like, hard to be here sometimes. Yeah. Is there something about you, because the objects from that sequence there on the on the right, when juxtaposed with like, some of the things are happening, is this guy’s gap, is there something about like, I don’t know, chaos of objects, what the interesting thing is now is like, you have so many like digital objects at your disposal now, like, as a digital creator in a way that like, before you start to make everything by hand. And now it’s much easier, I can get things from the Unity Asset Library.
What is it about kind of like some of these? I don’t know something well, chaos of objects in space that’s attractive for you?
I really appreciate the question because it does relate to the way that the, the virtual reality piece with the, with the meme, like from I Came and Went as a Ghost Hand the those photogrammetry scans, which were all similarly like memories, or felt like hyper objects, or whatever you feel, things that felt felt important to me. So the objects in the scene are mostly taken from a reservoir that I have, that I’ve had with me since I first started making 3d objects. So since I was, I don’t know, if I would have been 14, or how old I would have been. So I’ve carried around like, all these hacks, on like, hard drives my migrating them from hard drive the hard drive.
Like, there’s a hacked video, like I basically like, and I mixed some of the things that I also just downloaded, like, it’s not like, there’s, there’s like, any strict rules. But so the objects in this, for the most part are all things that I felt really strongly about, that felt like very so in virtual reality. You can read the text on the pages as they’re moving past you and you can like a very quick walk, it’s one to one. So like when you’re moving at about Yeah, a quick walk pace that is like when time becomes like real time, so you can see how fast it moves past you. So you have to really like to slow your body down and pay close attention to the things that are, as they’re, as they’re, being destroyed in front of you. So you have to like to slow down and see and pay attention to each work. Yeah. Why? I mean, I guess thankfully all those objects are now like in the cloud now but I’m sure that was very stressful and moving them from someone who moved many in mp3 collection from one hard drive. Or when the hard drive to another predefined my like FireWire 400, like dongles, like connect my.
Well, I did what I asked about because this has come up with a couple artists I have worked with in the past. When you’re looking to try to pick things out, does it like because at this point, you have so many things that you can pull from Digital Object wise, how do you go about deciding like, what things that you would like to work with? Or do you find things in your collection and then like, sort of build, sort of build an experience around that? Again, it’s project based, so it’s like if something for Okay, so for instance, the skin suits project with the produce effect, where I’m using 1000s and 1000s of avatars, humanoid avatars, because the the point of the piece is that your your I’m trying to explain this in a non technical way, because I want to talk about the IK rig, but it’s like doesn’t make any sense. There’s an ik rig that’s like, it’s a virtual reality piece. And the, there’s just an IK rig that’s doing inverse kinematics on whether your hand is in space are you basically just to have your field of view filled about you have, like, have the sense of that you have filled a body and it can be like there’s, there’s just so many avatars that are packed from so many video games. So it’s like you fill the body of Kermit and then it gets, like, instantiated and gets blown away. Sort of like ripped away from you. And then that’s like your body as Kermit leaves the next one that comes to fill this will be like Cinderella, so it’s just like, like hacks, video games from like, from everything from like, Kingdom Hearts to GTA to the Star Wars games.
I probably had avatars for sure. I had the Kingdom Hearts avatars, I somehow ripped or torn to them that I had to go out and find. Finding those though, I had to go out and like, find those from people that had also hacked and like mod games. And so there was like a bit of seeking because it was a project based virtual reality piece. So it kind of I usually have the idea, and then I find the tools. Or I’ll sculpt them myself. I do 3d modeling also.
Well, I do want to chat about VR chat in a minute. The last few pieces that I’d like to discuss is, Stalking the Trace. Let me share this here. You were pulling from some of the things that you were working with, in terms of The Sky is a Gap, tells us a little bit about this piece in terms of, because there’s a lot obviously going on here for folks who haven’t, haven’t been able to take a look at it.
So this is, so The Sky is a Gap is the virtual reality piece in which we can say that in the alleys, which is behind these zoetrope app apertures. And as you’re walking through the piece, there’s this
strange phenomena where you scrub your like act sort of accidentally in the way that zoetrope works is your this the almost like sleight of hand trick with your senses, like with your visual center. So as you’re moving forward through the piece, physically in this, in this lobby, we’ll call it where the video production is. The expectation of the person in virtual reality sort of comes after if that makes sense. So there’d be these ghost images that would happen as you’re physically walking around. And then as when you would go to enter the virtual reality piece, you’d have that sort of same sense that you would scrub forwards and backwards, if that makes sense. So and so that was a really interesting thing because I had mocked up the space in virtual reality first, and came upon that on accident because I thought that I wanted these, like Walter Benjamin like arcades, and I had came up on accident when I Shrunk the arcades, and I was like, Okay, this is lovely. And then the and then this The video is a 15 minute is a 15 minute piece, it’s six channels, and it’s linked to the theater software, the lighting. And that is a tone poem for the virtual reality piece. So I’m using a lot of the same assets, and it’s all animations. It basically is exposing the back end of the virtual reality piece with a lot of Houdini simulations and other cutscenes from video games that I packed from and my own animation. Yeah, this ball and chain animation, for instance, is something that I’m in.
It seems like people come into this space, because it kind of operates on two different levels, right? There’s the piece itself, but they’re also sort of in this like, like, it’s like a waiting room as well. How are you drawing a connection between? Is it just a crossover material as well? Or like, I guess, how are you thinking about the flow of someone coming into space, experiencing VR and then coming back out of the space, what sort of the journey that you’re looking to try to take them on, like through this particular piece.
So because so much of this has to do with interview individual, the piece itself has so much to do with individual sovereignty that like, and that’s a way of just what I’m talking about as the game mechanic and the virtual reality piece, you’re consenting to move forward, this virtual reality, these these, these disaster scenes, these like cataclysms, you’re moving forwards and backwards.
There’s moments of respite and there’s moments that become more intense, that have to do that. But basically, it’s just the, the animations are linked, and the assets are linked to the virtual reality piece. So there’s a sense of deja vu, it’s likely that you’ve experienced it as a person. So there’s a lot of using the production mapping, also in a way to simulate virtual reality. So there’s like some sort of breakout point where there becomes that hole, that becomes projection mapped. And there’s like this, this Tr p lawyer effect of seeing virtual reality, like virtual virtual reality spaces. So you have this feeling of enormous smallness. And, and you notice that there’s a lot of scale ships that I play with in between the lobby and the virtual reality face.
Just like can you tell us a little bit about your new show boo stamina at magenta planes? Because interestingly, you mentioned that even in the past year, you’ve been doing, you’ve been doing it sounds like you didn’t even know you had had a 5050 split in the past between server VR, and oil based work. It sounds like at least over the past year, at least over the past year it sounds like it’s skewed more towards stuff in the physical dimension. Tell us a little bit about this particular show and love and how it connects to maybe some of the digital work that you’ve been doing in the past as well.
This show ends on Saturday, if anyone is in New York the last few days of the show. And there’s hologram combines and all in, all in braces that are built into the paintings that are also combined, and the show is reflecting on the like, the ways in which we use technology. And the question that I’m asking is, whether or not technology is better used as a splint or a graph. And what I mean by that is like, the way that we use technology now, for the most part is that we’re treating it like a cognitive peripheral, right like, and that came out of this work that I did another painting shows my last paying show previous to this, which was in September, which was titled The Sentinel tears tears. And for that show, I had trained a canary how to sing dubstep based on this, which you can all find on the internet.
That piece delights me, then it’s out of this, the idea that the way that we used to use technology is in the way that we It’s called the Sentinel species in ecology. And it’s when we used to, we used to use biology as a way as technology as a way to provide warning. That’s where the saying a canary in a coal mine comes from. And so that piece was about design, the idea of imminent danger coming in how to process loss or trauma, and this is the response. So this is part two. And there’s playing with both things, the structure of technology and my space in it. Yeah. The canary piece is interesting. It’s one of those examples of convergent evolution with h and as well, right, bird voices. The bird song is very similar to human human speech. So we definitely see a lot of resonance there, even though people didn’t know that at the time when, knowing that birds can mimic and they can sing and learn other other types of things as well. Yeah.
Specifically, electronic music is very, like it’s interesting. So we’re attracted to that. And it sounds like birdsong. I fed it through. It’s like, actually, it’s so strange, it’s like and then another thing that surprises me from that project is I tried to upload. So I added this Skrillex song, and I tried to upload and it didn’t, the canary sound didn’t sound so I was like, I can kind of hear the song, that at the very end of the show, I can kind of hear this at the end. When I finished the video, I was playing it back and and then I uploaded it to YouTube, and the copyright flags came up for the same song. So the algorithm had heard the song better than I could the end, the bird had done such a good job that it was actually tricking the algorithm. Now, I would have thought that would have been a good, like workaround for the content ID system is just teach birds to and then you can upload videos of birds. Very, very shrill. Yes, for your shrill enjoyment. Next time next time. But the show closed on Saturday. And so definitely run, run, don’t walk.
Adam Lane was asking about violent video games being used to treat PTSD, which we talked about earlier, and letting soldiers experience combat in a space. Let’s see, the question for you is, how do you think about the agency of people experiencing your VR pieces? Are you taking agency away? Giving participants agency that wouldn’t normally have? So tell us a little bit about the role of agency, the agency there in some of your work? Has that is something as unique about VR, in that sense, you’re certainly giving participants more control over their experience there.
That’s a really, really good question. And it’s also in 2017, when I was showing this work at Sundance, and so much of the conversation was around, how do we sort of shoehorn filmmaking into virtual reality. And I was having this, like a lot of these uncomfortable, like, conversations around this, because I was just like, I was like the idea because I’d never approached, I’d never approached virtual reality, like filmmaking, I’m not a filmmaker. So this was interesting, because so much of virtual reality, or so much of filmmaking is obviously there’s a frame, which means you’re controlling someone’s agency, you’re controlling someone’s attention, because we’re giving perspective, right? And so agency becomes an even larger frame or context, because obviously, in virtual reality, you have so much more of it. So then how do you, it does start to act more like the structure of a video game, right? It’s like, it’s like, it changes so much. It’s almost like, the way I think, like I think this is often cited when you talk about virtual reality or virtual spaces. Flatlander, the allegory of the 2D shape finding the 3D reality, it’s like, you can’t it’s the it’s not the same type of framing technique. Right. So you can program attention, but it’s going to be a different experience. Right. So I think. yeah, I, I like to do both, like I like to give away, which in the case of like the, the piece that was working with the death and rebirth mechanic, right. It’s like I was taking agency away and giving it and taking it away. So I think that that’s actually a really interesting part of the media is being able to take it away and being able to give it. I think that’s something that’s like your relationship to the agency. I think someone who works with games or plays games is very different from other mediums where that’s not it’s not like a given. I do think that yeah, it’s been a struggle. I think art for me, for filmmakers or other artists who aren’t accustomed to thinking about agency as being pretty central to liking what you want out of the VR experience. I think it’s been harder for them to recognize what makes those things? Good. Exactly. And I think for game developers, it’s really intuitive or even like artists that understand installation. I feel like yeah, that, which is, that’s a little bit more foggy. Because there’s usually no programming involved, you’re kind of just like, here’s a space, you know. And it looks like a control control where someone enters a controller or someone exits. And that can be like, and then it’s up to them to investigate, which is sort of the premise of most installation work. And then obviously, when you put that in a virtual reality context and approach it as art, it becomes very different because there’s so many more things you can do. And so that’s what I love about it.
We had a question about your work at Zabludowicz Collection. And the question was about whether or not you acknowledge that collections being in the news aren’t news this week for it’s rolling Gaza right now as? And how do you acknowledge that as being a funding body of your work, just because Stalking the Trace was shown there back in 2019?
Yeah. It’s like, it’s such a non it’s not nuanced at all. What’s actually happening? It’s like, what? Israel’s role in apartheid and all that. But that aside, the private conversations, the reason I was able to do that show, was because I was assured that that wasn’t happening. And at this point, I need to collect my thoughts to actually figure out what’s going on. Because there’s not a lot of evidence to suggest that I was, I don’t know if that was correct information, if that makes sense. So I’ll, I would like to address this soon. I just need to have some conversations, figure out what is going on, actually. Yeah, it’s an issue. And I’m sure folks will be looking forward to your collective thoughts on it. Yeah.
The last question I had for you was working with collaborative technologies, like, how are you looking at the intersection between game engines and artificial intelligence? And how are you when you’re looking to combine technologies as a creator? How do you go about trying to make things maybe fit together that don’t always seem like they fit? You mentioned a developer friend kind of being skeptical about something you want to do in your work? So as curious when you’re picking out tools, how do you make sure that they’re compatible? And how do you make sure that doesn’t limit ideas that you have in your head?
Just break it and see if I just keep breaking it, just keep breaking it and a whole bunch of different ways. And I think that’s the best way to do that. Because that is the question more whether or not something’s going to work, or is it? Because then, I mean, yeah, there’ll be an idea I have. It’s just, it doesn’t work. I’ll just be like, not gonna work.