Jakob Kudsk Steensen: Dreaming up a digital swamp
Deep inside Berlin’s most exclusive clubs lies an immersive media installation that harkens back to the city’s history. Berl-Berl is also “an instrument, a networked video game, a virtual environment, and a song for swamps.” Spanning 1400 square meters and two floors at Halle am Berghain in Berlin, Danish-born, New York City-based artist Jakob Kudsk Steensen created the immersive installation by wiring 13 computers to one another and to nine LED-powered walls. Inspired by the word Berl, which means ‘swamp’ in an old regional Slavic dialect, each sound and object has been digitized from local swamps, as well as the archive at Berlin’s Museum of Natural History. You can explore Berl-Berl virtually here.
We speak with Jakob about his collaboration with musician Arca on Berl-Berl’s soundscape, working between the worlds of games and contemporary art, and how classic Japanese designer Hideo Kojima inspired him to construct virtual worlds.
What role has first-person games have in shaping your creative practice?
I grew up in a rural town of 1,000 people near the coast. This was the time studios Epic Games started to make expansive virtual landscapes that went on to become cultural icons. Games like Metal Gear Solid, Unreal Tournament, Quake, and Fallout 1 were the early foundations of games becoming major productions, and pop cultural household names during the late 1990’s and early 2000. At this point, I was a teenager. Living in a rural environment, access to these vast and narrative virtual worlds absolutely blew me away—with their imaginations, stories, interactive dimensions. As did the sense of access to a different world than the one I had immediate access to. Where I lived for a good deal of my life was a bit of the ‘Bible Belt’ of Denmark. I moved there with my mom from Copenahgen, and I always felt a little outside of, or alien to, the families and cultures of where I grew up, so games gave me access to vast video alternative ideologies, pictures, characters, and creators. For many, games have a power to show the world can be different than what is immediately around you.
So I started to mod video games. The map Facing Worlds, a multiplayer level in Unreal Tournament 1999, became an obsession of mine. It is a level where two teams compete. The landscape floats above earth, with our blue planet on the horizon and at the edge of your view. There are two tall towers with references to ancient Egyptian architecture. You can be at the top of the tower, with a holistic worldview of the level and earth beyond, or you can be on a bridge connecting the two towers—immersed, in the thick of it. I began taking maps apart, remaking them, making my own, and uploading them to forums on the web where I started to chat with other level designers.
Several people I know today who have gone into the video game industry—owning studios, making music videos or working on titles—all started modifying games back then. There weren’t many educational formal routes you could take. Facing Worlds is an example of widely imaginative, but also clever and conceptual virtual world-making. It’s a kind of collage iconography, explorations of movement and vision, and how to be imaginative with technology. Hideo Kojima is a pioneer of more cinematic approaches to video games. I became interested in his games when he announced, out of the blue, Death Stranding at Tribeca Film Festival. I was showing at the festival the previous year, and got interested in how it placed game, film and experiential technology makers in the same room. It signals the rise of a new format of which I think Death Stranding is a part. Games today are barred behind many convoluted conventions of gameplay, interaction, male-gaze, stereotypical action heroes of which Death Stranding also belongs. Still, it signifies something new. A third format is on the rise, which I think many artists like myself explore, or like Team Rolfes and Pussykrew also start to venture into, in their own ways.
I first thought I wanted to be an animator in video games, but learned when taking the formal test that I had to draw the same character 17 times. This immediately pushed me to be an artist, exploring these tools and technologies in much more visceral and at times chaotic ways. Being introduced to the tools early on means I have deep technical knowledge and I know how to design, create and tell stories through virtual environments. It was always primarily the virtual landscapes, sceneries, that interested me, and the ideologies behind them.
This is also how I work today. I venture from research to emotion and imagination—and try to create different perspectives and worldviews with each artwork and virtual landscape I make. One projector never replicates the same perspective as the previous. How you think about how to design—how to create, and the ideologies behind them—will completely alter feelings, moods, spaces and how people interact with them digitally. Behind every virtual space created is an ideology.
My work is connected to recent video game history, but also quite removed from it, as I primarily work in the field of exhibition-making; in contemporary art. What interests me about virtual world making is a sense of morphability, spatial poetry, and the ability to manifest in space new worldviews. I get to create with video game technologies, but for very diverse audiences who may never have played a video game. I find it appealing to both make for them, and for people like me who grew up with 3D. You get to make your own rules in a way, when using familiar tools like video game engines outside of video game fields. Though I am working on a game, and have published my work online previously on game platforms, I am interested in what lies beyond the cultural conventions of games,’ using the same tools.
In making ‘Berl-Berl,’ how did you push past creating ‘just an immersive show’—and holistically think about immersion in real, physical spaces? What, if anything, were you hoping to take from the world of games into physical installations?
The concept of “immersive” shows has gotten a bit of a boom the last five years. It is often associated with large projections, colors in space, and large spectacles and screen effects. For me, I try to make things slow. Extremely crafted, and about a sense of meditation and rhythm. Every work I create is literally created the way I described Facing Worlds as a map in UT1999. In that way, my work when showing in physical space, actually follows a bit older game design principle.
I create entire virtual landscapes. They have physics—things move up, down, to the side, but also through each other, and different plants, rocks and entities can morph into each other, change texture, sound and transform into something else. Things are never static in my work, which is how I try to break the otherwise more traditional approach I have to making maps in games. I draw out on paper the design for different environments, and their feelings, their moods and colors. Is this location supposed to feel enclosed or open and welcoming? How does it connect to the next space, and give a sense of progression in a story (Without words) or a transformation? More traditional 3D world-making can tell stories without words, but through architecture, color, tone, speed and movement. These are qualities you work with in physical space, when the work shows as spatial sound, or when you connect physical performers to virtual worlds, or even how an audience moves through a physical space.
Making game levels is like making puzzles for the mind and senses. This logic I take with me when making a work like Berl-Berl. You move through two floors of the building. Each area has a theme: some feel grand and with a holistic view, like the top tower in Facing Worlds. Others feel intimate, emotional, closer and immersed. I think carefully of what you see and hear in the physical space, connected to what is shown and played in sound from the virtual. Material might feel closeup, of fungi underground, roots or leaves, intimate. Or, a camera in the virtual landscape might show an entire landscape on a huge screen, giving a more cinematic and holistic connection to the virtual landscape.
I also literally built the entire exhibition first as a multiplayer first person game I use with my team to explore possibilities and to pre-visualize and get a feeling for the final show. I always do this. So any physical show actually also exists as a multiplayer map in first person and VR. Whatever you access a virtual world with, be it in VR, screens, AR, or first-person, connects to the physical site you are in when doing so—especially for museum exhibitions.
Museums are often in possession of large open physical spaces you can modify. Modifying them and making a choreography of movement IRL, is very similar to doing so in first person multiplayer games, where multiple synchronized paths of movement are necessary rather than linear ones. I often also adjust the virtual spaces to the physical ones. In Berghain, I have literally built the virtual swamp on top of a 1-1 model of the building I made—so the scale of architecture and imagery connects. Behind the scenes, Berl-Berl is actually a multiplayer game 13 computers play together, moving through the landscape at different paces and in different movements, depending on what screen a computer is connected to, and if it is supposed to feel more intimate or removed and objective, giving different ways of perceiving and feeling connected to the digital swamp of the work.
How did you get to collaborate with Arca on the soundscape for ‘Berl-Berl,’ and how did you factor sound into the overall sensory experience?
Berl-Berl is made as an instrument. It includes 29 speakers, 13 computers and nine LED walls. Everything is connected to each other, and sends information back and forth. As a result, the building becomes a living instrument. If a branch appears on one screen, the computer tells the server, and it then specializes and playback sounds of that specific branch, in that specific part of the building. Other sounds, like the weather, exist in the entire building as a holistic environmental effect.
As screens, each connected to one computer, journey through the virtual landscape together, they trigger different hyper-spatialized sensory sounds, alongside what we refer to as global effects like wind, clouds, frost, and larger transformations that happen in the work as it journeys from forest to undergrowth, and from summer to frosty conditions. These changes happen all the time, at their own randomized pace, creating a kind of living instrument. Everything is very rhythmic and musical in how they have been edited. We connect Unreal Engine to MAX MSP, to Spat—a software that can hyper spatialize sound across the 29 speakers. We used impulse responses—measurements of the building’s natural echo of about seven seconds—to make the sound. Sound naturally bounces around space, making it hard for the audience to pinpoint exactly where sonic aspects come from.
Arca is a key component in this instrument. Berl-Berl is based on research into mythologies of lost languages, songs and concepts about swamps and wetlands. From this research, I created notions and words shared with all collaborators. Arca sang words and vocals which refer back to these swamp mythologies. She also sang to the building, as if it was a cathedral, playing with the natural echo. Her voice fills the entire building at times, like when in a church, and at other times appear like soft whispers in different hyper-local areas. The songs are triggered under specific conditions and places in the virtual swamp, determined by the server.
So song, and language, goes hand-in-hand with the virtual transformations. The singing also is very sensory, but often without words, but sung as if in a cathedral. This mixes a kind of animistic feeling with something more modern and Christian. Modern religion demolished many of the older wetland cultures and their worldviews, so we mix them into a new form. I am also moved by Arca’s talent and precision, and found it interesting to work in a format outside more conventional music. To me, this has been one of the most rewarding and interesting collaborations to date. It brings song, language and a sense of improvisation to an otherwise fairly complex and large-scale creation using technology. This is where I want to go in the future… towards musicality, performance, and life in technology.
In what ways did the specific qualities of Berghain’s physical space shape the development of the piece?
The building itself has been central to the creation of the work. I normally live in New York, where I am part of the “immersive” media community, which I love and enjoy. But I am more of a foreigner to Berlin. I have been based here since November making Berl-Berl. I found it important to embrace the iconic building itself, associated with nightclub culture and the more recent post-cold war version of Berlin. This is also why I worked a lot of musicality into Berl-Berl and reached out to Arca. It was important to me that the project felt like it embraced local sensibilities in Berlin, and in Berghain. But I also didn’t want to make something flashy, fast, and clubby.
I wanted to make something of a third format, something new, more poetic and slow. The virtual world of Berl-Berl is spatialized exactly to the building itself physically also. But I think the most important thing was for me to make something that doesn’t try to be what Berghain has been in the past, but also not just take it over with an “art” exhibition. It is a tricky place to operate, between different fields and expectations. After the long COVID lockdown, I felt it necessary to both embrace what was in the past, while also signalling something new.
Credits
Interview conducted in August 2021. Headshot by Pablo Castagnola.
Links
