When we sit down (or stand!) to play games, we give little thought about the devices in our hands. In the same way that the QWERTY keyboard is disconnected from our experiences browsing the web, the game controller feels disconnected from the activity of play.

Fernando Ramallo sees that interaction between player and play devices as central to their work. Ramallo was born in Argentina and now teaches in New York City. They are intensely focused on custom physical devices and creative systems that can engender new forms of play. 

Their game Panoramical with David Kanaga pushed the boundaries of what you can draw from interaction with music as a play space. It even supported a MIDI controller as it uses 18 different inputs to control each scene.

Making a way into play

Do you want to tell me a little bit about how you made your way into games? 

I never really thought about making games up until I left home. I went to school for what was called audio-visual design. For example, it’s a mix of animation and video film. In other words, it’s just a broad spectrum of different things related to science and technology, and explicitly outputting work into a screen.

When I was studying in Buenos Aires, a local company did this workshop, and I ended up working with them as a designer. They had a lot of faith in me because I wasn’t a game designer at all. Afterward, I started making prototypes for them and got to learn the whole industry while working on all these little experimental games.

My parents are programmers. They make software, and they do accounting software for local companies where I grew up. When I was a kid, I would ask them if they could make a game for me with that accounting program.

But they were outliers. None of my classmates, none of their parents, even worked with computers. Because there were no computers around, I felt like computers were this rare thing that a few other people and we knew about. There wasn’t a lot of access to it, and it was still very new. But since my parents had already been working with technology, it had been more common for us.

Could you tell me a little bit about maybe some of the things that were happening during your childhood in Argentina? There was a massive depression towards the end of the ’90s, and since then there have been monetary issues.

Every 20 years, everything goes to shit. People get to the end of the month, and they can’t pay their bills. As a result, everything costs more currency, and there’s inflation. In other words, there’s always this presence of a depressed economy, having a hard time feeling like the future is not very certain, or you’ll be able to stay.

Growing up with that around always made me curious about what other ways people lived, what other countries were like. Growing up with that uncertainty made me more curious about what life is like in places where you may have less economic insecurity or just a different way of living.

It’s hard because, of course, I really love it over there. There’s a lot of culture and food and family, and even the way you interact with people that I really miss. However, it’s hard, especially with technology, since it feels a little bit isolated.

I certainly see that millennial creators and designers. The backdrop is a financial depression here in the United States, and just generationally, millennials in the United States are way behind where their parents are. You’re doing worse than your parents are, and that’s only the calculus about even becoming a creator. It changes a lot when you’re having to think more actively about your own future. I can certainly imagine that.

Well, tell me a little bit about some of your early work.

Panoramical was this period of my life when I had encountered the independent game scene and all these creators that were doing the work that I wanted to be doing. For instance, it felt exhilarating, and I really wanted to be a part of that. I wanted to prove to myself that I could make a bigger game, where I have control over it. 

David Kanaga and I made this little prototype. I actually didn’t like it at first. I thought it wasn’t enough. It needed more gameplay, or it needed more of a complicated mechanic or something to do.

David convinced me that it was good enough, that it was good; that it was interesting; that the interaction was interesting enough. 

We had a conversation with Cassie McQuater, who’s a digital artist here in Los Angeles. She said one of the things that’s nice, compared to some of the gaming conventions or selling games on the App Store, is that people at art galleries have more patience and tolerance for things that they don’t understand.

If I had shown the game as an installation, I would have felt like it was enough. I would see how I could create something that I can show to people in that environment. My challenge for Panoramical specifically was: how do I bring the things that I would like, or the things that I find interesting about making an art installation, and try to sell it to gamers? Can I actually make a commercial game that’s basically an art installation? That was the challenge.

Music is often thought of as being something in the background, and you’re not making just a visualization, right? It’s more complicated than that.

I used Winamp a lot growing up. At one point, I was super into the visualizer, and I thought they were great. With Panoramical, I really wanted the visuals to not just be hooked into frequencies. I wanted the movements and the shapes and everything to be more curated, or more handcrafted; this tie between the image and the sound that I wanted. I wanted my hand to be in it, and not necessarily have to come up with maps to drive whatever sound I was getting from David.

A lot of people would ask us, “How did the reactive visuals work, and what was the code that drove the visual play from the sound?” And there’s actually no connection between the shapes and the sounds. 

They’re just running in parallel and responding to the same input. That alone makes it feel like there’s a secure connection, and there’s more complexity behind it than there actually is.

I went to piano school for my entire childhood. The next logical step would be to go to a conservatorship or an actual music school and make that my life. But then I was already playing with computers. I swore off music altogether because I was overwhelmed by the pressures that everyone was putting on me about making it part of a career. I was just doing it for fun.

With Panoramical in particular, in my head, it was music that was David’s thing, and I was going to focus on the visuals. But it was informed by the fact that I appreciated music a lot, and I like music, and I always thought it would be half of the experience. 

You describe yourself as creating playful interactive art. 

 I’ve been playing a lot of flight simulators. There’s this flight simulator-ish project in the back burner.

Flight simulators, and sims, simulators as a category, are excellent. They’re open-ended. It’s really just about personal experience.

What I like about them is that you have this very complex simulation, or system, or machine, and it only takes a little bit to learn to actually know how to control it. But then if you want to learn more, there’s an infinite depth of knowledge that you can have about how to communicate, and use the radio, and flight planning, and different planes, or whatever.

But at the same time, there’s this space for introspection, and just the experience of actually flying the plane is so slow. Nothing happens that you don’t have a choice but to be content with it and contemplate the landscape. I just really love that relationship with landscapes, and with beauty, that you have when you’re flying a plane.

You’ve been working in VR, and also with Looking Glass holographic displays. Tell me a little bit about the process of moving into those spaces as a creator.

When VR came around, I liked the technology, of course, but I noticed how it was being sold as this big next thing, so I was also skeptical. 

There’s optimism about the tool itself, even if you’re skeptical about uses. 

The interface itself is a game-changer. When I first started playing around, I just got so many ideas for different interactions. I was really hoping that it would become more common, so you could experiment with more different interactions. 

Unlike creators in other mediums, you’re creating with commercial tools at the end of the day that is bought and sold and change over time. If you’re a sculptor, you have to worry about clay no longer becoming a feasible medium or becoming obsolete because someone bought the right to it.

I think about that a lot; all the time. I’m always low-level, in the back of my head, thinking about how can I make the work that I’m doing and the tools that I’m learning more stand the test of time? I’m starting to feel less and less comfortable with the pace at which things become obsolete in whatever, anything related to technology.

I interact with all these tools like 3ds Max and Unity, Blender, whatever. These tools are designed to let you do as much as possible and be very professional and precise. What I’m interested in is having a tool that’s less precise, where precision and control isn’t what the tool is trying to do. 

I was looking at a lot of creative tools for kids that came out of the ’90s. Microsoft went through a series of tools, including one called Creative Writer. There’s a history of tools made for children. The goal here is it’s not like Photoshop. The goal here is not necessarily accuracy, it’s just pure expression. The tools I am using every day—what are the values that those tools have? And how can you make tools for other people, and for yourself, that is more aligned with the values that you want to have when you’re creating.

Unity’s going to be very easy for you to make a third-person video game. But maybe it’s going to get in your way when you’re trying to use it differently. 

There’s s also a problem with ownership, right? When you’re working with the tool, you’ve made the most fantastic thing, but if people can’t take the artifact that they made out of it, and own it, then there’s something lost there.

We worked with MoMa on taking games into the permanent collection of the museum. Their design curator Paola Antonelli said, “Code is more fragile than porcelain.” It’s such a massive challenge for archivists, and even for the artists themselves, to ensure that their work is still going to be available for future generations.

It’s just an extra burden for being a digital artist, having to think about your legacy, or what happens to your work after it’s gone. 

I’m thinking about tools, thinking about access.

It’s like you have to build your easel, your paintbrush, and figure out how to create the pigments and the paint before you start thinking about what you’re drawing. There are all of these things that you have to stress about when you’re working with digital stuff.