What if music controlled a videogame instead of the player?

The idea of the Music Video(game) Jam was for its participants to spend a week exploring “non-player-centric” ideas. That is, videogames that don’t rely on the player to perform actions in order to move events forward. It’s an idea that James Earl Cox III has been exploring as of late, through games such as Bottle Rockets, and so this game jam is his way of encouraging others to also practice what he preaches. 

“I believe we need more games that are environmental/world and plot driven (rather than character/plot). Games where the player character isn’t the hero, nor does the story, or world, bend for them or wait for them,” he wrote in a Gamasutra blog.

Predictably, some reactions to this proposal were along the lines of “how is that still a game?” To which, I might reply, perhaps it isn’t, but it could still be a videogame. We’ve discussed this before: perhaps videogames could be more like videos than games? What Cox writes about is along the same lines. 

perhaps videogames could be more like videos than games? 

Further, Cox posits that shaping the required interactions of a videogame around the beats and arcs of a song is a viable format for his experiment. Importantly, Cox tells the participating jammers that the music must take the spotlight in their creations, and not any of the game components. “Create an experience that alters the way players will hear the song from now on. Build your game to match the flow of the song. Simply put, make the song visually interactive,” Cox wrote.

The only other instructions were to select one of the songs prepared for the game jam, use no other sounds outside of that song, and to provide a satisfying denouement to round out the composition. You can see all the results here, but I’ve picked out four of the entries below that all riff on the same entrancing song. So, if you’d like to come along…


The song that this small collection of videogames I’ve gathered takes as their conductor is “Last Level” by Two Parent Home. It’s a downtempo space flight of a song: heavy, tinny beats pound alongside a whistling alien synth, while choral gasps tremble underneath.

What the small game collective known as Barnaque do with this tune in Haelius is extract its eerier, wonder-filled tones to weave a strange, incomprehensible virtual dream. It starts from a first-person perspective, where you cut up a large green fish into screen-filling blood splats. Then you throw a ball between two friends at a lakeside. The ball matches the song’s beats, at first slow and then erratic, it starting to show the first signs of the sinister mystique that pervades the rest of Haelius.

You know it’s got going when you’ve had a man swim through a lake of dead fish and mucky detritus. After, you watch a woman being knocked over by a truck, over and over. With Barnaque’s influence, “Last Level” becomes a disturbing fever dream, somewhere between the twisted worlds of Salad Fingers creator David Firth and a fan-created Boards of Canada music video.

The Sinking Feeling

While The Sinking Feeling also finds some of the oddity in “Last Level” that Haelius does, it ends up being more pleasant. Funny that, especially as it seems to depict a steady descent towards death.

The use of debris works particularly well as you crash into it, splaying it into parts, when sinking uncontrollably into the oceans. The way that fish, octopus, eels, and jellyfish are used to create patterns to dodge reminds me of the way Busby Berkeley turned dancers into geometric shapes with an overhead camera angle in the films he worked on.

Yes, there’s a great sense of synchronicity between the visuals and the music. Which, really, is what many music videogames strive to achieve. It’s not as strong in the scene of a young girl chasing a butterfly across hills that the ocean descent is broken up with, but overall it’s a striking piece of work.

Fractal Gate

Now we enter ultra-space. Well, something like that. Fractal Gate‘s title tells you what it is. It’s the star gate sequence in 2001: A Space Odyssey, essentially.

It’s much more abstract than what Haelius and The Sinking Feeling offers. It’s as if a music visualizer had a go at making a videogame based upon listening to “Last Level,” matching the spacey chords with a mesmerizing journey through an endless purple passage. It doesn’t tell a story, it shows you what the music would probably look life if its sound waves were a substance.

All you can do is press the Space bar on each beat to make the two purple walls zooming past your eyes pulse with white light. That’s all, and you don’t even have to do that much if you don’t want to. Still, it’s enough.


Finally, we have space whales and a virtual 3D grid that you can make lumps in with your mouse. The idea in Abyss is to swim along with the whale by pinching the blue net that runs across the screen. It’s as if you control the waves of a digital ocean. 

This is the same style of interpretation that Fractal Gate takes on “Last Level.” Except it’s closer to how we’re used to seeing music represented visually in graphic equalizers and sound waves on screen. The whale can be explained by the drawn-out, serene notes of the song, as they are also found in the CDs that music shops would label “chill out,” right next to the 8-track CD of whale song.