What does a videogame sound like? That’s the question Pippin Barr has been trialing as of late. It might seem absurd at first—we know what games sound like, don’t we?—but he doesn’t refer to diegetic sounds or sound effects that go towards making a convincing world. What he is interested in is the idea of gameplay as an instrument.

David “Proteus” Kanaga can be pointed to as the source of this concept, at least for Barr. Before even Proteus, Kanaga was scoring videogames in a way that gave each verb or action an auditory identity. His idea was that the play of games and the creation of music was an isomorphism; that is, our experience of creating meaning in both is the same. Hence, in Proteus our exploration of the lo-fi island doubles up as an improvised piece of music elicited by the creatures, plants, and weather.

Kanaga’s scores were a way to put his theorizing into action. Take a look at his “Let’s Score Assassin’s Creed” above as a quick case study. The horse’s trot and Altaïr’s walk are matched by plodding hits, and the pitch of the sustained otherworldly synth, which starts when Altaïr dismounts, rises after the assassination has taken place. We’re inclined to use the sounds as much as the actions we see (and that we’d normally enact) to substantiate meaning. “For every process or change of state in a game, there should be a corresponding process or change of state in its soundtrack,” Kanaga writes about scoring games. “So, don’t think in terms of background music and sound effects, but rather events, states, processes, textures, rhythms, and forms.”

Barr wanted to draw across from Kanaga’s work after Sound System I, which was his experiment in using physics to generate “natural” musical compositions. What he has come up with is the imaginatively titled Sound System II. The only connection between this sequel and its predecessor is that it’s a continuation of Barr’s exploration of the application of sound in games. Oh, and the William Turner painting in the background, which is there only to make a visual connection, and nothing else.

Essentially, Sound System II is Atari’s classic arcade game Breakout. But the difference is that when the ball (represented by a pixel) hits a surface it creates a sound specific to that surface. When it contacts the paddle this sound might be a bass hit. If it’s the borders of the playing space then perhaps a snare drum is heard. The rainbow of blocks that you chip away at with the ball have the most variation between the ten levels. Sometimes the blocks only react with a single noise, but in the higher levels the ball sends sonic waves across them as if someone is playing up and down the scale on a piano, with each colored layer having a different timbre. 

By playing Sound System II, then, you generate music from all of the different game elements. Barr also relates the challenge of the game to the skill required to play an instrument—”the amount of time one can ‘survive’ in a game might correspond to the length of a piece of music – that they are one and the same thing,” he proposes.

So what a videogame sounds like, in this case, is entirely affected by what you’re doing and how well you’re able to do it. The player becomes the composer. The game is their instrument.

You can play Sound System II in your browser.