Ever since the Walkman, ordinary people have had the ability to soundtrack their lives. You can play familiar music to reinforce or change your mood, music that either fits the tone and speed of your situation or is purposefully set against it. Earbud listeners can seek solace in this private musical headspace, an environment of their own making.

Now imagine this, what if your personal soundtrack could change dynamically? What if technology existed that took into account weather, time of day, and your exact location and activity? When you breach the threshold of your door, one song crossfades into another. When you bump into an acquaintance on the street, the volume pitches down to allow for easy conversation. When you approach the entrance to your place of work you hear the theme song that you assigned to that location. A dynamic personal soundtrack would take the traditional playlist to a whole new level.

With augmented reality, the potential for context-aware playlisting exists, and game developers and sound artists are already working on projects that push these ideas forward. Although AR is a term mostly tied to visual alteration, experiments focusing on audio have been expanding the palette of possibilities for the technology.

Ed Key and David Kanaga’s video game, Proteus, and sound artist Halsey Burgund’s installation, Scapes, are two recent examples of projects that use the infrastructure or AR audio to house new methods of exploration that reflect on the player’s/participant’s own involvement in the work.

Proteus is a video game that simulates an AR audio world. Everything and everywhere in Proteus is musical, and exploring the retro-styled 3D island, triggering sounds is the game’s primary mechanic. If you approach a frog, it will hop away with a two-note chime. Reach a mountain summit and serene strings will neutralize the noise of lower altitudes. Synthesizer tones cascade as leaves fall from 8-bit branches. David Kanaga, the composer of Proteus, said of the game’s musicality, “The ideal is to have every object making music. Every object is singing. And you can interact with it. You can feel the presence of the objects.” In Proteus, musical discovery and physical exploration are tied together like never before.

Games have been piping in location-sensitive music for decades, but in Proteus, this system of player-driven music composition is more than just background noise, it is the game. There’s no clear “goal” to Proteus at its outset; you’re simply plopped onto a procedurally generated island to do as you will. Controls are limited to walk, look, and sit. In this free-roam setting, the exotic sounds of the island and its wildlife provoke curiosity and drawn you in further.

Proteus’ functionally tight, interactive soundscape is an impressive feat, artistically and technically, but what about applying those same ideas to real world tech? An IRL Proteus system would require a sophisticated hybrid of geo-locative and image recognition instruments, a technophile fantasy that is, at best, in development in a secret Google lab. While we await sci-fi’s magic headgear of the future, artists and designers have taken to smartphones as a logical platform for AR.

Enter musician Halsey Burgund. Burgund developed a smartphone-based participatory audio platform called Roundware that allows users to record sound and assign it to their GPS coordinates. When Roundware listeners wander in reach of an audio clip’s location, they will hear it, along with any other samples in range. In his Scapes installation at DeCordova Sculpture Park, Burgund composed several generative pieces of ambient music for specific placement around the park. “As participants walk around the sculpture park, the individual path they follow creates their own personal version of the Scapes audio experience. The music is directly influenced by the landscape and is composed using custom algorithms which constantly generate new music; there are no repeated loops,” Burgund explains on his website.

Then, at some point while wandering among the sculptures, voices can be heard as well. You see, park visitors were able to download the Roundware app on their phone and contribute their own audio to the installation. Participants added 850 recordings to Scapes over a 6-month span. Some were descriptions of what they saw, others were meditations on the art, and still others were reflective of the Scapes project itself.

In a video documenting Scapes, Burgund ducks under a domed sculpture and the sound of rain emanates from his phone. Perhaps the audio posits a subtle commentary on the sculpture’s physical presence, both formally and from an earlier, soggier point in time. The person who originally recorded the audio may have just been trying to stay dry during a storm, but the audio remnant provides content for further insight into the sculpture than did not exist without it.

While Proteus is very much concerned with living in the now, Burgund is “trying to connect people to what is around them, past, present, and future.” Playthroughs of Proteus are ephemeral; there’s no way to record music in-game, leaving you to enjoy it in the moment. Proteus is about letting go while Scapes is about writing clues for the next person to consider.

When discussing technology, it’s easy to get wrapped up in the specifics of how systems work and lose sight of whether the latest development is being used for anything worthwhile. Games like Proteus present an idealist vision of where AR could lead while sound artists like Halsey Burgund apply those ideas to the real world within the constraints of current tech. However, what’s more important is that both present interactive artistic visions that outshine their techie novelty.

Your iTunes Genius playlist should be rightly concerned.

Proteus gif via Matt Glanville.