What if there was a type of graffiti that was completely legal, could be added to anything, didn’t destroy property, and was itself potentially indestructible? Such a refreshing medium might lead us to rethink our physical environment and even use our handheld devices—so often accused of disconnecting people from their environments—to engage further with the world around us. Andy Biar and Andrew Bueno’s Peek app uses augmented reality, paired with filmed performances and 3D renderings, to do just that. Biar’s video compiling his experiments and process turns his city of Pittsburgh (he is a student at Carnegie Mellon University) into a treasure trove of unexpected easter eggs, some of them permanent and others as ephemeral as a pile of dead leaves. The city’s famous Cathedral of Learning, for example, is satirized by a performance of an actor rapidly kneeling and pretending to ‘pray’ in front of its stained-glass windows.

Each superimposed element, whether rendered or filmed, is attached to a specific physical ‘fiducial’ in the actual world that is marked by the app and cannot be removed unless the physical space is significantly altered. Through the RGBD workflow, filmed footage is overlaid with points in 3D space, giving the app some flexibility in recognizing where to place performers and objects. However, there are still many limitations in the app’s functionality, many of them possibly the result of limited mobile device image sensors. For example, Biar and his collaborators found that they could only place fiducials at certain times of the day, and the app didn’t recognize the same locations when the lighting was different.

There are also significant barriers that must be surmounted before ordinary users of the app can create content for it. Creating assets that respond to specific video cues is still a complex process, involving a green-screen process similar to the kind used by professional movie and TV studios. For now, however, it’s more of an art project than a tool. There’s a certain beauty to an experience that relies on finding your way around familiar settings in a new medium and allowing others to discover a common experience with you. See it in action below.

This post was originally written by Rachel Pincus for PSFK