How wearable interfaces could free videogames from the tyranny of screens

From the first motion picture houses of the 1900s, to the ubiquity of mobile phones these days, the past century could be described as an invasion of screens. But what if we don’t need that many of them? It’s a common assumption that screens are a necessary component of modern life: we have computers and need visual feedback to communicate with them. However, a tidal wave of wearable technology could wash all these flickering, flashing squares out to sea.

“People think that the interface has to be attached to the display, but you already have a lot of interface on you,” says Julia Schwarz, who’s currently working on a PhD in computer human interaction at Carnegie Mellon. “Why not just use that?” With this sentiment as her m.o., she founded Qeexo, a company striving to liberate us from the tyranny of screens. “Basically the way your phone or tablet sees you is that you’re a point on the screen,” she continues, which is ridiculous considering the extent of articulation of a living human body.

We can knock, strum, poke, rub, wiggle our nose, breath lightly, scream. The problem is that our computers can’t decipher these things. Her research is in making our interfaces smarter so we can compute au naturale. For example: she created some of the gestures used for Kinect 2.0. Another of her inventions allows you to rap on your mobile with a knuckle to snap a screenshot, instead of performing the awkward motion of hitting the home and power buttons simultaneously.

You could make a case that the more organic computing becomes, the less it will rely on screens. This may sound rash to our monitor-adled brains, but it’s an inevitability if wearable devices hope to get the rest of our bodies quantitating. “Wearables like smart watches have these tiny screens. There is almost no real estate on the screen itself,” Schwarz elaborates. “They’re kind of assuming that all the interaction needs to happen on this tiny display.” But rather than making the screen bigger, she insists, we should just use the rest of our bodies. “Gestures can be very powerful that way,” she says.

We can knock, strum, poke, rub, wiggle our nose, breath lightly, scream. 

This isn’t crazy-futuristic stuff, but it does require creativity. Qeexo’s user-friendly interfaces use acoustics to listen to the sound you make with your tap while relying on proximity sensors to figure the location you’re actually tapping. This goes a long way towards downsizing screen-space, but it doesn’t totally eradicate the need for screens. Some tech, like videogames couldn’t exist without a display, could they?

Not so fast. A few years ago, Patrick Kersten and Christiaan Ribbens of the Netherlands invented Woven, a sensor-laden prototype of the first wearable gaming platform. Save for a glowing montage of LEDs on front, it looks like a sweatshirt you’d pick up at Dick’s Sporting Goods. Instead of button-mashing, you interact with it through jumping, kicking, walking, punching, and, you know, hugging things. What’s more, there’s not a single screen in sight.

Currently, its lone game is a physical hunt called Spooky that has players grabbing ghosts out of thin air. It sounds fun—a bit like the street game Ninja, or the graphic-less Johann Sebastian Joust, if a little spastic. “We can’t stop the player from moving his arm in a particular direction. At least, not yet. This makes most ‘shirt’ games feel like you’re waving your arms and legs in the air without any physical feedback,” Ribbens tells me over e-mail.


The way it works is thus: the shirt uses a gaggle of bend sensors to register the positions of your arms and knees, then feeds this data into the game. But building onboard interfaces is not easy. It’s even hard to initiate a game without a screen telling you to press start. “We are so used to visual feedback that when we take it away, we are quite limited. How do you tell a player what to do… if you don’t have an explanatory text, a digital button, or a graphical icon?” he asks, before answering his own question: audial cues and vibrations gave them just enough control to guide the player.

Ribbens says that the best application of this warm and comfy cotton tech is probably in live-action roleplay. He thinks his shirt-games, and wearables in general, are a good fit for augmenting the imagination of those dressed like knaves and mace-wielding ogres in the wilderness. He acknowledges that LARPing isn’t for everyone, that people will probably always want to play videogames in front of the TV. But as game developers figure out ways to naturally interface with our bodies, he says, “being connected wirelessly with a game will take on a new meaning.”