This article is part of a collaboration with iQ by Intel.
Today’s biggest games, like the Witcher 3 and Metal Gear Solid 5, are graphical masterpieces, set in huge open worlds where every leaf and pebble is rendered in painstaking detail. They are truly sights to behold—as long as you can see them. If you can’t, then it’s nigh-impossible to explore these worlds. This puts people who are sightless or visually impaired at a severe disadvantage when it comes to the games they can play and enjoy. But this may only be a temporary setback. Researchers are developing technology that could help the blind navigate videogames effortlessly.
The future of sightless navigation is made possible by 3D cameras such as Intel’s RealSense. As has been written, these cameras have the cool, Matrix-y ability to scan an object in 3D as opposed to the flat images we see in Polaroids or on a TV screen. This has far-reaching applications, enabling robots to move gracefully through surroundings without bumping into things, and drones to fly through dense forests, avoiding trees. And when these cameras are used in tandem with wearables, they can do things like give humans extrasensory abilities.
At CES 2015, Intel unveiled a prototype of the RealSense Spatial Awareness Wearable, which is more or less a vest that sends impulses to its wearer’s body based on cues from the environment. You can think of it as the ringtones on your phone, only way more intricate, and attached to your torso. According to Rajiv Mongia, director of the RealSense Interaction Design Group, “we’re using a sense of touch as a way to give people a sense of sight.”
For an easy to understand example, Chandrika “Shani” Jayant, a Senior User Experience Designer at Intel, explains that “if you’re walking down the street and there’s an overhanging branch, then the top motors on the person will go off. After getting closer to the tree branch, the vibrations will get stronger and stronger.” That way you will know to duck or walk around it.
All that is well and good, but the big, critical, life-changing question is: how could these awesome human augmentics let the visually impaired play video games?
“There’s been a lot of work done on video games for blind people in the last 10 or 20 years with sound, but there’s no reason it couldn’t be done with haptics (i.e. vibrations that convey a sense of touch) as well,” Jayant says. “Right now it’s for a real physical environment, but there’s no reason it couldn’t be for a virtual environment.”
The most obvious usage is mapping the virtual environment into a system of nuanced vibrations: The Spatial Awareness Wearable gives immediate feedback, so a sightless player could dodge a virtual tree branch in the same way they dodged the real tree. In this way, the buildings and landscapes of the game world could be revealed to them.
That’s a compelling starting point, but games present other complexities, like picking up items and equipping a weapon. How could a blind player distinguish between a laser gun and lightsaber that was lying at their character’s feet? Well, going back to the example of a phone’s ringtones, specific patterns of vibrations that convey identifying information can be programmed into the suit, sort of like braille, but for your whole body.
The RealSense Interaction Design Group are currently fine-tuning the tech for identifying people, which can notify the blind whether a friend or stranger is approaching them. “If a person walks by they actually feel it as a wave going by them,” Jayant says about the wearable. A nice side effect is that it also solves problems that will help them play games. In a video game you need to pinpoint enemies that you are aiming at, dodge a punch, or deliver the coup de grace, and this could help the visually impaired do it. And with facial recognition, the identity of other characters could be translated into haptic pulses, so you know if you are dealing with a guild mate or foe.
This is technology that is still waiting to be realized, but the potential for the blind to join everyone else in virtual worlds is huge.