Shining in the Darkness
My first experience with the audio game Swamp is an eye-opening one, in spite of the fact that I can’t see a thing. The game lacks graphics, and I have blindfolded myself so that I can focus on playing. This comes at the expense of mastering the complicated controls, which make use of the mouse, the arrow keys, and a panoply of letter and number keys. Yet I do better when I block out the ambient lighting of the room and the clutter of papers on my desk. I’m wearing a pair of monitoring headphones, which convey to me the game in its entirety: the map of the swamp, the other players, and the undead creatures. The results are ethereal. I feel I have tapped into an invisible otherworld—as if a battlefield has vanished into thin air, but the sounds of fighting can still be heard.
I’m lost in literally thousands of softly trilling beeps. Imagine Predator vision, aimed at the ears instead of the eyes. The sounds are various types of radar, and the frequency and pitch of the beeps relate how near or how far I am from a mini-mart, a shelter, or a parked car. In this way, a soundscape of the city swells around me. I’m not very good at reading it, however, and tend to get stuck in corners. As I find my way out of the parking lot, I notice that textures of sound are emerging under my feet. My footsteps convey that I’m walking across cement, then grass, then marsh. I hear another player’s footsteps running past. I follow them, but can’t keep up. Gunfire patters behind me, so I go in that direction. Before I get there, I run into a zombie, who greets me with the typical zombie moan. I turn until the moaning is balanced equally between the left and right cups of my headphones. Then, I shoot into the darkness.
/ / /
Swamp is an online, cooperative first-person shooter for those with vision impairment. You can think of it as Left 4 Dead for the blind. The game has more than 1,500 accounts, and on an active night, you will find around 20 to 30 players on its server, running through the dark and exploding zombies’ heads. Audio games have largely gone unnoticed by the gaming public, nestled away in small pockets of the internet—overlooked by not only the sighted, but also by their intended audience. People don’t tend to talk about things that can’t be seen. The earliest written languages were cave paintings and pictographs—visual representations of feet and vaginas and buffalos that eventually were stylized into cuneiform. Even music reviews tend to focus on the visual. They are littered with metaphors, musicians’ personal histories, and cultural references. Simply put: If it’s out of sight, it’s out of mind.
The relative obscurity of these games illustrates an unfortunate truth: The disabled are usually not invited to the party.
Your average Call of Duty or Resident Evil fan has likely never heard of audio games, such as the simple versions of Pong, Monopoly, and hangman created by Jim Kitchen, a programmer who was diagnosed with the eye disease retinitis pigmentosa as a child, and completely lost his sight by the age of 31. Nor do they know of AudioQuake, a mod of the classic shooter Quake that makes it accessible for the blind. Terraformers, which won an Independent Game Festival award for sound design in 2003, constructed a space colony using the 3D sound technique known as binaural recordings. Recently, the iPhone audio game Papa Sangre used similar technology to plot the bowels of hell. When players pivot in a circle, they hear the snores of sleeping beasts rotate around them.
The relative obscurity of these games illustrates an unfortunate truth: The disabled are usually not invited to the party. It’s not that they don’t have the capacity to play, but that games typically aren’t designed in ways that allow them to. Blindness is an extreme case. Even so, blind people can perform admirably at games with quality sound design, such as the Nintendo 64 shooter Perfect Dark—once they memorize how to navigate the menus. Other people with disabilities face different barriers. According to Bill Donegan of SpecialEffect, a charity that modifies game hardware so that the disabled can use it, “The biggest obstacle is the controller. The main obstacle is finding the right hardware for them to use.” Whereas a modern gamepad has 19 different buttons, people with spinal injuries, or those who are in the later stages of muscular dystrophy, can generally handle very few.
Still, there is a silver lining, even if it is unintentional. The videogame industry’s recent trend of roping in broader audiences has resulted in a windfall of accessible games. Companies have made a push to simplify. “With Forza [Motorsport 4], having the option to auto-brake can be a big help,” says Donegan. “In FIFA, over the past couple of years, they have put in a two-button mode so you can play the game with just the joystick and two buttons.” Beyond that, he is happy with the increase in games that only require one button, such as Fotonica for PC and Canabalt for iPhone. He is encouraged by the newfound abundance of easy-to-use controllers, such as the Wii Remote, the Kinect camera, and iPhone’s touchscreen. They make it easier for one-switch and two-switch players (the number denoting how many buttons a disabled player can use) to join in the fun.
/ / /
Jeremy Kaldobsky, aka Aprone, the creator of Swamp, began by making one-switch games. The idea started as a joke. During a game of Babo Violent 2, a bloody shooter that is half Smash TV and half Halo, one player made the crack that a teammate was playing so badly that “Aprone’s cat” could beat him. Kaldobsky took the remark at face value. Holding degrees in mechanical engineering and computer science, he rigged up a little helmet, equipped with a video camera, for his cat. It turned out that the cat could play. Though the cat thought it was eyeing bugs on the screen, it was in fact chasing other players around as it moved its head. Though the stunt was done for laughs, and the cat wasn’t very good, Kaldobsky realized the potential. This inspired him to begin developing accessible games.
Although Kaldobsky has been making small games as a hobby for 15 years, he only recently got into audio games. When he was introduced to a small community of blind gamers, he saw that they were stuck playing badly outdated games. Because most audio games are made by blind people who lack the coding know-how, or by programmers who lost the ability to see long before the modern era of games, their scope is limited. Simply looking at a list of audio games reveals how miserable playing games while blind can be. Games like Yahtzee, Simon, trivia, tic-tac-toe, and casino games originating from the ’80s remain popular. It’s as if the history of videogames never happened. Blind players rejoiced when a version of Colossal Cave Adventure, a text adventure from 1976, was released with speech recognition on the iPhone. Kaldobsky wants to help bring audio games up to date. In addition to Swamp, he has made a tower-defense game and a life simulation, and plans to create a dungeon crawler.
“Objects in the game are constantly making you aware of them. This means the types of gameplay you have can be different.”
Swamp isn’t simply an ordinary first-person shooter without graphics. Audio games rarely, if ever, make for a straight translation. Although Swamp has a graphical mode (it displays an overhead view of the marsh, with less detail than Frogger for the Atari 2600) so that sighted friends and family can play along, the game is infinitely more compelling with the lights off. Conventions such as running and shooting have been supplemented with tools that enable players to find their way in the dark. One key addition is an audial compass, which announces whether your character is facing north, east, south, or west when you press the “W” key. There are also beacons of sound that can be toggled on and off. Think of these as lighthouses—sans the light. Each numeric key on the keyboard will trigger a sound cue, which resonates from different spots in the game. The result is similar to checking the mini-map to establish your orientation.
These sounds convey a big picture, while others show the world that is immediately around you. The game utilizes several types of radar that sweep in front of you within arm’s reach. You can imagine these as a sonic walking cane. The beeps simulate echolocation, the technique that animals like bats and dolphins use to navigate—and a skill that some blind people claim to learn. In Swamp, radar sent in front of you will beep five times. It begins at the far left and moves to the right until it is directly in front of you, and then continues onward to the right. The radar can be sent in four directions (forward, backward, left, and right) via the arrow keys. It makes different tones depending on whether it detects a wall, empty space, or other structures.
Another type of feedback is plotted onto the map itself. Kaldobsky noticed that players were having trouble escaping from buildings when under attack by zombies. They weren’t able to find the exits under duress. So he implemented a clever system that would lead players to safety in a jam. He mapped each building to a grid, and assigned the sound of a heartbeat to each space on the grid. As you go deeper into a building, your heart rate increases, and as you come out, it slows down. It is, in effect, a breadcrumb trail of sound.
If that sounds complicated, believe me, it is. But even after an hour, I was making great strides at moving around. Swamp’s regular players have navigation down to a science. Kaldobsky told me an impressive story about a squad of players who made a synchronized sweep down the aisles of a grocery store, returning to a single-file line before marching on to the next set of aisles. Kaldobsky claims that players learn to visualize the game. He compares playing to hypnogogia, the hallucinatory mental state that occurs when you are on the verge of falling asleep. “You can ‘see’ dream images in your mind. When you open your eyes, you realize you are not seeing anything. But while your eyes are closed, and you’ve given it long enough, you forget what things really look like, and the dream images seem real,” he says. “In the game, you will start to picture the environment just based on sound. You forget that you aren’t really seeing.” Perhaps with more nuanced sound design, a vivid world would emerge, like the image of la Grande Jatte in Seurat’s famous pointillist painting.
Holding degrees in mechanical engineering and computer science, he rigged up a little helmet, equipped with a video camera, for his cat. It turned out that the cat could play.
According to Paul Bennun, a director at Somethin’ Else, the company that created the acclaimed Papa Sangre, audio games “are the ultimate first-person games…. You’re right inside it.” He doesn’t consider audio games as being strictly for the blind. He believes that the exploration of sound expands the broader field of game design. One difference is in perspective. “Objects [in audio games] have a persistent presence, even when you’re not looking at them,” he explains. “That’s quite interesting, in terms of mechanics. Objects in the game are constantly making you aware of them. This means the types of gameplay you have can be different.” Even though your character is facing one direction, you are aware of everything around you. You can hear in every direction. It’s like having eyes in the back of your head.
Kaldobsky agrees that audio games could evolve the way we play. “Before Dance Dance Revolution, if someone came up to you and said that there is going to be a videogame that you have to control with your feet, you’d probably think they were stupid,” he says, “[But] it became huge.” He thinks audio games have the potential to create a paradigm shift, like Guitar Hero and touchscreens and motion controllers. Picture a version of DJ Hero that doesn’t rely on the television at all. You sit in front of the turntable controller, slip on your surround-sound headphones, and perhaps close your eyes.
More importantly, however, Kaldobsky believes that a similar sort of sound design could one day be used to help the blind see in real life. When I asked if physical space could be conveyed in sound, the same way virtual space is in Swamp, he said, “If the microphones were powerful enough … and you had a system that positioned them properly, I see no reason why you couldn’t. It would be like virtual reality.”
But even if that never happens, the blind can still shoot zombies.
Illustration by Michael Rapa; zombie waveforms from Swamp
