Imagine a machine that allows you to project an avatar into alien space, to receive sensory feedback as you traverse and manipulate a virtual world, controlling the action with a series of remote tactile controls. This description may sound befitting one of the techno-toys populating a William Gibson novel, but it more accurately describes Ming Mecca, a modern hardware device that jacks game worlds into a matrix of sound and voltage.
The device has the look of a well-machined switchboard, a combination of two silver rectangles decked out in a shiny symmetry of input jacks, dials and switches. Branded as a “videogame synthesizer,” it is designed to generate digital environments on the fly. The larger rectangle creates the world, translating analog voltage into real-time rules and landscapes, while the smaller box enables in-game commands, housing an NES controller input that allows users to manipulate the voltage interface. Everything gets output via a composite video signal, but in order to see it in action, you’ll need more than just an old TV.
“The first thing to understand — and I think the most common point of confusion for people not familiar with modular synthesis — is that Ming Mecca needs to plug into a larger Eurorack system in order to be useable,” Ming Mecca’s creator Jordan Bartee says. Eurorack is one of several standards for today’s synth rigs, dictating size measurements and voltage power to ensure all intermingling modules can connect and communicate. Other standards are available, but trying to squeeze Ming Mecca into a non-Eurorack synth set-up would be sort of like trying to connect a Sega CD to an SNES.
Derived from the early analog computers of the 60s, modern modular synthesizers still operate with a similar basic structure. Like building a car, your rig needs to start with a frame and an engine. For synthesizers, that means getting a case and power supply. The case is where you arrange your modules, connecting their ribbons to a main chip board, and the power supply pumps juice through it all. The modules are patched together with cables, and communicate via a language of analog voltage. Through it, they synthesize music and sound. The twist with Ming Mecca is that, rather than synthesize sound, it processes voltage to synthesize game worlds.
“The easiest way to understand Ming Mecca is to think of it as an experimental programming language optimized for play, improvisation, and free exploration,” says Bartee. “The crucial difference is that instead of writing lines of code, you program using a tactile, physical interface.”
Just like you can tweak dials and switches on a synth module to augment the tone and style of sound, Ming Mecca allows users to mess with the rules of virtual worlds via the same interface. This means you can change the placement of objects on the screen, cycle colors, and morph the boundaries in which everything operates via the same kinds of controls. The way those objects act and relate to each other is further manipulated by interconnecting a series of input jacks with cables. “These voltages act similarly to the dials and switches, manipulating various system parameters—but do so procedurally based on the voltage signals and their interconnection” Bartee says. “By interacting with a physical interface composed of dials, switches, and switchboard jacks, interactive graphical worlds can be improvised in realtime.”
Ming Mecca’s creation is derived from a very west coast perspective, according to Bartee. He explains that the east coast synth scene—defined by companies like Moog and Arp—had a focus on live performance, attaching traditional instruments such as keyboards to the rig to create music. On the west coast, however, the dominant companies were Buchla and Serge, who focused on creating analog computers capable of “programming generative systems using algorithms and automated behaviors.”
“So from an east coast perspective I think Ming Mecca must seems pretty counter-intuitive,” he admits, “but from a west coast perspective I think it’s almost inevitable.” Connecting videogames to the world of modular synthesis seemed like a natural extension. “Synth heads definitely have an intuitive understanding of coding in a broad sense, since modular synth patching is really a physical form of programming,” he says. “Videogames are essentially systems, and modular synthesizers are interfaces for creating systems.”
However, you aren’t going to use Ming Mecca to create a traditional game, which should seem obvious considering the interface. There’s no way to “export” a creation, and the design is more geared towards free-form exploration and play. “If you come to Ming Mecca expecting to recreate Zelda or Mario,” Bartee warns, “you’re going to be pretty disappointed.”
So how does it all work?
“Right now Ming Mecca consists of two modules: the Control Core, and the World Core,” Bartee says. The World Core generates the game world—creating graphics, processing physics and collision and outputting an analog NTSC video signal. The Control Core, conversely, allows NES controllers to work as voltage manipulators, translating button commands into “modular-friendly voltages.
“The basic idea is that graphics data (sprites, tiles, tile maps, palette definitions, etc) are loaded into the World Core via an SD Card. So the actual pixel-level asset construction happens on a laptop, either by directly editing our ASCII-art-like text format, or by using our spritesheet conversion tool,” Bartee notes. “Once the assets are loaded into the system, the World Core’s interface and voltage patching is used to move those assets around, determine how they’re displayed, and how they interact with both each other and with user-input via the controller.”
For instance, sprite behavior is dictated through “control voltage.” Patching a cable into a sprite’s X-axis input jack, for example, allows users to move that sprite left and right via voltage. If the voltage is zero, the sprite sits all the way to the left, and will begin to move linearly as the voltage increases. “0V will place it at the extreme left side of the screen, 2.5V will place it in the center, and 5V will place it on the extreme right; ditto for all the other values, sweeping across the screen in 1-pixel increments,” Bartee explains.
Other parameters, such as gravity, are dictated by a more simple “gate input,” which Bartee describes as a sort of on-and-off switch. Pump in something above .5V and gravity turns on. Drop it below .5V and everything floats.
When patched into other modules, collision events—such as sprites hitting tiles, colliding with each other or running into the edge of the screen—can create more gates which Bartee says can then trigger “sounds, increment counters, feed Boolean logic structures, etc.
Seeing this all in action is jarring and mesmerizing. The rig itself looks like something ripped out of the chest of a robot, all cords, dials and flashing lights. On-screen, sprites get violently whipped about a shifting game world, sound and image reacting to these sharp, visceral jumps in voltage. According to Bartee, Ming Mecca (and modular synthesis in general) lends itself to such madness. Even on a traditional synth rig, one of the first things people do, he says, is make really harsh, chaotic noise. “Which is great, obviously—I love that stuff, both in the musical domain and in visual art and games.”
The desire to create Ming Mecca stems, in part, from Bartee’s early experiences with the Commodore 64. “There was an uncanny brokenness to the C64 that got under my skin” he says. “Just loading a game was an exercise in esoterica—you had to enter in a sequence of BASIC commands to access the floppy disk drive, load a program into memory, and then run it. And once you ran it, typically the machine would go a little crazy, flashing random colors or filling the screen with garbage characters.”
Creating the project offered him a way to “capture and extend the feeling I had messing around with the C64 when I was little.” Ming Mecca is partly a way to investigate that glitchy underworld, to dive into the underlying workings beneath the screen. “It takes parameters and systems of videogame worlds that are normally completely hidden and closed,” says Bartee, “and splays them out across a physical interface to be interrogated.”
“Glitch is deep in my bones,” Bartee says. “My formative experiences with technology were pretty much glitch-oriented, mostly involving the Commodore 64 as I mentioned, but also in the form of my early art-making. I had a casio keyboard when I was 10 years old or so, and if you under-powered it, it would just freak out and make the most incredible sounds. When I got into music as a teenager, I sampled it and used it in a lot of my music.”
For Bartee, glitch art is more than nostalgia—it is an avenue into new ways of thinking, a way to explore the ontological world. “I love the aesthetic itself,” he says, “but that’s always been secondary to the interesting problems it puts forward. The relationship of the visible to the invisible, the waking world of normal consciousness and the dark complexity of dreams, the hidden systems underlying and inside of nature—I see all of that reflected through the language of glitch.”
Images courtesy of Jordan Bartee/Ming Mecca