What big data can’t teach us about videogames
Big data has finally come to the world of games. As designers now study players with more and more granularity, games themselves become mirrors of our own play preferences. But with big data comes big questions: If games become personalized experiences, how will we have water-cooler conversations about them? Where is the boundary between collecting information to better experiences and a deeper understanding of the real you? Over the next few weeks, iQ and Kill Screen will explore these questions.
On December 10, 1993, trained pollsters working from a central monitored location in Indiana started telephoning Americans to find out what they wanted to see in a painting. They asked them 102 questions. What colors, shapes and scenery? Realistic or abstract, serious or light-hearted? This first ever comprehensive poll of American taste in painting was commissioned by two artists, Soviet Union expats Vitaly Komar and Alexander Melamid. They also held town hall meetings where they talked to “regular Americans” about what they liked in painting and what kind of painting they would commision themselves.
What emerged from the data they collected and the centerpiece of their project was a painting they titled America’s Most Wanted. Most people polled thought that blue was the color they would like to see most in a painting so the painting is mostly blue. Most people wanted natural outdoor scenes featuring bodies of water, so that’s the landscape. Most people wanted to see animals, people and historical figures, so there is a few people, some deer, and George Washington.
It’s a terrible painting and a great joke. The juxtaposition of regular, contemporary Americans and our first President in his powdered wig and breeches. The Bob Rossian happy little bushes, so skillfully executed, so innocuous. It’s a grand scale design by committee.
Leave a painter like Pollock or Picasso alone with the canvas and they’ll create an entire new dimension for painting, a new way of seeing that reshapes the art world. Invite the audience to tell the painter what to paint and you’ll end up with the kind of utilitarian, decorative painting you would find in a cheap motel. It’s not expressing anything as much as it is just filling an empty space on the wall. It pleases or at least doesn’t offend guests that stay in the room, but it doesn’t do much else.
In the art world this kind of measuring of the audience is a big joke. In the games industry it’s considered best practices.
Everything all the time
I thought about Komar and Melamid after I met the CEO of Fuse Powered, Jon Walsh, at the St. Regis Hotel in San Francisco during GDC 2013. He opened his laptop and showed me a simple dashboard, one that may look familiar if you’ve ever managed a website or fiddled with Google Analytics. There were charts, graphs, sliders, drop menus. The service is called Fuseboxx 2.0, a tool for mobile game developers that collects player metrics and allows them to quickly modify their game based on those metrics.
“It’s very important to know where you’re getting these players and how they behave in the game after you’ve acquired them,” Walsh said. “We can tell you, oh, you’ve just bought 17,000 players from Tapjoy. The next day take a look at those players and check how many of them actually opened the app, how long did they play, did they convert at a normal rate to in-app purchases, or did you actually buy just a bunch of low quality players?”
If a lot of players are having trouble getting through level five the developer can increase the amount of health they start with in just a few clicks and that change will take effect the next time they play. “If players only last a few days and you’re waiting weeks to make fixes you lose all those players. . . That’s what it’s really about. Collecting the data, analyzing the data, and the ability to act on it instantly.”
I was confronted with the possibility that the game experiences I cherished most could be reduced to a science.
This isn’t a part of the game making process that’s paraded in front of the press often, and it’s even less visible to average players. It’s odd to see and think about games as a series of purely technical processes that can be optimized, and to think of myself, the player (I’m not sure if I’m considered “low quality” or not), as a consumer whose desires could be quantified and addressed with a simple click of a mouse. I believed or at least wanted to believe that there was something more to it.
It’s not just that I was seeing how the sausage is made. I was confronted with the possibility that the game experiences I cherished most could be reduced to a science. Think about your favorite game, whatever feeling you associate with playing it. Was that experience achieved or improved with something resembling a spreadsheet, or is there something slippery and unquantifiable at its core that’s more critical to the overall effect it has on you?
The developers I talked to, exceedingly reasonable people, agreed that it was some combination of both.
Craig Fryar coauthored the first 3D multiplayer PC game back in 1990 when he was working for Apple. He did this on the side, on nights and weekends. It was a tank battle game called Spectre, funnily enough, since Fryar is now Head of Global Business Intelligence at Wargaming.net (now 70 million registered players strong), creators of the hugely popular World of Tanks.
He told me that Wargaming isn’t trying to create an equation that squeezes the maximum value out of a property. “I know of other companies that use data and it trumps everything. That’s not the case here. The data serves the purpose of inspiring us and helping us get insights on how we can make things better, more efficiently, how we can do things that hopefully surprise and delight our players and actually engage them at a deep level.”
Ramin Shokrizade, a Game Economist for Wargaming, reiterated that big data is at best just a tool that serves the developer and player, that it can’t replace true innovation. “When you bring the two fields together, the creative and the quantitative, that’s when you get your best results. When you just have a product that’s quantitative, the symptom of that is you see a product that looks just like 20 products before it, it’s just been optimized a little bit.”
Fryar says that analytics always starts with a question you need to answer or a hypothesis you need to validate. Then you go out and find the relevant data. When Fryar first meets with stakeholders he asks them what they hope to accomplish with the data, what they need. “I would get this all the time, it’s fascinating, they would say, I need everything all the time. I understand where that came from. It’s a fear of missing something.”
Now, Fryar said, we’re closer to seeing everything all the time. He estimates that 20 percent of the data Wargaming collects is structured, quantitative data, meaning how people play the game, how they move through the maps, the weapons they use, kill/death ratios, etc. The other 80 percent will likely be qualitative, unstructured text. It’s the voice of the consumer, the comments people make on Facebook, Twitter, blogs, forums, customer service tickets and in-game chat.
“In former companies I’ve been with, we may have put up a press release, and looked at the response to that in our text analytics and see that we didn’t really expect this one little phrase would cause commotion, so we were able to course correct really quick. I refer to it as being able to throw a pebble into the pond of social media and watch the ripples in real time, so we can respond to that.”
The local maxima
Roland Barthes identifies two separate kinds of pleasure a reader can derive from a text: plaisir, commonly translated to “pleasure,” and jouissance, commonly translated to “bliss.” The former describes the pleasure in a text that satisfies our expectations. The latter describes the pleasure in a text that shatters these expectations. He writes:
“Text of pleasure: the text that contents, fills, grants euphoria; the text that comes from culture and does not break with it, is linked to a comfortable practice of reading.
Text of bliss: the text that imposes a state of loss, the text that discomforts (perhaps to the point of a certain boredom), unsettles the reader’s historical, cultural, psychological assumptions, the consistency of his tastes, values, memories, brings to a crisis his relation with language.”
Replace “text” with “game,” “reading” with “playing,” “language” with “interactivity” and you get the gist.
The evolution of all media is the dance between these two types of pleasure. A new idea, which provides bliss, disturbs the existing order of things. It reshapes our understanding of that medium and opens the door to new possibilities, new forms of expression, new audiences. That idea is then codified and optimized (the jump cut in film, electric guitar distortion in music, etc.). Where it provided bliss it is now the norm, an expectation to be met, a text of pleasure that will in turn be disturbed by a new idea, a new text of bliss.
Barthes writes that “bliss may come only with the absolutely new, for only the new disturbs (weakens) consciousness,” but that doesn’t mean that everything new and unsettling to us is blissful by default, “nine times out of ten, the new is only the stereotype of novelty.” Or, as indie developer of SpyParty Chris Hecker put it to me, “statistically, the likelihood is that you just have a crappy game.”
“I think you have to make unpopular decisions as a designer if you want to actually make cool stuff.”
Hecker wrote about what he calls “Metrics Fetishism” in 2010. He told me that with so many opportunities to get excessive feedback these days, a developer needs a strong internal compass or he’ll go crazy.
“I think you have to make unpopular decisions as a designer if you want to actually make cool stuff,” he said. “Especially if you already have players, because you’ll piss them off. If you have an intelligent player base, they’ll know that you have to hurt them sometimes.”
Metrics, he said, allow developers to polish existing solutions to perfection, but only intuition leads to new solutions. “When you’re at the top of a local maxima you’re at the top of this hill, and you hope that there’s another, higher hill over here, but the only way to get there is to first go downhill. Sometimes that’s the only way to get to the game you’re trying to make. You might be screwing your game up a little bit but you’re trying to get somewhere.”
Scientific illusion
Matteo Bittanti is a writer, artist and teacher in the Visual Studies program at California College of the Arts. He told me that the games industry and culture in general is stuck in a feedback loop because of big data. “It’s basically a conservative and reactionary power. What really depresses me is that it keeps giving me things that I liked in the past, so I’m playing Forza 5, Gran Turismo 6, Battlefield 4… We don’t have this kind of demented return to formula even in mainstream cinema.”
The problem, he said, is the almost mystical value we attribute to big data, the illusion that science provides all the answers. He compares it to our ongoing obsession with neural science, brain imagery that can supposedly teach us everything there is to know about human behavior.
“It’s the same kind of bias you had with photography where we thought it was the medium that delivers the truth, that it was reality. As if the positioning of the camera didn’t matter. We always forget about this discourse, as if now we’re using science so the debate is over. Big data is the same thing. It’s also marketing. You have to sell all this stuff. You have to sell this ideology. . . Journalists and critics don’t look at it and doubt it and say we had this kind of conversation before.”
Everyone I talked to at some point reminded me that video games are also a business, and that there is nothing wrong with that. I agree. Fuseboxx 2.0 seems like a useful tool and I’m sure it works to the benefit of many developers and players alike. I’m also aware that play-testing and focus groups have always existed in the games industry, and that many accomplished developer’s whose work I admire have been informed by data they collect.
What we seem to forget is that big data is also a business, and as such, it presents itself as something developers need. It does this by offering supposedly definitive answers. It employs data scientists who work in labs that produce empirical evidence on player behavior and taste.
But you can’t apply this kind technological optimism to an inherently creative process. It is and always will be messy, frustrating and personal. If we just accept data as fact, if we don’t challenge and at times intentionally go against what it tells us people want, eventually we’ll end up with nothing but the type of paintings you find in cheap motels.
Derived header image via Tim Sheerman-Chase, See-ming Lee
Scientist image via x-ray delta one
Digital landscape image via Gwen Vanhee
