Composed roughly of an architect, a software developer, an art and tech writer, and an electronic engineer, Foreign Objects coalesced as a multi-disciplinary research and creative studio that explores what it means to live in a networked world. Through participatory workshops, designing games, and interactive installation spaces, Foreign Objects’ multifaceted portfolio brings us closer to critiquing through play the Internet’s strong presence in our lives. 

Their latest project, a web-based game called Bot or Not, challenges players to figure out if their online chat partner is—as its title suggests—a bot or not. Here, we speak with FO’s members based between Toronto and London about their origins at MIT, deception inherent in chatbot interfaces, and translating across creative disciplines.  

FEELING FOREIGN

Can everyone share a bit about their independent creative and intellectual practices outside of Foreign Objects? 

Sam Ghantous (SG): My background is in Architecture. I teach at universities between Canada and the US. My independent practice doesn’t have a particular name but generally deals with the entanglements between contemporary media and architecture.

Kalli Ratzepi (KR): I work as a designer and a software engineer. I’m interested in questions about interfaces and going back and really reading the history of computation, and understanding the interfaces we interact with—their histories and how we can re-imagine them. 

Agnes Cameron (AC): My background is in electronic engineering. More broadly, I’m into computers through a real interest in simulation, which also bears an interest in games and zero player games and ideas about that. For the last year and a bit, I also had an independent technical consulting practice where I worked with artists to realize artworks.

I work for the Knowledge Futures Group, which came out of the MIT Media Lab, where Kalli and I studied. They’re broadly working around semantic and decentralized web stuff, which is also something that I’m pretty into and interested in. I mostly do software development, but I’ve also had a past in fabrication and engineering.

Gary Zhang (GZ): I originally studied painting, but broadly, my education is somewhere between that and literature and philosophy. I also freelance as a writer within art and technology—that cursed pairing. And an art practice that is adjacent to technology but a little bit more conceptual. At the moment, I’m interested in social infrastructure, especially around finance and insurance.

I am a researcher at the Berggruen Institute. I run a research cluster called Non-human Networks and Markets, which is part of a program called the Transformation of the Human, which is broadly art technology philosophy. Just trying to restart this great synthesis of sorts that started in the 60s.

How did Foreign Objects emerge? Were you influenced by existing collectives, or was a space that you needed to carve for yourselves—something you didn’t see in the world?

GZ: We all met at MIT. Sam was already a year out of architecture and teaching by the time that the other three of us met at the same time as Agnes and Kalli were in the media lab. I was downstairs in the visual arts program in the same building, in the basement of the old media lab.

 We started as friends, really. It wasn’t obvious that we’d start working together. We’ve been friends for much longer than we’ve been working together. The Decentralized Web Summit in 2018 was probably a big moment for us, both practically and methodologically. So meetings and also community-wise. That community brought us to that and continued to be the discourse and a group of people we would work amongst.

SG: We should also acknowledge that we’re also two couples, maybe.

[Everyone laughs]

GZ: It’s like a long double date.

AC: We didn’t name ourselves until quite a long time after. We did this first project, Internet As a City, a facilitated game and workshop made for Decentralized Web Summit’s art track. Kalli and I had both been doing a lot of work in the crypto space. We had many frustrations with this idea of everyone inventing social systems and assuming that you can name a set of rules in a technology, and the social system will follow.

We had been frustrated with that and wanted to trouble that a bit. The way we chose to do that was really to look at spatial practice. People have been thinking about how spaces could use rules and protocols to interact with one another for such a long time. We used that metaphor as a way to explore. You can make a structure, but it doesn’t mean that people will use it like that. So we brought people to collectively make all these spaces together.

SG: It took you all finishing school for us to recognize that we all were in a sort of precarious situation when it came to visas and work, which prompted us to formalize this collaboration to practice.

AC: The name Foreign Objects also had its roots in the office that Kalli and I shared, which our adviser used to refer to as the foreign office. I really trashed a whole day, very elaborately vinyl cutting the foreign office logo, sticking it on the glass. It’s still there because it doesn’t really come off.

GZ: That was some kind of medium specificity fallacy in naming ourselves Foreign Objects, as though by identifying ourselves as by nature foreign in the US, that would by revealing that somehow change the situation, and we would no longer be subject to all of the pitfalls on that. We just ran out of Visas. It’s been impossible to get back in because all the Visas are on hold.

INTERDISCIPLINARY EXPLORATIONS OF THE INTERNET

There’s so much that Foreign Objects encapsulates—interactive design, research, software development, and more. Can you explain in your own words what Foreign Objects is?

SG: It is the coming together of four people from completely different disciplines. 

AC: We all have come to the places that we are through quite circuitous routes and have never really felt like particularly at home in a discipline or a practice. Because of that, we enjoy working together and have a fairly weird perspective on things sometimes—which is the other half of where our name comes from. We’re both foreign in the US and somewhat foreign to the places we’ve ended up. 

GZ: On different days in February, we were a branding agency, an architecture studio, a game maker, a space maker, web designer. Back end, front end, whatever.

When you end up defining yourself negatively against so much of the things that you’re supposed to be doing, you have to make a space. At least some kind of little shop front that you can hide behind to be recognized as an entity and have a logo and build an LLC so you can pay taxes. You want to be creative and critical in all these ways, but it doesn’t seem to fit anywhere, necessarily.

SG: We’re also generally committed to working around referencing and awareness about the Internet.

GZ: The more genuine line we have used somewhere is that we’re interested in making artifacts for ways of living in a networked world. What we gravitated towards with Internet as a City, was ways of learning together—to look at the Internet, look at this piece of technology as a question of how we want to live together. That’s obviously a social and political question. It’s also a question of how do we build infrastructures, what those infrastructures mean, and so on.

You’re coming from these different backgrounds, and some sort of consensus has to be reached initially to start a project. How do these conversations happen, and what compels you all to say, “We want to do this project” and follow through with it? There must be many ideas being tossed around.

KR: If the project contains a technical component, Agnes and I will take the lead there. We’re all pretty opinionated. This back and forth is very much how we did things. In the end, we just converge to something. 

AC: Everyone has their shtick. There’s a period of initial touzle, and then eventually, things take shape, and everyone goes along with them.

KR: Humor is the way to go forward because I would assume my ideas would be, “Oh yeah, let’s think about interfaces.” Then, as we play, “Oh, it’s a simulation for sure.”

AC: “Let’s make an operating system.”

KR: Then we think about the space. Then I think, “Okay. What can we design?”

SG: It’s four languages that require some degree of translation. Be it references or ambitions for what we care about in the world and the format it should take. So there is this period of trying to understand the other person’s thing or find shared ground, shared affinities.

KR: More concretely, writing things down or just creating images, sketches is integral and has been very helpful, in particular cases. Obviously, you get excited about something, and then you cannot get it out of your head.

GZ: We’re all researchers. We’re all discussing together while also pursuing our own research, which may or may not actually result in work in our minds, but through sharing Are.na channels and throwing ideas together. Agnes will be looking at the history of operating systems, and Kalli will be studying interfaces. I’ll be looking at some random bullshit in finance or something, and Sam will be thinking deeply about space. 

What might mark our difference is that we are really interested in the archives, in representing this piece of technological history that really defines the development of so many of the ways we use computers today. We think about networks, and so on, and that has built America’s infrastructure more or less. A lot of it comes from wandering around and talking to people.

HOW BOT OR NOT WAS MADE

The game Bot or Not is a playful critique of online chatbots. What drew you all to the medium of a game for this project? 

AC: The genesis of the project started as a joke. Mozilla had put out this open call for proposals relating to truth, AI, and disinformation. One of our lines was, “Oh, you can have a truth or dare Turing test.” That was the original title of the project. 

From there, it became more of a fleshed-out idea of a game. A lot of what goes into making chatbots effective is not the chatbot itself, but it’s the context you put it in. It’s a clever engineering of context, and a bit of sleight of hand is what can make these things really effective. 

Games are wonderful because everyone knows the rules. You can pick a well-known game. You already have a lot of infrastructure put in place, so you’re working within this medium where suddenly questions and answers are expected, or there are expected ways to digress from a model. 

The framing of a game is really helpful in creating any work with a chatbot. A lot of what you have with it if you’re talking to the bot is kind of a scam in that there are lots of things that pretend to be smart. AI is not very smart at all. A lot of AIs are humans pretending to be computers. Or these bots rely on an assumed idea of context. Or really on 50% of the time they’ll get it right and 50% of the time it won’t, but half of the people leave with the impression that this thing is really smart.

What were the different technical steps involved in building the game? 

KR: It’s essentially a Dialogflow implementation. The app is a simple front-end that is mobile. It’s talking to Dialogflow in the backend. I remember one detail that we had to omit, which was in the initial proposal, that this would be a game where you would be paired up with either human or bot. You didn’t know what it was, but you could also record audio.There were dares in [the initial version].

AC:  The backend was actually two components, and it got quite sticky at one point. Basically, whenever you’re chatting to the bot, things you say go through this pipeline. Initially, whatever you say is screened for really common sentence structures. For example, if you say something really shitty, the bot will also pick up on that. So there’s a handler at the front, which responds to very recognized, very structured questions.

For example, if you propose a ‘would you rather’ question to the bot, Dialogflow is really not very good at dealing with structured syntax. It won’t recognize keywords from a sentence structure. There are some little bits on the front end that will recognize shapes like that and then respond. So if you ask it a would you rather question like, “Would you rather kiss your sister or eat a brick?” It will just pick out one of those—sometimes with horrible results.

Dialogflow is a weird platform because it’s intended for writing customer service bots. Our project really tested the limits of Dialogflow as a medium. By the end, we had almost written our own chatbot. 

COMBATING AND CRITIQUING THE AUTOMATED WORKSPACE

What did you learn from the process of creating the bot and writing the content?

AC: Dialogflow is a nuanced context passer, like a dialogue tree. It was interesting to see the assumptions that get built into how these platforms are meant to work. Something we wanted to talk about with this project, is the labor dimension of automated agents, especially in a workplace—for example, a customer service workplace.

Half of the people that you encounter on a day-to-day basis won’t be people at all in the next 25 years. It’s already happening with a lot of workplaces that are becoming semi-automated—Google Home making calls to hairdressers; things like that. All of these things are very rapidly coming to dominate particular professions, automate them, and make working conditions, in some ways more precarious and tougher (not that they are great in a lot of these places).

It was interesting to work directly with a medium by which people are being automated because Dialogflow is a Fire your secretary? Use us! kind of platform. You can then just get something to respond to you really quickly. It’s interesting to prototype with it. 

We were reading a lot about ELIZA, which is Joseph Weizenbaum’s psychotherapist chatbot, which is a wonderful example of using context to make something that’s utter shit. ELIZA works so well because there’s this very nuanced exploitation of contextual tools in that. That project really gave us a lot of good ammunition—if the bot doesn’t understand what’s happening, it just asks you a question and tries to change the subject. 

It sounds like the bot is an educational tool to train humans to recognize what is human and technology. Were there any surprising conversations within the game or responses from players? 

AC: One of my big regrets was that we took quite a moral stance to record people’s conversations in that we didn’t. Other than people telling us, we don’t know what people have said to the bot.

GZ: It was something to do with doing the right thing [laughs]. 

AC: A friend of ours sent it to his friends, and one was like, “Wow, I thought this was going to be my new boyfriend. What the fuck?” So with some people, it worked really well.

GZ: So much of [reception] relies on—concerning games—the idea that people want to have an emotional connection with technology. Broadly speaking, our game always requires you to have some kind of effective connection with this otherwise entirely artificial thing. You’ve got to suspend disbelief. The wider chatbot industry is that how many of these technologies that are being massively invested in are about exploiting this innate desire to believe, rather than disbelieve in the authenticity of another voice or another agent. To an extent, that was also the trick that we were playing—to raise a little bit of awareness as well.

AC:  We ran a workshop called Botchat, which was also very informative in the early bits of development. We did it at NEW INC and with undergraduates at Rutgers University. We got people to embody bots, so we would, throughout the workshop, talk about different philosophies as bots, and then get people to essentially train themselves as art critic bots, and then respond to artworks.

They got a bunch of artworks to critique and would come up with programmatic ways of responding to them, and then they would be shown as a list of artworks that they’ve never seen before and would have to use their program to critique them. You got some great ones like if someone stared at the paintings about Jesus. It’s silly, but we were thinking—what does it mean to be one of these agents, your embodiment, and how do you occupy a person’s space online? Especially since a lot of these bots are being used to replace people.

REIMAGINING THE WEB

Can you elaborate more on Internet as a City? What compelled you to imagine the web as a space?

KR: Sam is an architect, and I was always interested in finding parallels between what we see in the tech world, and specifically the web, architecture, urbanism. When we started, I went through a phase of reading from one discipline and finding out if there are parallels in a completely different discipline. Talking with Agnes and later, the four of us led to asking questions like: If a website is a building, how would the building be? Or, Is the web some sort of city in the same way that historically cities were created? People inhabited them, and they developed specific behaviors. Do we see that on the web today? We also took a more clear political stance against whatever building Facebook would be, for example, or tried to identify behaviors that we see in real life.

SG: We are using architecture or objects or cities here for metaphors for the purpose of communication. The biggest thing with Internet as a City is how do you have a conversation or even an interaction about this topic, web access, and privacy, these issues with people of different disciplines.

Quite literally, the workshop we did was with blockchain technologists and artists. The city is something everyone has a familiarity with. Streets and roads and how things connect, and it’s used as a way to produce access or communication or have a conversation about these internet issues. 

GZ: The city is an interface to live in general for most of us is a familiar metaphor. It makes it easier to understand the Internet, not as the other things it is: a bunch of wires, a series of corporate structures, a series of communication protocols. We understand it more as a space that we engage with through interfaces, socially and personally.

And the project took the form of a participatory workshop and game, right?

GZ: The workshop centered predominantly around this stakeholder game essentially a simulation. We designed these 300 cardboard pieces, which were these differently-shaped blocks which could be used to literally build up a space using these cardboard objects, and they could be used to represent different things that the players intended. In each space, there were four roles: the public, the private sector, the governments and was there like an anarchist kind of role [laughs]

It was a two-by-two of interested stakeholders in this collective space that everyone shared, or it was like a governance experiment. People took turns to build—to put down a cable to put down a part of the city and the Internet, but in their own kind of vision. Then the next person would come over and maybe dislodge that and say, Actually, we need all of the communications running in the direction of my building. 

Eventually, people started inventing their own roles. There were definitely more destructive ones that came about by the time that. Half an hour in, people had built up these quite massive Lego structures using different blocks in this space, and it was carnage by the end. We didn’t really reach an equilibrium.

AC: The last part of the game was that all of the different groups had to collaborate with one another to make a meta-structure. Collaboration is definitely one word for it, but it descended into mayhem pretty fast. There were some generative and interesting conversations that people were having during. And there were many ridiculous little play-fights that people would have but also equally, there were some genuinely nice things that people took away in terms of social collaborations around networks and what that can look like.

What is Foreign Objects currently working on? 

SG: We’re the first digital artists-in-residence at the Bard Graduate Center. We’re working on a project about the kitchen’s social politics and how that changes under smart technologies. We discovered that, historically, the kitchen was this sort of centralizing space. It was not only the kitchen—it was a hearth. People slept around the hearth—they cooked food together. There wasn’t a social hierarchy; everyone was contributing and working collectively. In architectural history, that becomes more separated from the other functions from the house and is more distributed into smaller heating elements.

Now, you have a toaster, a microwave, a stove, an oven, and a crepe-maker. Smart technologies have a social function, but do not require our presence or co-presence in this space. They barely require our contribution! We hope to find a way to regroup all of these individual eating elements. Maybe we produce a party around a new kind o hearth, a set of social rituals, or habits for making food that really requires a group of people in the space and out of the space.

We’re taking the worst of these new technologies, finding redeeming qualities for social coexistence and new environmental consequences—’environmental’ being the space of the house in domesticity and so we’re trying that out across different mediums. Still, it will likely be a narrative website of sorts that has mildly animated images that depict this new set of relationships and rituals and spaces. Hopefully, it will also involve a recipe for distributed cooking: if it takes 15 people to cook a grilled cheese, what would that look like as a recipe?