Lecture 21: Games as Teaching Tools

Related Resources

[CGD] Chapters 18 and 19

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

Some content in this audio is not covered under a Creative Commons license. For more information, see the course materials on the MIT OpenCourseWare website.

PROFESSOR: OK. So today we're going to, I guess, kind of [UNINTELLIGIBLE] folks from [UNINTELLIGIBLE] which is actually very closely related. These are folks that, [UNINTELLIGIBLE] the Comparative Media Studies Research Group, based in the old Media Lab building right now. And you work with the shared education program.

For folks who're unaware of this, MIT actually has a teacher education program. So if you want to be a certified teacher to teach--

AUDIENCE: Middle school math and science. Secondary math and science.

PROFESSOR: Then you can take classes and basically get your certification of [UNINTELLIGIBLE] certification at the end of those classes. But they also make games. Games using portable devices like iPhones and PocketPC games that run on Flash to go and prototype stuff before. And focusing a lot on games for learning. So the education [UNINTELLIGIBLE] the research group. The Learning Games Network is kind of like a spinoff company of the [UNINTELLIGIBLE]. It basically sort of works on projects that will be too expensive to do inside the university. Because the university is really inefficient.

Anyway. Maybe you can start by introducing yourself?

GUEST SPEAKER: I'm the latest in the group, so OK. I am Constantine. I come from Austria. I'm at the moment a visiting researcher at the Education Arcade. And so I'd say I come from an educational background. So I did my Ph.D on learning from failure. And I worked on theories that explain why we use, why we learn through mistakes and why most of them don't want to learn from failure. But in games we think it's really funny, or entertaining, or challenging.

So I was really interested in how games can be tools to understand learning and also, is it possible-- this is one of my research questions-- is it possible that people may learn, through playing games, how to cope with failure better? Like, being more open to realizing that sometimes in life we make failure and our expectations are not true, or sometimes maybe wrong.

So I do research on the theoretical level with that, and I do interviews with players on learning biographies and on how learning experiences through games were important in their lives. And in the summer I was happy to work for Gambit and design a game in the summer production. Maybe you heard about that anyway, or--

PROFESSOR: [UNINTELLIGIBLE]

GUEST SPEAKER: So it's just like 8 weeks. I was on a team and my theory was used as the basis to develop a learning game called Afterland. And of course it's free online. And I think our students did a really good job on this game. And it's a platformer where-- I will not tell you what it's about because then I would ruin it, but there is a twist. And what I'm going to do now in the next year is, it looks like I will try for a few months and do research on how the game is used by players. How they learn in the game, how they reflect on the game and I will work with [UNINTELLIGIBLE] Johnson and hopefully even [UNINTELLIGIBLE] in that. Thank you.

PROFESSOR: [UNINTELLIGIBLE]

GUEST SPEAKER: Hi, I'm Dan Reilly. I'm a CMS alum. I did a masters in, finished in 07. Worked with Philip a little bit here and there. Mostly I worked with the Education Arcade during my time at CMS and with Scott. And we worked on a project called Lunar the Labyrinth which you may have heard of. It teaches middle school math and literacy skills, particularly to the at-risk population that I believe [UNINTELLIGIBLE]. It's free and online.

PROFESSOR: [UNINTELLIGIBLE]

GUEST SPEAKER: Better than after me. Got to keep it interesting.

PROFESSOR: Is there a twist?

GUEST SPEAKER: Is there a twist?

PROFESSOR: Yes, well-- [UNINTELLIGIBLE] twist that you originally wanted.

[INTERPOSING VOICES]

GUEST SPEAKER: There was a twist in the development process. So, that one was feted by the Department of Education. And it was a larger scale project, but the grant didn't go to MIT, it went to Maryland Public Television. So that was part of the, I believe, impetus, for the spinoff of the Learning Games Network, is to house a structure that can accomodate the sorts of larger scale projects and think about them in more sustainable ways. I'll let Scott talk about that.

So, how did I get into this? My interest in learning games is mostly around engagement. I feel like people who are extremely motivated and enthralled with a particular subject will find a way to learn it. And that a game may or may not be the best way to learn anything in particular. But that there are many people out there who struggle to get interested in a topic they're even trying. And so a game can be useful there.

So that's an exploratory tool to say, well, I never really thought about history, but I like this game and I want to spend some time with it and I want to be good at it, and it's social, so now I'm finding myself asking the kinds of questions that other people are telling me historians ask. And I don't know what label they want to put on it, but I think it's interesting, and just sort of having that introduction to a topic.

Also I think games are helpful for-- I guess I'd call it identity exploration. So for people, or particular learners, who may not think of themselves as learners or as confident learners, that they can put that enemy behind them when they enter the game because, it's a game. And anyone can play it. Or, maybe I'm particularly good at games, so I can play games. And then I can suddenly realize that I might be good at other things as well, or I might be interested in other things that I wouldn't allow myself to discover whether or not I was interested, because it was a risky sphere to enter.

So, current projects. We're doing a language learning game. It's attached to a massively multiplayer online game, with a grant targeting a population of Spanish-speaking high school students in the US. And that grant is from the Hewlett Foundation. And then we have a couple of other projects going as well. We've got science and some other stuff. That's about it.

GUEST SPEAKER: So I'm Scott Osterweil. I'm the research director of the Education Arcade. I have two titles, research director and creative director of the Education Arcade. But also one of the co-founders of the Learning Games Network. And I don't want to repeat too much of what's been said. Except that at the same time as I do this for a living, I still struggle with the term educational game. Because it's not the way I think of what I do, primarily.

The first game I designed, before working in this setting-- the first game I did was a commercial game. It wasn't designed as an educational game, it was designed just as a puzzle adventure game. It happened to be about math and logic, but that was just because that's what we thought was interesting. It got marketed because of the changes in the marketplace as edutainment. But when I started in the business there was no such word as edutainment.

Most games, if you go back to the early 90s, were sort of exploratory environments. Either they were twitch games or they were exploratory environments. And I have nothing against twitch games, but the exploratory environments were all about thinking and problem solving. And to me that's what's interesting. No matter what domain you're working in, whether it's mathematical or whether it's historical, the processes are the same.

And one slight twist I want to put on what Dan said, I think we're on the same page, but because he led with the idea of engagement, it's possible to think that I'm talking about games as an inducement to do something you don't want to do. And I think that games are an opportunity to discover how much fun something is. Which is very different from sort of, ah, we tricked you . But rather, this is really interesting, don't you like doing this and you can do more. And if that's all you can get out of it, that's fine. If you later realize, oh, I like doing that and that's math, and you do more of that, that's awesome. That's even better.

And I think it's not so much because I just said it's fun, it's because I think the very process of thinking and problem-solving [UNINTELLIGIBLE] People do get that out of games, and I just think it's important. It's how people realize, if you enjoyed it in a game, you can enjoy it in life. And maybe this comes from my own realization . But when I was in sixth grade, instead of them saying the scientific method, and writing this list of dull steps on the blackboard, they said, play this puzzle. Now let's talk about how you solved the puzzle. That's science. I think that I might have ended up working at MIT.

So anyway, that's my spiel.

PROFESSOR: So talking a little bit about identifying how you're going to locate what's enjoyable about the thing that you want the people to engage with. Can you talk a little bit about, maybe in a recent project or in a past project, how you went about that process. Let's talk about the things that you worked out. And maybe sometimes it was more difficult and you had challenges. Maybe sometimes it was just pretty obvious.

GUEST SPEAKER: I'll take it. So--

GUEST SPEAKER: [UNINTELLIGIBLE]

GUEST SPEAKER: I'd like to congratulate Scott on his personal growth. So, with the language learning game that we're working on at the moment, there are approaches to language learning out there in the world that emphasize vocabulary memorization or grammar. Grammar, meaning, memorizing tables of conjugations and it's pretty much stripped of any sort of meaningful context. So, and a meaningful context could either be a text, and I use that word loosely, that is engaging. It could be a film that you want to see, that happens to be in a language that you are trying to learn. Or it could be a means of communication. After all, that's what language is.

So, if you're thinking about using language as a means of communication, then of course you need to think about, well, what are you communicating? And, fortunately, games are full of communication. Between the game and the player, and the player and the game. And if it's a multiplayer game, between the players in the game. So it was almost-- if you start from that philosophy, it's almost a trivial exercise to say, OK, well, what's something interesting for people to talk about in a game? And so you come up with an interesting game.

And instead of having the mechanic be, click this button to activate whatever the nuclear missile, you have to, you build in collaboration. So, I have part of the piece and you have part of the piece, and we need to talk about when to use what and in which context, and on whom. And so, suddenly we're talking and it doesn't even feel like we're learning a language, it just feels like we're playing a game. And then, if you're doing it in a language that you're attempting to learn, then there's the barrier there where you might want to say something that you just don't have the skill yet to say.

So it's not quite as simple as just saying, hey, go talk. But there's slightly innovative ways that you can give people tools to allow themselves to express themselves in a fairly constrained context. So, because you've designed the game, you sort of rig the system. You know that people aren't going to talk about absolutely anything. Maybe they will, in the chat, but primarily in the game you're incentivizing their discussion around a particular topic.

So if you give them tools to discuss that particular topic, not only does it make the gameplay more efficient, but that's also how the learning can happen. Because there can be some sort of selection mechanism, whereby I am able to say something that I want to say, but without needing to bring all of the knowledge of how to construct that on my own. There's some amount of, oh, I can just recognize that that's more or less what I want.

And then, by seeing it repeatedly and by seeing other players use the same tools to communicate, and knowing that the communication in a certain context is valid, that I can pay close attention to that, and start to become familiar and learn. But it's not, I have to memorize it so that I can use it. It's, I'm using it and I'm using it so much that I just can actually learn it.

AUDIENCE: Question on that. So, if you're trying to encourage me to talk Spanish and you talk Spanish and we're trying to work something out, what prevents you from just typing something in English and having you read it in English?

GUEST SPEAKER: Right, so it depends on the game. And in some games that's totally fine. In other games we have teams moderate other teams. In other games, we have some sort of communication tool that's not free text entry. So it's not-- so one extreme is, here's a list of 10 sentences that you could say. Click one of them. And that's not the direction we tend to go in. But that's an example of how to communicate something without free text entry. So there could be some sort of sentence construction kit, let's say. Where you have a pool of different words that you can assemble and a variety of orders to express a small set of meanings.

GUEST SPEAKER: And if you think about it, games like Charades or Pictionary, the need to communicate with constraints doesn't stop us from necessarily enjoying the game. It could make it interesting. So, that's part of the game. Part of our insight was thinking about our own experiences learning language in terms of the game of say, and this gets at my core idea, which is to find the game that's already there. So, a game that I always play when I'm traveling with [UNINTELLIGIBLE] when I travel to France, where I speak the language sort of, is how far can I get into a conversation before they figure out I'm American. So that becomes a game.

And so the whole, and how can I have or if I notice someone who's French, how much of a conversation can we have in French? That's a game for me. At that point it becomes kind of fun. So I think we were just really trying to access that. And I think that really gets back to your larger question. When you take on something new, I want to know where's the game already in this?

And I [UNINTELLIGIBLE] the first game I think, it was math, and the colleague with whom I co-designed the game, we both spent a lot of time saying, when we were kids, how did we play with numbers, what kind of things did we do, what was interesting us with numbers? We really try to access our own play. And similarly, now, if I can try to read practitioners in the field and say, what do you think about the shower, when you run it-- where's the-- because that's where the play is actually happening in the field.

And I think that's where people love their jobs, it's the play in it that they love about it. And that's what I think you're trying to access in any simulation. So Dan, talking about the language one, we're doing a game with the Royal Shakespeare Company, if we can get funding, around The Tempest. And we're trying to figure out how to make a game of it. And I realized that part of the change with Shakespeare is, to some degree, the language, too, and sort of making sense out of the language which is now somewhat archaic but really interesting.

And so we tried to make a game that's sort of a puzzle that you assemble by finding the pieces of the play and associative things. So you find a hedgehog and you find some text that makes a reference, that somehow makes you think of the hedgehog and you put them together and you actually start building a visual puzzle.

So it's really a challenge of making sense of the significance of text. It [UNINTELLIGIBLE] you and your picture of Shakespeare's play, The Tempest, starts to emerge through the gameplay. So a story sort of appears that way, the way a picture appears in a jigsaw puzzle. So it's a game you play without knowing, without ever having seen it. But part of the experience of theater is the way stories emerge from different patterns in the game. We're still trying to make a story [UNINTELLIGIBLE PHRASE] through exploration.

GUEST SPEAKER: Maybe if I talk about my career as coming-- I think I come from an area where we make really bad games. There's a saying that education games are like chocolate-covered broccoli. It's not healthy and not sweet. And I actually come from a background where teachers or educators are asked to build games for somebody. And what you have is contents.

And I can say that, because I think most of my colleagues seem to know better. They thought, OK, I have content. That content is good, and now I just make do with a song and the kids are going to love it. And I think, as you all see, Jeremy, we all know how quick kids realize it is not fun. And I think there are tons of bad games out there that ruin the reputation of educational games in a sense. And coming here to MIT and working with Scott and people like Dan, that I realized more, the question is more, what makes us curious about the problem, about the learning content. Of course, we can have the content of the game, something we don't even have to call it. It's more abstract.

We look for metaphors that are kind of like, if you can understand how to be interested in history without really learning all the real data maybe you learn in other history, but you get interested in how history works.

And I think that's kind of like what I learned here and I realize now that it's really useful, that we don't use the chocolate covered broccoli any more. We ask more about, what do we really want? Do we want to have something that is healthy or sweet, or something different, let's cook it. Let's not just combine two things. And I think there's something that also-- the good educational games that are out there are heading in that direction.

PROFESSOR: Do you have counter-examples from your work, of something that seemed fruitful but then kind of maybe you were trying to address some subject area and then were trying to marry some specific game mechanics you thought might work, but were disappointed?

GUEST SPEAKER: I could talk about one that's not resolved yet, that I'm struggling with. We're doing a game with a professor here at Ocean, in environmental engineering, about microbes. He does all his work study on microbes in the ocean. Which I've learned about. Half the biomass of the ocean are microbes. And we know almost nothing about them, because until people like him, he's one of a few, built apparatus to study ocean-like conditions in the laboratory, we had no way of knowing what was going on with them.

So that's interesting if you get into it. It's not something that middle school kids are dying to know about. So we're doing a simulation where the premise is that you've got a part of the ocean that's dead and you're trying to re-engineer it by introducing nano-bots, which you can build. And these nano-bots would in effect simulate the functions of what looked like processing oxygen. Or rather, plants processing either oxygen and producing CO2 and energy, or the reverse. Taking CO2 and sunlight and-- Anyway.

So, the challenge, we made a sort of interesting playable simulation where you build the stuff. But I haven't figured out yet I'm struggling with, have we found a way to entice you to do this, though, without having to give you too much information? And I'm a big believer that a game-- this game does not start with instructions, because I don't believe games should ever start with instructions.

But I'm still struggling with the information we're providing along the way. There's too much read this now, as opposed to-- and I think the problem is, because simulations are very wide open environments, and it's very hard to constrain a simulation enough that the player goes, ah, now I understand what variable I just used. And particularly because kids have a tendency to see simulations as black boxes. So they'll change three variables at once and not notice they've changed three variables, and not think about what each variable was. They just don't have the practice of changing-- really, controlling three variables. And if you don't do that, simulations do become black boxes, and I'm still struggling with how to-- not to, say, mark the kid down the road and say, OK, now make this choice, now make that choice. Because that's too constrained and not fun. I want to have them have a lot of choices but also have them understand the effect of the choices that they've made. Maybe that's the best way to sum it. So just processing the [UNINTELLIGIBLE PHRASE]

GUEST SPEAKER: I can give a better example. So, I was working on this game. I wasn't even working on that, to my excuse. I was beginning to work with these people, then realized it's not the way I want to go. And then I stepped out of it. And I think it's a good example of what you shouldn't do.

So this small town in Austria is using their waste to produce renewable energy. It's really interesting. The town is somewhere in the countryside and nobody was interested. It makes so much power now that they can give power for free in their town to companies and the neighboring companies in neighboring towns. So a lot of people are interested in how they do that. And we thought, well, we want to be a cool town, what if we make a game instead of just providing the content.

But what they did, they gave us all the content and said, this is what you should teach the people. And make it fun. And in the end, these people were sitting together, OK, what do people like? Comics? And you have to a character that is moving around. And they would make small multiple choice tests. And afterwards they get their rewards, and it says your name and it says you're trained in that discipline.

And what was interesting for me, they built the game. And the game's been bashed. And it's very sad and it's horrible. But the funniest thing for me was, who was the target group? And they never talked about that. They always thought who cares about target groups, we make this game, everybody's going to love it, everybody's so interested. The idea was who decides if you can do that in the majors of towns. They say, well, we want to have a company like that.

And so now we're talking about Austrian, European, majors of towns. And they are normally fat and like to drink wine. Normally they sit with another mayor and have lunch and then they say, wow, this works. And then they decide to do it or not. But now they were sending these games to majors who did not know at all what to do with this. They couldn't get into the game.

And so, not that the game was bad. They didn't even think about their target group. And I think that one aspect that interested me the most is, you have to have the context of the game as a learning experience. It has to be suitable. If this is not happening, you can play the most interesting game. If someone doesn't get the problem, doesn't get engaged to the problem, it's just something happening. It's not really interesting. And I think that's a major problem that this example was totally .

PROFESSOR: So, know who you're designing your game for. And how do you think they get engaged in the games.

GUEST SPEAKER: Yes.

GUEST SPEAKER: Just to do a simplistic example, but it's one [UNINTELLIGIBLE] example. If I wanted a game for my mother, who occasionally would play a casual game. I wouldn't do a massively multiplayer game with lots of stuff going around. All these senior citizens are going to get into this massively multiplayer game. I mean, it sounds like it would be really great, to come up with a massively multiplayer game for senior citizens, because of their eyesight. But you also know that they didn't grow up with computers, and they're not going to be able to navigate a 3D space. So, that's a simplified example of why it's important [INAUDIBLE].

PROFESSOR: And what is really good about it is that game like [? Revish ?] the cancer game, maybe some of you know about. It's a cancer game [UNINTELLIGIBLE].

GUEST SPEAKER: It was actually mentioned in the reading [UNINTELLIGIBLE].

GUEST SPEAKER: But it's hopefully one of the best [UNINTELLIGIBLE] learning games out there. But the good thing about them is, people that play that game have cancer. So you have the problem to the game is so close that people who do not play the game say oh, I don't care about cancer. They care about cancer because they have cancer. The game is about cancer, and the way chemotherapy works.

So the context of the engagement is automatically, the problem is solved. But what about this cancer. What about a learning game where it's about language learning? Are these people already interested to learn the language? Do they come there with their language problem, or do we want to introduce them to the problem or show them maybe you don't have to be afraid to learn a language or whatever. This question has to be addressed at a very early stage. Otherwise, if you start designing and then address the question, it's late.

The question is sort of about the explicit versus the [UNINTELLIGIBLE]. So if you play [UNINTELLIGIBLE], that was really designed to introduce, basically, an idea that people have struggled with, which are what position and velocity graphs actually mean. But the game doesn't mention velocity and it doesn't mention graphs. And in fact in the same reviews that it got lots of good reviews when it came out, a lot of them said, I had no idea this was educational. I wouldn't argue that that's fine, that people don't necessarily know that as they're playing the game, that they are still building up some conceptual knowledge.

I think that some large percentage of the people who play a game in that setting will probably play the game and do nothing else but then three months later they'll have gone. That experience for most people will not be resilient. Not because they didn't enjoy it, but because they have lots of things going on in life and there's more information coming out of [UNINTELLIGIBLE]. On the other hand, the kid plays that game and then you sit down and have a conversation about velocity and position graphs, there might be some real meaningful learning that occurs. And so I think it's really important to not overreach in that sense. I try to avoid, although we still do the saying that our games teach. Because I think teaching is a much more active and explicit activity to learn games too. I think our games let people play with them, let people learn from them.

And then just like if you read a book, the book doesn't teach anything. The book puts information [UNINTELLIGIBLE] And I think that's really part of, I think, we have to be modest and humble about it as we do this stuff. But that was [UNINTELLIGIBLE] as the challenge [UNINTELLIGIBLE] of thinking about, if you do want some learning to occur, where do you want the learning to occur. Is this for peoples' personal lives, is it a mediated setting like a hospital? Or is it in a school-- those are really critical questions . It's not just the learning but the setting.

PROFESSOR: And who's introducing a game to them, for instance. Is it a teacher, a doctor.

GUEST SPEAKER: And were they playing. Like with Labyrinth, which is this middle school math game that Dan and I were working on. We knew that if we didn't push in another direction, the implementation was going to be, well print this game and it gets played in classrooms.

If you've ever seen-- if you remember your own computer labs in school, that that was [UNINTELLIGIBLE], which is true actually for a lot of [UNINTELLIGIBLE]. It meant every now and then you had to line up and walk down the hall to this other place, that you would only get to occasionally. And by the time you settled into your chairs and booted it up, you had maybe 20 minutes on the computer and then it was time to go.

So if you design something for schools, you're talking about a 20 minute experience once every two or three weeks. I'm not sure that was worth thinking about anything for that. So we design the game, we play it online at home. And encourage teachers to let the kids just play it. And there was another advantage of that, which was to go back to who introduces it. If you play that-- even if we have the perfect 20 minute game for the computer lab, if the teacher's walking around-- I shouldn't say.

I've seen good uses of that space. But there's some greater probability you're going to feel like this is just a lesson. Whereas if the teacher says, go play this at home, there's some greater probability that you will sort of forget about the school and get [UNINTELLIGIBLE].

And again, it's not that we want to trick you into doing math. It's that we want to engage with the math in the most pleasant spirit possible. And the less we give the wrapper of school to it, the more playful you're likely to be. Even if you're not denying that it's math or science or [UNINTELLIGIBLE]. So that was [UNINTELLIGIBLE PHRASE].

GUEST SPEAKER: So Constantine, you had said the cancer game is really well evaluated. So did any of you have stories or experiences from seeing your games out in the world that way. You've gotten feedback from teachers who have tried this in the classrooms.

GUEST SPEAKER: Yeah.

GUEST SPEAKER: [UNINTELLIGIBLE] so from the game I just talked about, about renewable energies that totally failed. People played it but didn't understand it, hated it, or [UNINTELLIGIBLE]. So I think, total failure. And I think just to start with this conversation is, this is a problematic thing.

Because there is very little research going on on the iteration of games, because it's also very hard. It takes lot of effort and it also could take a lot of money to evaluate the research, that the success of the game for learning purposes.

And I just know very few studies that do that, in the end. And I think we've just reached that moment where people start to think that this is an interesting way to do it. And Remission, as I said, is probably-- maybe you'll correct me in a second, but I think it's the only game I would put my hand into the fire for it.

But the question is, what did they learn in the game? They learned about [UNINTELLIGIBLE]. They learned that there is something they can do. They learned a more positive perspective, a more positive relation to medicine. They didn't learn anything else. In the research--

GUEST SPEAKER: It doesn't really change their illness.

PROFESSOR: And they didn't get happier. It's not that they said, wow, now, I couldn't cope better with their illness. They just took the medicine and so the doctors were really happy with that. But--

GUEST SPEAKER: That's the stated goal of Remission, actually. It's, this is a 3D game where you play a woman in space or something. Shot down and [UNINTELLIGIBLE] but the stated goal of the game, at least of the people who were making it, was to try to encourage people who are suffering from cancer to keep up with their medication. Don't fail keeping up the schedule of taking your chemotherapy drugs, basically. And the game was designed to show what happens, what these drugs actually try to do in some sort of fictional-- or fantastical, I'm not going to say fictional -- so that's Remission.

PROFESSOR: Is this a free play game as well?

GUEST SPEAKER: Yeah. Yeah. [UNINTELLIGIBLE].

GUEST SPEAKER: So, on Labyrinth the original grant include an excess of one year evaluation on the game use. And then when Congress cut back the funding of the larger program, of which Labyrinth was just one small piece, that got cut off. And so the research didn't happen. We're now looking for a grant, because Labyrinth has been used aggressively in Maryland schools [UNINTELLIGIBLE]. So we are going to look for funding.

But generally, there's less money. As little money as there is you're developing this [UNINTELLIGIBLE] there's even less for researching them. And I think some of what I think our work is designed [UNINTELLIGIBLE], we figure out what we can learn by making the game. And we publish what we've learned in the hope that other people will pick up the ball, people who have more of a research bent in the traditional sense. Will start to do the research about it. I think we're on the brink of that. I think with stuff like the Quest to Learn School. Increasing numbers of educators are using [UNINTELLIGIBLE].

I do have only anecdotal evidence, but the very first game that I did, which was this massive logic oriented game, ended up being not particularly well marketed. It nevertheless sold a million copies. And the anecdotal evidence I got was that it was one of the rare cases where parents would bring it in and recommend it to teachers, and teachers would recommend it to parents. Or kids would come home from school recommending it.

And there seemed to be this fairly broad acceptance by both parents and teachers and kids, without kids having the sense that, oh, my teacher's making me do this. And I hear that now that everybody's rich and famous in their 20s, I occasionally hear from them. And some of them say it's what interested them in math and stuff. So that's all anecdotal, and maybe [UNINTELLIGIBLE] has some [UNINTELLIGIBLE]. But I think the million copies is beating towards the--

PROFESSOR: [UNINTELLIGIBLE PHRASE]

GUEST SPEAKER: Sure. I'll just add a little bit to that. I also have some anecdotal stuff I want to bring in. But the most interesting thing that I've discovered was with Labyrinth and a target audience that I've seen play it, that it was not specifically designed for. Which is kids with some sort of attention disorder. Whether it's ADD, or autism or something. And seeing these kids engage with the game, even engage with the game in a social way, and be hyperfocused on the game. And be highly competent with the game, and to really calm down and adopt pro-social behaviors that were not characteristic of them. Or that their peers or families hadn't seen with them before. I think there's a lot of power there.

GUEST SPEAKER: The first two teachers I've met who have used Labyrinth said they were seeing performance with kids they didn't expect to see before. Because our sense was that lots of kids have competencies and skills that they don't show off in the format of school. School being its own kind of game which only certain people are good at. And so having a kid be able to demonstrate a competency at a different kind of game, and let the teacher see it. Because part of the design of Labyrinth is that teachers can track progress online. That was one of our goals. And that was [UNINTELLIGIBLE].

GUEST SPEAKER: This is a very good laptop actually. So is it possible to look it up and show some of Labyrinth? Because we've been talking about it in this aspect.

GUEST SPEAKER: Yeah, sure. Absolutely.

[INTERPOSING VOICES]

GUEST SPEAKER: It's just a notion of a game.

GUEST SPEAKER: But other people can [UNINTELLIGIBLE].

[INTERPOSING VOICES]

PROFESSOR: While we're setting that up, I'd like to ask a bit about prototyping. [UNINTELLIGIBLE] So, in the process of designing-- these are mostly digital games that we've been talking about all the time. But I'd like to talk about the early stage, when you're working with paper or simple digital prototypes. Maybe you can talk a little bit about how that might be similar or different from designing other kinds of computer games, or even board games? Do you test it differently because you're designing it explicitly for a specific audience? Do you test it with instructors as well as players, or [UNINTELLIGIBLE]?

GUEST SPEAKER: I think it's always best to try to test with your target audience. The things you'd learn from [UNINTELLIGIBLE] there's thing you learn, you can learn from any audience. Particularly usability, which is important. Frequently the flaw in a game is not the core mechanic, but the ease with which the player can understand what they're supposed to do. But you can test that with any wide range of people. Anyone who you think is roughly as computer game literate as your target audience. But I think if your goal is to interest middle-school kids like we do with this game-- let me just turn this on-- then you do want to, because there are questions about age appropriateness. Or [UNINTELLIGIBLE PHRASE]

All right. So we're about to get this screen here. So this is a login screen. This is not the first screen you would come to if you'd started the game from scratch, you get some more information. But I've already logged in. So what you'll notice-- let me just log in. So it's online. You can play from any computer. That was important because we were concerned about underserved populations who either don't have computers at home or who have less control over their time, don't have a computer in their room, certainly. Or might do all their work in an after-school setting or a library. So basically, once you have an identity you can log in from anywhere.

I'm going to briefly enter the game space but [UNINTELLIGIBLE]. So here we're already in the middle of the game. This is a puzzle adventure game. And the plot is, someone's stolen my pet. I've tracked it down to this underground factory that's being run by monsters-- and there's middle schoolers-- with the help of a mysterious woman with bluish skin and wings. I've been given a monster costume, so in the guise of a monster myself I can go around and do jobs in the factory. Find my pet, figure out what the monsters are up to, which is indeed no good.

And I'm in the middle of this game, so I've already advanced some. There is a sequence of rooms, which, part of the game is just finding the rooms where the puzzles are. For example, we've got it here. This is one of the wings of the factory. There's a hundred doors here. And I can click on every door if I have to. But I have a map, which is itself a mathematical tool, to find where I want to go. And as I go through, each room represents a different puzzle.

Anyway, having said that, we're going to actually go back-- And oh. The other big feature in the game is that if you play on teams, then you can communicate with your teammates through this communicator. And the reason there is to actually get kids reading and writing about what they're doing with math. Because if you want to help your teammmates, you can't feed each other the answers [UNINTELLIGIBLE] puzzle. Every puzzle, every time you play, it's a different solution. So you can't tell each other the answer, all you can do is talk about your strategies. We're going to leave the game and go to puzzle mode [UNINTELLIGIBLE].

PROFESSOR: This is a shortcut?

GUEST SPEAKER: Yeah, shortcut to puzzles, right. You don't have to play it in the context of the game. You can play any puzzle that you want. I'm going to try one that I've never played with a group before. So usually I pick the same game, play it, and get used to playing it with people. But let's just try this one. [UNINTELLIGIBLE]. I think I've tried level one.

So I'm trying to remember what you know here. You're in the back in the shipping area. And you've got to come up with a manifest to ship the number of objects that you've got. That's all you know. You've basically got to give them the right instructions to ship it. And so now, just tell me what to do here.

AUDIENCE: Click something.

GUEST SPEAKER: Of course. And that's [UNINTELLIGIBLE]

AUDIENCE: [UNINTELLIGIBLE]

AUDIENCE: [UNINTELLIGIBLE] Up into the slots up there. There's ones and zeroes.

GUEST SPEAKER: Can we put that crazy fish head second to the right on every one in the equals spot?

GUEST SPEAKER: Yes. You can.

[INTERPOSING VOICES]

GUEST SPEAKER: Want to try it in the equals spot.

GUEST SPEAKER: And go to the equals on this one.

GUEST SPEAKER: [UNINTELLIGIBLE] Just tell us what your thinking was.

AUDIENCE: Well, it looks like we're trying to build math equations. And usually they end with equals something.

GUEST SPEAKER: Right. And you saw that. Now, just so you know, the equation we're going to build-- you're right, and just as a shortcut-- is not about arithmetic. The equations are simple. I'm not interested in drilling kids on arithmetic. This is really about number systems, and number systems are just simple systems. But there's also some algebra that's going to go on this, we start to do this too. And then in higher levels of the puzzle, [UNINTELLIGIBLE] you start doing stuff in base two and base six. We sort of dovetailed with the notion that our representations of numbers are just symbolic. So zero, or one, or one zero, doesn't have to always mean 10. It's just a symbol. Anyway. So now what should we do?

AUDIENCE: Pac-Man should be minus. Because the only ones with six digits have Pac-Man as the third digit.

PROFESSOR: Sounds good.

GUEST SPEAKER: I'm not sure which one is Pac-Man.

AUDIENCE: Bottom right.

GUEST SPEAKER: This guy?

AUDIENCE: That guy's Pac-Man. Pac-Man with an underline.

GUEST SPEAKER: I'm not sure if we can actually put that one there.

[INTERPOSING VOICES]

AUDIENCE: If you look at the top left, then what is the number two digit number that, minus the last digit, gives a single-digit number. [UNINTELLIGIBLE]

PROFESSOR: [UNINTELLIGIBLE]

GUEST SPEAKER: Oh, that's a good point, yeah. It has to be divided by [UNINTELLIGIBLE].

GUEST SPEAKER: But now what you're really doing is thinking about the structure of math. Which is exactly what--

AUDIENCE: So when you say that's divided--

AUDIENCE: So, I'm going to guess the banana is four?

GUEST SPEAKER: [UNINTELLIGIBLE]

GUEST SPEAKER: You want to say why?

AUDIENCE: Because using that same logic, that first version, that something, n x divided by x equals some single digit number. And I'm guessing x is 4.

GUEST SPEAKER: [UNINTELLIGIBLE]

GUEST SPEAKER: By the way, you guys have avoided, and not noticed a piece of information which would make your work even more--

AUDIENCE: Oh, the thing at the top.

AUDIENCE: What's the thing at the top?

AUDIENCE: The thing at the bottom.

AUDIENCE: It could be two, actually, as well.

AUDIENCE: What are all the green things?

GUEST SPEAKER: Those are the things you're shipping. Those are jars of stuff, of pepper.

AUDIENCE: Oh, those make sense.

AUDIENCE: Banana something banana.

GUEST SPEAKER: Banana something--

AUDIENCE: Equals something banana. So--

[INTERPOSING VOICES]

AUDIENCE: Banana could actually be two. Because 12 divided by two is equal to six. six.

[INTERPOSING VOICES]

GUEST SPEAKER: Oh, and in that respect, that means a little--

AUDIENCE: Yeah, the four squares are signs. You try that?

[INTERPOSING VOICES]

AUDIENCE: It has to be zero on--

GUEST SPEAKER: Every time you play the game the symbols are different. They're the same symbols but they're a different--

AUDIENCE: Triple backlash should be-- oh.

[INTERPOSING VOICES]

GUEST SPEAKER: By the way, you're still missing a piece of information. But that's OK. That's what games are all about.

[INTERPOSING VOICES]

AUDIENCE: So what's this thing in the top left? Can we put things in there and press enter, and will it evaluate them for us?

GUEST SPEAKER: Yeah, but right now-- you're not done yet . [UNINTELLIGIBLE].

AUDIENCE: Yeah, triple dash is definitely one.

GUEST SPEAKER: Triple dash is--

AUDIENCE: Oh, triple dash is one, because one times banana is banana.

GUEST SPEAKER: Unless banana's zero.

[INTERPOSING VOICES]

AUDIENCE: Banana's six, I think.

AUDIENCE: It could be five.

GUEST SPEAKER: I've never done this game before-- no. The fact that you're naming all these symbols, because they were just symbols.

[INTERPOSING VOICES]

AUDIENCE: I think banana's [UNINTELLIGIBLE], because banana--

[INTERPOSING VOICES]

AUDIENCE: [UNINTELLIGIBLE] banana squared is equal to something times banana.

AUDIENCE: Not something times--

AUDIENCE: So, yeah, it could be 25 actually.

[INTERPOSING VOICES]

GUEST SPEAKER: Remember, I didn't know the solution to this when it came in, but I actually have enough data to know that you're wrong. It's on the screen.

AUDIENCE: Banana definitely is--

[INTERPOSING VOICES]

AUDIENCE: Five because banana squared is anchor banana. But flower banana divided by banana is also an integer. So if flower banana and anchor banana are local banana, then banana should be five and not six. because flower minus anchor times banana.

GUEST SPEAKER: I love your mental calculation.

[INTERPOSING VOICES]

GUEST SPEAKER: So you're saying it's what?

AUDIENCE: Five.

GUEST SPEAKER: Well, that could work, but you could have done it in a simpler way. Or you could have surmised it, not known it, but-- remember, this is only the first time you played, subsequent times you'd have other information.

[INTERPOSING VOICES]

GUEST SPEAKER: You're a member of the group, so don't give away [UNINTELLIGIBLE] but in fact you're making-- [UNINTELLIGIBLE]

AUDIENCE: Anchor is 20.

AUDIENCE: Anchor is two.

AUDIENCE: Sorry, two. Not 20.

AUDIENCE: Anchor's two. Claw is zero.

GUEST SPEAKER: Where's claw?

AUDIENCE: Claw's the one on the top left. Of the squares that are left. Left.

GUEST SPEAKER: Oh, you mean brass circles. That makes sense.

GUEST SPEAKER: Oh, that one.

AUDIENCE: That's zero. Has to be zero.

PROFESSOR: Wait, why is it zero?

AUDIENCE: Because if you look at the equation at the bottom--

GUEST SPEAKER: Oh, I'm looking at the wrong spot.

[INTERPOSING VOICES]

AUDIENCE: Yeah, sigma's nine.

GUEST SPEAKER: [UNINTELLIGIBLE]

AUDIENCE: OK, so [UNINTELLIGIBLE] scythe is plus. Scythe is plus.

AUDIENCE: No, it has to be minus.

[INTERPOSING VOICES]

GUEST SPEAKER: Anyone who is having trouble, and this happens in a group. So, anyone having trouble following it as people were making stuff up or finding stuff. Because if you did it's OK. In fact, sometimes you've got to play the game yourself for it to make sense. [UNINTELLIGIBLE] I could have told you-- I actually think I could have told you a lot about the game going into it. And there would still be a high probability that when you started playing it you would have not absorbed anything I'd said. That's my experience. I could almost give it all away but you still don't get it until they play it.

PROFESSOR: So they didn't know--

GUEST SPEAKER: [UNINTELLIGIBLE]

AUDIENCE: Beta delta upside down is eight.

[INTERPOSING VOICES]

AUDIENCE: Can we enter that now?

AUDIENCE: Oh, so now you have to type in one times whatever-- equals--

GUEST SPEAKER: You've got to type in what this is. So and this is actually getting at another representation.

AUDIENCE: Eight plus nine.

AUDIENCE: Eight plus nine is 17.

GUEST SPEAKER: How, here you don't have to actually remember the [UNINTELLIGIBLE]

AUDIENCE: Just divide it by six.

AUDIENCE: [UNINTELLIGIBLE]

AUDIENCE: It's puzzle mode.

GUEST SPEAKER: And by the way, it could take it in either order.

AUDIENCE: Yeah, I agree. Equals [UNINTELLIGIBLE].

GUEST SPEAKER: I think this level's only addition [UNINTELLIGIBLE].

[INTERPOSING VOICES]

AUDIENCE: Or we could do 10 minus two.

GUEST SPEAKER: Anyway. [UNINTELLIGIBLE] you were doing algebra. When you were solving for those numbers, you were doing algebra. You were really thinking about x and the structure of the math and the numerical quality of it. This is simpler, but it's still getting at the fact that there are multiple ways of representing numbers [UNINTELLIGIBLE].

AUDIENCE: Is there any mechanic to prevent brute force?

GUEST SPEAKER: Yes, those lights go out. But, you do earn points along the way in the game if you make some progress. And so, none of these puzzles are brick walls. Because you can always play multiple puzzles at once. You can leave this puzzle, go to another puzzle. And, if you make any progress at all, you're accumulating points for the end of the game. You could actually get to the end of the game story without having solved any puzzles. Takes a long time. The rewards are there for doing well. And there are rewards for doing well, but there isn't punishment for not doing well. At least in sort of a-- we don't say you can't, we don't make you stop in your tracks.

AUDIENCE: [UNINTELLIGIBLE] you can do it again.

GUEST SPEAKER: Yes, that's right. You can do it again instantaneously. And there is no cost in the game to having failed repeatedly. If you eventually solve it, you end up with the same score whether you solved it on the 20th time or the first time. That's really what problem solving is about. It's not about being the fastest. It's just about learning how to solve things. Speed is overrated [UNINTELLIGIBLE]. And in math it's very overrated. Kids [UNINTELLIGIBLE] calculations in their head.

It's just more of this. And the entire level's-- we came up with representations of subtraction, which is that you have a large number, you have 10 jars and five of them are empty. So it's 10 minus five.

[INTERPOSING VOICES]

AUDIENCE: How is division?

AUDIENCE: Are they 10 rows of jars? And the rows are empty?

GUEST SPEAKER: Yeah, I think it's the same-- I think that's it. I think that's the same. It's in a grid pattern with some [UNINTELLIGIBLE].

AUDIENCE: And I guess there's [UNINTELLIGIBLE] but do you have any algebra as such in this format?

GUEST SPEAKER: No. But what we encourage is that-- so what comes with this is material for teachers to relate this to the curriculum. And what we really want to happen is that kids have played the game before they get the subject. And when the teacher introduces symbol systems in base six or whatever relates to this, instead of saying, all right, we're starting a new unit, you again know nothing, you're starting again at square one and have to start all over again with something new.

Instead, the teacher comes in and says, I want to introduce a new concept, but it's one that you've really already begun to master. And it's called bases. [UNINTELLIGIBLE] so how did you solve this? What was going on in this game? And they have a conversation about what they already know, as a way of leveraging into-- I mean, things that I arguably don't know. Again, I would argue that [UNINTELLIGIBLE] you have things in cartons. So a carton of six jars plus some left over would be-- a carton of six jars plus three left over would be one, three, basically.

So anyway. The point is that the kids begin to build this robust conceptual understanding. And hopefully there's a playfulness so that when the teacher talks about it, that some of them may actually feel good about it. Instead of, here's one more reading --you'll never be a mathematician.

I think math education is largely designed to gate everybody so that at the end of high school, [UNINTELLIGIBLE] population is good at math. Which means that we're taking 12 years teaching 93% of the population that's no good at math and they should stop doing that. I don't know why we'd waste 12 years if you could just tell them to go to kindergarten. But that's what what we end up doing. Anyway, I think that's true of every subject. I'd like to turn it around. Everyone says there's things about math I can do. [UNINTELLIGIBLE]

PROFESSOR: Can you tell us a little bit more about how that process went?

GUEST SPEAKER: Sure. And I also wanted to provide a little extra nuance to one of the questions about what if you just brute-force your way through it. In the higher levels, it won't tell you when you put a symbol in a slot, whether it's correct or not. So--

GUEST SPEAKER: Which actually, you could learn something interesting. Because you could start doing something and [UNINTELLIGIBLE] something that doesn't add up, that's actually new. You're learning something. So, there's 10 puzzles or-- how many?

GUEST SPEAKER: Nine.

AUDIENCE: Nine puzzles, and we came up with nine plus x, and some of that was slated for school admission and we never got there. And some of that x was puzzles or activities that we decided were not as compelling as the others. In our process for that, we did a hybrid approach with paper and computer prototyping.

In general the rule is, if there can be paper prototypes, paper prototype it. Because it's so much faster. But there are certain kinds of activities that really can't. So for instance, this one is perfect for paper prototyping. You just take out a piece of paper and a pen and you create-- I mean, you have to create some of these equations so that you then have a system of equations to solve. But once you've created some of those is, as would need to be done whether it's a paper or a computer prototype, then it's trivial to put it in front of someone with paper.

So this is an example of something where we would, for the first round, test internally with just symbols and numbers. And then probably we would do-- we might even test that version with people in our target audience. But probably we would try to do some rudimentary art treatment before we put it in front of people, just so there's a little bit of context and it looks a little bit more like a game. And then--

GUEST SPEAKER: So that we try to-- if there's story, this one has less story about it, but if there's story, as there are in some puzzles, we actually try to act it out. [UNINTELLIGIBLE] resistance to doing it. So act out the character. You're the computer, when [UNINTELLIGIBLE] and so even if it's just a character giving a response to something that the player does. You try to do it in character, you try to get a sense of what-- because I think, even in that kind of rudimentary form you're engaging the player's imagination so you know whether or not they're interested in this character or the idea [UNINTELLIGIBLE].

GUEST SPEAKER: Because I do master [UNINTELLIGIBLE].

GUEST SPEAKER: And like a dungeon master you need to prepare, because it's hard to be the computer without also being the bottleneck if you're a human. And in fact we've done some games, not represented here, we've done some games that are real time, that if you want to prototype with people, then especially on paper, and you want to keep things moving, then you need to have more than one person be the computer, because there's just too much to track for one person to do it all.

Or if you want to do-- sometimes we do hybrids, where we'll do a computer prototype but there's aspects of the game that are not represented through the interface or that logically we have a model, so we'll do players directing human interactions with the computer. Not players, designers, influencing those interactions.

So anyway, this is a game that moved pretty much all the way to final design on paper, and then went straight to implementation. Some of the other games-- for instance, there's a game-- do you want to show [UNINTELLIGIBLE].

GUEST SPEAKER: [UNINTELLIGIBLE], yeah.

GUEST SPEAKER: That's an example of a game that's a little bit harder to think about playing just on paper.

GUEST SPEAKER: [UNINTELLIGIBLE] we implemented this early.

GUEST SPEAKER: Yeah, we did.

GUEST SPEAKER: So this is a game about modular arithmetic and common multiples and that sort of thing. So it's very hard to see on the screen. In the upper left there is an avatar.

GUEST SPEAKER: That's you.

GUEST SPEAKER: [UNINTELLIGIBLE]

GUEST SPEAKER: And you know from the story that your goal is to get to the golden eggs and avoid the [UNINTELLIGIBLE]. That much you know. And that you're in a factory where they make stuff.

AUDIENCE: Do you move your avatar?

GUEST SPEAKER: [UNINTELLIGIBLE]

[INTERPOSING VOICES]

GUEST SPEAKER: There is one flaw, by the way, in the [UNINTELLIGIBLE] interface, because people don't realize they have to click to get their guy onto that thing. And it was supposed to start with the [UNINTELLIGIBLE] right where he was, that people would [UNINTELLIGIBLE] click. Because now by-- what should I try now.

AUDIENCE: Four.

GUEST SPEAKER: We're going to go--

AUDIENCE: Four or nine.

AUDIENCE: Six.

AUDIENCE: Four. Four.

AUDIENCE: Six. Then you can let him get the [UNINTELLIGIBLE].

GUEST SPEAKER: I'll go for six. Bam. Four is a subject that--

GUEST SPEAKER: Oh, can we move [UNINTELLIGIBLE].

GUEST SPEAKER: But you'll notice that that's all in different amounts. And so it's really that modular arithmetic. So 12 will take me once around on one of them, but [UNINTELLIGIBLE] on another one. And you start doing simultaneous equations to solve this. This level's hard. I think maybe the hardest puzzle we designed in the whole game is level three of this, where every step you've got to align two bridges. It's serious work.

AUDIENCE: Are you trying to avoid that monster?

GUEST SPEAKER: Yeah, you've got to avoid the monster.

GUEST SPEAKER: Which isn't always hard. But you have to keep in mind, sometimes it needs, instead of going around once, particularly one on the nine thing, maybe you go 10 and he'll always go on [UNINTELLIGIBLE] shows up, to make [UNINTELLIGIBLE] to show up and [UNINTELLIGIBLE]. So this is really the [UNINTELLIGIBLE].

GUEST SPEAKER: As of what?

GUEST SPEAKER: Yes.

GUEST SPEAKER: Anyway, again [UNINTELLIGIBLE] you can say, hey, [UNINTELLIGIBLE] math, school math at all. There's no school math in this game. But I think there are really interesting things to think about.

GUEST SPEAKER: [UNINTELLIGIBLE] said with that character. If he walks or not. I think we [UNINTELLIGIBLE] anyway. But you have to keep in mind, there are two forms of learning. One is learning the game, how to play the game. And the other one is learning-- there's a problem solving, you've got a puzzle in the game. And I think it's sometimes like special education games. It's so easy to mix this.

Because sometimes you make the coolest educational game and it's really interesting. But [UNINTELLIGIBLE] character wouldn't move if you don't press the mouse button. So he would always stand there in the beginning and he would get attacked. And I think the mixture of both forms of learning is quite interesting to keep in mind when you're prototyping.

GUEST SPEAKER: Yeah, there's two answers to that. One is, I never think. just because we want people to think hard, I don't think the interface should be the thing that they're thinking hard about. It's always tempting to say, oh, that's a really hard thing to solve. How do I make him jump? But no, I think generally it should not be hard to figure out how to make him jump. Or walk left or right, or how to open a door. Unless that's a puzzle, and it's easy to delude yourself that by making the interface hard...

On the other hand, there is this other challenge, which is that kids do have different degrees of literacy around games. And it's hard to figure out where the sweet spot is. That's one I don't have an easy answer for. Because some large percentage of the audience naturally knows, try and get to the end. Knows what winning means. But there's some kids who don't.

And the question is, if you make the game, if you bend over backwards-- I watched my mother play this, and it was horrible. Because she just was totally clueless about where to go. But I think if I try to set up the game to work for my mother, you can only work with all this explicit instruction.

So hopefully with kids it's less of an issue. There's more, I think, uniform game literacy. But it's always a challenge. No game appeals to everybody. And you do want-- I don't think those of us who make learning games have a good answer to that. I don't think we can ever make a game that every kid would play with equal-- we should never claim that we're going to get 100% on that.

PROFESSOR: One thing that I know that you've noticed is the amount of tolerance that kids and adults have for frustration, when-- I've seen kids say, I hate this game. This game is too hard and they'll just keep going at it. Whereas the adults have this tendency to get really embarrassed when they can't solve it first time.

GUEST SPEAKER: It happens when you're 26. You suddenly sit down and say, well, I'm not very good at this. You may be a gamer all your life, but it's having another adult watching you. There's some point or another where you suddenly say, I'm not good at this. I don't know why it happens. [UNINTELLIGIBLE]

AUDIENCE: [UNINTELLIGIBLE]

AUDIENCE: Can you do negative numbers?

GUEST SPEAKER: Not in this one, no. That's a good idea, though.

AUDIENCE: [UNINTELLIGIBLE]

AUDIENCE: In much of the arithmetic you could always do it as put in a larger number and [UNINTELLIGIBLE]

GUEST SPEAKER: That's right.

PROFESSOR: I have a question for [UNINTELLIGIBLE]. Because in some of them we were hearing from you guys about coming, more or less from a game design angle into designing games as a learning purpose. And you kind of [UNINTELLIGIBLE] the other way around. You came in from the [UNINTELLIGIBLE]. Are there things to keep in mind when you're working across disciplines, that game designers know?

GUEST SPEAKER: So, I think one more difference that I still have, or see between the way I approach games and the way educational games approach games design, I'm more focused on learning as an experience, more in abstract form. That doesn't mean that I wouldn't be invested in that, but that's not what I've been working on yet here. So for instance, after that it was-- so the research question was how, how can players overcome expectations that are wrong.

And so the idea was not to use educational stuff like numbers or whatever, but more, can we develop patterns in games that players who realize very quickly, and are they ready to give them up if they realize that they think what they've seen is wrong. So, for me it was very interesting because I come at an abstract theory inside.

And so we were developing prototypes. And it's very interesting when you come from the abstract idea to prototypes. But paper prototyping is perfect because it really made me as a scientist come to the ground. Because of course I can read [UNINTELLIGIBLE] and all these people and say, wow, experiences this and that. And then you suddenly have pen and paper and sit there and think about a game that, is it a metaphor for your theory. And so I think my takeaway was that paper prototyping was very healthy for me.

And with the team to come down to the fact, what is really [UNINTELLIGIBLE] in the game. What can we really-- I think at the beginning I was a bit inexperienced in prototyping. Wow, we can all do that later. We can do the cool stuff when we get to programming. But then I realized, no, we can't. We cannot do it as a paper prototype, how should we do it in programming. But what is changing?

And I think for us it was very interesting. So I think one experience we had is, we had them develop different prototypes. And so I think that was healthy for us not to make one paper prototype but then think how we can do it better. But that's what [UNINTELLIGIBLE] a lot of prototypes. And to stay open. And even if you spend three or four days on a prototype, if it doesn't work with others the moment you show them. And they go like, what, these are the rules. Are you sure? I don't get it. If you explain the tenth time, maybe-- maybe it was a cool idea back home or with your friend, maybe, but it just doesn't work. And so this experience in the prototypes was very healthy for us.

And then coming to the point. And I realized for me it was important that somebody coming in the room, playing a prototype for five minutes, kind of understands the core mechanic. If this is not happening-- who takes the five minutes to understand a core mechanic? Nobody, today.

PROFESSOR: That would be, like in this game, just walking onto a [INAUDIBLE].

GUEST SPEAKER: Right. That's right. And so, yeah. I think, coming from a different perspective, again, prototyping is kind of like the way to get in the game. Even as the designer. Because from then on, you know what you're talking about. And so I had to lose a lot of my abstract thoughts in the process.

And then the playtesting. So it was interesting sometimes to see, even if the prototype is cool, then you have a prototype. And then people come in and say, [UNINTELLIGIBLE] again. [UNINTELLIGIBLE] it's so easy. But then you can start to stick with your idea or say no, maybe I have to go down with the theoretical aspect and really look [UNINTELLIGIBLE] for the players.

GUEST SPEAKER: And another thing about prototyping, it's not focus groups. So it's not a case of, --sure you want to put it in front of as many people as possible. But you really want to, I think, avoid stuff like what did you like, what didn't you like. Those kind of questions. Because people want to be helpful. And sometimes the piece they didn't like was actually the part where they were working hardest. So it's like I didn't like that I had to jump over the wall. [INAUDIBLE] jump over the wall.

And it may not really mean that if you took that out, that they would enjoy the game more. In fact, they might not enjoy it at all. So you have to be really careful about that. And people want to please you.

So they'll say oh, it's good, and stuff like that. I think it's really important that what you do with prototyping is watch the player. And then trust yourself as a designer. Rather than ask them what you should do, because they will try and be helpful and they'll come up with wrong suggestions. Watch them and use your best judgment as designer as to what they're reacting to, and why they're reacting the way they are. And try to fix it.

And then you try to get some questions, try to maybe get some access to their thinking. But don't think that they necessarily know what was interesting to them, or what would make them happy. Don't ask those questions. Those are focus group questions, and you shouldn't ask them because it's not going to give you the answers you want. You've got to trust yourself at some level as to what's happening.

GUEST SPEAKER: And something that I also had to learn was, if you have the focus test or the playtest, and then you realize you have problems. And it's really healthy to write all the problems. And then brainstorm different solutions.

Don't ever, when you discuss it, try to come up with a solution immediately. Because what happens is, you quickly develop one solution. And then you go and end up with another problem, but you don't realize that sometimes, some problems stick together. Or sometimes you realize, well, we have so many problems, just because of this little aspect of the game. Why don't we just let it [UNINTELLIGIBLE]. If you don't do that, you start to spend two hours in solving a problem then realize it was wrong. Or you never realize it again. So have [UNINTELLIGIBLE] problems and quick solutions where you [UNINTELLIGIBLE] or not. Make solution, solution, solution and then with the same problem a few weeks later again.

GUEST SPEAKER: And I just want to clarify, focus tests and focus groups are two very different things. The kind of resource we're talking about is focus testing, which is specifically identifying individual problems in the game. And a focus group is asking them what they liked, which is actually very not useful for most product design in fact. Games in particular.

GUEST SPEAKER: One piece of advice that [UNINTELLIGIBLE] really important if you're a smart college student, like an MIT student, is that you've probably learned in spite of your education, by having someone sit in front of a class and tell you what was going on. And you somehow absorbed it. And that's why you've got here. But lots of people don't learn that. And it's really hard to avoid the tendency in the game to just tell people what they should know. Which is, sort of for my money, like when you meet someone who speaks a foreign language. You don't understand each other so you both start speaking slower and louder. And I think that's what a lot of education-- it's all a bad education in general does. Speaks slower and louder. And [UNINTELLIGIBLE] educational games. So just remember that just because you got it the first time someone told you doesn't mean that everybody else-- it's not just about explaining it better. It's about giving them a different way of learning.

PROFESSOR: Any questions for our guests?

GUEST SPEAKER: I just want to say something. Have you ever heard of firehose games [UNINTELLIGIBLE] forum. So I told them, I know a guy who works there. And he said, we've got a focus group tomorrow night. They're PS3 set to release in a few months. Does anyone want to go tomorrow night? Seven o'clock, about an hour to an hour and a half.

GUEST SPEAKER: That includes [UNINTELLIGIBLE] and [UNINTELLIGIBLE] are part of Firehose Games.

AUDIENCE: Are they looking for people who haven't played the game before?

GUEST SPEAKER: Maybe. We're going to have [UNINTELLIGIBLE].

AUDIENCE: Can I ask a question, [UNINTELLIGIBLE]?

PROFESSOR: So the question I would have to use is, [UNINTELLIGIBLE] in his play biography, educational games? Have you played it, would you say it was an important experience to you? That even, maybe it's embarrassing now when the other ones are listening. But does anybody out there [UNINTELLIGIBLE] Zambonis or whatever that game was?

GUEST SPEAKER: Best game of all time.

[INTERPOSING VOICES]

AUDIENCE: It was similar in that you had a [UNINTELLIGIBLE] you were taking around and had to solve little puzzles. And underlying them was math and simple math equations. But-- [INTERPOSING VOICES]

AUDIENCE: Pizza. You divide a pizza, it's cool.

GUEST SPEAKER: We can't let that slip. That's Scott's game.

[INTERPOSING VOICES]

PROFESSOR: I think it's interesting to think about, if you have any experience like that, what would make you like the game. I think it's very rich, because of all the people who have that experience with educational games, and then think about them when you design it. Think about the influence.

GUEST SPEAKER: There's this series of games called Thinking Things. And one of them had a mode that basically was kind of like programming, in that you could place different types of units on a grid, and then give them instructions in a loop. So, say, go forward then left then right. And they could paint a square. And so what you'd end up doing is, if it's a starting configuration, a certain set of rules, you could get them to draw patterns and stuff like that. And what used to happen is, my dad would say, can you get two inverted triangles drawn on the screen. And I would sit there and try to basically program it. I realized that it was actually a programming game, so.

PROFESSOR: LOGO does a lot of that sort of thing too [UNINTELLIGIBLE PHRASE]

GUEST SPEAKER: There's a bunch of games that have done that. The last puzzle [UNINTELLIGIBLE] some of that stuff too. And there's a product out just now called, that I'm hooked on right now, called Train Yard, which is on the iPad and the iPhone.

PROFESSOR: [UNINTELLIGIBLE]

GUEST SPEAKER: What?

PROFESSOR: I thought that would be your game.

GUEST SPEAKER: And it really is a programming game, basically. It's kind of cool.

AUDIENCE: Robuzzle. [UNINTELLIGIBLE].

PROFESSOR: [UNINTELLIGIBLE]

AUDIENCE: I was just going to say that [UNINTELLIGIBLE] educational context. Where-- Dinosaur Discovery, which is an ancient game, I played it when I was in kindergarten [UNINTELLIGIBLE]. And it was an adventure text puzzle game. Which was Band Leader. And then Where in the World is Carmen Sandiego.

GUEST SPEAKER: Another classic game that I played, back when-- about the time when things started getting popular, is Midnight East. It was a math busters game.

AUDIENCE: [UNINTELLIGIBLE]

[INTERPOSING VOICES]

AUDIENCE: The Incredible Machine.

AUDIENCE: That's like the [UNINTELLIGIBLE]. It's really [UNINTELLIGIBLE].

AUDIENCE: I loved that game.

PROFESSOR: [UNINTELLIGIBLE PHRASE] back in the good days. I really enjoyed that game that sounds like, I think we all want to add to the education game. It's not like, oh my god, I had to play an education game, but more in, they could be really in [UNINTELLIGIBLE].

GUEST SPEAKER: [UNINTELLIGIBLE] and it came with almanac. They encouraged you to to do research, in fact. It wasn't cheating to look up the answers.