Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Description: Two students give a brief introduction to their augmented reality game prototype that combines audio and visual clues.
Speakers: Sophie Clyde and Pablo Villalobos

Student Project: "Night Hun...
SOPHIE CLYDE: Hi. I'm Sophie.
PABLO VILLALOBOS: I'm Pablo.
SOPHIE CLYDE: And this is our project, Night Hunter. So our early concept for this game was something like an invisible maze that you could only hear. So we had the idea of trying to-- the original idea came from the idea of superheroes who have enhanced senses. Since this was to be an audio-only game, we thought, how can we enhance your hearing? We thought, what if you could hear the things that you can't see?
So the idea was that you would try to move through a maze. You would hear what-- you would use something like sonar to try to-- you'd send out a wave. It would make a sound. You could tell where it was in space and avoid it.
That's pretty hard, as it turns out. One of the things we wanted to explore by making this game was how good at people-- or how good are people at actually figuring out where that invisible thing is. It turns out the answer is not very good. If you're trying to avoid something, that's very important that you are very good because you can't tell where the edges are. You can just tell it's vaguely over there.
PABLO VILLALOBOS: Our original-- when we did original play storming, playing around with physical objects to come up with a concept, our original concept-- we split two tables up and we put objects down to represent sound sources. And we had people walk through it. And when they got close to the sound source, we would ping. And that was our original ideating playtest. But after that, we transitioned into more of a collection kind of game because it's hard to detect those edges.
And so with our first physical playtest, we had one player wearing blackout glasses. So they were, effectively, blind. And then a bunch of our playtesting helpers, who are in this room, would act as sound sources that the player is trying to collect. And so they would move opposite the direction where the player is facing.
So it's as though they're always moving forward relative to all the other people. And all the sound sources would each take a step in cadence and then make some sort of sound. We did clapping at first. But we had a lot of, actually, very important findings just from this first playtest.
SOPHIE CLYDE: Yeah. One of the things we found out is that if everyone is making the same sound at the same time, it's really, really hard to figure out where they're coming from. It's just one big cloud of "what the heck is going on?" And it was also pretty hard to try to even-- we had to show people how to move at first. It was hard to understand just from a description. If he's looking at me, I'm going this way. And we had to have someone pointing which way the person was looking.
So we tried a second round, where the-- where everyone was making a different noise, and they weren't moving in sync. And it proved really helpful for our playtester, who could lock onto one sound, turn toward it until they got close enough. When the person got close enough, tapped on the shoulder, they were out-- collected, basically. And they got slowly faster and faster just locking onto that one individual sound, pointing towards it, collecting it until they were all out.
We had a timer for that second round. The timer was not necessary. So we're like, well, yay! This works so much better. So we moved on to making a unity prototype.
PABLO VILLALOBOS: So our original unity prototype was kind of like this. It was a duplication of this, where the frames were in the center of the screen. And then there was a cloud of sound sources around them. And they would move against where the player was looking.
So again, it was like they were flying around and trying to collect these sound sources. But we quickly found out that it was-- like the other team mentioned-- is that it's very hard to determine or to lock onto something when you can-- when there's variance on the z-axis when you can look up and down. So we flattened it. So they were all around you. But even then, it was still hard, pretty hard, to focus on each individual sound.
SOPHIE CLYDE: So we decided to change to an endless runner prototype, where the idea is that you're here. Things are constantly coming at you, and you have to turn your head so that they will move toward you. It's like there's something coming over here. Oh, I can hear that. Oh, now it's coming at me-- collected. Huzzah.
PABLO VILLALOBOS: So we transitioned to a linear experience.
SOPHIE CLYDE: So our final product is Night Hunter. The idea is that-- it's a little hard to see in this particular photo. But you're this tiny, little arrow ship thing. And there are small sound sources that you can hear they're headed toward you. And you have to turn your head, wearing the frames, to find them.
They will-- if you are looking at it, it will move towards you. If you miss it, it hits a invisible plane behind you and is deleted, and another one spawns because another thing we found was if there are multiple, you will hear one over here and one over here, and you can't tell which is which. Even if the sounds are different, it's still really hard. It's easier if the sounds are different. It's still really hard.
PABLO VILLALOBOS: And another massive change which-- we made is that there is a UI now. So before, we were just doing completely blind-- you're just-- you're only using your ears to play. But we actually made this UI for playtesting initially-- or for debugging so that we could see the sounds coming at the player. But we thought maybe for a tutorial, this would be really useful because then the player could conceptualize what they're doing before they go blind.
But we decided to actually add it into the game. So you can always see where you are, and you can see the sounds coming into view. They start out of you. So you still have to use your ears to localize it. But once you get close enough to it, you can see it. And then with more precision, you can actually collect it.
SOPHIE CLYDE: Instead of having a full range of vision and sight, it's more like you have a really, really limited range of vision. It's like being almost blind because this is-- you still have to get yourself in the general area using your auditory senses. And then you can see it. And then that helps with the final-- oh, it's right here. I thought it was here-ish. Oh, I can turn it. I can see it now. It's there.
PABLO VILLALOBOS: And with this version of the game, we felt like now it was-- we didn't want players to rely too much on their visuals. We still wanted to get that audio-first gaming experience.
So what we decided to do is have the screen occasionally go black. So you can't see anything. And then you have to truly rely on just your ears to actually collect the sounds.
SOPHIE CLYDE: So you'll be playing for a while. And suddenly, the screen's black and you're like, oh, what's happening? Wait, I can still hear things. It must still be going. Oh. Oh, I still have to use my ears.
PABLO VILLALOBOS: And we rationalized that as a radar system failure. So you're on this spaceship, and you have a radar of the-- your immediate surroundings. And then occasionally, for whatever technical reasons, it goes black. And then you're flying blind. But you can still use your other sensors, like your ears, to navigate.
Welcome!
This is one of over 2,400 courses on OCW. Explore materials for this course in the pages linked along the left.
MIT OpenCourseWare is a free & open publication of material from thousands of MIT courses, covering the entire MIT curriculum.
No enrollment or registration. Freely browse and use OCW materials at your own pace. There's no signup, and no start or end dates.
Knowledge is your reward. Use OCW to guide your own life-long learning, or to teach others. We don't offer credit or certification for using OCW.
Made for sharing. Download files for later. Send to friends and colleagues. Modify, remix, and reuse (just remember to cite OCW as the source.)
Learn more at Get Started with MIT OpenCourseWare