Bose AR Unity Workshop

• Download English-US transcript (PDF)

[CREAK]

[WHOOSH]

[TAPPING]

FILIP BABA: All right, so welcome, everyone, to our Bose AR Unity workshop. My name is Filip Baba. I'm a senior developer advocate at Bose.

So what that means basically is I work with developers, enabling them to build experiences for our platform. I host workshops. We do game jams, hackathons, also public speaking on behalf of the platform. And if you do end up building something with Base AR Unity SDK specifically-- I'm on the Unity side-- you'll most likely be talking to me, and I'll be helping you get your idea and your product out there and trying to bring Bose on board to help you out with that as much as possible. So I am your voice when it comes to these things.

So I'm just going to talk a little bit briefly about what is Bose AR. Some of you may already know or not, but I'm just going to get into it. So we have a new concept called audio augmented reality. Now, many of you probably already heard about augmented reality. First thing you probably think about is hololens or ARKit ARCore, Magic Leap, stuff like that.

So we are actually audio first augmented reality. Now, what that means is, think spatial audio, for example, where you have sounds placed in space around you, and you're kind of virtually hearing it spatially. So what enables us to do audio augmented reality are the sensors that are located in each of these Bose AR wearables.

You'll have a pair of frames, but then we also have our headphones as well, which have the same sets of sensors in them. And those sensors are an accelerometer, which allows you to get impulse and sudden movements; a gyroscope, which tracks rotation, so typically what we use the gyro for is to track the user's head orientation; and magnetometer, which is basically a compass, and we use that for our navigation apps. So if you know the GPS position of a user based on their phone, and if you use the magnetic heading from where their head is actually like pointing at, you can know what they're looking at on a map pretty accurately, actually.

So we have gestures as well on our devices. So the frames specifically have an input gesture here on the right side. It's a double tap, so it's not a specific, touch sensitive sensor. It's actually based on the accelometer, so it's the little impulse tap. And the tap actually works from any side, but it's recommended to do it from the right side because that's where the sensor kit is.

We also have custom gestures that we actually encourage our developers to explore, and it's a mixture of the other sensors. So a custom gesture would be something like, you're creating an experience, and it tells you, look up, and you look up, and then you track that rotation change. That would be a custom gesture. I know some of the games that have been built, sometimes, it tells you to duck, and there's a bullet flying over you or something like that. So that would be a custom gesture.

So Heads Up Hands Free is one of the models that we have. So the idea behind it Heads Up Hands Free is, a lot of the applications today, whether they're AR or just standard mobile applications, you're always staring at your phone. You're always walking around. Your neck as a little craned because you're always looking down at your phone.

The whole idea behind Heads Up Hands Free is that we want to create applications that you don't have to stare at your phone. The input is the wearable, and the output also goes through the wearable. So you don't actually have to look at a screen for most of these activities.

So I'll give you an example. Let's say NaviGuide, for example. What it does is it knows your GPS location based on a mobile device, and it knows you're heading based on where you're looking with the magnetometer. So what it can do is then it can tell you're looking at this restaurant, for example, or this cafe. You double tap the input gesture, and it'll actually pull a Yelp review.

Now, think about the non Heads Up Hands Free way of pulling up a Yelp review. What are you doing? You're most likely googling the name, or you're typing in the street address. But it requires you to pull out your phone, launch the app, find a name, type it in or the address or the pin on the map, and then you get to that Yelp review.

It takes you out of what you're doing. So if you want to look up five or six restaurants walking down the street, you're basically probably staring at your phone for like 15 minutes. The idea behind Heads Up Hands Free is you just walk by a place, tap, you get the information, you move on-- no phone required, at least not looking at it.

Now, it's a really cool concept I think, and a bunch of our apps use that as its guiding principle. It's not a must, but it's a design philosophy, and some of the developers have been really clever about implementing it. So the way that it works is the sensors-- actually, the data gets sent to the mobile app, so the wearable has to work with a mobile phone, iOS or Android, and that goes for the frames and the headphones as well.

So some of the products that work with it are the frames, for example, which you all have here. I'm going to show just a little brief intro in frames.

[VIDEO PLAYBACK]

[MUSIC PLAYING]

(SINGING) I look good. Good, good. Say, I look good. Good, good. This is sunshine dripping down my face.

[END VIDEO PLAYBACK]

So the frames were actually a concept that came out of our San Francisco office, so it's a very different kind of idea behind it. I, personally, think it's a super cool idea. I think frames are a really great outdoor wearable.

It's great while you're biking, for example, when you don't want to be interrupted. If you don't want to-- let's say you get a call. You could just pick up straight through the frames. You don't have to pull out your phone.

Or you could do turn-by-turn navigation. Again, you don't have to be mounting your phone, putting a waterproof case or whatever. You just have your frames on.

Also, it doesn't plug up your ears. It's open ear audio, so you can still hear the sounds of the street around you. And it's not on the safe in that way, like wearing like noise canceling headphones while biking, for example.

I just realized I got to do this QuickTime thing. Maybe click there. There we go.

Next up-- oh, skip that-- QC35 II's. Now, this product you might be more familiar with. You've probably seen people wearing these, usually maybe on an airplane. So this is the classic noise canceling headphone which also has Bose AR sensors in it. And a lot of people actually don't know that, so I believe it's all the models after October of 2018 were manufactured with the Bose AR sensors in it.

And then the NC700's, which are these-- so these are the brand new noise canceling headphones that just came out. I get asked what's the difference between the QCs and the NCs. There's a few.

Obviously, there's a redesign. It looks a lot more modern. It has a little bit more of a sleek aesthetic. I think that the QC35 have a very dad road trip look to them, especially the gray ones.

But yeah, it's a brand new redesign, and one of the main upgrades, I'd say, is the microphone. So for conference calling, it actually has a noise canceling microphone because it has eight beamforming mics. So it can actually pick up your voice as opposed to the environment, so I found it really good in loud environments. They've actually done comparison tests in a loud environment, and you can barely hear anything in the background while you're talking with them. So I personally really like that because I do a lot of conference calls in sometimes noisy environments. There's a little promo video associated with that, too, if you want to see.

[VIDEO PLAYBACK]

And these are the first product that was-- the first Bose product that has been marketed with the Bose AR features in it. So the whole idea behind the NCs is, again, the Heads Up Hands Free.

[MUSIC PLAYING]

FILIP BABA: So I don't know if you saw near the end of the video, but there's a little capacitive touch screen here on the side. So it's not touch screen, it's a touch--

AUDIENCE: Surface.

FILIP BABA: Surface, yeah. And it's, yeah, you can do volume up and down, skip track, or go back. And then there's also touch and hold, which you can actually get as a gesture inside Bose AR when you're building your app. So again, the whole idea is, you're not pulling out your phone. You have your wearable, your AR-enabled wearable, which is your headphones, which you carry around with you. If you're like me, I carry around my headphones everywhere. So you can talk to your Google Assistant or to Siri through it, receive messages. It'll actually read out the messages to you.

So again, the whole idea is to have this wearable be pretty much your main input and output device for your daily mobile uses. So one of our projections is that we're going to have one million AR devices in 2019 across all three of the products that I just showed you. Now, that's actually really huge, because most AR is single purpose AR.

If you get a hololens, it's more enterprise. You're not really going to be wearing it out on the bus, you know. Or a Magic Leap, same thing, they only work in certain environments. It can't be in bright sunlight. And AR, typically, is a very niche thing that's found in specific places for specific use cases.

But the idea behind Bose AR is that these wearables are actually being worn by people every day, already, just for their regular life, to listen to music, to do their conference calls. But now we have this AR platform built into it as well. And developers can actually leverage a very big install base. Because there's a lot of these devices out there. And they're constantly being sold and the user base is growing.

And they're going to be looking for Bose AR enabled apps. Well, what can I run on these new headphones that I just bought? So we have an iOS and Android native SDK. And then we also have a Unity plug-in, which is what we're going to be using today, which actually works with both iOS and Android.

So I'm just going to go into a few of the use cases, a few of the apps that are here, that might be a little bit inspiring so you can kind of get an idea of what people are doing. So Otocast is a really cool concept behind, like, how you can-- let's say you're at a music festival. They could set up a tour where you actually hear these audio pings.

So if there's a point of interest as you're walking by, you'll just hear, like, a spatial audio pinging. And it's very unobtrusive, it's not like it's yelling at you. Hey, there's the thing here.

It's just kind of like, it's a ping. And then if you look at that ping, you can actually double tap to enter it. And it'll tell you, oh, this is this stage. Or maybe if you're doing like a historical tour, it'll tell you this is this landmark, would you like to enter this tour? And then you enter it. And then you can go and actually explore and it'll kind of tell you what these environments are.

Walc is kind of a similar concept, also to Otocast and Naviguide. And Naviguide, I explained earlier. It's kind of like you're using a gesture to Yelp things, basically. So it's really cool to kind of, like, not bring you out of your travel experience. You're just kind of walking, and exploring, and getting information you need without, again, staring at your phone.

So all of these kind of use initially in the sensors what I said. So you the GPS from the mobile device. And then you use the magnetic heading from the wearable. And what's really cool about using magnetic heading from the frames, people always wear their sunglasses perpendicular. It's not like your phone, where the magnetometer sometimes-- you ever use the map and you're trying to look at the compass on the map, and it's opposite, or it's not very accurate? And you got to, like, do this and you got to recalibrate your phone. So with the frames, it's actually a lot better because you're never really putting that in a wrong direction. It's always kind of facing perpendicular. So it's really good to get a heading.

Sport and wellness, so another use case is some of these sport apps. So Golfshot's actually a really great example. People wear frames when they're out on the golf course. It's sunglasses, they might be listening to music as well, so it's a perfect fit. So Golfshot's an already established app that golfers use to get information about but their shot, and their stored game details. I'm not much of a golfer so I don't really know all the details. But I do know that they have a Bose AR enabled feature that actually allow you to measure things like distance to the next hole, again, without having to pull out your phone and typing in some information. It uses all the data from the sensors and from the mobile device. It knows where you are, it knows what course you're on. And then you can have this seamless experience, again, without pulling out your phone. You're just already wearing some glasses because you're playing golf, and it's sunny, and you know. Yeah, I hope I makes sense.

So New Balance I think is a really cool one. This is a demo. So they kind of created more of a workout app. It's kind of like a motivational workout app, where it'll count your sets and reps. So think, like, you're doing sit ups and you're actually doing that custom gesture. And you're counting how many sit ups you did. And it gives you positive feedback, oh, you're doing great, two more, or something like that. So again, if you weren't doing heads up, hands free, maybe you'd be checking your phone to see how many you've done. There's really no way to kind of do it seamlessly unless you're kind of wearing a little extra piece of sensors on you.

Headspace is also a demo. And this is actually getting a big revamp. But the idea behind this, it's a meditation app. So it's a perfect fit for things like spatial audio. They even have a little bit of an exercise you kind of like stretch your neck.

And then again, it's using the sensors in the wearable to actually be able to tell if you did your neck stretch. Then when you do your neck stretch, it kind of has this soothing voice. It's like, all right, so now, we're going to close our eyes and, you know. And it starts to play a little soundscape. And it helps you kind of meditate.

Another idea I heard brought up is, you can actually check someone's posture. So if someone's falling asleep, you can kind of slightly ping them to wake up, so things like that. Like I said, developers have been really creative with it.

And then music of course, I think music is one of the best use cases. Because I think spatial audio, which is what today's workshop is mainly going to be about, is a very unexplored area of music or sound in general. And Bose AR is a perfect fit for spatial audio, especially because of the sensors that I was just mentioning because you know the head orientation, you can actually simulate spatial audio just between the two stereo speakers. You can actually blend the sound between based on where your head is looking.

So Radar is a great example of this. So Radar is an app built by Bose, which actually connects to something that's called Creator Tool that's currently in development. It's in closed beta.

But the idea here is that Radar is going to be kind of almost like a marketplace slash area where you can browse lots of different spatial audio experiences. And they're going to be tied to certain areas on the map. So maybe if you're in a certain place, there's certain spatial audio experiences that wouldn't be in some other place.

And it can do things like it can ping you when you enter, let's say, some park. And maybe there's a spatial audio experience tied to that park. And you'll be able to enter it if you just kind of have the app running in the background.

Traverse is a cool example of a mixture between RAPI and existing ARAPI. So I think they're using ARKit. And what it actually allows you to do, it's kind of like you're in the middle of a concert. And you're hearing the different instruments playing around you.

So what you can actually do is, while you're holding your phone up, you can actually kind of close your eyes you just walk around the room. And you're actually exploring all the different instruments around you. You can walk up to the singer, you can walk up to the piano. And it's using ARKit to do positional tracking and Bose AR for spatial audio and your head orientation.

So it's actually using both the phone's positional and the head's rotational tracking. And it knows between the two. And it kind of simulates as if you're actually in the middle of a band or a concert that's playing.

iHeartRadio is a cool demo app, I'd say. It's basically another take on how you can tune in to different radio channels. So instead of pulling out your phone and seeking, you're actually using your head orientation. And you're kind of, like, your head's the tuner. And then when you like something you just kind of double tap and you go into it.

And then you can pull out of it at any time and tune into something else. And if you get a little preview and you like it, you tap into it and you're listening to it without having to look at oh, I'm 22.3 FM, or whatever. You're just kind of listening to it.

Games is predominantly in the realm of our Unity SDK. So we have some cool games that have been built. The three that I have up here are what I would call representative experiences. So we actually have a lot more games that have been built, especially this year.

And I'll show you actually some videos of the games to kind of get inspired. So all these games are actually out. So you can download them on the App Store if you have a Bose AR enabled wearable.

And like I said, they're representative experiences. Meaning that I believe these developers kind of created some design patterns out of these. They were one of the first to make games for it. And I think these three are kind of in their own category.

So OverHerd, it's a really funny game. You basically play as an Englishman that's taunted by Frenchmen at castles at night. It's very kind of, like, Monty Python-esque. Funny accents, the voice acting is hilarious.

And what you do is you use your head as, like, a catapult. And your head orientation like this. And you double tap and you basically launch cows into these castles. And you kind of hear the moo. And it's all spatial. So you're kind of listening for where they're at.

And when you're turning your head, you hear the castles over there. And then the Frenchmen are, like, taunting you. Like, oh, I bet you can't hit the broad side of a barn, or whatever. I'm not going to try to do the accent, but you should hear it. It's pretty hilarious. And it's a great example of, it's kind of using the spatial audio and using your head orientation as the controller, basically.

So Komrad is a cool design pattern. So it's basically like a spy novel. And it's a choose your own adventure spy novel. So you can actually make decisions in the story.

So the story goes on and there's yes and no decision trees that you go through. And at some point I think you have to duck and dodge a bullet. And it's got this whole AI theme to it. And the AI's talking to you and you're making decisions. And you kind of change the outcome of the story.

So that's a pretty popular design pattern, and one that I'm a huge fan of. I love choose your own adventure games. And I think the medium is perfect for it. Instead of just doing yes or no on the screen, you can actually do yes and no by nodding and shaking your head. So it's very immersive. When you get asked a question you're like, yes. And then it kind of goes on.

Dead Drop Desperado is a multiplayer game. It's a local multiplayer game. So what I actually really like about this one is it's got a little bit of a party game kind of aesthetic to it. So you play with a friend. And one of you holds the mobile phone and the other one gets handed off the wearable, so let's say the headphones.

And what happens is he's standing in front of you. And you shoot bullets at your friend. And your friend is trying to dodge the bullets by listening to where they're coming from using the spatial audio. And you're kind of leaning left and right like that.

And then it gets switched off. And then you're the one who's dodging, and he's the one that's shooting. And then in the end, there's kind of a score tally and it's got a leaderboard. And it kind of has a little bit of a competitive aspect to it.

What I personally like is that you could, let's say, you have your headphones, and you want to show someone something cool on them. You could just kind of hand off the headphones. And now it's this multi-player interaction.

So you have the headphones, I have the phone. And you switch it up. And it's also a little bit competitive with the leaderboard and such.

So like I said, representative experiences. You're definitely not limited to these design patterns. We're actually excited to see what our developers come up with. And there's always cool stuff that people come up with.

And I just wanted to show you a little bit. So we have this video that was produced. So we've posted a bunch of game jams this year. And this is one of the first game jams that we hosted. And these games that I just showed you were actually built at it.

So I'll just go ahead and show you. So this is from our Playcrafting partnership. Playcrafting is a small, game developer collective. But plus they also host events like Play NYC.

And they bring developers, they teach Unity classes, and they just have a general kind of game developer community across the country. So we've partnered with them to have some of these Bose AR game jams. And I'll show you the video.

[MUSIC PLAYING]

ERIC HAMEL: We're here this weekend at Bose's offices in Boston. And we're doing a game jam with their new AR technology.

MICHAEL LUDDEN: We've given the developers a brand new version of the Bose AR SDK and Unity plugin. It allows Unity developers to use and add Bose AR integration to Unity based apps.

REJON TAYLOR-FOSTER: They've built a very intuitive system for developers like me to be able to just pop it into Unity and just go.

MICHAEL CARRIER: We give you access to all of the sensors that are available in the device, the accelerometer, the gyroscope, two different rotation sensors. They provide orientation of the user's head and world space. We can actually provide spatialized audio, which just gives you an entirely new level of context when you're making any sort of experience.

ERIC CHAN: We're able to get all sorts of different types of information from the frames glasses, which we can use to help develop the new types of interactions.

ERIC HAMEL: We were challenged to create an augmented reality experience that put audio first.

ANNA SHABAYEV: It's very interesting to look at games from, like, a whole different angle.

JONATHAN SHRACK: Usually people think AR and think, oh, it's all visual. And this one, it's Bose, so it's audio.

CRAIG HERNDON: The challenge really made us put our heads together and be like, what can you really do?

ANNA SHABAYEV: There's a whole world of sound. And we shouldn't just be paying attention to just the visual component.

MICHAEL LUDDEN: So we've had a lot of really creative ideas that the developers have come up with so far. And I think some of the more interesting aspects are the different game mechanics with the user interface experiments that they're doing.

MIKE LEVINE: The game we created, we're calling sonic samurai. You're the samurai and you can't see. But there are all these monsters around you. And you basically have to use the Bose headset to determine where the monsters are. And then you actually use your mobile device as your sword to strike at the monsters.

REJON TAYLOR-FOSTER: It's a game where you essentially try to dodge bullets that are sent from one user towards you. Imagine the matrix, but you can't see the bullets that are coming at you. You can only hear them.

MARC HARPIN: Our game is called OverHerd.

ERIC HAMEL: You play a medieval catapult operator, catapulting poultry and cows as French castles.

MARC HARPIN: You're doing this all at night, under cover of darkness.

ERIC HAMEL: So you have to use your ears to hear where the castle is.

MARC HARPIN: When you point your frames in the correct direction, you can hear the French defenders taunting you.

ERIC HAMEL: Daring you throw things at them.

FRENCHMAN 1: Your efforts are all wastes of perfectly good livestocks.

MARC HARPIN: By turning your head and elevating it up, you actually modulate the angle the catapult with fire at.

ERIC CHAN: You are being bombarded by vegetables in a 360 degree radius. You can tell what direction the vegetables are coming from based on spatial audio. And you need to find and face the correct vegetable and eat it.

JONATHAN SHRACK: We're working on kind of a spy thriller game.

CRAIG HERNDON: When you put on the Bose frames, you start to get radio chatter from a secret agent. And you start doing instructions to locate radar pings, diffuse bombs. Like James Bond and Mission Impossible in real life.

WOMAN 1: Duck now.

ERIC CHAN: Augmented reality is really one of the most important future mediums for design.

MICHAEL CARRIER: And we're getting the tools set up in a way that allow people to start experimenting. And using something like Unity allows that iteration to happen very, very quickly.

REJON TAYLOR-FOSTER: It's an honor to be chosen to be here to work on this technology for the first time.

ANNA SHABAYEV: And also give developers a bigger idea of, like, what's possible with this stuff so that they can use our games as a stepping stone for what's next.

DAN BUTCHKO: Here we are at PAX East 2019. All five of these games are debuting for the first time anywhere.

ERIC HAMEL: People are having a lot of fun here this weekend. Lots of people dodging vegetables and throwing cows.

MAN 1: I always enjoy, like, finding new games that are innovative.

MAN 2: I've never played anything just based entirely on sound.

WOMAN 2: I don't think I've seen anything else here that relies completely on audio and motion, no visual at all. But it felt really immersive and cool.

IAN CUBIN: People are really getting a sense that the audio is read and spatial.

WOMAN 3: You felt like you're really there.

MAN 3: It's just a whole different experience.

WOMAN 4: It's more social. It really brings us back to where games really are, where it's with other people.

MAN 3: It's a fun ride, these glasses.

DAN BUTCHKO: We're changing the landscape of AR and games in America. And this is just a taste of what's to come.

CRAIG HERNDON: It's very exciting for us to have that chance to impact other people's lives with a new technology.

CHUCK FREEDMAN: The things that people were able to do over a weekend really inspired me to think we're going to see some amazing stuff this year.

FILIP BABA: And we have actually seen some pretty amazing stuff this year. I feel like the more of these experiences get built, and the more developers get creative with it, it's an early platform so there's plenty of room to kind of set the standard at what could be created. What else do I have here?

So I have a few actual gameplay footage that I'd like to show you on some experiences that I personally like. Developers cut up these videos to kind of show what the games are like. So just going to get into these. And again, just some more inspiration.

NARRATOR 1: The Worst Grim Reaper's Soulmates it's an AR music game.

FILIP BABA: That's this t-shirt.

NARRATOR 1: In which a singing grim reaper plays a song that responds to you and your world. The grim reaper Sebastian is a character from another one of our games. He's tired of his dead end job collecting souls and wants to pursue his dream of being a songwriter. Seeking inspiration, Sebastian asks you to let him write a song about your life.

[MUSIC PLAYING]

SEBASTIAN: (SINGING) You looking up, maybe lost in thought, kind of like me, you know I daydream a lot. There are things on the ceiling and in outer space. There's magic in almost every place. Hey, would you like to share you thoughts with me? Wanting to learn about the inside of the human skull. I ask what's going on in your soul. OK, mm, are you maybe thinking about food? I still have a lot to learn about human kind.

AUDIENCE: [LAUGHTER]

SEBASTIAN: Would be on their minds. OK, so if it's not spaghetti or chocolate cake. So maybe the fact that the sun will someday explode and everything we know will go up in an ocean of flames? The human mind is less dark than I would guessed. I'll try to see inside of you once more. Hm, oh, is it someone you like? As in, like like?

NARRATOR 1: Imagine players using this app while commuting and exploring or just relaxing. Through interacting with Sebastian and his song, we want to show players that even the smallest things in life can carry meaning. Sometimes we just need someone to talk to.

FILIP BABA: Cool, right? It actually works. It's actually really fun. So one of the things that I personally have learned over overdoing some of these jams and seeing what developers build is that when you're not focusing so much on the visuals, you have a lot of room for story and substance, if I could say it so myself.

It's like this developer, was we had all teams of four. So three of those people were just content creators. So one we brought was Mitty. He was he was actually producing music during the jam. One was just doing like the singing and the voice acting.

I forget what the third guy was doing. I think he was doing, like, UI. Even though I do recommend to have nice, polished UI. But it doesn't have to be crazy, that's the thing. Like most games are very 3D.

And one of the things that's a problem, I'm a game developer myself, nowadays there's so many tools and so many things you can do, it's kind of hard to figure out what you want to do. Because you're like, oh my god, just overwhelmed with all these things that our modern devices can do.

So there's a lot of room for this kind of content to be produced. So this is kind of like a little AI kind of companion game, you could say. And yeah, it responds. And there's other things he didn't mention in the video. Like, it actually changes based on the weather.

So it actually checks the weather. If it's a rainy day, you start the game, you'll have a rainy day mood. If it's a sunny day, you'll have a sunny day mood. So it's kind of a replayable thing. And I don't expect people to be running this app, like, nonstop.

But I could probably see people running this for that one hour of their commute during the day, or maybe like for their lunch break or something like that, which is actually a lot, in terms of mobile app usage. So it's fun. And it's music, so you're probably already listening to music. Why not have a soundtrack to your life, for example? That's kind of the idea behind the game.

This one is also a really cool example. So I'll just let him explain it.

NARRATOR 2: That is about bopping and grooving your head to the music. Similar to other rhythm games, you score points by staying on the beat. At the moment, we have about a dozen custom written songs. In the future, we want the possibility for players to download hundreds of songs or play with their own music library.

[MUSIC PLAYING]

We are going to play Party in the Washing Machine. Our foxy friend here is grooving along as well. You could follow it and he helps you stay on the beat. The line below is showing your head movements. It also changes colors to indicate how long you're doing.

So we really intend this game to be a phone in pocket experience. We added both audio and tactile feedback. For example, the high pass filter turns on when you're doing bad. And your phone vibrates with the beat when you're doing great.

So the idea of Hip Hop Hero came from us wanting to make a game that naturally played with Bose AR defenses, something you can play while commuting on the train in public.

FILIP BABA: Yeah, it's like a play on words of Guitar Hero, kind of. You know, you're kind of staying on the beat. You get rated based on how well you did, how well on the beat you stayed.

And you don't have to have a specific head gesture. It's just kind of how you would normally bob your head. And they have a pretty good algorithm for detecting that change. They're using both the accelerometer and the gyro, but I'm not I'm not 100% sure on that.

But yeah, I thought it was great how they visualized it as well. So that's an actual graph based on your head movement. And then you get rated and, you know, it's yeah. I mean, you've probably seen games like this.

And this is the last one. This one I like because you're kind of producing music in a way. Now, it's basically song loops that they've created. But the amount of song loops that are in this app, for example,

I mean, someone here can probably do the mathematical permutations of how many combinations of unique songs you can get out of it. But it's a lot when you include, like, 10 loops, for example, for bass lines, as you're kind of adding complexity to a song. And if you have maybe, like, 10 of each samples and in the end you got, like, maybe 40 samples in total, you could be creating unique music on every play through.

NARRATOR 3: Choose your own adventure musical experience of layering song loops, set in a world where flowers produce music. Players listen, explore, and nod along to the music, collecting song loops like pollen, in order to blossom a musical garden.

The game is played by looking around to find new sounds, and nodding your head along to the beat when you find one you like. This process can go on indefinitely, allowing players to explore countless new ways of layering together song tracks.

FILIP BABA: It's basically, she totally nailed the explanation, like, in the beginning. But basically, yeah, you look around. You hear a preview of a loop, you like it. You nod it. And then you go on, and you're just layering loops. And it works really well. I mean, the developer really figured out a way to make it.

If you've ever tried to make music, sometimes you could just mess up and it just sounds terrible because you're mixing way too many dissonant sounds. But they managed to figure out an algorithm to layer it and it always sounds good.

And I've tried to break it. Whenever I play any of these experiences, I'm pressing everywhere and nodding to everything, trying to make it sound bad. And I couldn't. It always sounded good. So super cool.

So that's our developer portal. It Developer.Bose.com/BoseAR. That's where you'll find the SDK. I'm sure most of you have already gone on here and downloaded it. But I'll show you where all that is in a sec.

And the last kind of, like, but not least important thing that I wanted to say. So we have a new program that we just launched recently, and that's Bose AR Certification. So what Bose AR Certification is, is say you build a Bose AR app and you submit it for certification.

Our internal team will actually help you get across the finish line and publish the app to the App Store or the Android Store, making sure you have all the right requirements. And then if it does get certified, and it's within our brand guidelines, which again, we'll kind of help you get through that finish line, you can get a possible feature on the Bose Music app or the Bose Connect app. And you might even get featured on the Bose website.

So it's a really great showcase and a really great way to get noticed, especially as an early game developer. It's an early platform, so obviously we're looking for great content to feature on our portals. And if the content's great, and it's, again, passes the certification.

Which all these games that I've shown you, they're all in the process of getting certified and they'll all be available when you have your frames, for example, when you hook them up in the Bose connect app. There's actually a section there, Bose AR enabled apps. And then you click that and there's a directory there of apps. Yes.

AUDIENCE: So this works with Android now?

FILIP BABA: So it works on Android as well, yes, it does.

AUDIENCE: OK.

FILIP BABA: We had a little bit of a, like, earlier start with iOS SDK. So a lot of stuff was mainly iOS.

AUDIENCE: In July it wasn't working on Android.

FILIP BABA: Yeah, now we're rectifying that. There's going to be a lot of Android experiences coming out. Sadly, it's still, some developers just build for iOS. And we kind of have to prod them. Be like, hey, could you also build an Android build? And sometimes they need to be incentivized to do that. Other times they just do it.

If it's Unity, then most of these experiences are going to be on both. But sadly, that's just kind of how native app development is, they sometimes just want to work in one and not the other. But we are fully Android supported now as well.

And then we also have a Get Inspired page. This is a place where we have some case studies written. Golfshot's there, if you want to read about it. Dead Drop Desperado's also up there. I think OverHerd.

And it's just kind of a little interview with the developers. They kind of talk about their process of developing. And it's a good place to get inspired and be like this guy, really happy, like, ah. Yeah, that's basically the intro. I hope that didn't take too long. But yeah, just wanted to kind of give you a little bit of an understanding of what this is. So does anyone have any questions or no? Good.

All right, so now that we've gone through that, what I'm going to do here, I'm just going to close my Unity. And let's go ahead and actually fire up Unity. And we're going to get our a little beginner workshop started.

So you should all have a copy of the Unity package, of the Bose AR SDK for Unity, which you can find it on our downloads page. Let me know if you have trouble finding that.

We're going to be using this 4.0.1 beta. And when you download that, it's just a Unity package that gets downloaded. So I'm just going to let everyone kind of get to that. And the version of Unity that I'm going to be using is just the latest one. I have 2019.2.5F1.

And I have Android and iOS build support as modules in it. And I recommend that if you're going to be building to mobile that you have your desired platform modules installed. I'm just going to let everyone go ahead and do that.

AUDIENCE: Hey, should we tell them that-- is it still that iPhone is only VDM?

FILIP BABA: It's always a thing, yeah, it's just Apple and iPhone. But again, if you don't get a chance to build, you'll still be able to preview the app and play with it directly from your laptop. For most of this workshop, we're actually going to be sticking to the creative side of things, and concept, and trying to build an experience.

And then, like I said, towards the end, we'll try to make sure that everyone has built. And if you don't have the right device, I'm sure we have some extra devices. And if it comes to it, we could potentially also build or maybe share devices if needed. But once you have it set up once, then you're good to go. I have my iPhone here. I usually have to have both.

All right, everyone got their SDK? And ready to go. So what we're going to do is we're going to actually make a new project in Unity. And I'll just make sure everyone has this before we do that.

I'm just going to do the latest. And I'm just going to call this project MIT Bose AR Unity Workshop. And I'm just going to pick 3D project and create. Bless you. Hm?

AUDIENCE: Isn't it stable core?

FILIP BABA: Yes it is. I think I just missed a little bit in the beginning. All right, so I'm just going to switch to the default layout here.

Anyone not have their new Unity project open yet? All right, just take your time.

AUDIENCE: I'm downloading [INAUDIBLE].

FILIP BABA: Oh, well good thing the internet's fast.

AUDIENCE: I'm using an early version Unity right now.

FILIP BABA: How old? What's the version?

AUDIENCE: 15.

FILIP BABA: That's fine.

AUDIENCE: OK.

FILIP BABA: That's fine.

AUDIENCE: I'm downloading the new one.

FILIP BABA: Anything that's past the LTS, the 2017.1.4 should be fine. Anything with, like, an f at the end of it, it's final, not beta should work.

AUDIENCE: OK.

FILIP BABA: If for whatever reason it doesn't, I know sometimes there's been some edge cases. But it should be fine.

[INTERPOSING VOICES]

So when you do have this, I just want you to take that Unity package that was downloaded and just click it or double click it to open it. And you'll get an import dialog. And I just want you to import the entire thing into your project. Everyone imported?

AUDIENCE: Yeah, what do you click to import again?

FILIP BABA: You should just be able to double click the Unity package. And it should just pull up Unity. If for whatever reason that doesn't work, you can always right click in your project here and then Import Package, Custom Package. That also works as a backup.

AUDIENCE: Yours is a CSC file, right? I have an MCS file. So--

FILIP BABA: For what?

AUDIENCE: Just for the assets.

FILIP BABA: Oh, yeah. Don't worry about that. Everyone imported, ready to go? Give it another minute.

So yeah, when you import the Unity package, you get the entirety of the SDK. There's samples in here, sample scenes. There's a lot of cool stuff in here that I'll get into. And the setup itself is actually really straightforward and simple.

We've tried to make this as easy as possible to either put into a new project or integrate with an existing project. So if you're familiar with Unity, or if you've worked on a Unity project before, you'll find that a lot of the, I guess, the user flow and the way that the SDK works is typical, kind of to a Unity workflow.

So once you're all imported, you'll see these three folders show up. So the first one that I want you to look at is the Bose folder. So you go inside the Bose folder, there's wearable. And then inside wearable, there's modules.

You go to modules. And the first module that we want is connection. So we're going to go to connection and then prefabs. So that's Bose, wearable, modules, connection, prefabs.

And I want you to pull in this first prefab that's available here. So just drag that into your scene. You should see this white rectangle pop up. Now, the first thing that I like to do when I start a Unity project, especially if I'm developing on mobile, is I like to pull my game view, which is usually up here.

You might have a different layout. And I actually like to take the game view, and I like to make it visible. So I actually drag that game view over here to the right side of the project panel. So I always kind of have a little preview window here of what my preview looks like, basically.

So if you go ahead and do that, then you'll see that the panel that we just pulled in actually shows up here. And I'll explain what the wearable connect panel does. It's basically the entire menu that shows up that you can browse all available devices. You click a device, and you connect to it.