Meta (Facebook) Connect 2021 Metaverse Event Transcript

Mark Zuckerberg: (00:01) Desktop to web to phones, from text to photos to video. But this isn't the end of the line. The next platform and medium will be even more immersive, an embodied internet where you're in the experience, not just looking at it, and we call this the metaverse. And you're going to be able to do almost anything you can imagine, get together with friends and family, work, learn, play, shop, create as well as entirely new categories that don't really fit how we think about computers or phones today. Now, since we're doing this remotely today, I figured let's make this special. Mark Zuckerberg: (00:39) So we've put together something that I think is really going to give you a feeling for what this future could be like. We believe the metaverse will be the successor to the mobile internet. We'll be able to feel present like we're right there with people no matter how far apart we actually are. We'll be able to express ourselves in new, joyful, completely immersive ways and that's to unlock a lot of amazing new experiences. When I send my parents a video of my kids, they're going to feel like they're right in the moment with us not peering through a little window. Mark Zuckerberg: (01:13) When you play a game with your friends, you'll feel like you're right there together in a different world, not just on your computer by yourself. And when you're in a meeting in the metaverse, it'll feel like you're right in the room together, making eye contact, having a shared sense of space and not just looking at a grid of faces on a screen. That's what we mean by an embodied internet. Instead of looking at a screen, you're going to be in these experiences. Everything we do online today connecting socially, entertainment, games, work is going to be more natural and vivid. Mark Zuckerberg: (01:49) This isn't about spending more time on screens. It's about making the time that we already spend better. Screens just can't convey the full range of human expression and connection. They can't deliver that deep feeling of presence, but the next version of the internet can. That's what we should be working towards. Technology that's built around people and how we actually experience the world and interact with each other. That's what the metaverse is all about. Now, the best way to understand the metaverse is to experience it yourself. Mark Zuckerberg: (02:18) But it's a little tough because it doesn't ... Emerge as we speak, we're starting to get a sense of how it could all come together and what it could feel like. So today we're going to do something a little bit different. Rather than just focusing on this year's products, like a normal keynote, we're going to talk about the future. So let's start by exploring what different kinds of metaverse experiences could feel like, starting with the most important experience of all, connecting with people. Mark Zuckerberg: (02:59) Imagine you put on your glasses or headset, and you're instantly in your home space. It has parts of your physical home recreated virtually, it has things that are only possible virtually, and it has an incredibly inspiring view of whatever you find most beautiful. Speaker 1: (03:16) Hey, are you coming? Mark Zuckerberg: (03:18) Yeah, just got to find something to wear. All right. Perfect. Speaker 3: (03:27) Oh, hey Mark. Mark Zuckerberg: (03:32) Hey, what's going on? Speaker 2: (03:33) Hey, Mark. Speaker 4: (03:34) What's up, Mark? Mark Zuckerberg: (03:35) Whoa. We're floating in space. Who made this place? It's awesome. Speaker 2: (03:38) Right. It's from a creator I met in LA. Boz: (03:41) This place is amazing. Mark Zuckerberg: (03:44) Boz, is that you? Boz: (03:45) Of course, it's me. You know I had to be the robot, man. Mark Zuckerberg: (03:48) I thought I was supposed to be the robot. Speaker 2: (03:51) Whoa. Speaker 4: (03:53) I knew you were bluffing. Mark Zuckerberg: (03:57) Hey wait, where is Naomi? Let's call her. Speaker 4: (03:59) Naomi. Mark Zuckerberg: (04:04) Hey, should we deal you in? Naomi: (04:05) Sorry I'm running late, but you've got to see what we're checking out. There's an artist going around Soho hiding AR pieces for people to find. Mark Zuckerberg: (04:14) 3D street art. That's cool. Naomi: (04:16) Send that link over so we're going to look at it. Speaker 2: (04:19) This stunning dude. That is something. Mark Zuckerberg: (04:22) That's awesome. Naomi: (04:23) I love the movement. Mark Zuckerberg: (04:24) Wait, it's disappearing. Naomi: (04:26) This is amazing. Hold on. I'll tip the artist and they'll extend it. Boz: (04:30) Wow. Speaker 3: (04:32) If you guys like it here, I have another roommate that you're going to love. Check out this forest room. Mark Zuckerberg: (04:37) Let's see it. Speaker 3: (04:38) Coy fish that fly. That's new. Mark Zuckerberg: (04:41) This is wild. Hey, one sec. Boz. It's Priscilla. Prescilla: (04:45) Hey, you have to see this. Beast is going crazy. Mark Zuckerberg: (04:48) Oh, I love that guy. We've got to show that to the kids. Can you also send that to my dad? Prescilla: (04:55) I'll message him. Mark Zuckerberg: (04:56) All right. See you at home. This place is great, Boz, but there's something I got to get back to. Mark Zuckerberg: (05:01) All right. So that's a glimpse of a few ways that we're going to be able to get together and socialize in the metaverse. It's a ways off, but you can start to see some of the fundamental building blocks take shape. First, the feeling of presence. This is the defining quality of the metaverse. You're going to really feel like you're there with other people. You'll see their face expressions. You'll see their body language. Maybe figure out if they're actually holding a winning hand. All the subtle ways that we communicate that today's technology can't quite deliver. Mark Zuckerberg: (05:32) Next, there are avatars, and that's how we're going to represent ourselves in the metaverse. Avatars will be as common as profile pictures today, but instead of a static image, they're going to be living 3D representations of you, your expressions, your gestures that are going to make interactions much richer than anything that's possible online today. You'll probably have a photo realistic avatar for work, a stylized one for hanging out and maybe even a fantasy one for gaming. You're going to have a wardrobe of virtual clothes for different occasions designed by different creators and from different apps and experiences. Mark Zuckerberg: (06:10) Importantly, you should be able to bring your avatar and digital items across different apps and experiences in the metaverse. Beyond avatars, there is your home space. You're going to be able to design it to look the way you want, maybe put up your own pictures and videos and store your digital goods. You're going to be able to invite people over, play games and hang out. You'll also even have a home office where you can work. Your home is your personal space from which you can teleport to anywhere you want. Mark Zuckerberg: (06:41) Now, speaking of teleporting, there are going to be all kinds of different spaces that people make. Rooms like the ones that we just saw, but also games and whole worlds that you can teleport in and out of whenever you want. Teleporting around the metaverse is going to be like clicking a link on the internet. It's an open standard. In order to unlock the potential of the metaverse, there needs to be interoperability. And that goes beyond just taking your avatar and digital items across different apps and experiences, which we are already building an API to support. Mark Zuckerberg: (07:13) You want to know that when you buy something or create something, that your items will be useful in a lot of contexts and you're not going to be locked into one world or platform. You want to know that you own your items, not a platform. Now this is going to require not just technical work, like some of the important projects that are going on around crypto and NFTs in the community now. It's also going to take ecosystem building, norm setting and new forms of governance. And this is something that we're really going to focus on. Mark Zuckerberg: (07:45) Privacy and safety need to be built into the metaverse from day one. You'll get to decide when you want to be with other people, when you want to block someone from appearing in your space, or when you want to take a break and teleport to a private bubble to be alone. You're going to be able to bring things from the physical world into the metaverse, almost any type of media that can be represented digitally, photos, videos, art, music, movies, books, games, you name it. And lots of things that are physical today, like screens, will just be able to be holograms in the future. Mark Zuckerberg: (08:19) You won't need a physical TV. It'll just be a $1 hologram from some high school kid halfway across the world. And you'll be able to take your items and project them into the physical world as holograms and augmented reality, too. You're going to be able to move across these different experiences on all kinds of different devices, sometimes using for virtual reality so you're fully immersed, sometimes using augmented reality glasses so you can be present in the physical world as well, and sometimes on a computer or phone so you can quickly jump into the metaverse from existing platforms. Mark Zuckerberg: (08:53) There are going to be new ways of interacting with devices that are much more natural. Instead of typing or tapping, you're going to be able to gesture with your hands, say a few words, or even just make things happen by thinking about them. Your devices won't be the focal point of your attention anymore. Instead of getting in the way, they're going to give you a sense of presence, the new experiences that you're having and the people who you're with. And these are some of the basic concepts for the metaverse. Mark Zuckerberg: (09:21) And while this may sound like science fiction, we're starting to see a lot of these technologies coming together. In the next five or 10 years, a lot of this is going to be mainstream and a lot of us will be creating and inhabiting worlds that are just as detailed and convincing as this one on a daily basis. So even though it's still a long way off, we are starting to work on some of these foundational concepts today. Horizon is the social platform that we are building for people to create and interact in the metaverse. One part of this is Horizon Home, which is our early vision for a home space in the metaverse. Mark Zuckerberg: (10:05) Horizon Home is the first thing that you'll see when you put on your Quest headset. Today, there are already a bunch of options to choose from. And in the future, anyone will be able to create one. We've just called it Home until now, because it's been missing something very important. People. Soon, we're going to be introducing a social version of Home, where you can invite your friends to join you as avatars. You'll be able to hang out, watch videos together and jump into apps together. Then there is Horizon Worlds, which is where you can build worlds and jump into them with people. Mark Zuckerberg: (10:42) Horizon is designed to make it possible for everyone to create, and we're already seeing people build some really interesting experiences from creating new games together to throwing surprise parties and VR that family and friends around the world can join. We started rolling out Horizon Worlds in beta last year, and we're adding more people and more worlds every day. And we also just launched Horizon Workrooms earlier this year for collaboration. Beyond Horizon, we are also making it easier to communicate with your friends across different layers of reality. Mark Zuckerberg: (11:16) This year we're bringing messenger or calls to virtual reality. You're going to be able to invite your friends to a messenger call and soon you'll be able to explore somewhere together or join a game. Now, these are the kinds of tools that need to get built so that you can jump into the metaverse with your friends from anywhere, and they're going to unlock some pretty amazing experiences. Now let's move on from some of these basic concepts and people just connecting to a completely different set of experiences on entertainment and gaming. Mark Zuckerberg: (11:54) Imagine your best friend is at a concert somewhere across the world. What if you could be there with her? Speaker 5: (11:57) Yo. Speaker 6: (11:57) You're here. After party passes? Yes. Speaker 5: (11:57) Yes. Speaker 6: (11:57) Jumping in now. Mark Zuckerberg: (11:57) What if there was an after party that anyone could go to no matter where they were? Speaker 6: (13:12) This is wild. Speaker 5: (13:20) Hey, check this out. Charity auction happening. Oh, there's swag. Mark Zuckerberg: (13:25) And when you go to the after party, you could connect with other fans, hear new versions of your favorite songs and check out the merch that just dropped. Speaker 7: (13:34) I like that one. Speaker 6: (13:36) Well, now you've got to get it. Mark Zuckerberg: (13:39) You can start to see how the metaverse is going to enable richer experiences by letting us add new layers to the world that we can interact with. Creators and artists are going to be able to connect with their audiences in new ways and really bring them into these shared experiences. Now there's a lot that needs to get built to create experiences like this, but we're working on some of these pieces right now with Spark AR. First, we're building tools that creators can use to place digital objects into the physical world and let people interact with them. Mark Zuckerberg: (14:09) And rather than just simple visual effects, new creator capabilities will support 3D objects that can respond and react realistically, including a realistic sense of depth and occlusion. In the next year, we're also adding the ability for creators to connect different physical locations into cohesive, augmented reality storytelling experiences like guided tours or scavenger hunts. We're also so building a horizon marketplace where creators can sell and share 3D digital items. And our hope is that this will enable a lot more commerce and help grow the overall metaverse economy. Mark Zuckerberg: (14:50) Because at the end of the day, it is really the creators and developers who are going to build the metaverse and make this real. And to make sure that there's an ecosystem that can sustain hundreds of thousands of people working on this, which is what we're going to need to bring this to life, then it's critical that creators and developers can make a good living doing this work. Now, if you ask people today what they thought the metaverse was, a lot of people would probably say it was a Spider-Man movie. Mark Zuckerberg: (15:19) But the people actually follow the space would say it's about gaming. And that's because gaming provides many of the most immersive experiences and it is the biggest entertainment industry by far. Gaming in the metaverse is going to span from immersive experiences and fantasy worlds to bringing simple games into our everyday lives through holograms. Maybe you'll play old games in new ways. Speaker 8: (15:43) So Barcelona, huh? Speaker 9: (15:47) Well, it's not New York, but I like it. Speaker 8: (15:50) New York misses you. Speaker 9: (15:53) What's that? Speaker 8: (15:54) I said let me put my game face on so I can beat you. Speaker 9: (15:59) Okay. Check. Speaker 8: (16:02) I got to try another game face next time. Mark Zuckerberg: (16:05) Maybe you'll go head to head with players from around the world. Speaker 10: (16:10) Take a shot. Speaker 11: (16:11) Some call it skill. Speaker 10: (16:13) Keep talking. Mark Zuckerberg: (16:16) Or maybe you'll do things that aren't even possible in gaming today. Speaker 12: (16:27) Hey, Mark. Down for a VR foiling sesh? Mark Zuckerberg: (16:30) Now this is more my style. Speaker 12: (16:35) Classic one. All right. Nice. Nice choice, Mark. Ready to shred. All right, here we go. Mark Zuckerberg: (16:41) Whoa. Speaker 12: (16:42) Whoa. Hang in there, Mark. Mark Zuckerberg: (16:43) That's not good. Speaker 12: (16:44) Hit this section. Boom. All right. Back flip. Yoo-hoo. Mark Zuckerberg: (16:49) I've got an idea. Speaker 12: (16:50) Where you going? Mark Zuckerberg: (16:51) Got to pump it to jump it. Speaker 12: (16:52) What? I didn't know that was an option. Mark Zuckerberg: (16:55) You're not going to catch me now. Speaker 12: (16:56) Take that unicorn. Tube city. Mark Zuckerberg: (16:59) God, you're out of control. Speaker 12: (17:00) Don't worry. I'll let you win next time. All good. That was a close one. You want to go again? Mark Zuckerberg: (17:05) Maybe later. I'm going to need a lot more sunscreen though. Oh man. Well, gaming is a lot of people are going to step into the metaverse for the first time. It already has some of the most fully built out digital goods, the most active creator and developer communities and major platforms like Epic are working to build out the metaverse starting with gaming. For our part, we're heavily investing in building a healthy VR and AR ecosystem, so the game studios can and keep building and gaming creators can keep creating. Now, Deb from our studios team is joining me. Mark Zuckerberg: (17:40) Deb, do you want to take us through some of the exciting games in the pipeline for Quest? Deb: (17:43) Absolutely. Over the years, we've had the opportunity to work with incredible developers like Vertigo Games, the studio behind fan favorite, Arizona Sunshine. Mark Zuckerberg: (17:53) I love Arizona Sunshine. That game basically got me and my friends through the first few months of the pandemic. Deb: (17:57) That's awesome. If you enjoyed that, Mark, I think you'll be excited that we're partnering with Vertigo on five more great games from Deep Silver and others. We'll share more about this lineup very soon. Mark Zuckerberg: (18:08) Nice. What else is coming? Deb: (18:10) Well, the metaverse is constantly evolving, so one of the most important aspects will be live service games that launch updates and new downloadable content regularly like Echo VR, Beat Saber, Onward, Pistol Whip, and more. We're focused on this a lot right now, making sure games can build out active communities. Beat saber has a passionate community. Mark Zuckerberg: (18:34) I love Beat Saber. Deb: (18:34) So do I. And Beat Saber just passed $100 million in lifetime revenue on Quest alone. It's a great example of a game that keeps releasing fresh content. They've actually been working on evolving the way that you interact with the tracks and feel the music. Also, the team has been working on something really cool. Check this out. I can't wait to play this and they keep partnering with incredible artists to release new music packs all the time. Did you play the Billie Eilish music pack last month? Mark Zuckerberg: (19:17) A little more than I should have. I probably should have been working more on this metaverse presentation. Deb: (19:20) Well, they have a great lineup of artists for 2022 and there's one more epic surprise before the end of the year, so stay tuned. Okay. have you played Population: One? Mark Zuckerberg: (19:31) I mean, yeah. I love the game so much. Deb: (19:33) For those who haven't, Population: One is a thrilling battle royale that is only possible in VR. Since it's launched at Connect last year, it has become one of the highest earning games on Quest and the biggest multi-player FPS on the platform. You can have up to 24 people in it once for a match. We're super excited to keep launching big updates, like an all new autumn event later in November and a winter wonderland update in December. Okay. Here's something I know our community has been waiting for. Mark Zuckerberg: (20:44) Lay it on us. Deb: (20:46) All right. This is a title from the WarpFrog team. This is the team that set the standard for VR combat physics when it launched on Rift in 2019. That's right. Blade and Sorcery Nomad, the built for VR, medieval fantasy sandbox that pairs magic with melee is launching on Quest later this year- Deb: (21:03) ... that pairs magic with [Maily 00:21:02] is launching on Quest later this year. For more gaming updates, look after the 2022 Oculus gaming showcase. It's going to be loaded with news you won't want to miss. But Mark, I believe you have some news for us as well. Mark Zuckerberg: (21:49) Yeah. You know, I have to say it's really impressive to see this lineup come together over the last few years, but there's one project that I'm really looking forward to. Deb: (21:58) Yeah. This is one of the all time greats, and we've been working for years to bring it to Quest. Mark Zuckerberg: (22:03) I'm excited to announce that the Rockstar Games classic, Grand Theft Auto San Andreas is in development for Quest 2. This new version of what I think is one of the greatest games ever made will offer players an entirely new way to experience this iconic open world in virtual reality. Deb: (22:20) That's it Mark. I'm moving to the metaverse. Mark Zuckerberg: (22:23) All right. Thanks Deb. This is going to be amazing. Now, a lot of the most interesting games out there take advantage of how you can move around physically. Being able to look anywhere, move freely. It's just a fundamentally different experience from staring at a screen. This quality of being physically embodied and able to interact with the world and move around inside it. Now that opens up some completely new experiences that didn't really make sense before on 2D phones or computers. And one example that we are seeing take off is fitness. And speaking of that, I think it's time for my workout. A lot of you are already using Quest to stay fit. It lets you work out in some completely new ways. It's kind of like a Peloton, but instead of a bike, you just have your VR headset and with it, you can do anything from boxing lessons to sword fighting to even dancing. You'll be able to work out in new worlds, even against an AI. Maybe you'll get some friends together for some three on three. Maybe play pickup with people on the other side of the world. Or imagine your Facebook cycling group does an AR charity ride. Speaker 13: (24:15) Let's go. Mark Zuckerberg: (24:16) Complete with a leader board. Maybe you'll be able to train with the best, like you're right there with them. Like Lee Kiefer, Olympic gold medalist. Lee Kiefer: (24:31) On guard. Fence. Don't be scared to stab. You seem like a natural. Mark Zuckerberg: (24:39) All right. That's a little too realistic. See you later Lee. That was fun. Lee Kiefer: (24:43) Good job, Mark. Mark Zuckerberg: (24:48) And that's what fitness will be like in the metaverse. Now I think we're going to see a lot more unique experiences emerging around fitness that take advantage of the full immersion in interactive training. Speaking of which, Supernatural just added boxing to its lineup. FitXR has new fitness studios coming next year, and Player 22 by Rezzil, which is currently used by pro-athletes is adding guided and hand tracked body weight exercises soon. We're making a fitness accessories pack that makes Quest 2 more comfortable with controller grips for when things been intense and a facial interface that you can wipe the sweat off of making your sessions more comfortable. And that's all coming next year. Mark Zuckerberg: (25:28) But enough with the fun and games. It's time for everyone's favorite. Work. Over the last year and a half, a lot of us who work in offices have gone remote. And while I miss seeing the people I work with, I think remote work is here to stay for a lot of people. So we're going to need better tools to work together. Let's take a look at what working in the metaverse will be like. Imagine if you could be at the office without the commute, you would still have that sense of presence, shared physical space, those chance and interactions that make your day, all accessible from anywhere. Now, imagine that you have your perfect work setup and you can actually do more than you could in your regular work setup. And on top of all that you can keep wearing your favorite sweatpants Speaker 14: (26:14) Looking good. Let's get together real quick for a debrief. Speaker 15: (26:26) I'm free now. Let's jump in. Hi. Speaker 14: (26:29) Hey. Speaker 15: (26:29) So what do we think? Speaker 14: (26:36) I think it's ready. Speaker 15: (26:37) Great. Speaker 14: (26:39) I'll prep it for the presentation. Speaker 15: (26:41) All right. Good luck. Mark Zuckerberg: (26:45) Imagine a space where you can tune out distractions and focus on the task at hand. And when you're ready to share what you've been working on, you can present it as if you're right there with the team. Speaker 16: (27:00) Wait, where's Mark. Speaker 17: (27:02) I think he's in the middle of something. Mark Zuckerberg: (27:06) You can already see some of these elements in horizon work rooms, which we launched a couple of months ago. Later this year, we plan to introduce a room customization. So put your own logos and posters in your work rooms. We're also introducing a new office space in horizon home, for when you want your perfect workspace to do some focused work or just cross a few things off your to-do list. We're also announcing 2D, progressive web apps for the Quest Store. And as a new developer frameworks, they're easier to build. So you can drop in and check on a work project while you're in VR using services like Dropbox, Slack, or stay connected with Facebook and Instagram. This starts bringing more of your 2D internet services into the metaverse. And as we've focused more on work, and frankly, as we've heard your feedback more broadly, we're on making it so you can log into Quest with an account other than your personal Facebook account. Mark Zuckerberg: (28:02) We're starting to test support for work account soon. And we're working on making a broader shift here within the next year. I know this is a big deal for a lot of people. Not everyone wants their social media profile linked to all these other experiences. And I get that, especially as the metaverse expands. And I'll share more about that later. Mark Zuckerberg: (28:22) But I'm genuinely optimistic about work in the metaverse. We know from the last couple of years that a lot of people can effectively work from anywhere, but hybrid is going to be a lot more complex when some people are together and others are still remote. So giving everyone the tools to be present no matter where they are, whether it's a hologram sitting next to you in a physical meeting or in a discussion taking place in the metaverse, that's going to be a game changer. I think this could be very positive for our society and economy. Giving people access to jobs and more places, no matter where they live, will be a big deal for spread opportunity to more people. Dropping our daily commutes will mean less time stuck in traffic and more time doing things that matter. And it'll be good for the environment. Mark Zuckerberg: (29:09) Actually, if you travel for work and working in the metaverse means that you just take one less flight each year, that's probably better than almost anything else that you can do for the environment. I think working in the metaverse is going to feel like a huge step forward. And these dynamics, like the ability to teleport places with people and interact around shared projects and virtual space, they're going to be valuable and a lot of other categories of experiences too. So now Marne, our Chief Business Officer is going to take us through or some of them. Marne Levine: (29:42) Thanks, Mark. We'll see you in a minute. What if you could learn about anything in the world just by bringing it closer to you. Speaker 18: (29:54) So, we're going to have an astrophysicist in the family. Speaker 19: (29:58) Actually, I have to write this paper. Will you help me? Speaker 18: (30:01) Let's take a closer look. What part of the solar system are we talking about? Speaker 19: (30:07) Saturn. Marne Levine: (30:12) If you were taking astrophysics, you could study in the metaverse. Speaker 19: (30:17) Did you know the rings are made up of billions of icy particles? Speaker 18: (30:21) Really? Speaker 19: (30:22) Look at this. Speaker 18: (30:30) You ready to do that paper now, right? Speaker 19: (30:32) Yeah. Marne Levine: (30:39) In the metaverse you'll be able to teleport not just to any place, but any time as well. Ancient Rome. Imagine standing on the streets, hearing the sounds, visiting the markets, to get a sense of the rhythm of life over 2000 years ago. Imagine learning how The Forum was built by actually seeing The Forum get built right in front of you. Marne Levine: (31:09) Hi everyone. I'm Marne Levine. In the metaverse learning won't feel anything like the way we've learned before. With a headset or glasses, you'll be able to pull up schematics for your studying, or maybe even the service manual for a vehicle you're learning to repair. Let's say you're a med student or a doctor. With apps like Osso VR you can learn new techniques and surgery firsthand practicing until you get it right. Or if you're studying earth science, you could swim through the Great Barrier Reef. Get up close to Earth's mightiest insects with your instructor, David Attenborough, whose VR documentary is playing now in Oculus TV. David Attenborough: (31:53) This is a world of intrigue. A world of wonder. A secret world hidden to the human eye. Explore the extraordinary world of micro monsters 3D with me, David Attenborough. Marne Levine: (32:10) This is just one of the ways we're going to learn in the future. But in order to get there, we're going to need to help build the skill sets of the people who build these experiences. So we're setting aside $150 million to train the next generation of creators to build immersive learning content and increase access to devices. And to help more creators make a living building AR effects using Spark AR, we're going to establish a professional curriculum and certification process, make it easier to monetize, and put our Spark AR curriculum on Coursera and edX. Mark Zuckerberg: (32:49) Thanks Marne. So, that's a glimpse of the kinds of experiences that you might have in the metaverse. From connecting with friends to gaming and entertainment, to work in education, and creation and commerce. Now let's talk about how creators and developers are going to build all of this, and about the economy that we all need to create. Mark Zuckerberg: (33:11) The last few years have been humbling for me in our company in a lot of ways. One of the main lessons that I've learned is that building products isn't enough. We also need to help build ecosystems so that millions of people can have a stake in the future, can be rewarded for their work and benefit as the tide rises, not just as consumers, but as creators and developers. But this period has also been humbling because as big of a company as we are, we've also learned what it is like to build for other platforms, and living under their rules has profoundly shaped my views on the tech industry. Mark Zuckerberg: (33:51) Most of all, I've come to believe that the lack of choice and high fees are stifling innovation, stopping people from building new things and holding back the entire internet economy. We've tried to take a different approach. We want to serve as many people as possible, which means working to make our services cost less. Not more. Our mobile apps are free. Our ads business models, and auction, which guarantees every business the most competitive price possible. We offer our creator and commerce tools either at cost or with modest fees to enable as much creation and commerce as possible. And it's worked. Billions of people love our products. We have hundreds of millions of businesses on our platform, and we have a rapidly growing ecosystem and a thriving business. That's the approach that we want to take to help build the metaverse too. We plan to continue to either subsidize our devices or sell them at cost to make them available to more people. Mark Zuckerberg: (34:54) We'll new supporting side loading and linking to PCs. So consumers and developers have choice rather than forcing them to use the Quest Store to find apps or reach customers. And we'll aim to offer developer and creator services with low fees in as many cases as possible. So we can maximize the overall creator economy, while recognizing that to keep investing in this future we'll need to keep some fees higher for some period to make sure that we don't lose too much money on this program overall. After all, while a growing number of developers are already profitable, we expect to invest many billions of dollars for years to come before the metaverse reaches scale. Our hope though is that if we all work at it and within the next decade, the metaverse will reach a billion people, host hundreds of billions of dollars of digital commerce and support jobs for millions of creators and developers. Mark Zuckerberg: (35:52) We'll try to lay out how some of this commerce model will work, but I also want to be upfront about the fact that there's a ton that we just don't know yet. What I can tell you though, is that we are fully committed to this. It is the next chapter of our work, and we believe for the internet overall. And our strategy and track record show that we will do everything we believe is sustainable to grow the community, the creator economy, and the developer ecosystem. And to tell us more about all of that, here's, Vishal, our Head of Metaverse Products. Vishal Shah: (36:26) Thanks Mark. As you said, our approach is to serve as many people, creators and businesses as we can by keeping fees as low as possible and offering choice. As a company, we have been investing in commerce and creators for many years, but let me take a moment and explain how this will all come to life in metaverse. For people, the metaverse will offer more choice than we've ever seen before. And most importantly, there'll be a real sense of continuity where the things you buy are always available to you. Today, much of what you buy on the internet is inside a single app, website, or game. You might buy a custom skin for your gaming avatar, but you can't take it with you when you move to a new space. In the physical world, that would be the equivalent of buying your favorite sports team's Jersey, and never be able to wear it outside the stadium. Vishal Shah: (37:12) In the future, you'll be able to buy digital clothes for your avatar and then wear them into the metaverse more broadly. For creators, our goal is to provide a way for as many creators as possible to build a business in the metaverse. One thing we've learned from today's digital platforms is that we can't artificially limit innovation. We need to enable as many different types of creators as possible to unlock the best ideas. There will be many different kinds of creators in the metaverse. Creators who make digital objects, creators who offer services and experiences, and those who build entire worlds like game creators do today. We also want creators to have the biggest possible audiences. If a creator designs, a signature effect to surround avatar, their fans should be able to buy it and then visit different spaces to show it off to their friends. Frankly, if a creator builds any experience, all of their fans should be able to enjoy it. Vishal Shah: (38:03) Businesses will be creators too, building out digital spaces or even digital worlds. They'll sell both physical and digital goods as well as experiences and services. And they'll be able to ads to ensure the right customers find what they've created. Speaking of buying things, we're also exploring new types of ownership, models, and entitlements to ensure people feel confident that they actually own something. This will make it easier for people to sell limited edition digital objects like NFTs, display them in their digital spaces and even resell them to the next person securely. Vishal Shah: (38:35) In short, the metaverse will remove many of the physical constraints we see on commerce today and make entirely new businesses possible. While this is very exciting, the metaverse is all about co-creating. We're building this together with creators, developers and entrepreneurs. So today I've asked Jackie Aina to join me, to start a conversation about what might be possible. Jackie is a beauty creator who launched her own lifestyle brand on Instagram called FORVR Mood, where she sells candles with scents like cuffing season and partners with major brands on makeup drops. So let's meet Jackie in her space. Here we are in this digital space, totally inspired by you and all the things that you love. Jackie, I've been such a big fan of yours for such a long time. Tell me a bit about how you got started in social media. Jackie Aina: (39:24) So rewind back to 2009, as a person who likes beauty, I would go to like a makeup counter or go makeup shopping. And I always felt rejected, always felt othered. So I got so frustrated that eventually I was like, you know what? Not only am I going to not listen to them, but I'm going to show other people how I'm making it work with my skin tone too. Vishal Shah: (39:44) How did you go from creating content, creating videos to a product? You just have a brand called FORVR Mood, it's huge on Instagram, Instagram shopping. Jackie Aina: (39:54) So, FORVR Mood was not only a bit of a love letter to me and my relationship with my mom. That's how we bond, was through fragrance ever since childhood, but it's been doing amazing and we've actually been getting a lot of support. Vishal Shah: (40:08) Jackie, the space is pretty magical, and I know it kind of reflects the stuff you love. Like tell me a bit about what you love and why this represents you. Jackie Aina: (40:17) Okay. So first thing, color is everything. I love pastels. You can see it in the brand. You can see it like a lot of times in what I wear. With the metaverse when we're curating collections or even just brainstorming, this would be a great way to gather us all together and literally kind of create atmospheres and environments and moods to just kind of enhance creativity. And then if I can invite people, it would be like inviting them in my home. You know what I mean? It'd be like bringing them into a piece of me. So I would want them to feel some of the things that I feel when I step into my house or when I light my candles. Vishal Shah: (40:49) Interacting with people in comments and having a conversation is a way to get your fan excited about interacting with you, but also you feel like you get a connection [crosstalk 00:40:59]. Jackie Aina: (40:57) I get excited too. Yeah. Vishal Shah: (40:59) Yeah. And so this idea of reaching into the comments section and pulling a fan in to having a conversation- Jackie Aina: (41:06) I love that. Vishal Shah: (41:07) It's pretty powerful, and it humanizes that person on the other end of that comment. And I think there's a really interesting opportunity to think about how we build community and how we engage with our fans in a really deep way, as opposed to just through a comment box. Imagine for your biggest fans, you could have an exclusive launch party where anyone could visit no matter where they were in the world. Jackie Aina: (41:33) I love that, because when we first launched the brand, everything was shut down, because obviously quarantine. Couldn't go anywhere. And I had this idea of doing popups for FORVR Mood, where we would have a room inspired by each scent. Vishal Shah: (41:49) And now anyone from anywhere in the world could come and feel what you just described. Have that feeling, have that sense, really get into that mood. Also, you have your greatest fans, the people that care a lot about you, the ability to reach new customers... Speaker 20: (42:03) ... you have your greatest fans, the people that care a lot about you, the ability to reach new customers, use advertising to keep growing your business. And you could also drop an exclusive product in the metaverse, where only available to your most ardent fans who pay the special access to get that product. Jackie Aina: (42:17) That's dope. Speaker 20: (42:19) Commerce is going to be a big part of the metaverse. You'll be able to sell both digital and physical products. The butterfly effect transports us someplace magical. Jackie Aina: (42:30) So beautiful. Speaker 20: (42:41) So, Jackie, as we walk through this amazing world, what does the metaverse mean to you? Jackie Aina: (42:45) I just feel like this is endless possibilities of my imagination. Speaker 20: (42:50) I can't even begin to imagine how meaningful the metaverse will be, thanks to creators like you. Mark Zuckerberg: (42:56) There's a lot here to be excited about. Just like the internet today, more people are going to have the freedom to find a business model that works for them, whether that's custom work, tipping, subscriptions, ads, or other monetization tools that may only make sense in the metaverse. Mark Zuckerberg: (43:19) Now, think about how many people make a living on the internet today. How many of those jobs just didn't even exist a few years ago? I expect that the metaverse is going to open up lots of opportunities for people in the exact same way. Mark Zuckerberg: (43:32) Now, we have confidence in this general direction and we're investing significantly in building for this future, but the reality is that no one knows exactly which models are going to work and make this sustainable. So, we're going to approach this with humility and openness. And we're going to work with anyone whose efforts will help bring the metaverse to life. Mark Zuckerberg: (43:54) So, now, Boz is going to join us to talk about what people are already building for the metaverse and what kinds of tools we can create to help out. Hey, Boz, I'm still kind of jealous of your robot avatar. Boz: (44:05) Hey, man, you got to have the robot when you're working your dance moves, right? Mark Zuckerberg: (44:08) I mean, of course. Boz: (44:10) Look, Mark. We've been working on Facebook Reality Labs for a long time, but we always kept the same vision, which is that we want to build tools that help people feel connected anytime, anywhere. And that vision's what drives us today to help people get together in virtual spaces. Boz: (44:24) We've come a long way since the early days of VR, thanks to the developers and the creators who see the potential to entertain and to educate, but, of course, we still have a lot farther to go. So, to keep the pace of innovation moving faster and empower more use cases, we do have a few exciting announcements. Mark Zuckerberg: (44:39) Today, we're introducing the Presence Platform, which is a broad range of machine perception and AI capabilities that empower developers to build mixed reality experiences on Quest 2. You want to tell us more about the Presence Platform? Boz: (44:51) We've said before that realistic presence is the key to feeling connected in the metaverse. And the Presence Platform's capabilities are what's going to deliver on that promise, things like environmental understanding, content placement and persistence, voice interaction, standardized hand interactions. Boz: (45:06) In fact, let's start with hands. I mean, the human hand is an engineering marvel. And bringing hands into VR was no easy feat. It required a lot of collaboration against product, design, research, but we continue to improve that product, finding new ways to navigate with gestures and interact with VR. So, today, we're introducing the Interaction SDK, a library of modular components that will make it easy to add hand interactions to your apps. Mark Zuckerberg: (45:31) That's pretty exciting. Boz: (45:32) Yeah. We really think that this is going to help people accelerate the build time for new and existing titles that allow people to use their hands more naturally in more virtual experiences. Boz: (45:42) Now, in addition to hands, there's another exciting interaction space for VR. That's your voice. The Voice SDK lets developers integrate voice input into their games and apps so that they can create new gameplay and navigation. So, soon, you'll be able to jump into a FitXR workout and shake up your routine just by saying, "Surprise me." Oh yeah. I've tried this one before. This one's really tough. Mark Zuckerberg: (46:03) Yeah. Me too. It is. Boz: (46:04) Since we launched the Passthrough API, we've already seen breakthrough experiments from developers that blend the virtual and the real world. Soon, with our next SDK release, developers will be able to ship their mixed reality apps on the Oculus store and in App Lab. Boz: (46:18) Now, of course, access to Passthrough by itself is not enough. To achieve that rich mixed reality experience, apps also need to be aware of things in the room and blend the virtual objects with the physical environment around them so they can co-exist in the same space. So, developers want to be able to place persistent world-locked content, like animated holograms or your Instagram feed in your real space. And tools like spatial anchors and scene understanding capabilities will help make these mixed reality experiences feel seamless. Boz: (46:46) Now, I'd gladly nerd out with you on the technical details of this with you. Mark Zuckerberg: (46:49) I know. Boz: (46:50) But I think it's better to show you. Load the world beyond. This is Oppy. Oppy, come. Come on, Oppy. Who's a good persistent state virtual object layered on an interactive Passthrough environment? Oh, you are. Yes, you are. Oppy, sit. Pets in the metaverse, man. Just as stubborn as they are in the physical world. Mark Zuckerberg: (47:15) You know, Boz? I always really wanted a forest in my living room. Boz: (47:18) Did you? Mark Zuckerberg: (47:20) Not really. I'm looking forward to how developers are going to be able to build a new generation of mixed reality experiences on Quest 2. And it also gives us a glimpse of the kind of worlds that people are going to be able to build. Boz: (47:33) Now, let's talk about augmented reality. With Spark, we really focused on those use cases that allow people to both stay engaged with the world around them, but also stay connected to the people that they're not around. And creators are really leading the way here. And we're doing everything we can to support them with our know-how, with our resources, and, of course, with tools. Boz: (47:49) And for more on that, I'd like to say hi to Sue, the product director at Facebook Reality Labs and the head of Spark AR. Marne Levine: (47:56) Thanks guys. Spark AR is focused on building AR experiences that empower people to be more connected and at the same time more present with the world around them. And creators. Creators are helping us build these experiences of today while exploring the possibilities of tomorrow. Fundamentally, we're democratizing AR creation and enabling a global community with the tools and knowledge necessary to develop the AR content and experiences that people love to use. Marne Levine: (48:27) Programs like AR curriculum enrolled over 22, 000 creators in less than a year. And we are adding additional programming to support more advanced creators. And the depth and breadth of content would not be possible without these creators. Marne Levine: (48:42) In one week alone, you'll see effects that bring moments like landing on Mars or the launch of a new album to life all by tapping into everyone's imaginations. We're also powering experiences across Facebook. So, imagine playing that AR battleship game from your phone with a friend in the metaverse or on augmented call. How cool, right? Marne Levine: (49:05) Right now, we have over 600,000 creators in our community with diverse needs and motivations for using AR to engage with their fans and audiences. We know people, more than 700 million monthly, across the Facebook family of apps are already using AR a ton. We see over 80 billion effects applied per month. And we want to make it even easier for anyone to find their spark. Marne Levine: (49:31) So, we created a tool, codenamed Polar, that makes AR creation possible for novice creators who have no prior experience in art, 2D or 3D design, or programming. Think paint by numbers. So, creators whose first love might be photography, video, or dance can enhance their content for their own special effects. And that's an important piece of this vision and this community. It's open to anyone and everyone who's curious. Marne Levine: (50:02) And the metaverse is well-positioned to be a strong digital economy for creators from all walks of life. This is primitive to a functioning metaverse. And we want to make sure creators are ready to share their creativity and capitalize on this emerging opportunity from day one. Mark Zuckerberg: (50:22) Thanks, Sue. So, that's some of the software platform that needs to get built to deliver the metaverse, but there are other important considerations here too. Mark Zuckerberg: (50:33) Hey, Nick. Nick: (50:34) Hey, Mark. I hope I'm not interrupting. You got a sec? Have you got Oppy with you? Mark Zuckerberg: (50:39) I think Oppy's still in the virtual forest, but I always have time for you. What's going on? Nick: (50:43) Look, I just love the presentation so far. It's such visionary stuff. But as you mentioned early on, with all big technological advances, there are inevitably going to be all sorts of challenges and uncertainties. And I know you've talked about this a bit already, but people want to know how we're going to do all this in a responsible way, and especially that we play our part in helping to keep people safe and protect their privacy online. Mark Zuckerberg: (51:08) Yeah, that's right. This is incredibly important. Nick: (51:10) The way I look at it is that in the past, the speed that new technologies emerged sometimes left policy makers and regulators playing catch-up. So, on the one hand, companies get accused of charging ahead too quickly. And on the other, tech people feel that progress can't afford to wait for the slower pace of regulation. And I really think that it doesn't have to be the case this time around because we have years until the metaverse we envision is fully realized. So, this is the start of the journey, not the end. Mark Zuckerberg: (51:41) Like I said earlier, interoperability, open standards, privacy, and safety need to be built into the metaverse from day one. And with all the novel technologies that are being developed, everyone who's building for the metaverse should be focused on building responsibly from the beginning. Mark Zuckerberg: (51:57) This is one of the lessons that I've internalized from the last five years. It's that you really want to emphasize these principles from the start. So, at Connect last year, Boz outlined our responsible innovation principles. And the first one was never surprise people. Nick: (52:12) Right. And that means being transparent about how things work, what data is collected, and how that data is used over time. It also means giving people easy-to use safety controls, as well as age guidance, and parental controls for when youngsters are using these products. Mark Zuckerberg: (52:29) And we're spending a lot of time talking with experts and getting perspectives from outside the company on what we're building, even before we build it. And this is about designing for safety and privacy and inclusion before the products even exist. And one example is what we're doing with Project Aria, our research device that helps inform the AR glasses that we're building. Mark Zuckerberg: (52:50) We're also funding external research on this. Last year, we announced grants for research on the impact of AR, VR, and smart devices on people who aren't currently using them, especially communities whose perspectives have often been overlooked, as well as best practices for creating inclusive environments in virtual spaces. And this year, we're opening up support for even more research because we need those independent perspectives to make sure that we're living up to another one of our principles: consider everyone. Nick: (53:20) And that point is so important because this is a collaborative exercise. And in particular, we need to make sure that human rights and civil rights communities are involved so these technologies are built in a way that's inclusive and empowering. Mark Zuckerberg: (53:34) One of the advantages of starting right now is that we can collaborate with people at the very early stages of development, like what we're doing through the new fund that you announced a few weeks ago. Nick: (53:44) Yeah. The XR Programs and Research Fund. It's a two-year, 50-million-dollar investment in programs and external research with organizations all over the world, like Howard University, Women in Immersive Tech, Africa No Filter, and universities in Seoul, Hong Kong, and Singapore. And this is crucial because, as we've said, the metaverse isn't something we're building so much as it's something we are building for. Mark Zuckerberg: (54:11) Across the industry, we need to bring that same imagination and commitment to building for interoperability, openness, safety, and privacy, as we do for all the other product aspects of the metaverse. These have to be fundamental building blocks, just like the other software and experiences that we've been talking about. Nick: (54:29) Right. Mark Zuckerberg: (54:30) All right. Thanks, Nick. Mark Zuckerberg: (54:32) So, one thing that we haven't talked about much yet is the future of hardware that will help bring the metaverse to life. We're working on multiple new products to advance the state of the art, unlock richer social interactions, and make it a lot easier to be productive. Mark Zuckerberg: (54:48) That must be Angela, our head of VR devices. Hey, Angela. Angela: (54:53) Hey, Mark. Mark Zuckerberg: (54:54) Is that what I think it is? Angela: (54:55) I sure hope it is. Mark Zuckerberg: (54:57) All right. Come on in. So, what do you think is the most exciting thing that we're working on right now? Angela: (55:06) That's hard to say. We have such an exciting pipeline, but next year we are releasing a new product that will push the boundaries of VR even further. We've codenamed it Project Cambria. Mark Zuckerberg: (55:18) So, this isn't the next Quest. It's going to be compatible with Quest, but Cambria will be a completely new, advanced, and high-end product and it'll be at the higher end of the price spectrum too. Our plan here is to keep building out this product line to release our most advanced technology before we can hit the price points that we target with Quest. Mark Zuckerberg: (55:37) All right. So, let's talk about some of the new advances here. Angela: (55:40) Yeah, sure. There's a ton of new tech going into Cambria. For example, your avatar will be able to make natural eye contact and reflect your facial expressions in real time. This way, people you are interacting with will have a real sense of how you're actually feeling. It does mean building more sensors into a form factor that's comfortable to wear for a while. And because we want VR to be for everyone, we also have to make sure avatars represent a diverse set of human facial features and skin tones, as well as paying attention to things like glasses and beards that might get in the way of some of the sensors. Mark Zuckerberg: (56:15) So, that's going to be a big step forward for social presence. And I'm really glad that we're focused on making it inclusive from the start. Mark Zuckerberg: (56:22) Now, what about unlocking more mixed reality experiences? I mean, imagine working at your virtual desk with multiple screens while seeing your real desk so clearly that you can pick up a pen and write notes without taking your headset off or you're doing a workout with a virtual instructor in your living room. Angela: (56:40) It's going to be so cool. We're already seeing the potentials of these kinds of experiences today, as people are building for our Passthrough API. But with Cambria, we'll be taking this to the next level with high-resolution, colored, mixed reality Passthrough. We essentially combined an array of sensors with reconstruction algorithms to represent your physical world in the headset with a sense of depth and perspective. Now, we're still a ways away from exactly matching what our eyes see in the physical world, but we're pretty encouraged by how far we've been able to advance the Passthrough experience so far. Mark Zuckerberg: (57:14) Definitely. But we also need to push the visuals to the next level. So, let's talk about the progress that we're making on optics. Angela: (57:20) Yeah. We're pushing the limits of display technology and form factor with something called pancake optics. They essentially work by folding light several times over to achieve a slimmer profile than current lenses. Angela: (57:33) Now, with several optical layers, we'll need to precisely control every aspect of design and fabrication to achieve that high-quality, artifact-free display, and really just deliver the best optics ever in one of our headsets. Mark Zuckerberg: (57:47) That's pretty awesome, but let's make sure that we leave some of the good stuff for next year's release too. Mark Zuckerberg: (57:53) All right. Now, I'm excited to keep building out this new product line over future generations so we can keep getting our most advanced technology into people's hands even before we can get it into our Quest product line. Angela: (58:05) That's our goal. We're starting to work with developers to build experiences for Cambria as we speak. And we're looking forward to sharing so much more with y'all next year. Mark Zuckerberg: (58:14) Sounds good. Now, beyond virtual reality, we are also focused on the hardware to make true augmented reality possible. In a lot of ways, augmented reality is even harder, not just because we need to invent a completely new optical stack that's not based on screens, but because we basically need to fit a super computer into a pair of normal-looking glasses. Mark Zuckerberg: (58:37) So, we are approaching this problem from two directions. First, how much technology can we pack into a pair of normal, great-looking glasses today? And second, how do we take the long-term tech stack to do everything and keep miniaturizing and improving it until it fits into a pair of normal, good-looking glasses. Mark Zuckerberg: (58:57) On the first path, last month, we launched Ray- Ban stories, our first smart glasses in partnership with EssilorLuxottica. They're not full AR glasses yet, but they let you take pictures and videos, listen to music, and take phone calls while you're out looking at the world instead of down at your phone. And we built leading privacy features into the glasses, like the LED light whenever you're recording, which phones don't even have. And we delivered this in the iconic Ray-Ban style for just $299. These are all steps along the path to an embodied internet. Mark Zuckerberg: (59:31) But the ultimate goal here is true augmented reality glasses. And we've been working on that too. And today, I want to show you an experience that we've been working on for Project Nazare, which is the code name for our first full augmented reality glasses. Here, you'll see you're chatting with friends on WhatsApp and planning a game night. You can select a game. And then as you walk over to your kitchen, you can easily just put your game onto the table and you're off. And that's the kind of experience that augmented reality will unlock. Mark Zuckerberg: (01:00:02) There's a lot of technical work to get this form factor and experience right. We have to fit hologram displays, projectors, batteries, radios, custom silicon chips, cameras, speakers, sensors to map the world around you and more into glasses that are about five millimeters thick. So, we still have a ways to go with Nazare, but we are making good progress. Mark Zuckerberg: (01:00:24) I'm excited about our future roadmap, but even that's still early in a journey that's going to go on for decades. Immersive all-day experiences will require a lot of novel technologies. And for the last seven years, our research team has been working on a broad array of technologies that are necessary for these next-generation platforms. Michael Abrash leads this team, and he's going to join me to talk about some of the future technology that we're developing. Mark Zuckerberg: (01:00:53) Hey, Michael, do you want to take everyone through our roadmap? Michael: (01:00:56) Sure. That covers a lot of ground though. It's going to take about a dozen major technological breakthroughs to get to the next-generation metaverse and we're working on all of them: displays, audio, input, haptics, hand tracking, eye tracking, mixed reality sensors, graphics, computer vision, avatars, perceptual science, AI, and more. Mark Zuckerberg: (01:01:19) Yeah, you're right. That's a lot. We probably only have time for a couple right now. You want to pick a few? Michael: (01:01:24) Well, I think the metaverse is really going to be first and foremost about connecting people. So, I'd say let's start with avatars. Mark Zuckerberg: (01:01:31) Yeah, I agree. The goal here is to have both realistic and stylized avatars that create a deep feeling that we're present with people. Michael: (01:01:39) Exactly. And we've shown Codec avatars before. Speaker 21: (01:01:43) Well, what can your face do? Can you show us? Speaker 22: (01:01:46) Well, I've always hoped you would ask me that question. Michael: (01:01:49) And it's a remarkable experience to see Codec avatars in VR. And photorealistic avatars will be a huge breakthrough, but they're only part of the picture. Mark Zuckerberg: (01:01:59) Yeah. Because you're not always going to want to look exactly like yourself. That's why people shave their beards, dress up, style their hair, put on makeup, or get tattoos. And, of course, you'll be able to do all of that and more in the metaverse. Michael: (01:02:11) So, this next video shows how we're making progress on part of that with hair and skin rendering and relighting for 3D avatars. Speaker 23: (01:02:22) She's going for the medical field. I think she was interested to... Michael: (01:02:26) Now, I want to point out this technology is still very definitely research, but it shows the really high level of fidelity that avatars are ultimately going to need to reach. Notice that you can see the individual pores on his skin. Michael: (01:02:40) Now, clothing is another important way to express yourself. And whether you want to simulate your actual garments, say you're making a quick call with your grandmother, or maybe you just want to be wearing a stylish blue Oxford shirt like me, you're going to need the ability to simulate clothing. So, here's an early look at hand-cloth interaction. And you can see how the movement of the cloth when the hand touches and stretches it is really... Michael: (01:03:03) How the movement of the cloth when the hand touches and stretches it is really remarkably accurate. Mark Zuckerberg: (01:03:05) Now, to use your avatar to teleport to a meeting or sit down for a chat in the metaverse, you'll also need realistic virtual spaces to be with people in. So let's look at that. Contrary to how this looks, this is actually not a video of the physical world. It's a 3D reconstruction of a mock apartment, which I guess just became clear when that bowl took off by itself. Michael: (01:03:26) Exactly. So what you can see here is that a researcher is moving around various objects on the right, and then on the left, you can see the high fidelity real time rendering of the space and the moving objects on the left without the researcher. So what's critical here is that this is all happening in real time. That's what's novel here, and that's what differentiates it from CGI. So what we've seen so far is pretty awesome, but we all know what we want. We want the whole package, a full body avatar in a virtual environment, so you're just there. Last year, we showed our very first full body Codec Avatar at Connect. Here to tell you more about our progress since then is Yaser Sheikh. Yaser Sheikh: (01:04:06) Hi there. My name is Yaser, and I direct the Facebook Reality Research Lab in Pittsburgh. Well, that's not completely true. What you're seeing is actually a rendering of my prerecorded Codec Avatar. Over the past couple of years, we've made quite a bit of progress with Codec Avatars. My avatar can now look left, look right, look up, look down, make facial expressions... Michael: (01:04:25) Here, you can see Elaine is in VR on the right. You can see her view of Yaser's avatar and the reconstructed real world space that it's in on the left as she moves around the avatar. What you see on the left is an entirely virtual reconstruction. This gives you a sense of a future in which you'll be able to interact with another person's realistic avatar in virtual reality in real time. Now, preventing others from using your avatar will be critical. That's why even though this is early stage research, we're already thinking about how we could secure your avatar, whether by tying it to an authenticated account or by verifying identity in some other way. Mark Zuckerberg: (01:05:02) All right, so we've talked about what we're going to be able to see, but of course, we also want to be able to actually do things in the metaverse. And while seeing your hands and VR and manipulating virtual objects is a big step forward towards that, we're going to need an even easier and more intuitive way to interact with virtual content when you're on the go wearing AR glasses. And we believe that neural interfaces are going to be an important part of how we interact with AR glasses, and more specifically EMG input from the muscles on your wrist combined with contextualized AI. It turns out that we all have unused neuromotor pathways, and with simple and perhaps even imperceptible gestures, sensors will one day be able to translate those neuromotor signals into digital commands that enable you to control your devices. It's pretty wild. So let's take a look at what EMG input is going to be able to do and where we are with it today. Michael: (01:05:58) This video shows some of the ways that we think that wrist-based neural interfaces can provide an important input tool for AR glasses. So at first, that input is going to just be around enabling basic gestures; click, scroll, select. But as the technology evolves, EMG input could potentially unlock full speed typing and it could give you subtle personalized controls that you can use in any situation. This is genuinely unprecedented technology. So let's look at an early prototype that shows how it could be used in AR. Michael: (01:06:31) This experimental UI shows for the first time how EMG input could one day allow you to send a message in AR with your hand resting comfortably at your side, without ever having to look at a screen. Notice how the researcher used an overt and highly visible click gesture to send the first message. We've made a lot of progress over the last year, and we're now able to do the same thing with the smallest of movements of the hand. EMG enables this by picking up subtle neuromotor commands with remarkable precision. Mark Zuckerberg: (01:07:01) So basically what you're saying is that you're going to be able to send a text message just by thinking about moving your fingers. That's going to be pretty amazing, but I guess it's just one part of the equation, because the other is AI that understands your context and can give you a simple set of choices based on that context. So let's take a look at an AI interaction model that we've built using Project Aria, our initiative to accelerate research into AR glasses and live maps that we talked about last year. Michael: (01:07:29) Remember the mock apartment we saw earlier? Well, we've indexed every single object in it, including not only location, but also the texture, geometry, and function of each one. We fed all that information into the Project Aria glasses Ming Fay is wearing in this next demo. And the reason is to simulate how AR glasses will ultimately be able to access data from a 3D map that will help them identify the real world objects and the space around you and better understand your context so they can help you. Michael: (01:07:58) What's happening here is that the glasses are calculating where Ming Fay is in the room, locating and identifying all the nearby objects based on the index mentioned earlier, and figuring out which of those objects Ming Fay is interested in by observing her eye movements. That contextual information in turn enables the system to offer proactive assistance to Ming Fay in a variety of ways. Here, the system knows that she may want to turn on the TV, so when she clicks while looking at it, the system turns it on. Ultimately, her AR glasses will tell her what her available actions are at any time. Ming Fay: (01:08:38) Hey, assistant. Where is my favorite mug? Michael: (01:08:41) We're also working on a voice interface that will let you locate any object. Here, the system is displaying the current location of the mug. In the future, it might tell her where it is using speech or text, or it might highlight the location in her glasses. Michael: (01:08:56) Really, it's an early look at the sort of intuitive, low friction interface that AI plus EMG input will enable in the future. This is how we'll all be interacting one day. Mark Zuckerberg: (01:09:05) Thanks, Michael. That was awesome. All right, so I hope that that gives you a sense of some of the technical challenges that we're working on to deliver deep and immersive experiences in the metaverse. A lot of what we've shown today isn't going to be available in the next year or two. Some of this is still a long way off. But this is some of the most exciting stuff that I've ever gotten to work on, and I'm just incredibly energized to be on this journey with all of you. And there's one more thing that I want to tell you about today. Mark Zuckerberg: (01:09:41) I believe the metaverse is the next chapter for the internet, and it's the next chapter for our company too. So I've been thinking a lot about what this means for our company and who we are as we embark on this journey. We're a company that focuses on connecting people. While most other tech companies focus on how people interact with technology, we focus on building technology so people can interact with each other. One of the reasons I started Facebook was that at the time you could use the internet to find almost anything; information, news, movies, music, shopping; except for the thing that matters most of all, people. Today, we are seen as a social media company, but in our DNA, we are a company that builds technology to connect people. And the metaverse is the next frontier, just like social networking was when we got started. Mark Zuckerberg: (01:10:34) Facebook was born in a specific time and place, a college campus, the web. It was what we could build at the time to put people back into our experience of technology. But connecting people was always much bigger. From way earlier on, I remember sitting in middle school classes, sketching in my notebooks ideas that I wanted to code when I got home. And even though I didn't have the skill or technology to build it yet, it was always clear that the dream was to feel present with the people we care about. Isn't that the ultimate promise of technology, to be together with anyone, to be able to teleport anywhere and to create and experience anything? Mark Zuckerberg: (01:11:15) Yet here we are in 2021 and our devices are still designed around apps, not people. The experiences we're allowed to build and use are more tightly controlled than ever. And high taxes on creative new ideas are stifling. This is not the way that we are meant to use technology. The metaverse gives us an opportunity to change that, if we build it well, but it's going to take all of us; creators, developers, companies of all sizes. Together, we can finally put people at the center of our technology and deliver an experience where we are present with each other. Together, we can create a more open platform with more ways to discover experiences and more interoperability between them. And together, we can unlock a massively bigger creative economy. Mark Zuckerberg: (01:12:09) I know the internet story isn't straightforward. Every chapter means new voices and new ideas, and yes, there will be challenges and risks and disruption of established interests, but there will also be opportunities and benefits that we can't even imagine yet; for connection, for creation, for learning and joy. We'll all need to work together from the beginning to bring the best possible version of this future to life, a future where with just a pair of glasses you'll be able to step beyond the physical world and into the kinds of experiences that we have talked about today. Mark Zuckerberg: (01:12:49) I've been thinking a lot about our identity as we begin this next chapter. Facebook is one of the most used products in the history of the world. It is an iconic social media brand, but increasingly it just doesn't encompass everything that we do. Instagram, WhatsApp, Messenger, Quest, now Horizon, Nazare and more; building our social media apps will always be an important focus for us, but right now our brand is so tightly linked to one product that it can't possibly represent everything that we're doing today, let alone in the future. Over time, I hope that we are seen as a metaverse company, and I want to anchor our work and our identity on what we are building towards. Mark Zuckerberg: (01:13:39) We just announced that we are making a fundamental change to our company. We are now looking at and reporting on our business as two different segments, one for our family of apps and one for our work on future platforms. And as part of this, it is time for us to adopt a new company brand to encompass everything that we do, to reflect who we are and what we hope to build. I am proud to announce that starting today, our company is now Meta. Our mission remains the same, still about bringing people together. Our apps and their brands, they're not changing either. And we are still the company that designs technology around people. But now we have a new north star to help bring the metaverse to life, and we have a new name that reflects the full breadth of what we do and the future that we want to help build. Mark Zuckerberg: (01:14:38) From now on, we're going to be metaverse first, not Facebook first. That means that over time, you won't need to use Facebook to use our other services. And as our new brand starts showing up in our products, I hope that people come to know the Meta brand and the future that we stand for. I used to love studying classics, and the word meta comes from the Greek word meaning "beyond". For me, it symbolizes that there is always more to build. There's always a next chapter to the story. And for us, that's a story that started in a dorm room and grew beyond anything we could imagine, into a family of apps that people use to find one another, to find their voice, to start businesses and communities and movements that have changed the world. Mark Zuckerberg: (01:15:27) I'm proud of what we've built so far and excited about what comes next as we move beyond what's possible today, beyond the constraints of screens, beyond the limits of distance and physics, and towards a future where everyone can be present with each other, create new opportunities, and experience new things. It's a future that is beyond any one company, that will be made by all of us. We've built things that have brought people together in new ways. We've learned a lot from struggling with social issues and living under closed platforms. And now it is time to take everything that we have learned and help build the next chapter. I am dedicating our energy to this more than any other company in the world. And if this is the future that you want to see, then I hope that you will join us, because the future is going to be beyond anything we can imagine.