[0:00:03] This Week in Startups is brought to you by Squarespace. Start your free trial site today. When you decide to sign up for Squarespace, use offer code TWIST to get 10% off your first purchase. Four Sigmatic. Awaken your mind and support your well-being. Special offer for TWIST, 15% off foursigmatic.com slash TWIST or use discount code TWIST at checkout. And LinkedIn.
[0:00:31] LinkedIn has marketing tools to help you target your customers with precision. To redeem a $100 LinkedIn ad credit and launch your first campaign, go to linkedin.com slash thisweekinstartups. Hey, everybody. Hey, everybody. Welcome to another episode of This Week in Startups. We're live at Wilson Sonsini with a great live audience. And thanks to our friends at Wilson Sonsini for hosting.
[0:00:58] Perhaps the most discussed topic of 2018 has been artificial intelligence. And my good friends Sam Altman and Elon Musk were very concerned about the direction in which AI was going and who would have access to it. So they invested tens of millions of dollars into OpenAI. And the CTO and co-founder, Craig Brockman, is going to join me today. He's the person who does all the work. I think Sam and Elon take the credit.
[0:01:26] Sam does quite a lot of work. Does Sam work there full time? Two days a week? He's around a lot these days. You don't want to pin him down to a certain number of hours? No, no, no. I mean, I think Sam somehow finds more hours in a week than I think most humans have. He's a hard working, he's a worker bee. He definitely is. So to set the stage, tell us what is OpenAI? What's its mission?
[0:01:54] And then how did you find your way there? So the mission for OpenAI is to ensure that artificial general intelligence, and by that we're talking about systems that are as smart as we are, that can outperform humans at most economically valuable work. Whenever we can build that, that it's something that actually benefits all of humanity.
[0:02:13] So I think when you look at AI, we really have this continuous spectrum of technologies. Traditionally, AI is the field that kind of over-promised, never really delivered. We've had AI systems doing work for us for decades. Check processing is something that there's kind of an early triumph that was sort of moved into the realm of artificial neural networks in the 90s. But really, starting 2012 is when something changed.
[0:02:37] And for the first time, you were able to take deep neural networks, so these systems that can just learn from data, and use them to solve problems way better than anything else. So you can solve image recognition. You throw out all the old systems to replace it with a deep neural networks better. You take the exact same algorithm, use it in machine translation. Suddenly, Google Translate.
[0:02:56] It works extremely well. And we just see this across this huge variety of different domains. And so at first, people were like, oh, OK. So now we can do perception. We can do these very basic tasks. And then machine translation, that feels a little bit harder than normal perception. But then you start to move to things like solving games like Go or Dota, these competitive esports games. And suddenly, you're able to have neural nets, these AIs that are able to solve interactive problems much better than humans can. And so looking forward, I think that
[0:03:25] When you just project for the progress, and I'll talk a lot today about just the kind of progress you've seen and what really drives it, that I think that you really can't rule out being able to build the kind of systems that we're talking about in the near term. And let's talk about the corporate structure since it's a very entrepreneurial show.
[0:03:41] OpenAI is a nonprofit, a for-profit? How many employees are there? Yeah, so we're about 80 people full-time. We've been around for three years. We're structured as a nonprofit. Why? And the reason for that is that our goal at the end of the day is not to generate huge profits for us, right?
[0:03:59] The goal here is to build a system that we think that the impact, if you actually succeed, is going to be in the order, at a minimum, of the agricultural revolution. And on a maximum, you're talking about going to the stars and solving all disease and sort of all the things that humans have wanted to do but seem kind of out of reach of us being able to accomplish. And so when you talk about something like that, it's much larger than any one company, any one country, any one set of people. It's really about humanity. And when you're doing something like that, at the end of the day,
[0:04:28] It really is incumbent on those who are going to be shaping that technology to make sure that they're thinking larger than just for their own benefit.
[0:04:38] Okay, and so with 80 people, that sounds like you're spending in the order of $10 million a year or something like that. We spend more than that. More than that. As a nonprofit, our 990s are all public, so you can go and look them up. I think our 2016 budget, basically we've been on an exponential ramp of I think like $10 to $15 million I think was like kind of year one. And Elon and Sam.
[0:05:02] Seeded that with $50 million, is my understanding, public information. Yeah, so we actually have a pretty wide base of funders, ranging from Sam and Elon have contributed, but Reid Hoffman, Gabe Newell, who's the CEO of Valve, and a large number of other individuals, and also the Open Philanthropy Foundation. And so it's actually really important to us to not be just kind of representing any one set of people or any one interest. We really do want to expand and broaden the base.
[0:05:31] And the intellectual property, the knowledge that comes out of this, is it all open sourced? Or is there a fear of releasing that too quickly? And how do you manage that? Because if you do your job and you spend this tremendous amount of money, perhaps some other actors could leverage that. How do you manage that? This is actually a really good question. I think that it's a really subtle thing when you look at how AI technology is developing today, where you actually have a very open field.
[0:06:01] The companies like Google, Facebook, Baidu, they're all doing AI research and they're publishing everything. And so it is the case that today AI really has this norm of publishing everything, which is super weird for a very valuable field. I think a lot of it really is about where it came from. You had this very academic route of maybe like three or four labs that believed in deep learning for 20 years. No one would fund them. It was really just the Canadian moonshot organization that was willing to give them any money. And then...
[0:06:29] Once this stuff started working, all the big companies were like, hey, we want to hire your people. And those people said, OK, but we'll only join if we can keep publishing. And so that ethos is still very much there in the field. But I think that this makes a lot of sense when the stakes are low, when what you're talking about are just kind of mundane, either scientific advances or even
[0:06:52] product advances, the things that can be built into products to deliver value to customers. But once you start talking about things that can actually be damaging to society, then it's something else. And so we released a document earlier this year called the OpenAI Charter. And so this is based on about two years' worth of policy research that
[0:07:11] Describes how we as an organization will operate over time. And one of the points there, which I think raised a lot of eyebrows for people, is that today, one of the ways that we contribute is by publishing most of our research. But in the future, we expect to reduce that due to safety and security. So it's almost like working on the Manhattan Project, to use an analogy. It's like, if you know, the reason it was created was to make sure that these important technologies and insights are available to everybody.
[0:07:41] But if they're available to the wrong everybody or the wrong segment of everybody, it could be cataclysmically damaging. That's right. And I mean, if you look today at examples of AI technology, it's not really clear if the world's better for them being out there. I think that a really good example of that is deepfakes.
[0:08:01] So if you're familiar with that, this is open source code that anyone can use to generate fake videos. You can take an existing video, you can put in different people, and it looks pretty realistic. And so this is something where people have used it for lots of different applications that if you add them up, I'm not really sure that the world's better as a result. Yeah. And the one you're talking about, of course, is adult content where they're taking celebrities and putting their...
[0:08:28] Likenesses on pornographic images. And if you were to look at, you would think it was a leak from their phone when in fact it's a computer building it. Or having Obama say things that are inappropriate or that might trigger the right wing people or taking right wing folks and having them say things that might trigger the left.
[0:08:46] Just chaos ensues. That's right. That's right. And so where we are with this technology is this piece of open source code that anyone who knows how to install TensorFlow is now able to use to generate whatever content that they have in mind and generate these very realistic videos. And I think we should think of this as a case study, like a first little taste of what does it look like once you start having powerful AI technologies that
[0:09:11] You actually really do have to do these sort of security and safety evaluations on. Yeah. So the first one could destroy a person's reputation and do a reputable harm to their character. And when we get back from this quick break, I want to talk about what do you think the good things are? And maybe you'll show a couple of videos of the massive progress you've made already when we get back on This Week in Startups. Now I'm going to tell you all about Squarespace, which I love. You need to turn your new idea into a website.
[0:09:41] We use it for everything here. Check out Launch Festival Sydney. Check out Launch Angel Summit. We love it because we can get a beautiful, customizable, mobile-friendly site up instantly. And it's all designed with search engine optimization built in. You don't have to hire another consultant.
[0:10:08] It's free and secure hosting with 24-7 award-winning customer support. We love Squarespace. They've been a long-running supporter of This Week in Startups, and the product is exceptional. Go to squarespace.com right now and get a free trial. When you're ready to launch, use the offer code TWIST, T-W-I-S-T, and you will save 10% off your first purchase of a website or domain. I know you don't need to save the money if you're listening to this program in all likelihood, but I want you to use that TWIST code if you can, and I want you to do me a favor right now. A lot of people say, hey, J. Cal, thanks for the podcast.
[0:10:38] Thanks for doing it twice a week for 10 years. Is there any way I can help you? Yes, there is a way. Go use Squarespace and go ahead and say thank you at Squarespace on Twitter for supporting independent media like This Week in Startups. It's an amazing, amazing product. And by the way, they now support over 200 extensions. So if you want to be baller like me and do founder.university or angel.university or use any of those cool new extensions, get on there. Try them out. There's lots of beautiful new extensions and, of course, all those great customizable templates.
[0:11:06] And one thing people don't know is that Squarespace, some people don't know, Squarespace put in incredible e-commerce functionality so you can sell stuff, open up a store, no biggie deal. Okay, back to the program. Hey everybody, welcome back to This Week in Startups.
[0:11:21] I'm your host, Jason Calacanis, and we are here at Wilson Sonsini, the fantastic law firm in the Silicon Valley and in other cities as well. And I'm with Greg Brockman, who is the CTO and co-founder of OpenAI, and he has been very generous to give us an hour or so of his time and talk about the massive progress they've been making.
[0:11:39] I watched the videos, I subscribed to the YouTube channel, and it's been fantastic to watch. What do you think is at the, I know you have a video about this, but at the core of this innovation is Moore's Law, perhaps GPUs, and the massive compute power that we've seen expand. Maybe you could unpack that a little bit. For sure.
[0:12:01] Deep learning is the algorithmic technology that's really been powering all of the AI results for the past six years. Starting in 2012, machine translation, speech recognition, image recognition, playing games like Dota, Go, all of these were powered by the same algorithm.
[0:12:20] And that's a crazy thing, to really step back and realize that at its core, there are lots of different ways that practitioners tweak things to make them easier to learn. But at its core, all that's going on is that you have a bunch of just like math, where you're just multiplying together a bunch of numbers, and somehow the system learns whatever task the implementer wants.
[0:12:38] And the magic, the secret sauce here, the thing that makes this work, the reason that we're able to do it and the people like Alan Turing and other people who were very, very smart in the 50s weren't able to do it is because our computers are a lot faster. So if you look at the way that these systems are built.
[0:12:55] You have massive computational power. And it's funny, because on the one hand, it's massive relative to the scales that we're used to, like the amount of compute that we have in a laptop on a CPU. But compared to the amount of compute in here, we're still not quite there.
[0:13:12] The real core innovation, the thing that really triggered this revolution was when people realized that they could train neural networks on GPUs. GPUs have a lot more flops. Graphical processing units, what you use to play a video game or to do self-driving in a car by companies like NVIDIA. That's right. NVIDIA stock price, if you look, has gone up quite a bit over the past couple of years. Deep learning is at the core of that. Why didn't...
[0:13:41] Maybe perhaps a stupid question. Why did GPUs accelerate faster than CPUs? Yeah, so the thing that's interesting about neural nets is they're massively parallelizable. It's kind of there in the name. You have this network of all these neurons, so you can compute them in parallel. Now, for normal programming applications, if you build your startup and you're writing your application in Ruby on Rails, you can't really parallelize things that well. You have a bunch of sequential statements, and so you can't actually utilize very parallel processing.
[0:14:09] And so the CPU in your laptop, it's oriented around, it runs at, let's say, 3 gigahertz. So you're talking it runs about 3 billion operations per second, which is very, very fast. This thing runs about 10 operations per second, maybe 100 if you're in burst mode. And that's great for something like running a Ruby on Rails application. But if what you really want is you just care about the total number of operations that you're running in this massively parallel way, then you need a different architecture. And it turns out the graphics has a similar problem.
[0:14:39] That you have all these different, very independent things going on. You need to do your ray tracing. Known as pixels. Exactly. Objects on a screen. That's right. And so it's actually really interesting that the gaming industry really subsidized all of the research and development to get to the point that we had AI supercomputers. So if not for gamers obsessively wanting to use bigger monitors, better resolution, and have more frames per second in details and less latency.
[0:15:05] The AI revolution would be how much further behind? I mean, it depends how much less people wanted to play the games. Yeah. But it's not really clear what other applications would really drive it. Yeah. And it's actually really interesting that I think kind of what happened with NVIDIA was that, you know, so 2012 is, you know, I think maybe it was 2006 is when they first developed CUDA. I might have the year off slightly there. So this is the programming language for their GPUs. They said, hey, we've got this great platform. We want people, we want developers to build on it.
[0:15:32] No idea what they're going to build. And then you fast forward, maybe 2010 is when people first started playing with it. And 2012 was the first major success, when suddenly all the computer vision people were like, all right, we're throwing out 20 years, 40 years worth of research. We're just going to do deep neural networks from now on. And all these grad students started buying all these GPUs. And Jensen, the CEO of Nvidia, was like, why are all these grad students buying GPUs? He was trying to figure out what's going on. And so he went around and talked to a number of people. And he realized that
[0:16:01] This is powering AI. This is the thing that's going to be the next revolution. They bet very heavily, and that's the reason that they're the market leader here. How fast is it continuing to grow? Because what we've seen is, I guess, the cryptocurrency short-lived revolution resulted in mining, and I think maybe the concept of mining is going away, but that resulted in a huge push.
[0:16:24] Self-driving cars is a continuing push, computer vision, and obviously AI continues. How fast is it growing? Is it accelerating the pace, or is it just Moore's Law? Yeah, so the sad truth is Moore's Law is dead. We have maybe one more die shrink, maybe two more if you really squint at it. Explain why. Because at this point, we're basically building processors that are on the order of single-digit nanometers.
[0:16:48] That's the scale that you're operating at. You really have to worry about quantum effects. Basically, it's like, I can't even imagine what it means to manipulate matter at that level. And so the technology that's required is extremely specialized, and it's very, very hard to go any smaller. So putting more transistors onto a chip, that's hard. So we're hitting the end of that game. That's right. Is there another game that has emerged? That's right, yeah. So the new game has emerged, and the interesting thing about this game is that it's always been possible, but no one had any reason to do it.
[0:17:16] Is to try to pack as many chips into a box as possible. Ah. And how does that occur? Yep. So basically the thing you do is that traditional computers built on the von Neumann architecture. So this is something that was designed by John von Neumann in the 50s. And the idea is you have a big RAM. And you have a big processor. And a tiny little memory bottleneck between them. And so your processor is able to grab any data element it wants. And it brings it across the bottleneck. And it processes it. And it puts it back into RAM.
[0:17:45] And that it's able to do that. But what this means is if you have this architecture, that you are fundamentally limited in the amount of data that you can process at a time. And so what you need to do instead is you say, get rid of the Von Norman architecture. Replace it with a bunch of tiny little cores, each one with its own little memory right nearby, its own little on-chip memory. And so you can only access your neighbor's memory, the neighbor right next to you. You can't access the neighbor way far away. But that's OK, because that's how neural nets work. And if you think about it,
[0:18:14] That's kind of how this thing's designed too. Let's set the stage with the definition of a neural network. How would we explain this to a child or just somebody who is a complete neophyte? So I actually always find that it's actually really interesting that I think that describing neural nets and describing what deep learning really is, it's a very hard thing. And I've yet to find anyone who has a perfect definition for someone who is a neophyte. I think that I'll do my best.
[0:18:42] And the idea is that you have a bunch of computing elements. You have a bunch of little, smart, little brains that are very, very specialized for just one task that are all just arrayed in a big chain. And the way that you set this up is that, at first, it doesn't know anything. You put in some data, and then things fire and spike and do all sorts of crazy things and pass along the signal all the way through the network.
[0:19:10] You put garbage in, you get garbage out. But the important thing is how you train, how you make the system learn. And the way you make it learn is that you say, well, you got the wrong answer. Here's the answer that I wish you'd gotten. And then you go and you tweak all of the configurations of these tiny little elements to make it a little bit more probable to have given you the right answer. And you just keep doing that over and over and over. So you keep doing guess and check, guess and check, guess and check. And if you do guess and check enough times across enough different examples, you'll end up with a system that has learned how to solve whatever problem you care about.
[0:19:40] And so what is the best, most illustrative example of this occurring in the history of AI in your mind? So, I mean, I think that actually a good example is our Dota playing system.
[0:19:55] So the way that this works, so this is a game. So Dota is one of the most popular games in the world. It's played by these professional esports athletes who live in these houses together. It's a five versus five player game. And the people who want to be the best have to live together, practice together year round. There's a $20 million prize pool in it for them if they become the best. So we were able to train a neural net to
[0:20:25] Be, at the moment, competitive with the top human players and hopefully soon to exceed them. And you can pull that up, I think, here. Maybe show us a little video of it. I think we actually don't have that one here. You can look it up on the OpenAI channel. What's particularly interesting about this example, I think, is, correct me if I'm wrong, one, it's a team sport. So you have five players. So in a way, you have five different...
[0:20:52] People trying to solve the problem on one side who are humans. On the other side, do you have one AI controlling all five? Or is it like five AIs collaborating with each other? How should I think about that? Yeah, so these things are not always very well defined. For humans, it's very well defined. But the way to think about it is that it's five AIs. We have physical meat containers. Exactly, exactly. It's five AIs that have been training together in simulation for 10 millennia. Wow.
[0:21:18] And their brains kind of, you know, they all get to see the world through each other's eyes. They know exactly what each other sees. Their brains are separate, so they're able to make their own decisions. But given that, you know, you kind of end up with something that acts like a hive mind. When you look through those millennia of behaviors, did you find the Leroy Jenkins moment? Thank you.
[0:21:39] Six people knowing that. Look it up later. It's a deep pull. That was the guy. Remember the World of Warcraft where the guy says, Leroy Jenkins, and he goes in and starts to fight before the other members of the clan are ready to fight? I see, I see. Got it, got it. It just all blows up. Right, right, right. It's a very famous meme from back in the day. But so suffice it to say, it goes in and it plays really shitty. And then it plays less shitty. That's right. And then all of a sudden he realizes, oh, I can beat this sucker player by just...
[0:22:07] That's right. So the thing that's interesting is the way that it learns to play is it starts out with no knowledge of the game, and it just plays a copy of itself that also knows nothing about the game. And so it's just button mashing. And the thing that we see is that the skill progression that it learns is...
[0:22:21] It's actually very similar to that of a human. This was a total surprise to us. We did not expect it. We thought it'd be something totally incomprehensible that we'd have trouble understanding. But it actually looks very, very similar in that first that you see that they start to wander outside of base. And then they learn, don't ever wander outside of base, because if you do, you'll get killed. And so then they head on base. And we always see this dip in our performance curves. But then they start to be a little bit adventurous, because they're just continuing to button mash. And they're kind of learning that, OK, when I button mash on this, good things seem to happen.
[0:22:51] Don't know why, don't know what it meant, but let's do that a little bit more often. Button mash means randomly hitting the buttons. That's right, that's right. I love it. Did you guys come up with that term? The button mash, or is that an industry term? Yeah, yeah, I think it's just one of those things that's in the air. And eventually this thing button mashes its way to the top.
[0:23:09] And it's very interesting that it goes from just kind of wandering around. You start to see that it learns these basic skills like last hitting and these other mechanical skills. And then it works its way up to strategy. And the thing that we find difficult is that we don't understand Dota strategy ourselves. Yeah, my understanding of Dota is this is like one of these games that takes, you know, five, six, seven years to become an elite player. Is that right? That's right. So typical guidance is it takes me about 100 hours to be able to understand what's going on in the game.
[0:23:36] Okay, so if we start now, in two weeks, we'll know how to play the game. That's right. And then to not be, to be a solid player who could compete in an esports arena. I mean, you're talking real dedication, right? It's like how long is it going to take to become a professional track athlete, right? Yeah, play like elite guitar level. That's right. So where are you at now? You started the Dota.
[0:24:02] Project when and when did you, how many months of OpenAI's work? Those 80 individuals working on it or whatever subset were working on it. How long did it take you to train the neural net, to build this all up, to hack into Dota or I guess Dota let you hack into it. How long did it take you to beat elite players or good players? So we first selected the game in November of 2016. So about two years ago.
[0:24:30] We really started working on the project. It was just me and one other person. At the end of the year, I remember it was December. It was New Year's Eve when I was hacking away frantically and managed to actually, we had to reverse our engineer our way into the game initially. So I was able to make that work. We got our first machine learning results in March of that year. So that's kind of the very first brain that was able to learn anything at all. And it was this very restricted mini game. And then by August, we were able to defeat the top player at 1v1.
[0:25:00] So, you know, just like think about basketball, right? You know, like a heads up version. And we were able to soundly defeat the top player then. And the curve is that it was actually the case that if we had played that same person one day earlier, we would have lost. And so the learning is really so fast that individual days matter.
[0:25:19] One year later is when we went up on stage at the International and played against one of the top teams there. And we actually lost. And we estimate, and it's always a little bit hard to tell. Did you put up a good fight? We put up a really good fight. That must have been a glorious moment. Are they going to make a movie out of it? Maybe they will. Definitely a lot of footage. It feels like either an HBO real sports movie kind of thing or a 60 minutes of...
[0:25:45] This is when it all went wrong. Right, right, right. Yeah, well, you also need the comeback, of course. Right, of course. Stay tuned, stay tuned. Oh, so you're going to go back and try to beat the same team? Oh, yeah, we've got our system is training every day. And this would be what we would call narrow AI because it's doing a specific task, not general AI, am I correct? Yeah, so the way to think about it is that narrow and general are really this spectrum that exists. And to some extent, you can almost,
[0:26:15] Say that general intelligence is an artificial point on that. And so it's more of a how far along are you? And so where we were in the 90s was that we had these very domain-specific systems, which were also not that competent, right? That they solved these tasks that were really easy.
[0:26:30] Like recognizing handwritten digits on a check if it's very, very carefully arrayed for you. It's fine, but I'm not really blown away by that. And that as we've moved up the ladder of how much computation we can throw at these models and the algorithmic innovations that we have, we now have single models that are able to solve a huge variety of different problems. And so in one way, it's very narrow, but the system that we built is actually applicable to other problems. So I'll show you an example.
[0:26:58] of a robotics problem which people are unable to program, but we were actually able to learn using the exact same system. So what we're seeing here is a robotic hand flipping a cube with letters on it. And what is complicated about this? Right, so it turns out that being able to program a robotic hand to do anything is totally out of the reach of human programmers.
[0:27:25] There's just too many degrees of freedom. You're talking like 20 to 25 degrees of freedom here. The company that makes this robotic hand has made them for about 20 years. They sell about 10 of them a year because no one can program them. The only thing people do is they teleoperate them. Wow. And so you said, hey, put the cube in this position. So we're looking at a six-sided cube. That's right. And you're saying, hey, just show us the letter A, O, and N. That's your next goal. That's right. And figure out how to do it.
[0:27:54] Just figure out how to do it. That's on you. How come it's not dropping it or is this after it's been trained? This is trained. The way that this was trained is that it has about somewhere between, I think it's about 100 years of simulated experience. The way we trained it is entirely in simulation. Got it. You didn't actually have it.
[0:28:14] Have a hundred of these sitting there for a year. No, no, thank you. You made a simulation that does this. That's right. Then once it was trained and you didn't think it was going to drop it, you gave the cube to it. That's right, that's right. And this seems slower than a human, am I right?
[0:28:28] Yeah, so we actually limit the torques because if not, you can actually destroy the hand. This is one of the problems with working with research equipment rather than real industrial grade stuff. But the thing is, industrial grade hands just don't exist right now. Got it. How long did it take to get it to this point?
[0:28:50] So this is probably about maybe six to 12 months worth of work by a team of maybe 10 people. When you showed this to the people who make the robotic arm, what was their reaction? They were pretty blown away. For them, just people being able to do this, that was always the dream. But you make it for 20 years, and no one can figure out how to program it. All right, when we get back, we know that Elon is working on a neural...
[0:29:16] Headset that will let you connect the brain. What's that company called that he's doing? It's called Neuralink. Neuralink. So he's working on Neuralink. You're working on this. That company's working on hands. When we get back, how long before a person without a hand could have this attached and have it connect to its brain when we get back on This Week in Startups? I came home the other day and my wife Jade had made me this delicious coffee with Four Sigmatic and...
[0:29:46] It has mushrooms in it. Yes, lion's mane mushroom. And I said, I don't taste the mushroom, but I feel alert. I feel productive. Stuff is delicious. How did you find out about it? She said, Tim Ferriss. My wife, she listens to Tim Ferriss' podcast. Sometimes she listens to mine. But I love this Four Sigmatic. I am crazy about it. I'm drinking it every day. Sometimes I have the hot cacao. Sometimes I have the coffee. Sometimes I mix the two and make a mocha. It is coffee, just to be clear. It's made with 100% organic Arabic coffee.
[0:30:16] And it does not taste like mushrooms. It's got that mushroom in there, but it does not taste like mushroom. You need to know that. And it's been long used by Buddhist monks to help them with focus during namaste meditation. All right, so get in there, get some Four Sigmatic, and you will be more productive, focused, and creative. It's like coffee without the jitters. And the energizing effects are stabilized with dual extracted chaga mushrooms that also help support your daily immune system.
[0:30:45] So go ahead and awaken your mind and support your well-being with Four Sigmatic. And you're going to get 15% off from our friends at Four Sigmatic. They made this special offer just for us. FourSigmatic.com slash twist. That's F-O-U-R-S-I-G-M-A-T-I-C.com slash twist. FourSigmatic.com slash twist. You're going to love it. Please use that promo code twist when you check out so you get the 15% off.
[0:31:14] And so they know that you came from This Week in Startups. And if you love it, which I know you will, I want you to tweet yourself, at mention me, at mention Four Sigmatic, and let them know that you found out about it here on This Week in Startups. Okay, let's get back to this amazing episode. Okay, everybody, welcome back to This Week in Startups. My guest today, Greg Brockman, who is the CTO and co-founder of OpenAI. Super impressive to watch the robotic hand rotate the cube.
[0:31:43] Neuralink is Elon's other company. He says he's maybe two years away from that being able to communicate. If we put these three things together, would people who've lost a hand or never had one to begin with be able to have a functioning hand that responds to their brain? Well, just to clarify, Elon's actually no longer involved in OpenAI. Right. So he stepped off the board earlier this year. Got it. Why is that? Because of conflicts? Yeah, conflict with Tesla. He's still a supporter and a fan of the project, but no longer involved.
[0:32:13] But all the open source stuff happening in OpenAI, he can use at his company, or anybody can use, right? Sure. Got it. JONATHAN ZITTRAIN- It's open source. Anyone can use, yeah, for sure. So just putting those two disciplines together, is this the future that we'll see at some point, you think? JONATHAN ZITTRAIN- So I think that it's a really interesting question to think about if you can really build. And again, there's really a spectrum, right? One of really being able to build good prosthetics and how to make that happen. But I think that what AGI is really about is something a level deeper.
[0:32:42] We as humans are, you know, I think we're used to building technologies where fundamentally the decision-making capability is still, it's up to us that we can make better decisions than our systems can. And that I think that the real transition point and the challenge and the benefit really is being able to build systems that can make better decisions than we can, that can understand problems better than we can. So if I read into that, you put this hand on somebody who...
[0:33:10] Lost their hand, tragically. And when they're going for a cup, it knows it's a cup, and the AI decides, oh, that's a paper cup. Let me see how much friction, and I'll just hold it perfectly. So I think that this is the kind of thing that you can imagine, right? And I think that... So you have a sentient hand. And so this is the question, right? It's like, what do we want? How do we want technology, right? You know, technology itself is something that...
[0:33:35] Right, because...
[0:34:05] If we got to that artificial general intelligence, it would understand us and our behavior better than we could ever understand ourselves. So I think of it from a different angle. So I think of what's the upside? Why do you want to build smarter and smarter systems? And I think that today that we kind of see some of the downsides of doing that. We're starting to see some of the upsides. But imagine if you can have a...
[0:34:31] A system that is actually able to go read all of the scientific literature from all the different disciplines and figure out how to put together all the pieces to actually go and solve cancer. That can actually go and figure out what experiments to do. And it doesn't even matter if this system has a little robotic scientific lab or whether it just advises humans on what experiments to run. The point is that decision-making apparatus, that figuring out how to synthesize all the information that's out there.
[0:34:54] I think that's something that maybe it's within reach, but it certainly doesn't seem to be. That's not in reach. So the reason you're doing theCUBE, the reason you're doing DOTA is because trying to cure cancer with it, we're not ready yet. Well, that's certainly true. But I think it's also the case that if you just took a bunch of humans, which is what we're doing right now, and tried to solve cancer that way, certainly it's going to take, I don't think that we're within a small number of years of being able to solve it.
[0:35:23] Big coordination problems like trying to solve climate change. The problems like this seem to be outside of certainly the abilities that we've seen so far. It doesn't mean that we can't solve them ourselves, but I think that thinking about how can technology help us solve those problems is a worthwhile thing. So that's the upside, is being able to solve hard problems that can really benefit everyone that have been thus far out of reach of humans alone. And so then the question is, if you have a technology that's capable of doing that,
[0:35:53] Well, what other implications does it have? And how do we make sure this is something that slots into our lives in a way that we like? And that's a decision that the person implementing it has to make. So I don't think that it's up to just who implements it. And that's actually the core thesis of OpenAI, is that it's really something that we're all stakeholders in. And so the way that these systems are deployed, who they benefit, where that value goes, that's something that we should all be really concerned about. And Google bought Deep.
[0:36:23] DeepMind. DeepMind. I know Elon was an investor in that company, and he begged them not to sell to Google. Google is massively ahead of everybody, slightly ahead of everybody. What is your take on how far ahead they are of everybody else in AI? So ahead is, again, a really funny term or fuzzy term in this particular industry, because everyone's publishing.
[0:36:51] So based on publishing, where do you think they're at? And do you think they're publishing everything they know? So I actually think that it's harder than you think to get people in this field to not publish. And so it's actually very, very hard to hold anything back. And so for example, when we just said that one day we're going to decrease our publishing due to safety and security concerns in our charter, that really raised a bunch of eyebrows. People said, you're going to do what?
[0:37:16] And I think that that's a really interesting thing about this field. I think it's actually a very positive thing, that this field is so collegial and that people view themselves as this community that's really trying to advance this technology that's going to shape our society. And they happen to work at Google. They happen to work at Facebook. They happen to work at OpenAI. And they move around. That's right. There's a lot of mobility. I'm sure you have some deep mind people at your company. Yeah, we do. And I think that at the end of the day, the
[0:37:44] The thing that really matters, and we actually build our company in a way that I think is pretty different from how others do in this space, that the way that traditionally this worked is that people really ran their companies like traditional research labs. So it looks like you basically have people who typically are PhDs, who've been thinking about this for a long time, who work.
[0:38:03] The highest value is academic freedom and the ability just to explore ideas. And we support that. We have people who work in that way. But I think that the big thing that we've increasingly found really delivers a lot of surprising value is having these large projects, where you take people who have that PhD or that machine learning background, but you pair them with people who have the engineering background. And we actually also find that you can take people from totally different fields, put them in OpenAI for
[0:38:32] Couple months, and that they're going to be contributing to the machine learning at an exceptional level. A great engineer can learn how to contribute to the AI movement, machine learning movement, in three to six months. That's right. Yeah, so we actually have a bunch of different ways that people come in. We have a program called the Fellows Program, which is explicitly a six-month program where you spend three months doing self-study, and in the next three months, you write a paper. Are those the videos in your YouTube channel where you have people?
[0:38:58] Just working on sort of what seemed like academic-like products, projects. I watched a couple of those. They're interesting. Yeah. And so we have a real spectrum of different projects. And I think that the thing that I think is really interesting, right? So for kind of these, let's say, smaller scale projects, here's one example. So here is an AI which learned to play Mario, but it was never told what to do. It was just told, don't be bored.
[0:39:24] If you can predict what's going to happen, you're bored. So go do something else. How many Easter eggs did it find? How quickly? So this is the thing. It actually finds the warp levels. It figures out how to defeat the boss. How do you define interesting to a computer? So this is the trick, right? And so the way that we defined it here was we have a neural net that's trying to predict what's going to happen next. And if it can predict it, then no good. Then that's bad. That's boring. Did it find errors in the game? In other words, like...
[0:39:52] Bugs. So this particular one, I actually think it might have. But we have other AIs that have found. And it's always kind of fun when you find these little exploits where your little character kind of runs to a weird place. You're like, this AI knows nothing. It clearly has messed it up. And then suddenly, it zips through the level, and it finds some bug. And it turns out that those are all over in games like this. So on a spectrum, we have
[0:40:22] Specific tasks that it feels like if we focus on a video game or Go seems to be particularly challenging since the board is so big and there's so many permutations. We're figuring out how to beat humans. And then we have maybe curing cancer and then eventually being sentient as the thing that people are super concerned about. Walk me through the timetable. Because it's been 10 years or so since...
[0:40:50] Deep Blue beat Kasparov? Is that about right? So Deep Blue and Kasparov were in the 90s. It was an extra 20 years. That's right. That's right. I actually have two minds on thinking about things with respect to Deep Blue. On the one hand, the technology that was in Deep Blue was a dead end. It turned out that, yep, you can solve chess by looking through more moves than a human. But so what? What else can you apply that technology to?
[0:41:15] How can you scale that to solve problems in the real world? How am I supposed to look through all the different ways that I can configure my robot hand to- So they basically just did every permutation, not learning. That's right. That's right. And so that's kind of good old fashioned AI. It was like, you know, kind of these- Is it even AI or is it just solving every possibility and picking the most problem?
[0:41:34] Ballistic one. Right. So the question of how do you define AI, I think, is a hotly contested one, right? But fundamentally, so on the one hand, you'll get how it works. And I think the how it works, that you've really just got to say, OK, I see why this isn't going to affect my life. But on the other hand, it solves a hard task, right? And so I think that it is the case that I always have a lot of respect for systems that are able to solve tasks that other people were not able to. Because it means there is something there, right? There is some new insight, something that maybe can be learned from.
[0:42:04] But then you look at how Go was solved. And that with Go, you have a deep neural net that acts like an artificial intuition that picks out the few positions that maybe are good positions. And then you look through a bunch of possible moves. So it kind of simulates how a real human Go expert works, where first, if you show a human expert a board, they'll say, oh, here are maybe four moves that could work. But then they have to game everything out in their head. And then you look at how Dota works. And Dota starts to look a lot more like the real world.
[0:42:31] Where it's continuous, that you don't even have discrete squares. You have this game that lasts for 45 minutes, and you have to take tens of thousands of actions. You have all these different units, things like that. And there's no look ahead at all. It's just a neural net. And the way that it looks, it's almost like an insect level intelligence. Where it's like you trained this being, this brain, on a huge amount of training games. And then you end up with this.
[0:42:57] Brain that's pretty cheap to run. It can run on one CPU that is able to operate very well in the environment it was trained in. It's not able to be taken out and put into a new environment and do anything well there. But you basically have these extremely well-honed instincts for the problem at hand. So part of this evolution is going to be taking things that are working on finite problems, I guess what some would refer to as narrow, and pulling it out and plopping it
[0:43:26] Into an adjacency. So taking Dota, AI, and putting it into Call of Duty? So that kind of thing, right? So right now, we're able to transfer the systems, right? So we take that AI system that we built for Dota, you point at robotics, and it works. But you can't use the knowledge, that background knowledge. But that's starting to change already.
[0:43:47] So we actually have a model that we trained earlier this year, and other people now have kind of follow-on work and similar work, where we just had a model that read 7,000 books. We didn't tell it what to care about in the books. We just said, predict what is going to come next. Predict what word's going to be the next word in the sentence. And so just read it, predict what's going to happen next. And then you can take this model, and you can apply it to...
[0:44:13] A bunch of different natural language processing tasks, like these question answering tasks, like read a little story and then answer some question about it. And it sets state of the art across all these different data sets. So it really learns some background knowledge that's useful. And then you can also ask it to generate text for you. You can ask it to make a little story for me. And it comes up with these stories that are actually pretty interesting to read. One of my favorites is
[0:44:38] It talks about these guards of the Red Temple that they're kind of fighting and they're protecting whoever the narrator is. And then it starts talking about how some smoke comes from this fire. You start to see there's some world knowledge that's really built in there. And so I think that this work is all very early. It's preliminary. It can't be any worse than The Last Jedi. Right, exactly, exactly. But I think what we're starting to see is that you take these models, you scale them up.
[0:45:05] Unlike with Deep Blue, where the model that you're working with is fundamentally hard to get in the real world. You need the simulator that's perfect that lets you look ahead. Here, you just need background data. You can take data of books. You can take data from YouTube. You can take data of just images. And now we're able to start to learn things that seem like we can actually generalize to different problems than what we were training on. All right, when we get back from this final break, I want to talk about nation states having access to this. We've recently seen in China.
[0:45:33] The misuse of a technology crisper to flip switches and create babies that have different genetic makeup than they were intended. What are the unintended consequences that keep you up at night and that you co-founded OpenAI to maybe avoid when we get back on This Week in Startups? Hey everybody, I want to show you how we market this very podcast on LinkedIn. Here we go. This is a beautiful video that my CMO Presh made.
[0:46:05] And he's on LinkedIn making a campaign. And he's targeting people who work at Google, Instagram, Twitter, Snapchat, Facebook. And he's saying, let's get people who have a certain age, maybe above 25 years old. And then he's going to look and say, hey, what's their job experience like? How many years? Okay, we'll say, you know what? We want people who have been in the job for five years. You could have picked over 10. And here he goes. He's going to upload a video ad. Now, why is this important?
[0:46:34] It's important because I want to get more people watching this show who work at the companies that matter. And he's going to upload a video of one of the talks we had at LaunchScale from our podcast. And he's going to tell people the call to action is to go to our iTunes account and subscribe to our podcast. Because people who subscribe at iTunes maybe stick around a little bit longer, okay? And the call to action is subscribe. He uploads a video. And it's this easy.
[0:47:01] To make a video that targets the right people. Now, if you were buying ads anywhere else, you wouldn't be able to target by their job title, their skills, their experience, because other people don't have that information. Who has that information? LinkedIn. And here is your call to action. If you want to do what I just did, you're going to go to linkedin.com slash thisweekinstartups. If you take the time to type that in, you're going to get a hundred bucks, a C note, a beanie from my friends at LinkedIn.
[0:47:29] It is a great place to do marketing because you can get high quality leads, incredibly targeted website traffic, and higher brand awareness. And that's what I'm going for. J. Cal, your boy is doing this himself. We are spending money on LinkedIn to get the right people to watch this very podcast. And we've doubled the viewership in the last six months for this podcast. You're crazy if you don't take this $100 right now. And I don't think it's going to last. So get to it this week in startups.
[0:47:56] LinkedIn.com slash This Week in Startups. Okay, let's get back to this amazing program. Hey everybody, welcome back to This Week in Startups. I'm your host, Jason Calacanis, and with me is Greg Brockman. You can follow him on the Twitter, GDB, and you can follow the OpenAI, just do a search on YouTube, on the YouTube, and you'll find their very cool YouTube channel. Bunch of different videos there, a lot from the interns. That's right.
[0:48:25] I guess the thing everybody's wondering about, because again, back to Elon, he was like, hey, listen, this thing needs to be controlled. I think that was something you also share, that this needs to be controlled. That's at the core of AI's mission and ethics. What concerns you in the short term, medium term, and even long term? So I think there are three categories of risk. I think the first one is that we build systems that do what their operator intends, but that
[0:48:54] are able to be subverted by malicious humans. And that's something that we've really started to see already, where you have systems like our social networks that have various AI algorithms running within them, and that they kind of operate as they were designed to.
[0:49:09] But that a malicious human who kind of knows what they're doing is able to get those to have undesirable consequences, things that were not aligned with the intentions of the operator. Great. So you have Facebook says, we're going to make an algorithm that makes people engage with this content. And we'll take the content they engage with most and put it up for their friends to see. And then somebody puts up lies or shocking Pizzagate, things that divide Americans at their core, whatever the issues are, guns, et cetera.
[0:49:36] And that's how it works today. But you can imagine the same category of risk applied to systems that have way more responsibility. Systems that are, let's say, controlling the electrical grid or systems that are responsible for making big decisions in how resources are allocated in our country or things like that. Self-driving cars. Self-driving cars, perfect example. So when you have these physical autonomous systems as well, if those can be subverted, we're not going to be in for a good time.
[0:50:04] So that's category of risk number one. Category of risk number two is systems that don't actually do what the operator intended. There's no malicious human in there, but just somehow they go off the rails and don't do what whoever built them wanted them to do. And this is something that we actually see all the time with normal software. And we shouldn't expect AI to be any different.
[0:50:28] So you program the AI to, say, eliminate cancer. And they're like, well, humans cause cancer, so just get rid of all humans. And I think that's the subject of Terminator. Right. So exactly. So there's, yeah, so exactly those kinds of things. And one thing we're actually seeing that I think is very encouraging, right? So I think the thing that's really important and the best way to mitigate that is as you do technical research, do safety research in parallel. Explain that, safety research. Yeah, so safety research, right? So this is something that I think people in the 90s, maybe it was the first time people started to think seriously about.
[0:50:57] How are we supposed to build really powerful systems that do what humans want if we can't even really understand the actions they're taking? Imagine someone who's just way smarter than you. Think of your smartest friend from school. And imagine that they're saying, oh, I came up with this brilliant idea. We're going to do this. We're going to do this. We're going to do this. You can't follow any of it. And you're like, OK, yeah, it sounds good. Go ahead and do that. And if you trust that friend, if you're really sure that their values align with yours,
[0:51:22] You're going to think whatever they're going to do is going to be pretty good. Yeah, it turns out to be Facebook. And if it turns out that their values aren't very good, then maybe you'll have just been tricked into doing something if you're falling along with them. Or maybe you just gave them permission to go do something you didn't like. And so I think that that question of value alignment, that's the real question. And so the real core question becomes, how do we get human values into these AI systems? How do we get humans into the training process? And that's what science fiction predicted, which was, hey, train the replicants. Train the androids.
[0:51:52] To do no harm to humans. Like prime directive, don't hurt a human. Don't allow a human to be harmed. Don't do something that would harm. If you see a human that's about to be harmed, stop them from being harmed. All that kind of stuff. And then you look at Asimov's stories, and that they're really about how the three laws of robotics don't work. And I think the real lesson there is that if what we're supposed to do is write down a reward function for what humanity wants, to write down in a very explicit fashion what we like, I think we're going to be in for a bad time.
[0:52:21] But here's the upside, right? I think that that really depressed people for a long time. People were like, there's no solution. We're totally screwed. But if you look at the research that we've been producing and other people in the safety field that we've been helping spur have been producing for the past couple of years is a way around this, which is you don't actually need to specify the reward function. We already have technology that's able to learn stuff that humans are unable to specify. And so, sure, why not learn the reward function? And if you think about it, this is...
[0:52:50] How humans operate, too. You have a child that's growing up. When a child does something that's good, you say, that was good. When a child does something that's bad, you say, that wasn't so good. And somehow we're able to have these very smart systems that grow up to have aligned values with us. And so I don't think that it's a perfect analogy. But I think that the thing we're starting to see is having AIs that learn from feedback.
[0:53:13] So you show, for example, a human labeler two videos of behavior. You just say, which one of these is more like what you're looking for? And they just click which one they like. And from a small amount of feedback, we're able to start learning tasks. So this is just one example of the kinds of safety research that we've been promoting. OK, so what's the third? So the third category of risk is that we actually solve those two problems. We build systems that can't be subverted, that do exactly what they're supposed to do, but somehow the world isn't that good.
[0:53:39] So they were bad actors. That's right. And so it's not even necessarily malicious actors, but just somehow the economy is set up in a way that wasn't that good for everyone. So you can imagine this is you have one corporation that owns everything. All the dollars, this one megacorp that has become the biggest thing ever, and everyone else is kind of left behind in the dust. That doesn't seem like a very good world. And so I think that technology fundamentally is about promoting human values. It's about increasing just the overall goodness of humanity and to make sure that's something that trickles down to everyone.
[0:54:09] So the bet is, if everybody has equal access to it, it negates a superpower, let's just say Google or Amazon, let's say they ran away with it, from dominating all other companies and governments because they accrued too much power, solved too many problems. Well, so and here's the key point, is that OpenAI's mission, it's about the benefits of this technology, right? That what we think is crucial is that the benefits of AI, of general intelligence gets shared with everyone. Now, does that mean that...
[0:54:37] Everything should be open source and that everyone should have access. Not necessarily. As we talked about, I think that there are real issues with that. But I think that the thing that is most important is coming up with some configuration where no matter how much powerful technology we create, that those benefits are shared not just with one company, with one set of people, but with everyone. Is the closest analogy that you talk about internally
[0:55:01] Nuclear proliferation, because it does seem like this is a technology that had the opportunity to power the world or have this incredible good. But then we realized, once it got out there, you know what? We're going to start containing it. We're going to start limiting the number of countries that could have it to 10. And the 11th can't have it. And it's like, well, why can't the 11th have it? It's like, well, because, well, we don't actually have a reason that the 11th, 12th, and 13th couldn't have it. But we just decided we're going to stop at 10. Right. So I think that this is one.
[0:55:30] One technology they think sheds light.
[0:55:33] On what we should expect, it's not a perfect analogy, right? And that I think that there are things that are very different. So with nuclear, it's actually much easier to control in the sense that you know if you want to regulate it and keep it out of certain actors' hands. Because of materials. Exactly, right? It's much more clear that you need actual uranium ore, you need these massive centrifuges, that it's something that's very clearly only, it leaves behind a very massive footprint that you can tell when people are testing. There are all sorts of things there. And also very importantly is that you can tell if someone
[0:56:02] No one actually used a nuclear weapon. Instantly, yeah. They have monitors. And you know who used it. And where. You know all these things. But with AI, you think of a different analogy, which is think about cyber attacks. Cyber attacks, not even clear if someone used a cyber weapon sometimes. It's certainly not clear who used it. And so is AI going to be more like nuclear weapons or like cyber? It's not clear. And so I think that there are these deep questions that you can draw some analogies to existing technologies. But you shouldn't expect it to map perfectly.
[0:56:31] It is fascinating because you think about it. It's not like you're going to be able to collect all the NVIDIA cards that are out there. If you want an NVIDIA card of this generation, you need to have a license. It needs to have a signal when you're using it, and we have to track it. One thing I find really interesting about this conversation is actually that I think for many problems, it's hard to think about the implications of them, how they go. But the thing I find for general intelligence is that there's a bit that you need to flip internally of taking it seriously.
[0:57:00] If you don't take it seriously, it feels like sci-fi then. It's all kind of out there, so it doesn't matter. But if you actually think this could happen, this could be something we have to think about, then suddenly all the implications become pretty clear. You start thinking exactly those thoughts of, OK, well, there's these chips. Those are the choke point. How do you think about those? Where are those made? You have this question of, OK, the algorithms are out there. And the question of, well, how are these things going to be deployed?
[0:57:23] Who's going to be in control of them? And what are the values that are going to be in them? And I think then you think about just like, you have this world, and it's not just us. It's not just people in San Francisco. It's not people in the US. We're talking the whole world here. Different people have different values. And so how do we make sure that those values end up being something that get reflected in these final systems in a way that people are happy with? And all of those, those are really hard questions. You shouldn't expect to just be able to solve them. Yeah, not to be alarmist, but the technology you've already built and other people have built,
[0:57:51] Could be used to fly drones into planes using AI to say, hey, find the plane and identify the engine and just put this DGI drone in the engine and watch them plummet out of the sky in the harbor. I mean, it's horrible to even think about, but it wouldn't be very difficult for a reasonably intelligent programmer to do it today, correct? I think the core thing you're getting at there is that AI is inherently dual use.
[0:58:20] With, say, Nukes, it's easy to work on the good applications, not work on the military applications. But with AI, you kind of work on one technology. Yeah. The same technology that keeps the DGI drone from crashing into a building is the same one that could make it crash into a jet. And the thing is, here's the upside. The upside is that with the technologies we build today, there actually is a barrier to building it. Because even though you have the same core algorithm that enables this wide range of different application,
[0:58:50] It still takes a dedicated team who knows what they're doing and to put real effort in solving any particular application. When will it not? When will it not take the 80 people with PhDs and other degrees at OpenAI to solve this? When will it take two kids in a basement to make a really bad decision? Well, so that's when we build AGI. And so then the question of, well, what's the timeline to that? That's a really hard question to answer. A decade or two, probably.
[0:59:15] If I set the over-under at 15 years, would you take the over or the under? You understand over-under betting? I understand over-under very well. You like to gamble? I try not to gamble on AI timelines. What do you gamble on? Your poker player? Sports betting? Horses? What do you like? The ponies? These days, all of my probabilistic bets end up being in training AIs. Fair enough. What I want to show you here...
[0:59:40] is a chart that I think should show you kind of emotionally how to think about the progress in the field. And so what this is, is this is the growth of computation in the field, not in theory, but as used in practice for breakthrough models. And so this is an exponential zoom out. It's kind of like those charts that you see with like, you know, that first there's a human, then you zoom out and you see.
[1:00:00] Earth, then you see the solar system. And so here, these were historical results from the 90s that all just fell away. And this is the computational power it took to solve those problems. That's right. The total amount of computation that went into solving it. So 2012, now we have these models that are kind of going away. These are the image models. 2014, you're starting to have some more image models. You'll have machine translation. It's seek to seek right there. Finally, 2015 is going. And the rate of improvement here is about a factor of 10 every year.
[1:00:30] So the total amount of progress from 2012. So that's a lot more than Moore's law. So much more than Moore's law. Yeah. Total amount of progress has been about 300,000 times between 2012 and today. Between 2012 and today? That's right. Six years. That's right. It's been how much? 300,000 times. 300,000 times. Times. So the amount of compute in this guy. The amount of compute. Versus in one of the big alpha guys. And that's because of parallel computing.
[1:00:56] Is that right? That's right. That's because of people using – it's two factors, really. The two factors are, one, people using faster neural net accelerators. Wait, what's a neural net accelerator? So GPU. GPU, got it. People are starting to move to other more exotic like neural net ASICs, special built neural net computers. Who's building those? So Google, for example, is building their tensor processing unit called TPUs. But there's actually – we track about –
[1:01:23] I think 40 odd startups or so that are doing different approaches at building these massive neural net computers. So these are, instead of saying, hey, these GPUs work good at this. That's right. Somebody's making a product that says, these people have a need. This is a better tool for it. That's right. That's a whole game changer right there, isn't it? That's right. Now, remember that, you know, that.
[1:01:44] If you're in startups and you want to know who makes the money, it's always the people who make the shovels. Yeah. So those companies, there's about 40 of them in your tracking. Are you using any of their technology yet? Is any of it in market? So they're not quite in market yet, but we're getting close. So you'll get like Graphcore was originally planning on shipping at the end of last year. I think that they'll be shipping relatively soon. There are a handful of others. And I think that we're really excited about the possibility. And I think that there's just a bunch of different technologies that people are starting to...
[1:02:14] We bring to market. And the most exciting thing in my mind is that that progress that we're seeing there, our projections are that we have about five years left of it. And it doesn't rely on any scientific breakthroughs. There's no scientific risk. It's not that we need smaller transistors. We don't need quantum. Just execution. It's just execution. Now, that's hard, right? You know, it's hard execution on a new product that needs to be tested, but that we have the raw materials and intelligence and background to do. That's right. That is phenomenal.
[1:02:40] I didn't ask you about quantum computing here, which is kind of the unicorn of unicorns. Is that going to make this go supernova, like some people are saying? Or do you think this new hardware that's purpose-built is the win?
[1:02:57] Quantum, it's actually interesting. I try not to couple my fundamental scientific risks. And so when I think about the fundamental scientific risk of trying to build an AGI is pretty hard. And if you throw in needing quantum computers. And you even think about not just getting quantum computers online, but building the whole software stack. Getting to the point that we can run TensorFlow on those computers seems like a pretty high bar.
[1:03:20] And so I don't feel that I know enough about quantum computing to have a judgment on it. But to me, it seems like a very tough, tough, tough, tough cookie to take to say we're banking on that in order to build AGI. Yeah. You'll wait and see. Maybe when they get those able to solve more basic stacks. So I'm very excited about quantum, very excited about optical. I think there is a lot of new breakthrough technologies.
[1:03:46] That could be helpful to us. Describe optical, yeah. So that's using optical interconnects rather than electrons on a wire. At the end of the day, the kind of thing that we really want is we want really high bandwidth, really fast interconnects. Because think about it. If you paralyze everything, then what really matters isn't making individual chips faster. It's really about
[1:04:10] Can you actually connect all this stuff fast enough that they can send all the data across? Now you're back to the bus we were talking about before that moves the data back and forth between all these processors. Build us a fast bus and we'll get on. There's a little bit of fear of China these days. How far along are your contemporaries in China in terms of AI? And do the scientists here with the publish first mentality, is that cheered by the Chinese PhDs and researchers? Well, so there's an interesting thing that if you look,
[1:04:40] At paper count, the total number of papers, the number of papers coming out of China surpassed that coming out of the West a couple years ago. So actually, China's publishing more than out here. But there's a flip side to that. Corruption, quality. So the question of where are all the breakthroughs coming from, I think that the only breakthrough that I can think of that came out of China came out of Microsoft Research in China. And you would attribute that to?
[1:05:06] The quality. I think that the causality is hard to assess. That's the fact. And so then I think that something interesting that's going on is that it is definitely the case that in China that AI is a huge national priority. That I think it is the case that, especially after Go is a game that is a really...
[1:05:29] Like, you know, sort of revered game in China that, you know, that in America, maybe people tell their kids to be football players. In China, people would tell their kids to be Go players. And so when you have the top Chinese player being defeated by an AI, that's something that wakes everyone up. Isn't that fascinating how, like, the defeat of a top player in a casual kind of board game, not so casual, but in a board game, essentially.
[1:05:54] Would inspire everybody to pay attention. It is interesting how humans work. It is interesting how humans work, for sure. And I actually think that it's been very inspiring in a lot of ways to see how well China's mobilized to try to work on these problems. And I think that the applications that are being worked on, I think that, again, that's a different question in some ways. But I think that there's no question that the execution machinery is extremely good.
[1:06:24] How often do the three-letter agencies poke their head around and ask you questions in the United States? Are they super focused on this, or do they just come in and go, what's going on in here? I don't think it's a question that, you know, regardless of what the answer is that I'd be able to share. I was talking about NBC and ABC, the networks in Los Angeles. I wasn't talking about any other three-letter agencies. Of course, yeah, yeah, yeah. Well, I mean, twists is one letter too long. Exactly, exactly.
[1:06:54] Well, how serious, putting aside the three letter H's, how serious do you think our government takes it? Super serious? Lightly serious? Medium serious? So we've done, as an organization, probably three or so congressional hearings in the past 18 months or so. Does that be serious? So it's actually one thing that's very encouraging. Actually, my first testimony was chaired by Ted Cruz. And it turns out that when he was a kid, he thought he was going to be an AI researcher when he grew up.
[1:07:24] Really? Yeah, that's what he told me afterwards. And I was like, I did not know that. And so it's actually surprising to me to see the amount of engagement from a lot of top government officials.
[1:07:35] But on the flip side, I think that it's really hard to see. I think that the priority in terms of funding and in terms of really having serious efforts haven't been there in the same way. And so what we're starting to see is that people are starting to pay attention and that I think that we're able to get the kind of years that we would hope. But on the flip side, I think it's the case that here we have a private industry that's doing an extremely good job of pushing this forward and that the public...
[1:08:01] The sector is not really playing the game. Is there a revenue model for OpenAI to keep it sustainably funding? I mean, the Mozilla Foundation makes hundreds of millions of dollars. It's all public now.
[1:08:14] Because they have a browser that a lot of people use, and they use Google Search or Bing Search, and they make some revenue. Is there a revenue model to make you sustainable because you're spending all this money? Or do you have to keep going hat in hand and ask billionaires who care to give you money to keep this research going? Well, so the short term, the progress is just so fast in this field that I think that working on anything besides trying to push it forward would actually make you fall behind. So to take a customer would kill it.
[1:08:41] That's right. And the way that I think about it is that when you're on this exponential ramp of development, at any given point, you could kind of exit that ramp. And in fact, we've seen people who were working at OpenAI and were really excited about some technology that we just built and went and started a company with it. But the flip side is that then
[1:08:59] You fast forward a year later, and in fact, the new technology that we have in, say, robotics has advanced such that if you really wanted to make a company, you should have just waited a year and used that. And the thing is, this is always going to be true. And so I think that for us right now, we're just focused on how do we continue to push forward this technology. And I think that for us, we will always be focused on how do we accomplish the mission. So you just have to rely on benefactors and hopefully a wide enough array of them to keep this going.
[1:09:25] Yeah, I think that one of the things for us in the long term, we'll be figuring out what is the right sustainability model. But yeah, for the short term, I think that we're in pretty good shape. All right. Let's hear it for Greg. Give him a big round of applause. And we'll take two questions from the audience. Choose a good idea, I think. Yeah. Hi, my name's Ryan Rucker. Thank you for doing this. One quick question. I saw during the demo that there was, while the hand was sort of
[1:09:53] Was that an Easter egg? Were you sending us a message?
[1:10:20] It's actually interesting. In the particular case of the robotic hand, it actually ends up, one thing that's surprising to us is it learns these grasps that are very recognizable. And so it turns out there's this whole taxonomy of grasps. And if you use them, you can actually recognize the way that it's doing it. It's pinch grasp and different things.
[1:10:35] And I think that the point that you make is that sometimes you're going to end up in states where you're doing something that is not quite what we wanted. And I think that sometimes it's something like the middle finger is sticking up, but sometimes it's that you have systems that are going off and saying things that are inappropriate. We've seen that a number of times, or making decisions that are biased or unfair. And I think that this is something where, first of all,
[1:11:04] The most common place this happens today is in the data. If your data is biased, you should expect to get a biased system out. But as we look forward, we're going to be building these systems that are going to be training across huge swaths of unlabeled data. And the question of how do you get our cultural sentiment in there in order to try to squash these kind of behaviors is something that I think is an open question. But that's something that I think is really important and a number of people are working towards.
[1:11:32] Great. Let's take another question. You have the microphone back there. Related to that, earlier on you had mentioned that it was going to be hard to actually enumerate the specific human values that these systems were going to have to go. And perhaps it would be through a learning system that would identify the human values that would be embodied in these systems that they don't go and wreak havoc on our societies.
[1:11:53] Our societies aren't necessarily in consensus about what a human value is. For instance, in this country, there's a very deep and very passionate split about, let's say, whether an abortion is something on behalf of a woman or whether you have to protect an unborn human child. Very deep split. You won't be able to arrive at an emergent human value. And that's just a very prominent example. There are many, many much smaller examples throughout our daily lives. So how do you expect the systems to come to some emergent?
[1:12:20] Consensus, when we as humans can't even come to a consensus. This is also a very excellent question. And I think that this, in some ways, is the most important question about the post-AGI world. It's this question of, let's say we solve all the hard technical problems. We build these systems. They do whatever's in charge of them wants. And we figure out how to make sure they're not going to go off the rails. We figure everything else out. There's this question of, who sets the values? And whose values are those? And I think that that's a bigger question than
[1:12:49] One organization, right? I think that's something where you need to have stakeholders that are actually able to represent, again, not just one country, but really the world. And I don't think that there's a simple answer, right? But I think that fundamentally that we need to make sure that we're involving in that process people from governments, people from around the world. And I think that that's going to be a really hard question. But fundamentally, we have a world.
[1:13:15] It's organized in a certain way that fundamentally there are decisions that are made. And so I think that there are existing structures that you can actually look to and say that even though we don't all agree, somehow we are able to continue forward and actually have systems that operate. Yeah, it does seem to me that governments and societies already have this in place. A perfect example would be peer-to-peer technologies, cryptocurrencies, and VPNs.
[1:13:38] In America, we had peer-to-peer and people were like, well, it's possible to have Napster, therefore it must exist. And we all saw how that worked out. And then they were like, oh, it's possible to do tokenization and have all these currencies. And yeah, the government can't stop you until the SEC showed up and was like, these are the rules and are you following them? And they're like, no, it's a utility token. It's not a currency. And they just started enacting laws. And then in China...
[1:14:05] I had a whole bunch of people were telling me how like you can't stop cryptocurrencies. And I was like, you realize in China, if you sell a VPN to somebody, you'll go to jail. Like they're literally putting people in jail for selling a VPN, which is kind of like what everybody uses at their company anyway. So society exists and has rules already. Technology might be, it might be, the SEC might've been late or some nefarious country like China or.
[1:14:30] You know, have different values, might not want people to have freedom or privacy, and they will pursue a different route. Let's take another question. Where's the microphone? Okay, go ahead. So I have a question, and once AI surpasses human intelligence, how can we make sure that it isn't used for bad? Yeah, and I'll punch that up. When will it surpass human intelligence? If we were going to make an over-under bet, 20 years? 30? What's your answer?
[1:14:59] I'm going to say, well, we have to define intelligence right now because if we defined intelligence as beating somebody at Go or poker, it would have been negative two years ago. So we'd have to figure that out. This is the thing about gambling. When you're gambling, interpretation of the bet is critically important, right? Yep. Well, and so for its worth, the reason that I try not to make...
[1:15:28] Predictions about the timeline is because I think that when you look at the way that most people make their prediction, you've got to ask, what's it based on? What information really goes into it? I think a lot of people make it off of gut, which I think is totally understandable. But one thing that we've actually done is put a lot of effort into trying to figure out, what are the real factors that are driving progress? And what are the risks to those factors stopping? You think about computation. In order to come up with that chart, we really just looked at publicly available data.
[1:15:58] publicly available information that anyone could have pulled. No one synthesized it together. We published this blog post where we say, hey, 300,000 times increase. And the community
[1:16:06] Flipped out. They were like, wait, what? That's what we've been doing? And I think that really spending the time to step back and think about, what does the progress look like? What are the missing pieces that we still have real fundamental problems we haven't solved in terms of actually having systems that can reason and that can learn from small amount of data like humans can? So the way I think about it is more in terms of the facts and lead people to draw their own conclusions as to what they think. And so to answer the question,
[1:16:34] Making sure these systems aren't used for bad, I think that that is, again, part of this fundamental question. Technology, building technologies, that's something that we've been doing for a long time. And I think that we've seen that technologies in general, they come with good, they come with bad. And I think that the question of ultimately how they're deployed comes back to those three risks that I identified earlier.
[1:16:58] And so I think that there's this question of solving all the technical problems. And that's a hard thing. But fundamentally, it's an easy to grasp idea of have a technical team that's working on the technology, have a technical team working on the safety. If it turns out the safety is really hard, slow down the technical stuff. If it turns out the safety is easy, which hopefully it'll turn out to be, then full steam ahead. And then there's a question of the deployment. There's a question of who's involved in it.
[1:17:24] And there's a question of who has a say and who has a stake there and how do you coordinate with other people around the world who are going to be building similar things. And all of that, that's hard to describe in one sentence, right? That's something that I think requires big buy-in and real coordination in a way that's very, very difficult. Back to general AI, we both would agree that it's under 100 years away and over 10. We would both agree on that, right?
[1:17:48] General AI. Under 100 and over 10, right? We're going to do a binary search. We do. Would you agree? I would. We should take a poll of the audience. How many people believe general AI will emerge in under 100 years? Okay. How many people think it will happen in more than 10?
[1:18:08] Great. Okay. So now we've got like our goal posts. And now you can just sort of bring this in a little bit. Depends how hard we work. Exactly. So you agree with this 10 and 100. It's between 10 and 100. You don't like to make predictions. I don't like to make predictions. And there's also a question here to think about. Let's suppose that, you know, let's say that, you know, to your point that AGI takes like 20 years or something. Is it the case that it...
[1:18:35] It was totally impossible to happen in 15 years. Or is it the case that the people who were building it, maybe should have just worked a little bit harder and stayed up a couple more nights. And so not saying that's exactly how it works. And I think that there's a lot of dependencies here. But I think that, again, it's a little hard to say. If you look at the Manhattan Project, when we needed the bomb, it got done pretty quick. Resourcing is actually, I think, one of the super important questions in this field. And they're like, hey, the Nazis are going to get this. We better get to work.
[1:19:01] And we got it. And one thing we really hope is that the way that AI development doesn't go that way, right, in terms of being a competitive race. Because think of what we've just been talking about, of all the ways that could go wrong, and maybe it's going to turn out that safety's really hard. And imagine now that you're in a race for national supremacy. Yeah. Are we? So I think today we're not. I don't think we feel that way. But you could imagine that if this becomes something like we've seen in the past, right, you think about...
[1:19:25] What space was. And you think about what the atomic bomb was. Those things were these national projects. And so actually, one of the other points in our charter is this other one that I think looks really weird at first glance, which is we say that if another organization that's value aligned comes closer to building AGI than we are, let's say they're like two years away, like with 50% probability, we'll stop working on it. We'll go and join and help them. MARK MANDELMANN Wow, that's fascinating. MARK MIRCHANDANI Right? Because the thing we're concerned about is that this is all going to devolve into competitive race. And it's going to be race to the bottom on safety. And whoever wins.
[1:19:55] I'm going to kind of win in the short term, but down in the long term. If you allocate all 80 people to going fast and you don't put brakes on, you will get there quicker. So it's like, yeah.
[1:20:06] I understand exactly what you're saying, yeah. Okay, we'll take a final question from Michael. Go ahead. Yeah, thank you. First thing, I appreciate you coming by. I run an AI research company out here now from New York, and it kind of pertains a little bit on the last thing you guys talked about between Russia and China and even the U.S. Looking at this at a holistic standpoint, from a government standpoint,
[1:20:27] We kind of are behind China when it comes to government policy, right? They kind of have open arms when it comes to the ability and what they're able to do. So what do you think being at those congressional hearings from what you hear from our policy thinkers for the next year, two years, the fact that we are completely climbing at the exponential growth of AI, why aren't...
[1:20:52] Policymakers actually doing a better job in this country than other countries. Also, educators, what is the policy of note that matters? Yeah, so AI policy, I think, is something that's very, very nascent in this country. And as you say, I think that in China, it's something where the government's like, yes, please, we want this. And I'm not sure that the applications that the Chinese government are using are ones that we really want, right? So I think that it's not even the case that we could just be like, let's just import all of those ideas and hope to have society end up the way that we hope.
[1:21:22] The thing that I see on the ground, and very recently there's been some movement in terms of some regulation around export control for AI. It's starting to be some waves there. There was an announcement within the past couple weeks. But the thing that we've actually been recommending to policymakers is just spend time measuring.
[1:21:44] Spend time figuring out what's going on. And actually, during the Obama era, there was an AI report from the White House, which we helped contribute to. I think that was a good first step. And I think with this administration, we haven't yet seen that same taking that to the next level.
[1:22:04] I think that there's a lot of other things going on in this country that I think are also very important. So I'm not that miffed about it. But we're here whenever people want to talk to us. We've got plenty of intelligence in the White House today. Much more intelligence and decision making and winning going on than anybody knows. Let's give it up for.
[1:22:30] Our friends at Wilson Sonsini for hosting us, and Greg Brockman, our incredible guest. That was amazing.