[0:00:01] For the past 60 years, despite all the setbacks, neural networks have been steadily improving. In our next session, we're exploring the forgotten history of the field and why we must dare to dream big for the future. Please welcome co-founder and CTO of OpenAI, Greg Bruckman. Hello, everyone. I think we're all familiar with the fact that AI technology has been progressing very rapidly for the past couple of years.
[0:00:54] What I'm here to talk to you about today is the logical consequence, the logical destination of the kind of progress we've been seeing, really the goal of the field since it started in the 1950s, which is artificial general intelligence. Artificial general intelligence is something that is on the same spectrum as the AI technologies we're all used to, but it's qualitatively different. And the mission of my company, OpenAI, is to ensure that artificial general intelligence is something that benefits all of humanity.
[0:01:23] And the way that we define it are highly autonomous systems which outperform humans at most economically valuable work. And you know, the creation of artificial general intelligence is a bit like the arrival of a hypothetical alien or extraterrestrial being.
[0:01:39] It's like something that can generate huge amounts of value that maybe there's going to be technology to be unlocked, social problems that we can hope to solve. But also, we should be a little bit nervous about it. It's something that carries a bunch of risks, something that's unfamiliar. And really, all of humanity should come together and make sure that it all goes well. But before you can even really have that conversation about how to capture the benefits and mitigate the risks, you need to emotionally buy into the fact that this can happen at all.
[0:02:05] And that's what I want to talk about today, is the fact that for the first time in the past 50, 60 years of AI, that there's a real shot that we're going to be able to reach this end goal. But first, I want to make it a little bit more concrete what an AGI would be like. The way to imagine it is as a company of the smartest humans you've ever met, all working together in concert towards one goal with no breaks and no ego. And think about what a system like that could accomplish.
[0:02:30] And the kinds of tasks that we're talking about are less about automating manual work and much more about creating innovative new products and services, much more about having an automated scientist that can read the whole scientific literature across every different discipline, understand how they all fit together, and then work on curing cancer. You should imagine this system to be good at things like resolving international disputes, being an automated diplomat.
[0:02:54] But it's not going to be some magical utopia, right? There's going to be new problems that being in a world with these incredibly smart systems are going to generate. And I think that's something we need to be extremely mindful about. But what I want to focus this talk on is about why we should even think that a system like this is possible to achieve in the near term, why it's even worthwhile to spend our time thinking about it. And the way that I'm going to do this is by walking you through the results, the actual things that we've been able to make work.
[0:03:22] In the past six years, right? You know, prior to 2012, AI was a field of broken promises. People got very excited about maybe this will work, maybe that'll work. They got little toy results, but it was never really the case that they were able to do anything dramatic. And that's really started to change. So the components of AI progress generally and AGI progress in particular, you know, everyone kind of has probably heard about these. You have algorithms for the ideas, you have computation and then you have your data and environment.
[0:03:50] And at the core of all of the advances we've seen for the past six years, it's all one thing, one simple idea, one simple algorithmic core, which is the deep neural network. And for these past six years, deep learning has repeatedly broken through these seemingly insurmountable barriers time and time again. So for one domain, let's think about computer vision.
[0:04:11] So this is a state-of-the-art vision system prior to 2012. This is the hog feature, which was a very popular way of doing it. And you can see that the detector has decided this little patch in this image of a duck, this little patch of water, is actually a car. Mistake a human would never make, and you can kind of see why. It sees some stuff that looks a little bit car-like when you run it through this algorithm. That's how it worked back in the day.
[0:04:35] Since 2012, we've had this massive increase, this exponential improvement in our vision system. So this is performance on ImageNet, which is the most popular image recognition data set. And in 2012, Ilya Setskever is one of my co-founders at OpenAI. His collaborators, Alex Krzyzewski and Jeff Hinton, came out with a neural net applied to ImageNet. And we're just able to blow all the previous stuff out of the water.
[0:04:59] Their error rate was 16%. And at the time, people were like, oh, maybe we'll be able to improve this to get down to, I don't know, something crazy like 10%. And we've gone way past there. Actually, this chart continues. And I think the most recent results are sub 2%. Human level in this data set is 5%. So Vision, we've been able to take just one core idea of a deep neural net and just keep making little tweaks, get better understanding of how the existing algorithm works so you can figure out how to train it better and how to get better performance.
[0:05:26] But people looked at that and said, okay, fine, you're able to do perception, but you're never going to be able to solve a really hard problem like machine translation. You just have to understand so much about language in order to make that work. But translation, same story. Translation prior to 2014 was basically stalled. So the blue score is the score of how good your translation is. Used to be that a 0.1 improvement on a blue score was a big deal. That was some real noteworthy work.
[0:05:56] And between 2014 and 2018, just by applying deep neural nets and continuing to make tweaks, we had a nine blue score improvement. So if you've used Google Translate or Facebook's translation service or really any of the big companies' translations recently, they've gotten really, really good. And the reason they've gotten good is because they threw out all the old stuff, you know, 40 years worth of humans thinking hard about how to do it and replaced it with a deep neural network, right? So computer vision, machine translation.
[0:06:22] You can literally have the exact same architecture, the exact same technology, and the neural net doesn't really care what problem it's solving. It just cares what data it's shown. Image generation, this is something that in 2006, Jeff Hinton at University of Toronto had some interesting preliminary results where he was able to generate these little handwritten digits. And 2014, there was an algorithmic advance called the Generative Adversarial Network, and the paper had...
[0:06:52] Generated faces like this. You can see they're kind of very blurry, but they're kind of face-like. And you can start to see some coherence in these generated samples. One year later, Alec Bradford, who's now one of my collaborators at OpenAI, and his collaborators came out with a paper called DCGAN, where they made an improvement on this. And now you can start to see these kind of misshapen celebrity faces, these little bedrooms where you can really see some real coherence starting to emerge there.
[0:07:19] The way that you actually do this is that you just show the neural net 30,000 examples of celebrity faces, and somehow it's able to extract the essence of what a celebrity face is. Two years later, NVIDIA came out with this. So it's the same exact paradigm. It's just a neural net at the core and made a few tweaks, a few improvements, and now you're able to get these celebrity faces that look like celebrities that you've seen, but you can't quite place them because none of these are real celebrities.
[0:07:47] And 2018 DeepMind came out with these results of generating ImageNet samples that are totally indistinguishable, at least to me, from images from the real data set. So none of these are real dogs or wolves. These are all hallucinated by Deep Neural Net. So across all these domains, same core algorithms, same core technology, and improvements in any one of those domains results in improvements across the board.
[0:08:13] OK, so neural nets to generate things, neural nets to do perception. But what about neural nets to actually take action in the world? So 2013, DeepMind, I read some initial results of playing Atari games, which was pretty cool. I was able to learn just by being given a reward. And no one programmed in what you're supposed to do for these very simple games. 2015, we were able to get these little humanoids to run in simulation.
[0:08:40] But no one had ever used reinforcement learning to solve a really hard task. And 2016 was the first time this happened. So you had AlphaGo, which, you know, Go is this game that's been around for millennia, was considered to be just like incredibly hard, has this huge branching factor, and using a deep neural net, DeepMind was able to solve this game. And just this year, we were able to, for the first time,
[0:09:04] I play professional esports at world competitive levels with OpenAI 5, which, again, it's just a neural net. The way that OpenAI 5 works, so this is something that, you know, a project that I've worked on at OpenAI, is you have two neural networks, each one's very, very small, that they use about the same compute as a bumblebee brain.
[0:09:25] And they play millennia of game against each other. So every single day, OpenAI 5 plays about 600 years of real world experience. And you run this off in the cloud at massive, massive scale. And by doing that, you're able to effectively create this neural net that's able to respond to humans and really understand this game in a very, very deep way. But OK, so these are all just games. What about taking your system out of simulation and doing something in the real world?
[0:09:55] So this is, again, work that we did at OpenAI where we took the Dota system, the exact same learning system, the exact same algorithm, and pointed at a robotics problem that no one else had been able to solve. So this task is asking the robotic hand to point this cube in whatever the goal state is showing. So you'll notice that it's going to rotate the cube to have the A facing kind of towards us and the E facing that way. And once it does, it moves on to the next one.
[0:10:25] Robotic hands are something that have existed for a long time. They've been around for about 20 years. The company that makes these sells about 10 a year because no one can program them. But just by learning and simulation, we were able to write a controller that was able to control this physical robot. And the main core of how it works is that we've randomized all these aspects of the simulation.
[0:10:50] You randomize the coefficient of friction. You randomize the size of the block and the thickness of the fingers. And so then once the neural net is pointed at reality, it just treats reality as another kind of randomized simulation. So that's what's been going on in the results front. Those are the things we've been able to actually make work. Before 2012, it was really hopeless and abysmal. And since then, we've just been able to blow through these barriers. And at each step of the way, people say, all right, you made this advance.
[0:11:20] You'll never make it to the next level. And somehow that has not been predictive. And again, at the core, it's all one simple idea of a deep neural network. Now, deep neural networks are powered by compute, right? Compute's almost the rocket fuel of AI. And the thing that is really important and special about neural nets is that they can usefully consume all available compute. And the reason they can do that is right there in the name, right? That neural nets are made up of neurons, so they're massively, massively paralyzable.
[0:11:50] And so we recently did a study at OpenAI looking just across the literature, so just using publicly available information, but synthesized it in a way that no one had done before, and found that the amount of computation that's been going into neural nets over the past six years, into those largest AI training runs, has been increasing at this insane rate that we've just never seen in history. So there's been a 300,000 times increase in the amount of compute between the results of 2012 and the results of 2018. Actually, this is the very beginning of 2018 here.
[0:12:18] And to put that in perspective, because this is just a large number, what does it mean? The way to think about it is like if your phone battery in 2012 lasted for one day, 2018 started to last for 800 years. And the thing is, our projections are that there's about five more years worth of this progress, which means that that would then be like five years later, your phone would start lasting for 100 million years. And so it's just night and day, the kind of computational power that's going into these systems.
[0:12:45] And so another way of thinking about it is with this diagram. So this is very similar to the, this visualization is very similar to those visualizations you've seen where you see a human, you zoom out, you see the Earth, you zoom out, you see the solar system, you zoom out, you see the galaxy. So this is doing an exponential zoom out in terms of the amount of compute used for various results. So up here we have results from pre-2012. Now this is the 2013 result. We start to see the compute for the 2012 results starting to fade out into the distance.
[0:13:12] Now we're on to 2014, and you start to have GoogleNet, so this is a vision system. Sequence to sequence, this is machine translation is now going. Down here, we're starting to see speech, so this is Deep Speech 2. The compute for that is now fading into the distance. 2016, 2017, 2018, still holding pretty steady. I'm sorry, this does take a long time. There's been quite a lot of progress. 2017, this is our Dota initial system. We're now zoomed out from
[0:13:42] 2016 results with neural architecture, neural machine translation. And very finally, we reach our Dota 5v5 system, which we unveiled this year, and the AlphaGo and AlphaZero results. And actually, the most recent version of our Dota system, which we'll be releasing probably early next year, will set a new high point on this curve. And so when it comes to the computation, that our computers, it turns out, have been abysmally slow.
[0:14:11] Right, that the amount of compute used for the largest AI training run of the 90s is about 30, sorry, about five seconds on a modern GPU. And our projections are that the amount of compute used for this thing is going to be about 30, for a month's worth of training this system, is going to be consumed in about 30 seconds of a future AI system. So, it's again the case that...
[0:14:36] There's these two really interlocking dimensions of progress that have really been driving what's happening and that we expect to continue forward, which is, one, the algorithms, the core of a deep neural network with different tricks to apply in different domains to make it work across different modalities. Secondly, coupled with building neural net computers and scaling up these systems very large. Now, it's not to say that we're done, that there still are formidable challenges ahead, and these are things that are going to be hard.
[0:15:05] We still have to solve reasoning and abstraction and things like that. But the thing about all of these is that we have ideas for them. There's only one problem with these ideas, which is they require a lot more compute than we have today. But we're starting to get there. We're starting to be able to prototype these and starting to get promising results. And so what this all works out to is that for the past six years, we've just been punching through barriers repeatedly. And I think that...
[0:15:29] It's always difficult to bet against trends, right? You think about Moore's Law, which is a 40-year trend worth of progress in computers that an entire industry set their watch to. And here, on the compute front, we can really project out. On the idea front, what we know is that our ideas have been working, that they have been able to continue despite all the reasons that they shouldn't. And so I think the real question becomes, where do you want to place your bets?
[0:15:53] And at OpenAI, we know where we want to place our bets. And I think that trying to rule out AGI in the near term starts to become very, very difficult. And what that means is then you need to start thinking about, well, what can go wrong? What are the risks, right? And there's a whole host of different things that we need to think about. And I think that they kind of slot into a few different categories. One of these is machines that pursue goals that are misspecified by the operator.
[0:16:19] So if you can imagine, we already have autonomous systems in this world that sometimes are pursuing a goal with a single-minded fashion and end up with externalities we don't like. For example, a company that's just trying to maximize profits sometimes will go and cause things that we don't like to happen in the world, if you can imagine.
[0:16:39] There's also malicious humans that are subverting deployed systems, which, again, is something that we see all the time today. If you think about hacking and what that will be like with an AI, if you could make it do things that it's not supposed to do. But a third thing that we also should really think about, and I think, again, we're starting to see the effects of this in the world that exists today, is an economy that serves not the needs of everyone, that's kind of growing and generating lots of value, but that that value is either locked up with a few...
[0:17:06] Thank you very much. Appreciate it.