[0:00:09] Breaking the wall to mind machines. How deep learning can give birth to general artificial intelligence. Denis Hassabis, Google DeepMind, London. On the 9th of November 1989, I was 13 and remember watching it live on TV. So I'm going to talk about artificial intelligence, specifically what we call at DeepMind general artificial intelligence.
[0:00:41] So AI is basically the science of making machines smart. And I started my journey towards AI with games, specifically board games, including chess. So I started to play chess when I was four years old and captained various England junior teams. And that got me on the path to thinking about thinking. How is it that our minds can come up with these moves and win these games of chess?
[0:01:10] So for a very young age, I was puzzling over these things. And then when I learned how to program and got into computers as I got older, my love of games and computers naturally combined into designing and making video games. So I did this from my teenage years for about a decade, and I wrote many sort of number one best-selling games, including Theme Park, probably my most well-known game. Now, all the games I wrote...
[0:01:37] had AI as a core gameplay component. So for example, in Theme Park, the idea of the game was that you designed your own Disney World, and then thousands of little people came in to your Theme Park and decided on whether you designed it well and how much they enjoyed it. And because the AI was powering the simulation underneath the game, that meant that every player playing the game had a unique experience. So no two games were the same, because the game with the AI adapted.
[0:02:06] to the way that the player played the game. Then after a decade of working on computer games, I decided to go back to academia and study neuroscience for my PhD. And really, this was the final part of the puzzle for me in terms of launching an effort to solve AI. I specifically looked at areas in neuroscience, like imagination and memory, where we were very bad at understanding how that worked in machine learning and AI.
[0:02:34] So I wanted to gain inspiration from how the brain solves some of these hard problems of intelligence to come up with new inspirations for new types of algorithms that we could implement. So all these experiences culminated then in 2010 in me co-founding DeepMind. And you can think of DeepMind as an Apollo program effort, if you like, for neuroscience-inspired AI. And what we've done is collect together well over 100 of the world's best research scientists in this area.
[0:03:03] And put them all together with a whole bunch of very brilliant engineers to try and make as fast advances towards AI as possible. So while we're experimenting on artificial intelligence technologies, we're also experimenting actually with our organization and a new way to do research and science. And our mission, the way we articulate it, is to try and solve intelligence and then use that technology to make the world a better place.
[0:03:32] And I'm going to talk a little bit at the end about the different ways that I think AI is going to impact the future. So how are we going to do this? Well, our sort of more prosaic mission is to try and build the world's first general purpose learning machine. And the two key words here I would stress are learning and general. So all the algorithms that we're interested in building at DeepMind learn automatically from raw experience. They're not pre-programmed or handcrafted in any way. They learn directly from the raw data. And they're also general.
[0:04:04] In the sense that a single set of algorithms or a single system can operate out of the box across a wide range of tasks. And of course, we have an example of such a system. It's the human mind. But most AI that we see around us today, AI is now a huge sort of buzzword. It's very fashionable. But actually, most AI around us is not of this kind.
[0:04:28] The AI, for example, for our phones or cars or even for various internet and games things are actually usually bespoke pieces of software that have been handcrafted for one particular task. They don't learn or adapt and they're not flexible. So we call the type of AI we work on artificial general intelligence to distinguish it from this normal type of AI.
[0:04:52] And I think the difference between general AI and narrow AI becomes very clear when we look at one of the most famous watershed moments in AI research. So back in the late 90s, IBM's Deep Blue beat Garry Kasparov very famously at chess. Now the thing was, of course, this was a very impressive technological feat.
[0:05:11] But actually, I came away from that match much more impressed by Gary Kasparov's mind than the machine itself. Because, of course, Deep Blue, whilst it was amazing at chess, can't do anything else. Whereas Gary, of course, can speak multiple languages, drive cars, and do all these other things with his mind, as well as play chess.
[0:05:29] And I think you can see that in some sense, Deep Blue is not intelligent in the way we would normally regard intelligence. Because, for example, Deep Blue can't even play a much simpler game like tic-tac-toe, which is strictly simpler than chess. Nothing in its programming or its knowledge base would help it play that much simpler game. Even though it was obviously world champion level at chess, it would have to be reprogrammed from the scratch. So in some sense, this shows there's a bit of a dichotomy here in terms of what is intelligence.
[0:05:59] So instead of that, we view intelligence in the framework of what's called reinforcement learning. I'm just going to quickly outline that for you with this diagram. So on the left-hand side here, we have the agent system. And the agent system finds itself in some sort of environment. And it's trying to achieve a goal in that environment. Now, that environment could be real world or it could be virtual. If it's real, the agent is likely to be a robot. If it's a virtual environment, it's likely to be an avatar. And the agent interacts with the environment in two ways.
[0:06:27] One is that it gets noisy, incomplete observations through its sensory apparatus. We normally use vision, but we could use other modalities. And the job of the agent system is twofold. One is to make the best possible model of the environment out there based on these noisy observations and update that in real time, and then use that model as a prediction engine to decide what action it should take now that will best get it towards its goal.
[0:06:55] Now, although this diagram is quite simple, it hides a lot of complexity. But if we could solve everything behind this diagram, we know that would be sufficient for general intelligence. And we know that because biological systems, including all mammals and humans, use some form of reinforcement learning to learn. In the human brain, it's the dopamine system that implements a type of reinforcement learning. So I'm just going to show you a couple of videos of the system at work. But before I do, I just want to make sure you're clear on what it is you're going to see.
[0:07:25] The second thing we believe in other than reinforcement learning is that a true thinking machine has to be embedded in a sensory motor stream, has to be embedded in a sensory motor reality. But we use actually games as a testing platform for developing our AR algorithms.
[0:07:43] And we think it's a kind of perfect platform for that because for many reasons. One is there's unlimited training data. Of course, you can run these games for as long as you like. There's no testing bias because they've been designed by other people, not the AI creators. You can test millions of agents in parallel. And progress is very easy to measure because most games have game scores.
[0:08:04] So what we did was started off with classic 8-bit Atari games from the 80s. And here the AI system just gets raw pixels as inputs. And the goal is to maximize the score, and everything else is learned from scratch, directly from the pixels on the screen. It knows nothing else about the game, what it's controlling, or what it's supposed to be doing. And then we have this idea of generality, so a single system to play all the different games.
[0:08:29] So normally I would show lots of videos here, but I only got time to show one, but I think it's particularly appropriate for falling walls, because actually this game, Breakout, is about breaking down walls. So here the AI controls the pink bat and ball at the bottom of the screen, and it's trying to break through this rainbow-colored brick wall, brick by brick.
[0:08:48] So I'm going to show you the agent getting better progressively over the games it plays. So this is after 100 games, so not very many games. And you can see that the agent is mostly, you know, it's not very good yet. It's missing the ball, but it's starting to get the hang of the idea that it should move the bat towards the ball. Now after 300 games, the agent is now about as good as any human can play this. So it almost always gets the ball back. So we thought this is pretty good.
[0:09:15] But what happens if we left it playing for another couple of hundred games? And then an unexpected thing happened. Basically, the system discovered the optimal strategy was to dig a hole around the sort of left-hand side here and tunnel around to the back of the wall and send the ball around the back. So the funny thing about that was that's the optimal strategy, but although the researchers on that who created that are amazing machine learning researchers, they're not very good at playing Atari. So they actually learned something from the program they created.
[0:09:44] A better strategy. So if you're interested in this work, this was in the front cover of Nature back earlier this year, so you can read the details in there. And now we're moving on to 3D games, games that go, and robot simulations and further in the next few years. So here's just a quick video of now a 3D game. Here, the same system you saw playing the Atari game is now learning to drive a racing car around the track. Just again, just from the pixels only.
[0:10:12] So it has not been told anything about driving or what it's controlling. You can see it's driving quite happily at sort of 200 kilometers an hour around the racetrack. So now we're moving on to all sorts of new areas. And they're all inspired, and we look to systems neuroscience for clues as to how the brain actually solves these problems. So memory, attention, concepts, planning, navigation, and even things like imagination. And as we're here at Falling Walls, I was thinking about our work.
[0:10:41] What sorts of walls does it bring down? I think there are at least three, actually. So firstly, I think in science, the biggest advances in the future, the next few decades, are going to come from combining two different fields together at a very deep level. Obviously, for us at DeepMind, we're doing that by synthesizing machine learning with systems neuroscience.
[0:11:01] The other thing that we do at DeepMind is try to blend the best from two sorts of cultural organizations. So one is take the best from startups, the energy and buzz that they have and focus, and blend it with what's best from academia, the long-term thinking and collaborative nature of academia. And try and bring this together to create a new environment that makes scientific research more efficient and more productive and enhances collaboration and cooperation.
[0:11:30] And then finally, I think the third wall we're bringing down is really in the applications of what we're doing with AI and what other people are doing with AI, applying it to many, many industries where there's a lot of data. And I'm especially excited about healthcare in that regard.
[0:11:45] So finally, I just want to end by thinking about the bigger picture. Why do I think AI is so important? Why have I spent my whole career working on it? Well, I think there are two big issues facing us as a society today. One is information overload. We're just deluged as individuals, but also as scientists and business people by the amount of data that's incoming into us every day. And you can see that everywhere.
[0:12:07] From genomics to entertainment and so on. Now, personalization technologies are one way to try and deal with that, but they don't work very well because generally speaking, they're based on technologies like collaborative filtering, which is really about averaging the wisdom of the crowds. It's not about tailing it to the individual.
[0:12:23] And the second thing is systems complexity. All the big systems we would like to master as a society, climate, disease, energy, macroeconomics, even particle physics, are now getting so complex, even the greatest team of human scientists are finding it difficult to master and comprehend these systems unaided.
[0:12:41] So I think solving intelligence and solving AI is potentially a kind of meta solution to all these other problems. If we can solve AI, we can use it to help human scientists work side by side in a complementary way to help them make bigger breakthroughs. So I dream about using this kind of general AI to make AI-assisted science possible.
[0:13:06] But of course, as with all powerful technologies, and AI is no difference in this regard, it must be used ethically and responsibly and deployed responsibly. And even though human-level AGI is decades away, we should start the debate on this now, and indeed we're supporting that with various academic institutes and conferences to talk about the technical and ethical issues behind this type of technology. And as a neuroscientist...
[0:13:31] And us working on neuroscience-inspired AI. I'm very excited about the idea that this journey that we're on, trying to distill intelligence into a better understanding of the mysteries of the human mind, and comparing that to the human mind, will allow us to better understand the deep mysteries of ourselves, including things like perhaps dreaming, creativity, and maybe even consciousness. Thanks for listening.