[0:00:18] So I go to a lot of startup pitch events, and last year, every startup had to be a big data startup, no matter whether they were making fishing rods or sandwiches. This year, every startup has to be a machine learning startup. The people who are doing this for real, though, are going to be on this stage in the next hour. So after the first session, we get David Kenney, General Manager IBM Watson, who's going to reveal a few interesting things. But to start with,
[0:00:51] A man who is co-founder and CEO of a business that Google bought a couple of years ago for 400 million pounds, as reported, building general artificial intelligence, which is the big one. Please welcome to tell us a bit more about some specific applications before we have a deeper conversation, Demis Hassabis of DeepMind. Hi, everyone. It's really great to be here.
[0:01:28] As David said, I'm going to be talking about general artificial intelligence, and I'm going to talk a little bit about the latest work that we've been doing, and then talk at the very end about some of the applications we plan on. So first of all, DeepMind, as David was saying, we founded the company in 2010, and then a couple of years ago, we joined forces with Google to sort of turbocharge our mission.
[0:01:56] The way we think about DeepMind is a kind of an Apollo program for AI. So we have around 300 people and more than 200 research scientists. And one of the things that we're trying to do here is to find a new way of organizing science. To try and combine the best from Silicon Valley style startups and the best from the top academic places. And actually combine that together into a unique culture that allows us to make research breakthroughs faster than ever.
[0:02:30] So our mission, the way we articulate it, is in two steps. Step one, to try and fundamentally solve and understand intelligence. And then if we can do that, we believe that we can use that technology to try and apply it to almost everything else. So we actually feel that artificial general intelligence could be one of the most important technologies that humanity ever invents. And that's why we work so hard, and I spent my whole career working on this project.
[0:02:58] So how do we do that more prosaically at DeepMind? Well, we try and build general purpose learning algorithms. So what does that mean if we break that down? So all the algorithms that we work on at DeepMind have two things in common. First of all, they learn automatically how to do things, how to master tasks directly from raw inputs and raw experience. So they're not pre-programmed or handcrafted in any way. The second thing is the idea of generality.
[0:03:29] This is the idea that the same system or same set of algorithms can operate across a wide range of tasks, including new tasks that the algorithms have never seen before, and operate correctly in those new environments. So we call this kind of AI artificial general intelligence. And the hallmark of AGI is that it's flexible, adaptive, and potentially even inventive. So it can come up with things that the programmers themselves didn't know how to do.
[0:03:58] Because it's directly learning for itself how to master these tasks. If we compare that to most of what's called AI out there at the moment, which we call narrow AI at DeepMind, most of the AI that you interact with every day is actually of a different sort of AI. It's handcrafted and a special case for one particular task. So because of this, most of the AI you interact with today is quite brittle.
[0:04:25] If you do something that's unexpected that the programmers didn't cater for, there's no way that the AI can deal with that unexpected event. And of course, humans are very unpredictable in the way that they use devices. So there's always something unpredictable happens that the programmers couldn't cater for beforehand. So we think this is a big problem in AI and why we work on these kinds of general learning techniques. So probably still the most famous example
[0:04:52] of a narrow kind of AI is IBM's Deep Blue, which famously beat Garry Kasparov in the late 90s at chess. And it's an incredibly impressive feat, but Deep Blue itself was only built to do one particular thing, play chess. It couldn't even do something much simpler, like play a much simpler game like Norton Crosses, without being totally reprogrammed from scratch.
[0:05:17] So nothing in that knowledge base that Deep Blue had about chess would be useful for anything else. So this is a classic case of a narrow kind of AI. Now after Deep Blue beat the world chess champion, the next kind of holy grail or grand challenge for AI research in games was another game, a much more complex game called Go.
[0:05:41] Now, I'm going to talk, for the rest of this talk, about AlphaGo, a program that we created very recently and was one of our big breakthroughs, our most sort of recent breakthrough. Now, AlphaGo is a program to play this game called Go. Now, for those of you who don't know, this is what a Go board looks like. It's a 19 by 19 grid. The board starts empty. It's played between two players, and they take turns to put down these stones, these pieces, black and white stones.
[0:06:10] And they put the stones down on the vertices of the board. And the idea of the game is to surround your opponent's pieces or surround empty territory. And the player that has the most territory wins the game. Now, the rules of Go are very simple, but it leads to incredible complexity. And in fact, Go has a very long and storied history. It's what they play in Asia instead of chess. So it was originated in China 3,000 years ago.
[0:06:40] More than 40 million players play it today. And there's 2,000 professionals. So there are professional Go schools that you go to as a child if you live in Korea or Japan or China, if you show promise in the game of Go. And you play Go seven days a week, 12 hours a day. And one idea of how the sort of complexity of Go, why it's so much more complex than chess, one way to illustrate that is that there are more possible board positions in Go than there are atoms in the universe.
[0:07:09] 10 to the power 170. So even if you were to use all of the compute power on the planet and run it for a million years, you wouldn't have enough compute power to calculate and brute force everything about the game. So this is why trying to crack the game of Go has taken 20 years beyond solving chess, beyond Deep Blue. So the two really big reasons why Go is hard for computers to play is that, as I've just said, the complexity is actually enormous.
[0:07:41] It's probably Go is one of the most profound games mankind has ever devised. So from these very simple rules comes this enormous and beautiful complexity. And this complexity makes brute force search, the type of search that was used in Deep Blue, intractable. So you can't use those kinds of approaches that were used for things like chess computers. They simply won't scale to Go. So there's two main challenges if you break it down. One is the search space is enormous.
[0:08:10] And the second thing that's even harder is that it was thought to be impossible to write an evaluation function for Go to tell the system which player, black or white, was winning in a particular position. So this is incredibly hard for Go. And I'll just delve a little bit deeper into both of those two problems. So in Go, in any particular position, there are an average of 200 possibilities, 200 possible moves. In chess, by contrast,
[0:08:40] In an average position, there are only 20 moves. So Go has got an order of magnitude bigger branching factor. So that's what leads to this huge number of possibilities. So it's much harder to figure out what you should do next if you're going to consider all the different possibilities. And the second thing is the evaluation function, telling the system, how does the system know whether a position is desirable for black or for white?
[0:09:07] Well, in chess, the way you can do that is you can handcraft a specific set of rules. You can talk to chess grandmasters, and you can figure out how they make decisions, and then you can codify that in a sort of database of rules. And you can use that database to tell you who is winning the game. Now, that's easy in chess because chess has a concept of materiality. So, for example, if you want to, as a first approximation, you can just add up the value of the pieces on the board.
[0:09:37] So the queen is worth nine points, the rook is five points, the knight is three points, and so on. And whichever side has the most value on the board, the first approximation is winning the game. So that's already a very useful heuristic. But in Go, there's no concept of materiality. Every piece is worth the same. Another issue with Go is that Go is a constructive game. So what I mean by that is that the board starts off completely empty, and the position gets built up piece by piece.
[0:10:07] So the game gets more complex as time goes on. If you compare that to chess, chess is a destructive game. So all the pieces start on the board, and as the game goes on, it simplifies. Pieces are taken off the board. So what this means is in Go, if you want to evaluate a current position in the middle game, you have to involve predicting the future, how the game might go. Whereas in chess, you can assess the current position in isolation. You don't have to worry about the future so much.
[0:10:37] And the final issue with Go is that very small changes, one piece moving by one position can actually change the whole evaluation of the position. So very small changes can make a big global difference. So I just want to take a moment before I show you the program as to how do humans deal with this kind of complexity? Because, of course, humans play this game to a very, very high level and have been playing it for 3,000 years.
[0:11:04] And actually Go is thought of as much more of an intuitive game than a calculation game like chess. So top players rely on their instincts and intuition to figure out all these complexities and feel the way to the right move rather than calculate out the right move like a chess player would. And in this sort of 3,000 year history of Go, very rich history, there's this concept of what's called a divine move. A move so profound.
[0:11:30] That it's not understandable by humans. And it sort of comes from a divine inspiration. And top Go players, nine damn professional Go players, spend their whole careers hoping to play one move that's worthy of the name. And that idea of this sort of divine move is unique to Go. So how do we overcome these issues and build AlphaGo? Well, we turn to a number of new technologies that we've been using at DeepMind, especially deep neural networks.
[0:12:02] which are all the rage right now called deep learning. What we did first is we trained two neural networks and we started off by downloading 100,000 games from the internet, Go servers. So these are games played by human amateurs, strong human amateurs on Go servers in Japan and Korea. And we trained our first neural network, which we called the policy network, to mimic the behavior of the human players. So we gave it a position and we told this network, we trained this network to
[0:12:32] Figure out and copy what the human player would do in that position. Now, the thing is, of course, we don't want to just mimic strong amateurs. We want our program to be stronger than the best players in the world, the best professionals in the world. So the next stage is we took this first network and we had it playing against itself millions of times on our servers. And each time, learning through its own mistakes and learning incrementally to make fewer and fewer mistakes with each game.
[0:13:01] And we used a technology called reinforcement learning to do that, where with each game, it updated its neural network depending on whether it won or lost. And after we have this new neural network, we freeze that network, and we play that final network 30 million more times against itself. And that gives us what we call our goal data set, the final data set that we need.
[0:13:27] And we use that data set to train a final neural network, which we call the value network here in pink. This is a second neural network. And what this network does is it takes a position, a board position from Go, and we know from the 30 million games that we've played out who ends up winning the game from that position. And we train this new value network to predict from a particular position who will win the game and by how much.
[0:13:54] So this network, this network in pink here, the value network, is the fabled evaluation function that people thought would be impossible to program for Go. And what we did instead is we sidestepped that problem by having a neural network learn how to evaluate a position rather than us programming it with the definitions. So we took these two neural networks and we applied it together into a search tree. So this first neural network, the green one,
[0:14:24] AlphaGo takes a board position and outputs probability distributions around which moves are most likely to be played in the current position. So you see here on the green board, the higher the green bar, the more probable that move is. AlphaGo thinks that move is. The second neural network, the pink network, takes the board position and outputs a single real number between 0 and 1, which gives the prediction about
[0:14:50] Which side AlphaGo thinks is winning? So 0 is white, 1 is black. And by how much? So if it's 0.5, it means that AlphaGo thinks that the game is even. And just pictorially, what you can see is this cuts down this enormous search space of possibilities that I already told you is more than the atoms in the universe. So if you imagine that AlphaGo is trying to make a decision at the top of this tree, how is it going to make a decision about what to do next?
[0:15:18] Well, what it does is it calls these two neural networks, the policy network first, the green network, to narrow down the breadth of the search. So instead of searching the 200 possibilities from every position, it only searches the top four or five possibilities. And then secondly, it uses the value network, the pink network, to cut down the depth of the search. So instead of having to search all the way to the end of the game to find out who won, it can call the evaluation function at any time to tell it, give it an estimate of who is winning.
[0:15:48] So after we built this AlphaGo program, it was time to test it against the best of the current Go programs, which are built using traditional methods. And we found that AlphaGo won 99% of its games against all these other programs, some of which people have been building for more than 10 years. So then it was time to challenge some of the best humans in the world.
[0:16:11] So we then took on the European champion, Fan Hui, who is three times European champion. He was born in China, became professional when he was 16, and has won the European Championship three times. And this is what happened.
[0:16:45] I fight all the time. Now it's a problem. Now it's a problem. But I lose all my games. So Fan Hui, AlphaGo won 5-0, which is the first time that a Go program has ever beaten a professional. So this is around about a decade before people were expecting. The experts in the field, AI experts and Go experts, a whole 10 years before this was going to happen. So this was published as a front cover article in Nature.
[0:17:20] And it caused a huge stir in the machine learning community. So in fact, it's one of the top 20 most downloaded papers ever in Nature, which is the top science journal in the world. So then we took our final step, which was to take on the world champion, a South Korean grandmaster called Lee Sedol. He's a legend of the game. He won 18 world titles and is considered to be the greatest player of the past decade. So the way I describe him is he's kind of like the Roger Federer of Go.
[0:17:47] And we took him on in a $1 million challenge match in Seoul earlier this year in March. It was absolutely a huge event. It was an incredible press conference with thousands of journalists. The whole country came to a standstill, and all of Asia were watching the game. And incredibly, we managed to win 4-1, and AlphaGo won the match, and it was an amazing experience. And we were able to beat the top player in the world.
[0:18:15] By four games to one, which was incredibly unexpected. But not only did AlphaGo win, it played some incredibly creative moves. And I just want to show you one example of those moves. So in game two, this is the position in game two. And AlphaGo is playing black here. And AlphaGo on move 37 played this move here that's marked by the white triangle. And I just want to take a minute here to explain to you why this move was so astounding to the Go world and why it was considered to be so creative.
[0:18:44] So Go, as I said, has been going for 3,000 years. Humans have been studying this game for 3,000 years. And there are two important lines on a Go board. The third line, which is marked here, if you play a piece on the third line, what you're trying to do is capture territory on the side of the edge of the board. Instead of that, if you play on the fourth line, what you're basically saying to your opponent is you're trying to get power and influence into the center of the board.
[0:19:10] And for 3,000 years, the received wisdom has been that playing on the third and fourth lines and exchange on the third and fourth lines is equal for both players. But what AlphaGo decided to do was to play on the fifth line and take power into the center. And this is completely unheard of and against all the principles of playing Go, the known principles of playing Go. So for a professional, this would be unthinkable.
[0:19:35] And in fact, about 50 moves later, it turned out that this move ended up affecting the fight in the bottom left-hand corner of the ball. These two pieces over in the bottom left-hand of the ball. So completely on the other side of the ball. So this is what the professionals thought about it. That's a very surprising move. I thought it was a mistake.
[0:20:03] So those are the commentators, the nine Dan professionals, commentating live on the game. And he was so surprised about the move, he thought that our computer operator had misclicked on the computer screen. That's how astounded he was. And, you know, you could see him, they didn't even know where to put the piece.
[0:20:31] So we think that this move 37 is going to go down in history. And famous moves in Go end up getting names and books written about them and all sorts of things down the years. So this move has been analyzed now by the whole Go world for the last three months. And of course, Lee Sedol himself.
[0:20:52] Then did his own incredible move in game four, this move 78, which will also go down in history, and I don't have time to go into today, that surprised the computer, AlphaGo, and he won game four because of it. Again, another incredibly intuitive and creative move. What's really interesting is now the culture impact of this match. 280 million people watched the five games across Asia and the rest of the world. 60 million in China just for game one.
[0:21:20] And there were 35,000 press articles written about the match during the week of the match. And the thing I'm most proud of, actually, is there was worldwide, online, there were sold out of Go boards and Go pieces because so many people around the world wanted to learn and try this great game. Another interesting postscript is that Lee Sedol...
[0:21:41] himself felt that he'd improved because of the match and come up with new ideas. This is a really interesting thing of machine learning and AI helping to improve human performance as well. And since this match, Lisa Doll has gone on an incredible winning streak himself against all his professional opponents. And he said many interesting quotes about how AlphaGo has allowed him to think about new ideas about how to play Go and rejuvenated his passion for the game.
[0:22:11] So I just want to end here by talking a little bit about intuition and creativity. So I've talked a lot about Go being a very intuitive and creative game. So what do I mean by that? So one way of operational sort of definition of intuition is that it's implicit knowledge that's acquired through experience, but that's not consciously accessible or expressible. So you can ask, well, if you can't access it consciously, then how can you know that that knowledge is really there?
[0:22:40] Of course, you can test for this implicit knowledge and that you can test for the existence of it and the quality of it by behaviorally testing for it. So for a Go player, you can give them a Go position and you can see the quality of the move and the output that they decide to play. And that tells you how good their implicit knowledge, their intuition is. What about creativity? Well, creativity, I think you can operationalize as the ability to synthesize knowledge to produce a novel or original idea.
[0:23:10] I feel clearly demonstrated these abilities, albeit, of course, within the limited domain of Go. So just to end about the critical difference, circling back to what I said at the beginning between narrow AI and general AI. And therefore, what is the difference between AlphaGo and Deep Blue? Both of these programs beat world champions at their respective board games. But although AlphaGo is a big achievement, the key point here is how we did it.
[0:23:38] So Deep Blue was pre-programmed with specialized handcrafted knowledge that was distilled from chess grandmasters. Whereas AlphaGo used general purpose algorithms, these neural networks, and reinforcement learning to learn how to play Go and improve itself. It's a modular system that combined several different aspects together, deep learning, reinforcement learning, Monte Carlo tree search. So it combined pattern recognition with planning.
[0:24:04] And I think that's what we're going to see in the future with AI algorithms, combining the strengths of different types of algorithms. And because of this, AlphaGo thinks and plays in a much more human-like way.
[0:24:15] So now, because of these general purpose algorithms, of course, the aim of DeepMind is not just to play games. We use games as a great testing platform to develop our AI algorithms. But ultimately, we want to apply these technologies to have real-world impact on very important problems. And we're looking right now at applying these to things like healthcare, robotics, and conversational personal assistance. Thanks for listening.