[0:00:11] Good evening everyone, I'm Matthew Taylor, I'm Chief Executive of the RSA and it's my enormous pleasure to welcome you here for this evening's special event. Can you make sure your mobile phone is switched to silent? We're filming tonight and live streaming over the web, so welcome to everyone who's watching online. The hashtag is RSAAI if you'd like to get involved in the conversation on Twitter.
[0:00:38] So housekeeping notice is over. It's my enormous pleasure to introduce this evening's distinguished guest speaker, Dr. Demis Hesavis. Demis is the co-founder and CEO of DeepMind, a neuroscience-inspired AI company acquired by Google in 2014, and he leads the general AI efforts of Google, including the development of AlphaGo, the first program to beat a professional player at the game of Go.
[0:01:03] Demis is a former child chess and coding prodigy. On graduating from Cambridge, he founded the pioneering video games company, Alexia Studios. After a decade leading successful technology startups, he returned to academia to complete a PhD in cognitive neuroscience at UCL, followed by postdocs at MIT and Harvard. His research, connecting memory with imagination, was listed in the top 10 scientific breakthroughs in 2007 by the journal Science. He's also five times World Games champion, the recipient of the Royal Society's prestigious Mollard Award, and I'm delighted to say,
[0:01:32] A fellow of the RSA. Demis has described his DeepMind project as, quote, an Apollo program for the 21st century. And in his talk for us this evening, he'll offer us some of the latest insights in the frontiers of artificial intelligence research. Just as we are here at the RSA, Demis and his team are interested in how we harness the incredible accelerating power of technology for the benefit of all humanity, not just an elite.
[0:01:58] And tonight, we're delighted that Demis will focus on AI's potential to help solve some of our greatest global challenges from healthcare to climate change. Demis is going to speak for 20, 25 minutes. We'll have a conversation up here and then we'll bring you in for questions. So without further ado, please join me in welcoming Dr. Demis Hassabis. Thanks Matthew for that introduction. It's a great pleasure to be here. I'll always love visiting the Royal Society of Arts.
[0:02:34] You know, it's great to have these kinds of dialogues between the sciences and the arts. So I'm going to talk about artificial intelligence and what's sort of happening at the cutting edge of artificial intelligence through the lens of the work that we're doing at DeepMind. So DeepMind was founded in 2010 and we joined forces with Google in 2014. And one way you can think about DeepMind, one way I describe it is as an Apollo program effort for AI.
[0:03:01] And what that means, what we sort of mean by that is to bring together the world's best scientists and best engineers and put them in a kind of perfect environment surrounded by all the resources they need to try and make as quick and a rapid progress on the topic of AI as possible. So we're around 350 people now and over 250 research scientists working at DeepMind. So it's probably the biggest collection anywhere in the world of brain power focused on this issue.
[0:03:32] AI in terms of pushing the boundaries of AI research. Another thing that we're sort of experimenting on is also a kind of new way of organizing science. And I haven't got time to sort of go into that today. That would be a whole other lecture in itself. But you can sort of, the summary is really trying to create a hybrid organization that combines the best from the sort of top Silicon Valley startup culture, the focus and the buzz and the energy they have.
[0:03:58] With the best from academic institutes when they're operating very well. Kind of blue sky thinking, interdisciplinary work, and creative thinking that they encourage. And trying to meld together the best from both of those worlds. So our mission, the way we describe it normally is sort of articulated as a kind of two-step process. So step one, fundamentally solve intelligence. And then step two, we think that...
[0:04:24] It would naturally follow, if you did step one, that you could use that power of that technology to practically help us as a society solve everything else. Now, that might seem a little bit fanciful and far-fetched, but I hope by the end of this talk, I'll have convinced you that at least it's plausible that this could happen. So in terms of approaches to AI, I think there's at least four dimensions that it's worth thinking about AI in.
[0:04:53] And then the different types of AI and the different ways different industrial groups and academic institutes approach trying to build AI kind of spans these four dimensions. So the first and probably most important one is the idea of learning systems, systems that learn directly from their experience or from raw data versus handcrafted heuristic systems. So systems where they've been specifically pre-programmed with a particular solution to a problem.
[0:05:23] So there's learning versus handcrafted. Second dimension is the idea of generality versus a special casing or specific purpose. So what many systems are is they're handcrafted and they're built for one particular purpose in mind. What we're interested in at DeepMind is the idea of generality. One system that out of the box can do a wide range of tasks.
[0:05:51] The third dimension that we think a lot about at DeepMind is the idea of groundedness versus logic-based. And what we mean here is that we think that for a true thinking machine to be able to think about and achieve high-end tasks, they need to be grounded in a sensory motor reality. So they have to experience the world around them through their senses and ground the knowledge that they acquire grounded in this sensory motor experience.
[0:06:19] And opposed to that are logic-based systems or symbolic systems that are hand-coded. And the problem with those systems is when they interact with the outside world, with real events, they find it very difficult to map the logical knowledge they have to these real-world messy situations that they find themselves in. And in fact, a lot of the AI research that went on in the...
[0:06:43] 80s and 90s at places like MIT were actually based around these logic-based systems and expert systems. And they only got so far and then they couldn't encode things like common sense because they weren't ultimately grounded in perceptual experience. And then the fourth dimension we think a lot about is the idea of active learning versus passive observation.
[0:07:11] So a lot of AI systems that you'll be using today, that you use every day, like image recognition or voice recognition, they're kind of passive observation systems. They get in some data, and then they try and classify that data in some sense. What we're interested in instead is active agents. So agents that have a goal in mind, that actually have actions they undertake, and direct their own learning. So actually decide what they should explore.
[0:07:41] So at DeepMind, we are kind of committed to sort of the left-hand side on all four of these dimensions, these continuum. And so what we're interested in sort of building is general purpose learning systems. So systems that are at their heart, they learn how to do things and master tasks rather than being given the solution by the team of programmers.
[0:08:05] And this kind of AI is often dubbed artificial general intelligence to distinguish it from the normal everyday AI that's out there. Now, one of the biggest and obviously most famous sort of watershed moments in AI history was when in the late 90s, IBM's Deep Blue beat Garry Kasparov. And of course, this is a huge technical feat, very impressive technical feat. But actually, Deep Blue was an example of one of these
[0:08:35] What we would call narrow AI systems that were special cased and pre-programmed for one particular purpose. In this case, playing chess. And the way Deep Blue worked was by a team of very smart programmers working hand in hand with a team of very smart chess abilities in chess to end up playing the game of chess very well. But Deep Blue, although it was incredible at chess, it was not able to do anything else. So not even...
[0:09:03] For example, play a much simpler game like Noughts and Crosses. None of the knowledge that it had was useful for anything else. It would have to be pre-programmed with a whole new set of rules and heuristics in order to do anything else. So in some sense, this was slightly unsatisfying from an AI point of view. In a way, is that really intelligence? Where does the intelligence of that software reside? It's not really in the machine or the algorithm. It's actually in the mind of the programmers who solved that problem.
[0:09:33] So we wanted to go sort of beyond this kind of narrow AI and work on this kind of general purpose learning system. So we came up and pioneered with a technique called deep reinforcement learning, which has now become all the rage in AI research.
[0:09:53] And we've been working on this sort of since the beginning, since the founding of DeepMind. And it's sort of combining two techniques together. Deep learning, which is hierarchical neural networks that are used to perceive the world around them. So this is what's, if you interact with any image recognition through photo search on your phone or voice recognition by speaking into your phone, it will most likely be using deep learning systems at the back end.
[0:10:18] We combined that with another technique called reinforcement learning, which is about trying to select the right action from the set of available options to you at that moment that will best get you towards its goal. So we combined these two techniques together and scaled them up. Now we used computer games as a training platform for developing and testing our AI algorithms.
[0:10:43] Partly that was my background from computer games. That's what I used to do prior to DeepMind was design and program AI for computer games. And I realized that virtual environments, including games, are a much more efficient way to test out the capabilities of your AI systems than, say, using something like real-world robotics, which are much slower and messier and more expensive to deal with.
[0:11:08] So we're very interested in robotics as an application, but we don't usually use it as a development platform. We use these virtual environments. And you can see why they'd be much more efficient, because you can obviously run millions of simulations at once in the cloud and train your algorithms on a much more fast iteration. So it gives us much quicker feedback as to whether these algorithms are working.
[0:11:32] Another important thing about games is that when you're on a very long-term mission like we are, you know, multi-decade mission, then it's even more important to know in the short term that you're heading in the right direction. And because games lend themselves very nicely to measuring progress in the terms of scores, or it's often very easy to have a proxy to a score if there isn't a direct score, you can know very quickly if the changes you've made to the AI algorithms are actually heading you in the right direction.
[0:12:02] So we started with probably the most iconic of the game consoles that really started off the whole gaming boom, which was the Atari 2600 computer from the 80s. And there's lots of very good emulators now for these, and we took one of those open source emulators and sped it up. And then we used it as our proving ground for our initial algorithms. So we took 50...
[0:12:27] Classic 8-bit games. For those of you old enough in the audience, you might remember some of these. Some of the most iconic games ever, like Space Invaders, these kinds of games. And before, I just want to show you a video of how these agents perform. But before I do that, I just want to explain what it is you're going to see. So the agents here, all they get as their input are the raw pixels on the screen. So it's around 30,000 numbers per frame, because the screen is 250 by 150 pixels in size.
[0:12:54] And the goal here that we set the agent is to maximize the score. Everything else is learned completely from scratch, from first principles. So it doesn't have any idea what it's controlling, what gets it points, what the rules of the game are. It doesn't even have any idea about how video streams work. So the idea that pixels next to each other are correlated in time. It has to find and learn about all this structure for itself from first principles by experimenting and experiencing the game, by playing the game. And then we add additional constraint.
[0:13:24] That we want one system to play all the different games out of the box. So the same system with the same parameter settings can actually master all these 50 different games. Some of them very, very different in terms of their objectives and the way they look. So this is the notion, again, of generality coming in. So I'll show you this one. I've only got time today to show you one video. And I'll show you my favorite one, which is from a game called Breakout. Our algorithm is called DQN. And in this game, Breakout, the player plays...
[0:13:55] You control this pink bat at the bottom of the screen, and there's this little pink ball, this pixel ball, that's bouncing off this rainbow-colored wall. And what you've got to try and do is knock out the bricks from the rainbow-colored wall brick by brick, but not let the ball go past your bat. If the ball goes past your bat, you lose a life. And what I'm going to show you now as I roll the video is the agent getting better over time after it's played a certain number of games, and it will carry on improving as it gets more and more experience with the game. So this is after 100 games.
[0:14:26] So it's not very good yet at playing the game, but it started to get the hang of the idea that it should move this bat towards the ball, and that letting the ball go past the bat is probably a bad idea. Now, after 300 games, so another couple hundred games, now it's almost perfectly mastered the game and can kind of play it as well as any human could play this, and it gets the ball back most of the time, even if it's coming back at very fast angles.
[0:14:51] So we thought that was pretty cool, but what would happen if we let the agent play for another 200 games? And what it did was this unexpected, it found that the kind of best strategy was to dig a tunnel around the left-hand side of the wall and send the ball right around the back. And with some kind of incredible efficiency and accuracy, sort of superhuman accuracy. And obviously realized that was the most effective strategy with the least risk. And so would aim for that strategy from the beginning of the game.
[0:15:20] And the funny story about that is that, obviously, the researchers who created DQN are absolutely amazing researchers and programmers, but they're not so good at playing these Atari games themselves. So they actually learned something about... They didn't know about this strategy, and they learned something from their own system. So that's pretty funny. If you think about the power of general learning systems like this, they can actually master things, complex things, that even the programmers don't necessarily know how to codify.
[0:15:51] So then we took this a big step further, these kinds of systems. And earlier this year, we created a program called AlphaGo. And AlphaGo is a program to play the ancient game of Go.
[0:16:04] Now, for those of you who don't know, this is what a Go board looks like. There are two sides, black and white, and players take turns to put these stones on the vertices of a 19x19 board. So this is what the board looks like, and the board initially starts empty, and then it starts filling up with stones. And the aim of the game is to surround your opponent's stones with your ones, or to wall off empty parts of the board, and surround empty parts of the board and capture it as territory.
[0:16:34] And the aim of the game is to, at the end of the game, it looks something like this, where the board has been mostly filled up. And then you count up the amount of territory that white and black have walled off with their stones. And you add that to the number of capture stones that you've taken. And the player with the most points wins the game. So in this case, it was a very close game, but white would win this game by one point after the count up. Now the thing about Go is...
[0:17:02] It has only two rules, incredibly simple. I could teach it to you in sort of five minutes. But it leads to the most profound complexity. So the history of Go is actually a long and storied one. So it's over 3,000 years old. It was invented in China. And it's considered to be much more than just a game. In China, and ancient China especially, it was considered to be something more akin to poetry or music. So a kind of art form.
[0:17:26] And in fact, it almost had a spiritual sort of dimension in terms of embodying the mysteries of the universe within this game. And Confucius wrote about it as one of the four arts that had to be mastered for any true scholar.
[0:17:42] So for 3,000 years, there's been this massive tradition of playing Go. And today it's as popular as ever. It's 40 million active players and 2,000 professionals. And in fact, in Japan and Korea and China, where basically they play Go instead of in the West where we play chess.
[0:17:59] There are professional Go schools that if you're a kid who's six or seven or eight years old and you're spotted to have talent at your normal school, then you'll be taken out of your normal school and put into one of these Go schools to play Go 12 hours a day, seven days a week, sort of live and breathe Go all through your adolescence. So it's kind of amazing how seriously they take this and how important this game is to the culture of these Asian countries.
[0:18:28] And one way to just quickly illustrate the complexity of the game that arises out of these very simple, elegant rules is the fact that there are more possible board configurations in Go than there are atoms in the universe. So there's 10 to the power 170.
[0:18:45] Possible positions in Go. And there are only around 10 to the power 80 atoms in the observable universe. That's estimated. So there's no way you can kind of brute force calculate what's going to happen in Go. Even if you took all the compute power in the world and ran it for a million years, that wouldn't be enough compute power to enumerate all the possibilities in Go.
[0:19:08] So that explains why Go is so much harder for computers to play than something like chess. It's because this kind of brute force search that was used to master chess is not tractable for Go. And this breaks down into two main challenges. Firstly, this search base of possibilities is really huge. In Go, in an average position, there are 200 possible moves.
[0:19:32] Whereas in chess, in an average position, there are about 20. So Go has an order of magnitude larger branching factor that's called in computer chess. So Go is an order of magnitude more complex from that point of view. But the second challenge, which is even harder, is that it was thought until AlphaGo came along that it was impossible to write what's called an evaluation function to determine who is winning in a particular position, whether black or white is winning and by how much.
[0:19:59] Now in chess, it's relatively easy to write an evaluation function because chess has a concept of materiality. So a queen in chess is worth nine points, and a rook is five, a knight is three, and so on. So if you add up the pieces on both sides of the board and you count out how many points there are, add a first approximation that will indicate who is winning the game.
[0:20:21] And so that's already a good heuristic for an evaluation function. And there are many others that you can codify that build up and taken together can give you a very accurate picture of who's winning. In Go, all the pieces are the same. They're just stones. So there's no concept of materiality. So there's no shortcut to figuring out who is winning. The other problem with Go is that it's a constructive game. The board starts empty and it fills up.
[0:20:47] So if you're trying to evaluate a complex mid-game position, you also have to project into the future about what it might happen. Whereas in chess, chess is a sort of destructive game, where in the sense that the board starts off full with all the pieces, and as the game goes on, it simplifies. Pieces come off the board. So at any moment, you can evaluate the position right now, and that has the full information. You don't have to project into the future as to how the board might fill up.
[0:21:15] So to get around this complexity, the way humans get around it is that they use their intuition and their instinct. So in fact, Go is a much more intuitive game than chess. Chess is much more about calculation and enumerating the possibilities. Go is much more about feel and instinct and intuition. And in fact, if you ask...
[0:21:34] A top Go player, why it is they made a particular move in a complex position. Often they'll tell you it just felt right. Whereas a great chess player will never say that. They'll tell you, you know, I'm planning A because I thought B was going to happen and then I would do C. Now that plan may not pan out. It may not be a good plan, but they'll usually have an explicit plan in mind. Whereas with Go players, it's much more intuitive and implicit. So these things, so even that idea of intuition, trying to codify that,
[0:22:03] As you all know, intuition is not normally associated with computers, whereas calculation is. And that's one of the reasons why it makes Go so much harder. So to get around these two challenges, we turn to neural networks. And we train two neural networks, one each, to get over each of these two challenges.
[0:22:23] So the first network we trained was called a policy network. And what this did was we downloaded 100,000 games from an internet Go server from strong amateurs playing Go on the internet. And we tried to train a neural network to predict in any position what a human player would do next. What move would they make next?
[0:22:44] And what that means is that instead of having to look at all the 200 possibilities every time you're in a position of legal moves, you can actually just narrow it down to the top three or top four most probable moves and concentrate on looking at those. The second neural network is what allowed us to have this fabled evaluation function. We call it the value network. And the way we created this is we took that first network
[0:23:11] And we had it playing against itself 30 million times to create 30 million training games. And with each game, we know who wins. We know all the intermediate positions. And from that, the system, AlphaGo, learns over time how to predict the end result from any of the earlier positions.
[0:23:32] And over time, after each of these games it's playing, it gets incrementally slightly better each time. It learns from its mistakes and its prediction errors that it makes. And each time it slightly improves itself, almost imperceptibly with each game. But after a few million games, you end up with a highly accurate evaluation function that's sort of as good as any human can evaluate positions.
[0:23:56] So armed with these two neural networks, this cuts down that huge search space I was talking about to something much more tractable. So it was time to sort of challenge some of the top players in the world. And in March earlier this year, we challenged a $1 million challenge match against LisaDoll.
[0:24:16] Who is a South Korean grandmaster and considered to be the best player of the past decade. So he's a kind of complete legend of the game. He's won 18 world titles. And I like describing him as the sort of Roger Federer of Go. So, you know, he's sort of been at the top for the last 10 years, won the most Grand Slams, but is still at the top even now. And so AlphaGo, we challenged Lisa Dole and...
[0:24:42] It was a huge deal in Asia, and especially in Korea and China, where the whole country actually came to sort of a standstill watching this match. Here are some pictures of... These two pictures here of the press conferences we had absolutely completely filled these huge ballrooms in the hotel. I think it was the biggest ever press conference that Google had ever had. I think it was 1,000 journalists or something. There was also, at one point during the middle of the match,
[0:25:10] It was on every single national TV station live. So literally, we were in the hotel flicking through all the TV stations and they were all covering AlphaGo. And it was even on these jumbo screens in the shopping centers and other things. It was quite an amazing experience, actually. And many of the Go world and the AI world had thought that it was at least going to be another decade.
[0:25:35] Before a program like AlphaGo came along that could master the game of Go and beat the world's best players. Because of these complexities I've just talked about. So it's thought to be at least a decade away. And even on the evening of the match when they interviewed Lee Sedol, he felt confident he would win 5-0. And even losing one game would be unthinkable for him.
[0:25:57] But in the end, as some of you will know, AlphaGo actually emerged victorious, winning 4-1, and therefore causing a huge splash in both the Go worlds and the AI worlds, a decade before its time. And if you want to read about the details of how the algorithm works, we published a front cover article in Nature that details out all the scientific advances we had to do to create AlphaGo.
[0:26:25] Now, the culture of the impact of the match was huge. 280 million viewers watched the five games overall. 60 million in China just for game one. And so there's more viewers, I think, than the Super Bowl. 35,000 press articles were written about AlphaGo in this match over that week. And the thing I'm actually most pleased about is it really sort of...
[0:26:51] had a big breakthrough in terms of the consciousness in the West about this great game Go and actually boosting its popularity. And one measure of that, there were more than 10 times more boards and pieces sold online from the online places you can buy these than normally around that time of year. So I hope this is going to lead to an explosion of people taking up Go, especially in the West. Now, not only did it win this match, the other thing that was pretty amazing was...
[0:27:18] The way that AlphaGo won and the kinds of strategies that it came up with. So I haven't got time to go into many. There were so many amazing moments. But I'm going to try and explain to you the significance of one of the famous moves that it did. And just as another point to the tradition of Go, because the game is so intuitive and so revered, and the best players are like legends of the past, often as well,
[0:27:46] When a really amazing move is played in a very important game, that move will go down in Legend 2, and it will get a name, and that game will get a name, and it will be sort of studied over hundreds of years by many, many students of the game. And we think that some of the games in this AlphaGo match will end up being remembered like that. And this is my favorite move. This is Move 37 from Game 2. And AlphaGo here is black, and LisaDoll is white, and it's quite early in the game.
[0:28:14] And AlphaGo, on move 37, played this move here on the right-hand side, that black stone with the white triangle there showing you where it's been put down. Now, I just want to try and explain to you in one minute why that was so potentially historical. So we'll see if this works. So there are two very important lines in Go that basically the whole game revolves around. So there's the third line, which is here. Now, if you play on the third line...
[0:28:43] What you're telling your opponent is that you're trying to capture territory, wall off empty parts of the board to the side of the board. So this sort of side. If instead of that you play on the fourth line, what you're telling your opponent is that you're giving up territory on the side of the board, but instead what you're going for is influence and power into the center of the board. And the idea of playing on the fourth line is that that influence and power that you pick up
[0:29:10] Later on can be converted into territory elsewhere on the board. And that territory that you get elsewhere on the board will make up for the territory that you're going to lose on that side of the board. And for 3,000 years, the received wisdom has been that the third and fourth lines are a fair trade. So if one player plays on the third line and the other player plays on the fourth line, the influence you get balances out the territory that you lose. And that's been the kind of proverb, if you like, of how to play Go.
[0:29:39] One of the key things of playing Go. So instead of that, what AlphaGo has done with Move 37 is it's played on the fifth line. So this is kind of completely unthinkable to do this. It's hard to explain how kind of unthinkable this is. To the extent that no human Go player would even consider this move. Because the fifth line means that you're giving away territory from the fourth line, which is a huge amount more area to the side of the board.
[0:30:06] And we think the reason why AlphaGo likes this is that perhaps humans have been undervaluing influence in the center of the board for 3,000 years. And maybe it's more useful than all these human experts have thought. And it turned out in game two...
[0:30:21] That this, of course, you know, anyone can come up with a random new move, right? We could all do that too. But the interesting thing about Go as an art form is that it's kind of like objective art in the sense of you can say, oh, well, I think this move is a good move. But ultimately, what it comes down to is did you win the game and was it related to that move? And in game two, this is actually what happened, is that around 50 moves later, so a long time later...
[0:30:48] That stone ended up affecting the fighting that happened in the bottom left of the corner, the other side of the board. And these two stones here marked by the white triangle, they ended up spilling into the center of the board and connecting up around 50 moves later with that stone that moved 37. So it was almost like AlphaGo had presciently seen this sequence of events and how that stone was going to be positioned just perfectly to help that fighting in another part of the board.
[0:31:17] You know, I said that that was a pretty astounding move, but you don't have to take my word for it. What I want to just show you is this quite amusing clip from the live commentary of the match. So just to explain who these people are, this is the English commentary stream. Obviously, there was the Chinese and the Korean and the Japanese commentary streams. This is the English commentary stream.
[0:31:34] And these two guys are both professional Go players. And on the right is Michael Redmond, who is 9 Dan. 9 Dan is the highest level of Go you can get to. And he's the best Westerner, the only Westerner to have ever reached a 9 Dan level. I think he left the US when he was sort of 18 and has lived and studied in Japan ever since. So he's the greatest Westerner sort of ever. And we had him commentating on this move. So let's just see his reaction to move 37. And bear in mind, so this is the commentary room.
[0:32:03] And he's watching what's going on in the game room via that laptop in front of him. The Google team was talking about is this kind of evaluation... That's a very surprising move. I thought it was a mistake. So you can see that he's not even sure where the piece is. And later on they go on to speculate that maybe our...
[0:32:33] Lead programmer who entered the moves into the computer had misclicked and put it in the wrong place. That was how unthinkable that move was. So that was the reaction. And since then, the whole Go world has been studying all these five games for these nuances. And now we're seeing human players, expert players, starting to play on the fifth line and use these kinds of motifs.
[0:32:58] So Lisa Doll's reaction was quite interesting too. So this is a view of the match room over there on the right. And this is the top-down view of the actual board with the Move 37 played. And you can see there's our programmer, Adjo Huang, on the left, who's entering the moves. But on the right, there's an empty chair where Lisa Doll was sitting. And he basically got up and left for 15 minutes after this move. No one knew where he went. We thought he might have left the building.
[0:33:23] So it turned out that maybe he just went to wash his face or something. So that's how sort of shocked he was too. But then, of course, this whole match spurred Lisa Doll onto even greater heights, and he came back in Game 4, which is the game that he won, and he played his own incredible move on Move 78. And I haven't got time to explain why this move was amazing, this white move here with the black triangle in the center of the board, but it was an amazing move that...
[0:33:50] It really surprised AlphaGo. And later on, when we looked at the logs of what AlphaGo was thinking, it only gave this mover less than 1 in 10,000 chance of happening. So it hadn't really analyzed this move at all. And it caused some error in its evaluation. So I think this move also will go down in history as well, most likely. So some really incredible Go that was played during these matches.
[0:34:13] And what was great is after, you know, Lisa Dole is this amazing guy and we had several sort of dinners together during the match and before the match and after the match. And he told me that this AlphaGo match was one of the greatest experiences of his life and it renewed his passion for the game. You know, he's coming towards the end of his career now. You know, he's been playing this game for, I think he's 33, he's been playing for nearly 30 years. And this completely renewed his passion for the game and sort of made him feel like there was this huge greenfield spaces to it.
[0:34:43] What's interesting is he used some direct quotes from his interviews afterwards in the Korean press. You know, my thoughts have become more flexible after this game. I have a lot of new ideas and I expect good results. And I decided to more accurately predict the next move instead of depending on my intuition. So that's really interesting. Playing against AlphaGo sort of made him think that he shouldn't just automatically go with his intuition that he's trained over 30 years, but should more explicitly think about new ideas.
[0:35:12] And in fact, he's ended up over the last few months since the match having an amazing win record against and won a couple more titles against the other top human opponents. So I'm just going to sort of end this section a little bit. We're just talking about intuition and creativity. So I've talked quite a lot about intuition, but what I mean by it, and what I want to just unpack a little bit, especially for this audience as we're at the Royal Society of Arts here, and I'm not saying these are full...
[0:35:39] encompass everything about intuition and creativity, but just sort of operationally within the domain of Go, I think these are reasonable definitions. So by intuition, what I'm meaning is the implicit knowledge that's acquired through experience, but it's not consciously expressible or accessible. So you can't explain how you know this stuff to someone else, and you can't even explain it to yourself or access it yourself.
[0:36:03] So you might ask, well, if you can't explain it or access it consciously, how do we know it's even there? Well, of course, we can test the existence and the quality of this knowledge behaviorally by giving probing tests. So in Go, this is easy. You can just give someone a Go board and evaluate the quality of the output that their intuition is going to come up, what the kind of move they come up with is. Secondly, with creativity...
[0:36:28] One way to operationally define that would be the ability to synthesize the knowledge you currently have, these building blocks you have, to produce a novel or original idea in the service of some goal. And I think, you know, if you've seen with Move37, I think it's pretty clear that AlphaGo demonstrated both of these abilities to quite a high level, albeit, of course, with the caveat that this is still a very constrained domain of the board game Go.
[0:36:57] So one other important aspect of our research at DeepMind is systems neuroscience and getting inspiration from not just from mathematics and machine learning, but also from how the brain works. And we really stress the word systems because what we're interested in is the algorithms and the representations and the architectures the brain uses, not necessarily the low-level biological details.
[0:37:20] The implementation details. Because obviously the substrates are different. You've got computers on silicon and you've got our wetware which is made out of carbon. So there's no reason why necessarily the implementation should be the same. But certainly the algorithms and the architectures could have some important clues as to the capabilities the brain has that we would like these machines to have. So we talk a lot at DeepMind about systems neuroscience inspired AI.
[0:37:49] And here's a list of, I haven't got time to go into these, of areas, high-level areas that we're actively researching now that are partially inspired by the latest thinking about how the brain accomplishes these things. Memory, attention, concepts, planning, navigation, even imagination.
[0:38:05] And I studied quite a few of these areas, memory and imagination, for my PhD, which was in cognitive neuroscience. And in fact, I honed in on one of the areas of the brain called the hippocampus. And the hippocampus is at the center of your brain, and it's well known that if you have damage to the hippocampus, then you become amnesic. But actually, the hippocampus is responsible and heavily involved in all of these other capabilities as well, not just memory.
[0:38:31] And one of the projects that we have ongoing at DeepMind and one of the ones I'm personally involved in, you can think of loosely as trying to create an artificial hippocampus with all the capabilities that it has to link in with these neural networks that we've already built for things like the Atari games.
[0:38:52] So I just want to end by talking a little bit about real world applications. So I've talked a lot about games, and I've explained why we use games, because they're a very efficient domain to develop these algorithms in. But of course, we're not just interested in solving games for its own sake. What we want to do is actually build these general purpose learning algorithms in such a way that they're general enough.
[0:39:14] that they can be transferred into the real world and actually applied to all sorts of really important problems that have high impact for the good of society.
[0:39:23] We've already announced some collaborations with the NHS, looking at things like image recognition to help diagnosis of head and neck cancers, also eye retinal scans to look at macular degeneration. We're interested in robotics, and we're also interested in sort of slightly more left-field things that you might not expect these sorts of AI algorithms to appear in, including most recently, which I think is pretty cool, is our work with data centers. And what we did...
[0:39:52] Of course, Google has huge data centers, some of the biggest ones in the world, perhaps the biggest ones in the world, and they consume a lot of power. I think the latest assessment I read somewhere was about 2% of the world's power is currently used in data centers around the world by data centers, cloud computing. And that's, of course, only going to get more as more and more of our compute power is going into the cloud.
[0:40:15] And what we did is we used a similar system to AlphaGo, but instead of playing Go, we applied it to the cooling systems in the data centers to try and increase the energy efficiency of these data centers. And what we managed to do, which was quite surprising to the data center engineers, was we managed to save 40% of the energy that was used by the cooling systems, which ends up meaning that the whole data center is 15% less power usage.
[0:40:43] And obviously that's worth tens of millions of dollars a year, but it's also very good for the environment, of course.
[0:40:49] And what AlphaGo does, or our data center optimization system does, is it controls all the cooling systems, the fans, opening the windows, the cooling water, even where the compute power is being routed to within the data center, the calculations. All of those things are optimized, and its inputs are all the sensory data, the thermometers, the temperature gauges, and the fan speeds, and so on. And here's the graph of what happened.
[0:41:18] This is the power that's being used in the data center, the power usage efficiency. And you can see the sharp spike down when we turn on this AI system. And now that's controlling the data center. And then when we turn it off, it spikes back up to where it was originally, the power. So it makes a huge difference. And what we're thinking is now is that...
[0:41:39] Well, why don't we optimize something like the energy grid at national scale? There's no need to just think about a data center. There must be huge inefficiencies also at grid scale. So if that's true, and we're investigating this now, maybe we could save 10% of the energy consumption of a country, which obviously has huge implications for climate change and other things. I'm just going to end with our most recent word, which I thought would be fun.
[0:42:07] For this audience as well, because it's sort of creative too. We also just last couple of weeks, hot off the press, we announced that we now have one of our models called WaveNet. It's now the best text-to-speech system in the world. So text-to-speech systems are used for speech synthesis. So if you speak to your phone and it tells you back the result, the voice that it uses is speech synthesis.
[0:42:32] And most of the time, the current state-of-the-art models are called concatenative models. And what they are is that, basically, is that they get an actor to speak for 30 hours, lots of dialogue, and then they chop up that dialogue into syllables. And then when you ask it to speak something new, it stitches back those syllables together. And that's why they sound a bit warbly and also why they sound a little bit robotic. Instead of that, WaveNet...
[0:43:01] That's one small step for man.
[0:43:36] 50% better than these concatenative systems. So perfect would be a human actually reading it out directly, and that's the green bar. And the current state of the art is the red bar. And you've seen that WaveNet, which is the blue bar, reduces the distance between the best models out there today to perfect human level by more than 50%.
[0:43:58] So I'm just going to play you a couple of samples so you can hear for yourself the difference, hopefully. So there's two samples. One is going to be from the concatenative system and then from WaveNet. And it's about a strange film, I think, because this is one of the hundred test phrases that Google uses to evaluate how good a system is. So this is the concatenative. The Blue Lagoon is a 1980 American romance and adventure film directed by Randall Kleiser.
[0:44:24] And this is WaveNet. The Blue Lagoon is a 1980 American romance and adventure film directed by Randall Kleiser. So hopefully you agree that the second one's better. And at least 86% of people did in our user tests. And more natural sounding. But just a fun thing that I haven't shown before is you can use this to not only model waveforms of speech, but you can also model waveforms of music. And one of the kind of...
[0:44:53] Holy grails, I guess, of AI has always been, can we generate music and composition? And I think we may be on the cusp of that now. And the beginning, this one I'm going to show you is bits of piano music. It was trained on, I think, a bit of Rachmaninoff and that kind of music. And it's just free form.
[0:45:13] Constructing, creating. So we haven't really told it to... It doesn't understand anything about long-term melody yet or these kinds of things. That's coming. But this is a sort of free-form composition that it's doing. And then the second one... So it's a bit like a drunken pianist at the moment. But it's creating... That sound, that's not a synthesized sample. It's creating that sound, that piano sound you can hear, all the nuances of that sound.
[0:45:58] And it's creating, to our surprise, we've just started working on this, it's already creating sort of musically interesting sounding compositions. Of course, it's changing style too quickly, but if we can make it sort of stick to one style for longer, we think we might have something there. So I just want to end by coming back to my last slide on this idea of...
[0:46:22] Solving intelligence and then using it to solve other problems. We really think about AI as a kind of meta solution to all these other issues. And if we can sort of fundamentally solve AI and...
[0:46:33] make intelligence abundant in this learning fashion, then we think there are all sorts of areas where we can apply this to. The two I'm most excited about are science and healthcare, and actually having systems that can deal with huge amounts of data, more data than any human experts can keep in mind and make sense of, and actually try and surface interesting insights that then the top human experts can take on and theorize about and make quicker breakthroughs with. So I see AI as one of the most powerful tools we can create.
[0:47:02] to help aid these research programs and the best scientists and clinicians to do their jobs even better. Thanks for listening.