{"url": "https://www.youtube.com/watch?v=Psk5DLpqp3o", "page_url": "https://www.youtube.com/watch?v=Psk5DLpqp3o", "title": "Autumn Lecture 2016 - Demis Hassabis, DeepMind", "show_or_channel": "Royal Academy of Engineering", "date": "2016-10-07", "duration_s": 4112, "type": "talk", "speakers": ["Demis Hassabis"], "confidence": "high", "notes": "Royal Academy of Engineering Autumn Lecture 2016 (Silver Medal).", "people": ["demis-hassabis"], "id": "2016-10-07_royal-academy-of-engineer-autumn-lecture-2016-demis-hassabis-de", "audio_duration_s": 4111.04, "worker": "jc-w05", "transcribe_s": 55.4, "model": "large-v3", "segments": [{"start": 0.5, "end": 26.08, "text": "Now, it's a huge honor to be here tonight to introduce Dr. Demis Hassabis, who's remarkable in so many ways. He's remarkable for his work on memory, for his contribution to that huge and hugely unappreciated slice of modern culture known as the video game industry, and of course, he's very well known for his pioneering work on artificial intelligence."}, {"start": 26.08, "end": 52.88, "text": "Demis is the co-founder and CEO of DeepMind, which was bought by Google a couple of years ago. He leads Google's general AI efforts, which made headlines earlier this year with a self-taught AI software that attained human level performance in half of the 49 games that it tried. Then he made even more headlines with AlphaGo, which is the first program to beat a professional"}, {"start": 52.88, "end": 79.63, "text": "player at the ancient game of Go. Now, when you look over his life, you are filled with a deep sense of inadequacy. He's a former chess prodigy. He finished his A-levels two years early before coding the multi-million selling game Theme Park when he was aged just 17. He got a double first at Computer Science in Cambridge."}, {"start": 79.63, "end": 106.0, "text": "He founded Elixir Studios, which produced award-winning video games. Then after a decade of technology startups, he decided to go back to academia to complete a doctorate at University College London with the high priestess of memory, Eleanor McGuire. He did some research connecting memory with imagination, and that was listed in the top 10 scientific..."}, {"start": 106.0, "end": 134.03, "text": "breakthroughs of 2007 by the journal Science. And if that's not enough for us mere mortals, he's also a five times World Games champion, a recipient of the Royal Society's prestigious Mallard Award, and this year was given the Academy's Silver Medal. Now there's going to be some time for a few questions after his presentation, but it's a huge honor to invite him."}, {"start": 134.03, "end": 169.87, "text": "to come up to give the autumn lecture on the subject towards general artificial intelligence. Please give him a very warm hand of welcome. Well, thanks, Roger, for that lovely introduction, very generous introduction. And it's a real honor and pleasure to be here giving this lecture at the Royal Academy. So today I'm going to talk about artificial intelligence, specifically general artificial intelligence through the lens of the work that we've been doing at DeepMind."}, {"start": 170.19, "end": 193.49, "text": "So DeepMind was founded in 2010, right here in London, and we're still based here in London. And it's one way you can think about it as a kind of Apollo program effort for AI. So we now have around 350 people, more than 250 research scientists, 250 PhDs working on this subject. So I think it's the biggest collection of brainpower anywhere in the world working on AI."}, {"start": 194.77, "end": 224.64, "text": "And another thing we're doing is obviously we're pushing the boundaries of AI research, but we're consciously also trying to come up with new ways of organizing the scientific process in a way to try and speed up scientific endeavor and make sort of research breakthroughs more quickly. So I haven't got time to go into that today, but maybe we can talk about that during the Q&A. So I'm going to concentrate here on the technology. So what's our mission at DeepMind? Well, the way I like to articulate it is as a kind of two-step process."}, {"start": 224.64, "end": 252.1, "text": "Firstly, fundamentally solve intelligence. What we mean by that is understand how intelligence works and then recreate that artificially. And then if we were able to actually complete step one, I think step two would naturally follow, which is you could then use that technology to literally solve everything else. I mean, after all, we built civilization, all of this stuff around us, cities and where we sit now, all this technology with human intelligence. That's what's created that."}, {"start": 252.1, "end": 282.32, "text": "So if we can enhance that and make that kind of intelligence more abundant, then imagine what the possibilities would be. So more prosaically, the way we're going to try and do this at DeepMind is by creating what we call general purpose learning algorithms. So there's two key words here. There's the idea of learning. So all the software we build, all the algorithms we develop, learn automatically how to do things from raw experience or raw inputs. They're not pre-programmed or handcrafted in any way."}, {"start": 282.86, "end": 313.28, "text": "The second thing is this notion of generality. This is the idea that a single system or a single set of algorithms out of the box can operate across a wide range of tasks, perhaps even tasks that the system has never seen before. Now, these kinds of general learning algorithms, we call this type of AI artificial general intelligence or AGI. And the sort of hallmarks of this kind of AI are that it's flexible and it's adaptive and perhaps even inventive."}, {"start": 313.28, "end": 341.71, "text": "And it's built to be flexible from the ground up. Now, if you compare that to, I mean, AI is obviously a very hot topic at the moment. And if you compare that to most kinds of AI systems out there, which we call narrow AI to distinguish it from this kind of general AI, most of the AI systems out there today in your phone, even driving cars, doing recommendations on Netflix, places like that, they're kind of examples of handcrafted AI. They've been built."}, {"start": 341.71, "end": 364.62, "text": "Special case for one particular purpose. And they've been handcrafted with a set of rules. And because of that, they're often very brittle. If the human users do something unexpected, and they always do because we're humans and unpredictable, then they catastrophically fail. There's no way for them to adapt to the new situation or deal gracefully with this unexpected behavior."}, {"start": 365.78, "end": 394.21, "text": "Now, probably still the most famous example of a narrow AI system and a real watershed moment in AI was when Deep Blue, IBM's Deep Blue, beat Garry Kasparov at the game of chess in the late 90s, in 97. So, of course, this was a hugely impressive technical achievement. But Deep Blue was a handcrafted AI system. So what happened with Deep Blue is that a team of brilliant programmers talked to a team of..."}, {"start": 394.21, "end": 419.1, "text": "Brilliant chess grandmasters and effectively distilled the knowledge they had in their minds into a bunch of rules and heuristics that then Deep Blue could execute. So there wasn't really any learning here or in some sense perhaps not even any true intelligence. One way to sort of view that is that Deep Blue was obviously world championship level at chess but it was not able to do anything else."}, {"start": 419.1, "end": 449.07, "text": "So for example, it couldn't even play a much simpler game like noughts and crosses without being totally reprogrammed. So nothing in that knowledge base was general in the sense of being useful for anything other than playing chess. So by contrast, with our kind of general learning algorithms, the way we think about intelligence is actually through the framework of what's called reinforcement learning. And I'll just quickly explain what reinforcement learning is for those of you who don't know."}, {"start": 449.07, "end": 479.25, "text": "So let's start off with the agent system, the AI system, here pictured by this little figure. And the agent finds itself in some sort of environment, and it's trying to achieve a goal in that environment. Now, that environment could be real world, in which case the agent would be a robot, or it could be a virtual environment, in which case the agent would be an avatar. Now, the agent only interacts with the environment in two ways. Firstly, through its sensory apparatus."}, {"start": 479.25, "end": 507.46, "text": "And we mostly use vision, but we're starting to use audio as well now. You get observations about the environment. And just through these observations, which are always quite noisy and incomplete, the agent has to build a model of the environment out there, the best possible model it can build. And what the agent does with that model, once it's built that model, is use that model to make plans into the future and try and figure out what action it should take."}, {"start": 507.46, "end": 534.13, "text": "From the set of actions that are available to it at that moment in time, that will best get the agent towards its goal. And once it's figured out which action it should take, it outputs that action, that action gets executed, and then that may or may not make a change to the environment, which then drives a new observation. And with reinforcement learning, this goes on in real time, in a feedback loop."}, {"start": 534.13, "end": 563.34, "text": "The reason we know that this is enough for intelligence is because this is the way that biological systems learn. All animals, including humans, have reinforcement learning as one of the learning systems they use. And in humans, it's actually the dopamine system that implements these kinds of algorithms. Now, although this diagram looks pretty simple, there's actually huge complexities hidden behind this diagram that need to be solved. And if we could solve all of those complexities..."}, {"start": 563.34, "end": 588.72, "text": "Then we know that would be enough for general intelligence. So this is one way we think about intelligence. The second big thing that we committed to in a sort of philosophical level when we started DeepMind was the idea of grounded cognition. And what that means is that it's this idea that a true thinking machine has to actually be grounded in a sensory motor reality for it to be truly thinking."}, {"start": 589.36, "end": 609.46, "text": "And we actually use games because we believe they're the perfect platform for developing and testing AI algorithms. So a lot of people who subscribe to this view of grounded cognition end up working with real robots because obviously robots are clearly situated in a sensory motor reality, the real world. But actually what we figured out was..."}, {"start": 609.46, "end": 632.93, "text": "At the start of DeepMind is that you can use much more efficiently use virtual environments, including games. And as long as you're strict with how you use it, so you make sure that the agent doesn't have access to any of the internal states of the game, only has access to the sensory data that it would normally be getting if it was a real robot, then games are just as good a testing ground as using real robots."}, {"start": 632.93, "end": 659.12, "text": "And if any of you who've worked with real robots will know that they're quite a nightmare to use in terms of they keep breaking, they're very expensive, and they're also quite slow. So if you compare that to using virtual environments in the cloud, then you can develop your algorithms much more quickly. The other thing you can do is create unlimited amounts of training data because, of course, you can run these virtual environments for as long as you want."}, {"start": 659.28, "end": 688.32, "text": "There's no testing bias because if you take games that are commercial games that are already out there, then the game designers and the programmers who built those games didn't build those games with AI in mind and testing AI algorithms. They built it to be challenging and fun for human players. So often I think this is one of the things that's held back AI development over the last 20 years is that usually it's the AI developers that also create the benchmarks and the tests."}, {"start": 688.32, "end": 711.31, "text": "And you can't help but being unconsciously biased about what tests you create that sort of roughly suit the types of algorithms that you're thinking about. I've mentioned parallel testing. So obviously, because these are virtual environments, especially now we're at Google, we can run hundreds of thousands of these tests simultaneously. And also, progress is very measurable."}, {"start": 711.31, "end": 739.44, "text": "So this is very important in the sense of if you have a long-term big research goal, multi-decade research goal like solving intelligence, you need to make sure that you have lots of short-term measurable goals to make sure that you're going in the right direction. And games are kind of perfect for that because most games have scores, or if they don't have scores, you can easily have a proxy for progress. So this then yields the kind of idea of an end-to-end learning agent."}, {"start": 739.44, "end": 766.38, "text": "And we kind of coined this idea. And what we mean is starting from perception and going all the way to action. And if it's a computer game or virtual environment, then perception will be starting with the raw pixels. And we're interested in that entire stack of problems, all the way from processing your perception and making sense of that to building a model of the world out there to deciding what the best action is to take to get you towards your goal."}, {"start": 768.56, "end": 797.63, "text": "And at DeepMind, this is one of our big contributions to the AI field, is pioneering the use of games and virtual environments for serious AI research. So we call this kind of algorithm deep reinforcement learning. And what this combines is two of the kind of most important algorithms at the moment in the AI field. Deep learning, which is all the rage right now in terms of processing perception. So if you use search..."}, {"start": 797.63, "end": 823.89, "text": "With Google Photos, or if you use voice recognition on your phones, it will be deep learning now that is powering that. And deep learning, for those of you who don't know, is neural networks that are stacked into a hierarchy. So that's where the name deep comes from. They're deep neural networks. And we combine this with reinforcement learning, which I just talked about. And deep learning does the perception processing, and reinforcement learning figures out what action to take."}, {"start": 824.3, "end": 844.24, "text": "And by combining these two things together, this allows reinforcement learning to work at large scale. So as Roger mentioned, the first thing we tested this on that became a pretty well-known piece of work for us was testing it on classic Atari games from the 80s. So we took 50..."}, {"start": 844.24, "end": 872.5, "text": "And for those of you who are old enough to remember the Atari system, you'll remember some of these games. And just before I run the video showing you what the agent does, I'll just explain exactly what it is you're going to see. So the agents here, they only get raw pixels as inputs. So they get 30,000 inputs, 30,000 numbers per frame. And the goal is simply to maximize the score. And everything else..."}, {"start": 872.5, "end": 899.31, "text": "So we don't tell the system anything else. It's got to figure out everything else from first principles. And the other thing is we insist on this generality notion, so one system out of the box without tweaking any of its parameters to play all these different 50 games. So just to be clear, the system doesn't know anything about the rules of the game. It doesn't know how to get points. It doesn't even know what it's controlling."}, {"start": 899.31, "end": 929.57, "text": "It has to figure all of that out for itself by playing the game and experiencing the game. So we built this system which was called DQN. And I'm just going to show you here playing a different variety of Atari games. And you'll see the huge variety of games that it can master. So this is Enduro, which is one of the early racing games. It's River Raid. You can see the really different complexity of the visuals as well as the objectives of each of the games."}, {"start": 929.57, "end": 957.62, "text": "Classic Pong, it's playing the green bat here, and it always wins 21-0. So can't get a point off of it. Sequest here, it's figured out that you have to go to the surface to get more oxygen. And then in boxing here, it's the left-hand boxer, the red boxer. And you can see it does a bit of sparring with its opponent, the inbuilt AI, and then it figures out how to corner it, and then it racks up an infinite number of points. And it'll just carry on doing that until you switch it off."}, {"start": 957.62, "end": 988.64, "text": "So here you can see the vast differences in the visuals and the objectives, and yet the same system, DQN, can master all of these 50 games. So this work was featured on the front cover of Nature last year, and we also released the code publicly. So if you're a programmer and you're interested to play around with DQN and make your own enhancements to it, it's available on the Nature site. So then we took these systems further in the last year, and we built a program called AlphaGo, which Roger also mentioned."}, {"start": 988.64, "end": 1017.58, "text": "So AlphaGo is our program to play the ancient game of Go. And this is what a Go board looks like. Now, ever since Deep Blue beat Garry Kasparov, one of the holy grails or the grand challenges of AI research was to beat the much more complex game of Go. Now, this is a Go board. There's two sides, black and white. And you play Go on a 19 by 19 grid. And black and white take turns in putting down these stones on the vertices of the grid."}, {"start": 1017.58, "end": 1046.03, "text": "And initially the board starts empty and eventually the board fills up over time. Now the history of Go is a long and storied one. It's actually over 3,000 years old and is invented in China. And it's regarded in Asia as more than just a game. It's something more akin to art or poetry. And in fact, in the classical period, in Chinese sort of antiquity, it was considered to be one of the four arts that had to be mastered by a true scholar."}, {"start": 1046.86, "end": 1061.87, "text": "And Confucius wrote about it and it became a really big part of Chinese culture. Since then it's expanded all across Asia and now it's just as popular today as it's ever been. So there are 40 million players and more than 2,000 professionals."}, {"start": 1062.0, "end": 1088.72, "text": "And there are Go schools in Japan, Korea, and China, where if you're a child who's spotted at a young age to have talent in Go, you'll leave your normal school and go to Go school at the age of six or seven, and then start studying Go seven days a week, 12 hours a day. So they take this pretty seriously. And one of the ways you can see the beauty and the complexity of the game is that..."}, {"start": 1088.72, "end": 1119.42, "text": "There are more possible board positions than there are atoms in the universe. So there's 10 to the 170 possible board positions, and there's only about 10 to the 80 atoms in the observable universe. So this is just one of the hints at why this game is so complex for computers to master. You can't just brute force a solution to this. There are just too many possibilities. Now, despite this kind of complexity, the reason that Go is considered one of the most elegant games ever devised is that the rules are actually incredibly simple."}, {"start": 1119.42, "end": 1145.1, "text": "In fact, there were just two rules. And the main rule is the way that stones are captured. And I'm just going to quickly explain to you how that works so that you can appreciate what AlphaGo did later in the matches that it played. So here's a board position from a game of Go, fairly early opening position. Now, let's just zoom into this bottom right here of the board. And you'll see the pieces set up here on the bottom right."}, {"start": 1145.81, "end": 1166.51, "text": "Now, this white stone here that you'll notice is surrounded by three black stones. And if it was black to play, and black played in this final vertex position A, this final empty position A here, now that white stone would be completely surrounded by the four black stones, and the white stone would be captured and removed from the board."}, {"start": 1166.86, "end": 1192.51, "text": "And that's the main idea of Go is that you're trying to surround your opponent's pieces while they're trying to surround you. And you're also trying to surround not just your opponent's pieces, you're trying to wall off empty parts of the board. And both empty parts of the board that you surround and the pieces that you capture count as points towards your final score. So this might be a final ball position. And you can see here that white has walled off."}, {"start": 1192.51, "end": 1223.71, "text": "These areas of the territory, and black has walled off those areas. And in this case, white wins by one single point, has slightly more territory than black, and white would win the game. So from these simple rules, that enormous complexity develops. So why then is Go so hard for computers to play? Well, there's two main reasons. One is the search base is really massive. So in an average position in Go, there are 200 possible possibilities of where to move."}, {"start": 1223.71, "end": 1247.89, "text": "By contrast, if you think about chess, in an average position in chess, there's only about 20 possible moves. So the branching factor is an order of magnitude bigger at 200 for Go. The second thing, which is even more challenging, is that it was thought, until we built AlphaGo, that it was impossible to write what's called an evaluation function to determine who is winning, black or white, in a particular position."}, {"start": 1248.02, "end": 1277.38, "text": "And obviously it's critical to be able to estimate that if you want to write a program that's going to be any good at playing the game. And the reason that's so much harder than, say, chess, is in chess there's this concept of materiality. So a queen is worth nine points, a rook is five points, knight is three points, and so on. And as a first approximation, if you just add up the number of points on each side, that will tell you roughly who is winning in a particular position."}, {"start": 1277.38, "end": 1304.53, "text": "So it's quite easy to codify some rules, a set of heuristics, to tell the computer who is winning in any particular position in chess. In Go, this is impossible because there's only one type of piece, a stone. And even like a small change in the position of a stone can change the entire evaluation of the position. So it's much more difficult to write a handcrafted set of rules to tell you who is winning in a particular mid-game position."}, {"start": 1305.52, "end": 1335.41, "text": "So the way we got around these two problems is by turning to neural networks. And we trained two neural networks to get around both of these two issues. So firstly, we trained a policy network that observed 100,000 strong amateur games downloaded from the internet. And what the policy network did is it learned to predict in any particular position what the most likely moves were to be played. So what that means is instead of having to look in any one position at all the 200 possible legal moves,"}, {"start": 1335.41, "end": 1365.33, "text": "You could just look at the top three or top five most likely. So that really reduces down the breadth of the search that you have to do. The second neural network that we train is called the value network. And this is the network that evaluates a particular position and tells the system who is winning, white or black. And the way we train this is by having our AlphaGo system play against itself 30 million times on the Google Cloud."}, {"start": 1365.33, "end": 1392.59, "text": "And with each step, using reinforcement learning, it incrementally improved itself and learned from its own mistakes. So slowly over time, it got better and better at evaluating who was winning in a particular situation after experiencing millions of games of seeing, from this position, white ends up winning. From this position, black ends up winning. And over time, it started to generalize and figure out what features about positions predict the eventual outcome."}, {"start": 1393.74, "end": 1423.22, "text": "And so by using these two neural networks, we can overcome this huge complexity that Go has and make it tractable for a machine to play Go. And in fact, this actually mimics the way that humans play Go because that's how we as human players overcome this enormous complexity. We don't search everything. We actually use our own natural neural networks to figure out what's the most likely moves based on our experience of those kinds of positions and also to evaluate those mid-game positions."}, {"start": 1424.18, "end": 1452.14, "text": "So once we had this system, we played it against all the best computer programs of the handcrafted type, and it beat all of those 100% of the time. So then we decided to challenge some of the best human players in the world, which culminated in March earlier this year, us taking on a real legend of the game, Lee Sedol, who is a South Korean grandmaster and acknowledged as the greatest player over the last decade."}, {"start": 1452.14, "end": 1479.84, "text": "He won 18 world titles, and I call him the kind of Roger Federer of Go. So he's been at the top of the game for 10 years, and is still at the very, very top, winning world titles. So we played this in March as a million-dollar challenge match, live in Seoul over a six-day period. And in fact, the whole of Asia, and especially Korea, really came to a standstill during this match."}, {"start": 1479.84, "end": 1509.46, "text": "So here are some pictures of the crowded press conferences. So I think this was the opening press conference, which is the biggest press conference that Google had ever had. And at one point, it was so popular, the matches on every TV channel was being displayed, the live commentary from the game, literally in the hotel, flicking through all the national TV channels, and every single one was showing the games. And AlphaGo, as many of you know, actually won the game 4-1. And this is a huge..."}, {"start": 1509.46, "end": 1540.34, "text": "So really, AlphaGo was a decade before its time, which, as you all know from engineering and research, is a very unusual occurrence to be a decade before consensus on when something's going to happen. So the cultural impact of the match was huge. We had 280 million viewers overall."}, {"start": 1540.34, "end": 1566.61, "text": "60 million viewers in China just for the first game. And 35,000 press articles over that week. And the thing I'm actually most proud of is the fact that it popularized Go in the West, which I think the game really deserves. And actually, there were 10 times more boards sold online than normal for that time of year. So I really think it's given a huge boost to the game of Go and its sort of profile in the West."}, {"start": 1568.46, "end": 1595.3, "text": "So that, if you want to read more about how we did that, that was also featured on the front cover article in Nature earlier this year. And we go into all the details of how we built it and all the technical details of what we did with the, how to train those neural networks. But I just want to take a couple of minutes to explain to you about not only did AlphaGo win the match, but the way it won it really astounded the world too in terms of the creativity it exhibited."}, {"start": 1595.3, "end": 1622.53, "text": "So this is my favorite position from the five matches. And AlphaGo is black here. This is game two. And Lisa Doll is white. And AlphaGo played this move here, this black move here. On the right-hand side there, I'll move 37. Now, I just want to take a moment to explain to you why this was so astounding to the Go world and why I think this will go down in history as a very famous move."}, {"start": 1622.53, "end": 1652.64, "text": "So normally in Go, there are two important lines on the board. There's the third line. And if you play on the third line here, shown by this vertical line, what you're really telling your opponent is that you're trying to grab wall-off territory to the side of the board. So that's one thing you can do in Go. Another thing you can do is you can play on the fourth line instead, one line higher up. And if you play on the fourth line, what you're basically saying to your opponent is that I am going to take influence and power into the center of the board."}, {"start": 1652.64, "end": 1681.5, "text": "And what you hope is that later on that influence will result in territory somewhere else on the board. Now for 3,000 years, the received wisdom has been that playing on the third and the fourth line are about equal. So the territory that someone gets on the third line is about equal to the influence that somebody gets by playing on the fourth line. But what AlphaGo did is it sort of overturned this 3,000 years of tradition and knowledge by playing on the fifth line and taking influence from the fifth line."}, {"start": 1681.5, "end": 1707.52, "text": "Now, you're giving away fourth-line territory, so your opponent is going to get that much more territory. So this is kind of considered to be unthinkable to do that. And yet, that's what AlphaGo decided to do. And what happened is, turned out, that 50 moves later, that stone there, move 37, ended up affecting the fighting that broke out from the bottom left-hand corner of the board, completely another part of the board. 50 moves later..."}, {"start": 1707.52, "end": 1735.7, "text": "Those stones down here, these two stones here marked by the white triangles, ended up spilling out into the center of the board and then eventually connecting with that move 37, 50 moves later. And that's what won AlphaGo the game. So it sort of somehow intuited this long-term plan and figured out that this very unconventional move was worth playing. And just to show you, so you don't have to take my word for how surprising this was, I just want to show you a video of..."}, {"start": 1735.7, "end": 1758.85, "text": "The commentators, the live commentators who were commentating on the match as it was going on. So these were the Western commentators. So this is a guy on the right-hand side called Michael Redmond. He's a very famous Go player. He reached 9 Dan level, which is the highest level, and he's the only Westerner to have ever reached that level in Go. And obviously he studied and lived in Japan for two decades to get to that level."}, {"start": 1758.85, "end": 1786.22, "text": "And we'll see here, he's watching the monitor, and he's observing the moves going on in the match room, and then he's commentating on it on this demo board here. The Google team was talking about, is this kind of evaluation? Because he's so surprised. That's a very surprising move. I thought it was a mistake."}, {"start": 1787.09, "end": 1816.43, "text": "So what he was saying there is he thought that our computer operator, the lead programmer who was moving the stones on the board and then entering it into the computer, had misclicked. He thought that he'd actually put the piece down and misinformed the screen as to where that stone was. That's how astounding and sort of unthinkable that move was to him."}, {"start": 1817.81, "end": 1845.36, "text": "And then this was Lisa Doll's reaction. So this is the actual live game board on Move 37. And you can see on the right-hand side is the actual camera on the match room. And you can see Adra Huang, who's our lead programmer, who's entering the pieces onto the screen, the moves onto the screen, and playing the pieces on the board. And Lisa Doll has basically, after that Move 37... Oops."}, {"start": 1846.51, "end": 1876.61, "text": "Left the room for 15 minutes. And we were actually wondering where he'd gone to and whether he just disappeared and left the whole hotel. But actually, I think he just went to the bathroom to clear his head for 15 minutes. So that's pretty incredible in response to how has the computer come up with this move. Now, the thing about Lisa Doll is that he's an incredible games player, one of the best games player I've ever seen. And he actually came back in game four to win game four."}, {"start": 1876.61, "end": 1903.55, "text": "And with his own incredible move on move 78, which I haven't got time to go into today, played here this white stone in the middle of the board that confused AlphaGo and was a kind of brilliancy that allowed him to win game four. In fact, AlphaGo only gave the probability of that move as a 1 in 10,000 chance, so it hadn't really considered it before Lisa Dole played that move. What's really interesting is that since the match..."}, {"start": 1903.55, "end": 1932.7, "text": "I think AlphaGo has spurred Lisa Dolon to even greater heights. So he's had a kind of 90% win rate in championship matches since the AlphaGo match. And he said some very interesting things in the interviews about how his thoughts have become more flexible after playing AlphaGo, and he has a lot of new ideas, and he thinks he'll get good results with that. And he told me after the match that it sort of rejuvenated his passion for the game and his creativity about what is possible. It sort of freed his mind."}, {"start": 1932.7, "end": 1952.9, "text": "And the other interesting thing is that he said that he's decided to more accurately predict the next move instead of depending on his instincts and intuition. So Go is very much a game about intuition rather than calculation. So if you ask a great player like Lisa Dole if they're playing a very difficult game against an even opponent, why they made a particular move."}, {"start": 1952.9, "end": 1973.84, "text": "Often they'll just tell you it felt right. Whereas a great chess player will never normally say that. They'll normally tell you, you know, I planned A because I thought B was going to happen and I was going to do C. Now, that plan may not be a good plan, may not work out, but they had a specific plan in mind and they had calculated that out. In Go, it's much more about feel and intuition."}, {"start": 1974.32, "end": 2004.69, "text": "Which is surprising, and that maybe explains to you why Go is that much harder for computers to play, because intuition and feel are not two things you would normally associate with computer programs. So just to cycle back to what I was saying at the beginning of the talk about Deep Blue and AlphaGo, so Deep Blue, as I mentioned, had handcrafted chess knowledge. These rules and heuristics that were distilled from human chess grandmasters and then encoded by this amazing team of programmers."}, {"start": 2005.3, "end": 2033.09, "text": "By contrast, AlphaGo learned its knowledge by observing expert games and also through practice and self-play. Deep Blue also does full-width search. In other words, it searches all the possible moves in a position. AlphaGo, by contrast, did highly selective search that was guided by these two neural networks. So it only looked at the most probable moves and the most likely moves. And because of that..."}, {"start": 2033.09, "end": 2052.5, "text": "Deep Blue, in order to beat Garry Kasparov, needed to look at and calculate 200 million positions per second in order to win that match. Whereas AlphaGo only needed to look at 100,000 positions per second. So several orders of magnitude less search than Deep Blue needed to do."}, {"start": 2054.9, "end": 2084.16, "text": "I'm just going to briefly mention intuition and creativity. I've talked a little bit about that in relation to Go. So I should perhaps define a little bit more about what it is I mean by these terms in this context. So intuition, I think one way you can kind of operational definition of that would be implicit knowledge that you've acquired through experience, but it's not consciously accessible or expressible. So you can't describe it to someone else. In fact, you may not even be able to"}, {"start": 2084.16, "end": 2112.22, "text": "Verbalize what it is that you know yourself. So you might ask, well, if it's that kind of implicit knowledge and you can't really access it consciously, how do we know it's there? Well, of course, you can probe that knowledge and the existence and the quality of that knowledge behaviorally. So for a Go player, you can just give them a Go board and then see the quality of the output that this implicit system, their intuition, will come up with. Secondly, creativity."}, {"start": 2112.22, "end": 2139.34, "text": "So again, an operational definition in terms of this context might be the ability to synthesize knowledge that you have to produce a novel or original idea. And I think AlphaGo, with its Move 37 and several other moves that astounded the Go world, clearly has demonstrated both of these abilities, of course, albeit in a constrained domain like Go. So I'm not suggesting that this is the be-all and end-all of creativity."}, {"start": 2142.22, "end": 2169.84, "text": "So just talking about intuition and creativity, one other important thing about our research program at DeepMind is the use of systems neuroscience. And I did my PhD in neuroscience, and we tried to combine some of the best ideas of systems neuroscience with computer science and kind of cross-fertilize those ideas together. Now, we talk about systems neuroscience specifically because we're interested in this kind of high level of neuroscience rather than the biological details."}, {"start": 2169.84, "end": 2200.61, "text": "So I talk about we're interested in the algorithms and the representations and the architectures the brain uses, rather than the specific synapses and wetware and the implementation details of how the brain does things. So this is not reverse engineering the brain, but it's being inspired by some of the algorithms the brain uses. So we're looking at this kind of systems neuroscience-inspired AI. We're looking at all sorts of areas and capabilities that we would like our AI system to have that we don't currently have. So memory."}, {"start": 2200.61, "end": 2216.34, "text": "Attention, Concepts, Planning, Navigation, Imagination. These are all active areas of research at DeepMind where we're also, apart from doing machine learning, we're also looking at how the brain solves some of these problems. And it turns out..."}, {"start": 2216.34, "end": 2238.91, "text": "They're actually a key area of the brain that's relevant and implicated in all of these capabilities is actually the hippocampus, which is here noted in the middle of the brain in pink here. It's right in the center of your brain. And it's important for memory. This was kind of been known for 50 to 100 years now. If you have damage to your hippocampus, then you're amnesic. So you don't have episodic memory."}, {"start": 2238.91, "end": 2268.72, "text": "But part of my PhD work was to show that you also need hippocampus for a bunch of other things, including imagination, imagining the future, as well as remembering the past. So it's really key for a lot of these very important high-level tasks. And what you can think of one of the projects we're doing at DeepMind is effectively building an artificial hippocampus to fit together with the neural network systems that we already have."}, {"start": 2268.72, "end": 2298.11, "text": "We're playing games and we've mastered Atari games. We're now going to much more advanced 3D games. We've also mastered the game of Go. But the reason we're developing these algorithms is not obviously to ultimately just play games, despite the fact that that's obviously a very efficient way to develop AI. We want to apply these technologies to the real world. And we're starting to make progress with that now in many directions. And because the kind of algorithms we work on are very general,"}, {"start": 2298.11, "end": 2325.89, "text": "We hope that we can apply it to a really wide range of real-world tasks that are going to make a huge positive impact in the world. So we started looking at healthcare, medical diagnostics in partnership with the NHS, starting with medical imaging of things like head and neck cancer and also macular degeneration and 3D images of your retina. We're also looking at robotics as an application area, and we're even looking at things..."}, {"start": 2325.89, "end": 2355.7, "text": "As interestingly as data center optimization and energy efficiency. So I'll just end by talking a couple of specifics about things we've done recently. So with the data center project, what we did is we effectively applied AlphaGo-like techniques onto the cooling systems of a data center, a Google data center. And bear in mind that the Google data center is already some of the most efficient in the world in terms of the power they use."}, {"start": 2355.86, "end": 2378.96, "text": "And what we were able to do is, by using our learning systems, we were able to save 15% of the power they use by more effectively controlling the cooling systems in the data center. So the fans, the cooling water, even opening and closing windows, balancing the load of where the computation comes in, and by tracking all the sensors dotted around the data center temperature and so on."}, {"start": 2378.96, "end": 2400.46, "text": "And you can see the graph here of power usage. So this is the amount of power on the y-axis that the data center is using. And you can see clearly when we switch on the machine learning system, our machine learning system, how much the power drops. And then when we switch back off the machine learning system and go back to manual control, how much the power usage jumps back up."}, {"start": 2401.3, "end": 2429.95, "text": "So obviously this is worth tens of millions of dollars of saving in energy, but also it's great for the environment, obviously using less energy for these data centers. And we actually think now we would like to apply this to perhaps a national grid scale and try and optimize an entire power grid, not just a single data center. A very different application that we just announced a couple of weeks ago as one of our latest breakthroughs is actually in a completely different area."}, {"start": 2429.95, "end": 2454.75, "text": "So this is text-to-speech systems. So you take some text that you've typed in or you've got from an e-book or the web, and then you generate speech out of it. And this kind of system is used in your phones all the time. If you use Android or Apple or the iPhone, they have these speech systems built in."}, {"start": 2454.75, "end": 2480.24, "text": "So that the phone can actually talk to you. And most of these systems use what's called concatenative models. So those are kind of state of the art models today. And what those are is that you can think of them very basically as they get an actor to come in and speak for about 30 hours of different forms of dialogue. And then they chop up that speech, that 30 hours of speech, into individual syllables."}, {"start": 2480.24, "end": 2507.23, "text": "And then when you now give it a new piece of text that it's never seen before, you patch together and stitch together those different syllables to create those new words and those new sounds. But because this system is stitched together, that's what makes it sound a little bit robotic. Even the best systems still sound quite robotic. Instead of that, we created a new type of neural network, which we called a WaveNet, that models the raw waveform directly."}, {"start": 2507.23, "end": 2544.0, "text": "The raw speech waveform. And it can generate new waveforms directly. So not by stitching together example syllables from a pre-described sort of data set. And just to show you what a waveform is, so this is what a waveform looks like, an audio waveform. And if I play the waveform, it's a pretty famous piece of text. So that's what the waveform of Neil Armstrong's phrase looks like. So we basically take raw data like this, feed that to the neural network,"}, {"start": 2544.0, "end": 2559.2, "text": "And it starts to figure out how waveforms are created. So it can generate new waveforms for text that it's never seen before. So WaveNet, I haven't got time to go into details about the actual underlying technology."}, {"start": 2559.2, "end": 2576.74, "text": "But basically, there are two key things in new innovations about the WaveNet from the other kind of neural networks I was showing you. So one is this idea of dilation. So the neural networks, they normally take inputs, and they try and figure out what those inputs, what features are in those inputs, say a picture."}, {"start": 2576.74, "end": 2602.83, "text": "What we do here is, because this is audio, we increase the dilation, the size of the input channel, so that it can take more and more time into account as you go higher up the hierarchy of a neural network. So you can see here, at the bottom of this diagram is the input, and then as you go higher and higher up, each of those nodes takes into account a longer amount of timescale. So that gives you the coherence that you need for speech."}, {"start": 2602.96, "end": 2632.03, "text": "The second thing is this idea of autoregressiveness. So once you generate the next little sound for the speech, we actually, if you see the animation, we actually output that sound, and then we use that output as the next input into the neural network. So you can think of what the neural network, WaveNet, is going to generate next is conditioned on all the past things that are generated. So that's what allows it to create coherent speech and words."}, {"start": 2632.03, "end": 2658.85, "text": "So in a way, you can kind of think of it as memory. So it creates a whole pattern of sounds that actually fit together. So it's these two innovations that allow WaveNet to work. And the result is that you get 50% improvement over the state of the art in terms of the gap between the state of the art, current state of the art, and perfect level, which is actually humans speaking directly those pieces of text."}, {"start": 2658.85, "end": 2675.94, "text": "And just to explain to you what these bars are, it works in Chinese and English, and we're about to do it for all of the other languages. And you can see here on the red bar is the concatenative model. These scores are out of five, so five would be perfect. Now, you might wonder why the human speech..."}, {"start": 2675.94, "end": 2706.86, "text": "Which is the green bars is not 5 out of 5. And that's because after you've encoded it and decoded it digitally, you actually lose some sound quality. So people even rate the actor themselves speaking those words as only 4.55 in English. And you can see that WaveNet has increased this user score from 3.86 to 4.21. Which is a huge jump in performance. And it's even bigger in Chinese. Now I'll just play you two samples so you can hear it."}, {"start": 2706.86, "end": 2730.08, "text": "It's a strange piece of text talking about a book because the reason this text is this is from the canonical test set of 100 phrases that Google uses to test the quality of their speech systems. So this is the first one I'm going to play is the current state of the art. The Blue Lagoon is a 1980 American romance and adventure film directed by Randall Kleiser. So you can hear a little bit of the warbling that you get."}, {"start": 2730.08, "end": 2759.62, "text": "And this is WaveNet. The Blue Lagoon is a 1980 American romance and adventure film directed by Randall Kleiser. So hopefully you agree that the second one is the much more natural sounding one. And so it's a huge difference in terms of how much more natural and sort of quality level of this. And that also makes a big difference to things like how much users want to engage with these systems. So that's all I've got time for. But I just want to finish on just sort of talking a little bit about the big picture."}, {"start": 2759.62, "end": 2787.2, "text": "And going back to this idea of, you know, if we could solve intelligence in a fundamental way, then we can use it to solve everything else. Now, the reason I've worked my whole career in AI is that what I'm really looking forward to applying these AI systems to is science and healthcare. So try and drive breakthroughs in those areas by working with the best human experts in those fields and actually help them manage the amount of data they get."}, {"start": 2787.2, "end": 2819.01, "text": "And make sense of that data so that they can make breakthroughs much more quickly. Thanks for listening."}, {"start": 2819.01, "end": 2840.58, "text": "Is artificial intelligence going to give us that ex machina moment when you and I can be replaced by a humanoid robot? Cheesy question, but I've got to ask it. Well, you know, the thing that's held robotics back actually is nothing to do with the physical capabilities. You know, we have robots that are very dexterous now, can run fast and so on. And if they're controlled..."}, {"start": 2840.58, "end": 2867.22, "text": "By a human controller, they can do a lot of things. What we'd be missing is the algorithms, the intelligence. And I think we're still many, many decades away from anything like ex machina, assuming we want to build humanoid robots. But I think we will start seeing some very interesting industrial applications of robotics in the next sort of five plus years. Let's have our first question then, just over here. If you could stand up and introduce yourself."}, {"start": 2868.75, "end": 2895.9, "text": "Hello, thank you very much for a wonderful presentation. My name is Zaza Zaleki. My question is about the applicability of this logical intelligence system also to humanitarian conflict prediction, food security and humanitarian, to what extent it is possible that..."}, {"start": 2895.9, "end": 2926.05, "text": "This type of technology would be able to coordinate all the UN bodies together in order to, if not prevent conflict, but prevent food security shortages. Well, I think, you know, actually one thing I should mention is that the AlphaGo prize money that we won, the million dollars, obviously we won the match, so we donated it to charities, and we gave $450,000 of that to UNICEF, and we got them in."}, {"start": 2926.05, "end": 2955.86, "text": "To talk about exactly, you know, all the data they have, they really need help with using that for things like food security or child protection and these kinds of things that they do. And we're starting to explore how we can maybe be able to help. I think when it comes to politics or, you know, conflict resolution, I think that's going to be a lot harder and that will be one of the last things that I think AI will be able to help with because it's so complex and it so, you know, involves emotions and..."}, {"start": 2955.86, "end": 2985.14, "text": "And human motivations that are difficult to see just in data. So I think to the extent that we can codify that in data and the results are there in data, I think we might be able to help. I think part of the problem with conflict resolution is that people are not necessarily rational on either side, even if the AI came up with the right solution. Based on data, it doesn't mean that the powers that be would actually take any notice of that. That seems to be the evidence of today's world. So I think we might look at NG."}, {"start": 2985.14, "end": 3000.59, "text": "We have CEOs as a first starting point who actually are really keen to engage with this technology and really want to help with things like disease control and poverty and water security. So actually we're very passionate about that and we hope in the next couple of years to start some projects in that direction."}, {"start": 3000.59, "end": 3025.18, "text": "Very nice to start off with a positive use of AI, which is great. Let's have our next question. Do introduce yourself. I'm John Taysom. I'm a visiting professor in the Department of Engineering at UCL. And I was very struck by the engineering approach that you've taken to find solutions. And my question actually follows quite nicely from the previous one because it's really about the second area where perhaps we can make significant contributions to..."}, {"start": 3025.18, "end": 3053.54, "text": "Preventing humans compromising lives of other humans. And that's to do with car crashes and automotive problems. Clearly a lot of work's going on in autonomous vehicles. But from an engineering point of view, I'm old enough to remember the early days of the internet and the engineering decisions that had to be taken then, whether you had intelligence in the packet or intelligence in the switch. And of course it was a dumb packet intelligence switch design that won. Are we going the wrong way with autonomous cars?"}, {"start": 3053.54, "end": 3067.66, "text": "I think there's a lot of very interesting work going on in autonomous cars. Most of the work that I've seen, they use learning systems for some aspects of the system. So perhaps pedestrian..."}, {"start": 3067.66, "end": 3094.67, "text": "Identification, things like that, like vision systems are using learning systems. But the planners, the things that actually control the cars, are usually handcrafted. So they're normally rule-based, and they're getting very large because obviously there's all sorts of exceptions that can happen on the roads that they have to try and cater for. So it remains to be seen whether you can actually handcraft and hard-code every possibility that's going to happen, or whether you need to learn it in the way that human drivers learn."}, {"start": 3094.67, "end": 3116.08, "text": "Now, the debate's a kind of open debate because it depends on the level of autonomy that you're after. So if you just want to drive on a highway, then that's much simpler and probably a rule-based system will be enough. If you want to drive on any road under any condition, even dealing with an unexpected thing like the police closing off..."}, {"start": 3116.08, "end": 3146.93, "text": "Let's have the next question. There's one just down the back and there's one at the front. Fire away. Adrian Seccombe. I research into cyber agency. I'm very interested in understanding what you're doing to be able to identify the first glimmer of virtual life and how to manage it."}, {"start": 3147.86, "end": 3175.14, "text": "So, I mean, we've got to define what virtual life means. So I don't know if you have a definition in mind. I presume you've got one of your algorithms looking for that. Actually, I think a simple definition is, do you feel a pang of guilt if you turn one of your AI systems off? No, I can tell you categorically no. But I actually think, you know, so certainly not the current systems, the Atari programs or AlphaGo. I do think that..."}, {"start": 3175.14, "end": 3191.58, "text": "It may not take that much for you to feel quite attached to these systems. If you think about your pets, like your pet dog and your pet cat, they don't do that much, but yet they elicit huge amounts of loyalty. No, I'm just saying."}, {"start": 3191.58, "end": 3221.84, "text": "Given that they don't do that much, they elicit a huge amount of emotion in us and loyalty. Now, of course, perhaps they've co-evolved with us to elicit that. Cats purr and dogs wag their tail. We find that very attractive. But I don't think it would take much behavior for us to start thinking that these systems are exhibiting some interesting behavior that maybe is important. But for now, right."}, {"start": 3221.84, "end": 3246.78, "text": "But that also comes down to a design choice too about how much freedom we're going to give these systems to design things like their own goals. And that's an open research question at the moment. There we are. Next generation AI is going to have big eyes and a waggy tail. You heard it here first. Okay, let's have our next question. There's another one here as well. Hello, my name is Nicole Roberts. I just wanted to ask, futurists and theorists always talk about..."}, {"start": 3246.78, "end": 3276.24, "text": "The problems that they predict in the future with AI. And I was just wondering how technology now is trying to mitigate the problems with imperfect systems trying to create perfect systems. Because human beings are inherently biased. We have our faults. And are we possibly, with the algorithms, putting these into the new AI, possibly? And how are we mitigating the future problems with that? Yeah, it's a really good question, actually, that."}, {"start": 3276.24, "end": 3289.47, "text": "I think with these algorithms, we have a chance actually of trying to mitigate for that bias because one of the important things that we're building and we need to do is to build analysis tools for these."}, {"start": 3289.47, "end": 3308.54, "text": "Currently kind of black box systems so that we know what it is that they're learning and why they're learning and why they're making their decisions. And if we have those tools and lots of people are building those tools now and we've got a whole project on that called Virtual Brain Analytics, then we can maybe mitigate those biases if we know they're going on."}, {"start": 3308.54, "end": 3335.95, "text": "And actually compensate that by putting in some other numbers or another adjustment to that system so it kind of balances out the inherent bias. But it is a good question because what happens if the bias is there in the data? Then it will pick that up, right? But I think that there are always top-down controls we can do to try and mitigate and balance that out as long as we understand those systems well enough, which we really need."}, {"start": 3335.95, "end": 3350.86, "text": "Another question just down here. I think we've got a microphone just here. Oh, sorry, I beg your pardon. It's all the way at the back. I beg your pardon. Fire away. Thank you, Demis, for your wonderful talk."}, {"start": 3350.93, "end": 3379.5, "text": "We've heard in the first speaker, sorry, Vanessa Lawrence, Location International, we heard from the first question asking about humanitarian work and we know that the United Nations Sustainable Development Goals are very much going to be successful if we're able to measure and monitor what's exactly going on and if countries have the data in order to provide to the decision makers."}, {"start": 3379.5, "end": 3405.47, "text": "And much of that's going to be done by satellite and imagery and drones, where absolutely the wealth of data, very much there's many of us already trying to work to think about how AI can actually make a difference so that countries and regions really are able to submit these for the measuring and monitoring that's going on. And I wondered if you've been involved in this."}, {"start": 3405.47, "end": 3431.18, "text": "Yeah, that's good. Actually, I was talking just last week to a CEO of a microsatellite company that's putting lots of satellites up very cheaply so they can monitor 24-7 land masses on countries for things like NGOs. And he was asking me this exact question, and I think we're probably going to do something with them to monitor promises that governments make about various things they're going to do."}, {"start": 3431.18, "end": 3451.95, "text": "You know, you want to be able to monitor that. Like, are they, you know, are they really protecting this area of the forest? Or, you know, have they put down new irrigation? And actually the best way of doing that is often with satellite imagery and tracking that 24-7 so that you can hold them accountable. But the amount of data is so huge that, you know, I think you probably will need something like AI to help."}, {"start": 3451.95, "end": 3482.19, "text": "Help with that monitoring. Otherwise, you're going to have to have armies of people just looking at satellite imagery, which obviously most NGOs can't afford to do. So I think AI is going to be a very critical part of that solution. If you've got any ideas for specific projects, we'd love to hear that. But we are starting to talk to satellite companies about this. We've got another question just here. Thanks for a great speech, Demis. My name is John Lazar. I had a question about..."}, {"start": 3482.19, "end": 3509.12, "text": "There's this very oversimplified debate in the AI community about, is there going to be a singularity? Do you subscribe to the notion that that may happen? And if so, when that super intelligence happens, is it 2050? And is it likely to be benign or a danger? Yeah, I don't really subscribe to that word singularity. I think it's..."}, {"start": 3509.12, "end": 3515.95, "text": "It's a bit of a science fiction concept. But I do think that, and partly a lot of these assumptions..."}, {"start": 3515.95, "end": 3544.93, "text": "Coming from philosophers and other people that may not have held out to be true once we start engineering these solutions and understanding much better about the systems that we're dealing with. One example of that is this idea that machines will be built that can then write themselves. Maybe that's not something we'd want from our machines. There's a design choice there. I think what is important, though, is to make sure..."}, {"start": 3544.93, "end": 3555.44, "text": "that we start thinking now about the ethical and responsible use of these systems as they get more powerful. You know, I think technologies are like AI and kind of inherently neutral."}, {"start": 3555.44, "end": 3584.78, "text": "But it depends on how we decide to use them and what applications for that end up governing whether they're good or bad for the world. So I think we need to start thinking about this and also start researching important questions like setting goal systems. How do we set the right goals? How do we specify what exactly we want these systems to do and nothing else? And how do we keep those goals secure when the rest of the system is developing and adapting? And there are tough research questions there."}, {"start": 3584.78, "end": 3601.5, "text": "PhDs need to be done in this field. But I'm very hopeful and I'm very optimistic that with the right amount of human ingenuity focused on this problem, we'll solve all those issues in the coming years, way before ahead of time anything like a singularity moment would happen."}, {"start": 3601.5, "end": 3619.54, "text": "It's always seen as AI or humans, but of course you can see the eventuality where we could get an AI upgrade, for example, and start to merge as well. So there is a middle way. I always think about actually what we're building as perhaps one of the most powerful tools for humans to enhance what it is they can do."}, {"start": 3619.54, "end": 3647.38, "text": "It's kind of like a multiplier in human ingenuity and capable of coping with things that even the best human scientists are not good at, which is dealing with these vast amounts of data and keeping up to date with that whilst trying to find the insights in that. I think computers will be very helpful with assisting with that. Now, the microphone's headed over here. Hi. Yeah, so my name's Michael. I suppose my question is a bit around kind of..."}, {"start": 3647.38, "end": 3674.67, "text": "I'm going to struggle to express this, but knowability. So the kind of algorithm you described seems to kind of, I suppose, learn and imbibe what to do from training itself and experiencing things. I'm just wondering, does it matter that we possibly struggle to convey how exactly it does that to each other? So, I mean, if a plane, we taught a plane to fly itself by going in a virtual environment and flying itself."}, {"start": 3674.67, "end": 3704.54, "text": "10 billion hours. But then, do we have any guarantees that, say, the first time a plane actually flies off with this algorithm, that it won't fly and do loop-de-loops in the air or something? I suppose, yeah. Yeah, so it's related to the question that the lady asked me at the front here, where I think the solution to that is we need to... Well, firstly, you would never do that. You would test it another 10 billion times in simulation first to check what it was going to do. And then you would..."}, {"start": 3704.54, "end": 3718.96, "text": "Control test it in a test plane like you would with any kind of new technology before rolling it out. But the other thing we need is better tools for interrogating and understanding what these systems are learning."}, {"start": 3718.96, "end": 3748.3, "text": "And the representations they're creating. So actually, just to expand on my last answer about that, one of the inspirations that we're getting from that is from neuroscience, where I did my PhD in fMRI imaging. And there were lots of visualization tools and statistical tools, very powerful ones, for analyzing what's going on in your brain activity. And you get all these pretty pictures of brain activations. And we need, I think, the equivalent of that for artificial neural networks, which is why we call that whole project."}, {"start": 3748.3, "end": 3775.84, "text": "Virtual Brain Analytics. And we actually hired some of the world's top computational neuroscientists to help create that tool. And we've already been using it in some of our recent Nature papers. We published a few of the insights into that. And we need to go a lot further in the next few years with that. I think we've got time for a couple more questions. One chap here. This chap's been desperate to ask a question. Help him as well. No, no, no. Go on. I just want to get the microphone queued up afterwards. There we are. Fire away."}, {"start": 3775.84, "end": 3801.84, "text": "Two very friendly questions, great presentation. What do you say to AI boosters like the Guardian who say that Siri is artificial intelligence on an iPhone and that we are increasingly reliant on driverless cars? I don't know whether you're reliant on driverless cars, but I hadn't noticed the second trend. And then going a bit to Roger's point about the virtual pet."}, {"start": 3801.84, "end": 3829.23, "text": "The Japanese had that 20 years ago, if I'm not mistaken. I've forgotten the name, but that's it. That you feel guilty about not feeding or turning the plug off on. What do you feel about the Turing test and the hard AI thesis versus the soft AI thesis? You mentioned narrow AI, and I think your second category was wide AI? General AI. General AI."}, {"start": 3829.23, "end": 3858.48, "text": "Which do you subscribe to? Just because it looks like a human being, is that our level of intelligence, which is the soft, let me get it right, the hard AI thesis? Or do you think, yeah, well, you can put on a display, but it's still electrons and it's not human. Where do you stand on that? So just to address the first question, I think it's great that there's this huge excitement about AI. I don't think it really matters that, I mean, a lot of people don't,"}, {"start": 3858.48, "end": 3880.8, "text": "Don't see the distinction between learning AI and sort of narrow AI, but I think that's fine. I think it's great that there's a lot of excitement and many of the brightest students now are going into machine learning for their PhDs and master's courses. So I think that's really good. On the second question, the Turing test. So I don't think the Turing test was really meant..."}, {"start": 3880.8, "end": 3900.7, "text": "by Alan Turing to be a rigorous test. And the problem is, obviously he was thinking about that in the 40s and 50s, and we didn't have much in the way of computers and graphics and other things. So all he could think about was text. And it turns out that that's not a good test because it can be gamed very easily."}, {"start": 3900.7, "end": 3921.2, "text": "And there's systems that periodically say they've solved the Turing test and they're just parlor tricks. Actually, it turns out humans are quite easy to fool if you have a short five-minute dialogue with them and they have some kind of expectations. And they're not necessarily computer science experts."}, {"start": 3921.2, "end": 3950.46, "text": "It's interesting from an inspiration point of view, but I don't think it's really a rigorous test that should be taken seriously. Having said that, the deeper question it gets at, which I think you were asking, was at what point... It's probably related, I guess, to the consciousness question. At what point is a system really human-like, depending on the behaviour it exhibits? And actually, we're kind of agnostic about that at the moment, in the sense that we're looking to build intelligent machines, and I think that..."}, {"start": 3950.46, "end": 3979.14, "text": "Consciousness is not a well-defined capability. Even philosophers haven't agreed on the definition. So I think that's one problem with asking, are machines conscious? Can they be conscious? But my betting is that intelligence and consciousness are dissociable. So what I mean by that is I could imagine building a very smart machine that doesn't feel conscious in any way in the same way you do to me and I do to you. So I think they are going to be dissociable. But interestingly..."}, {"start": 3979.14, "end": 4008.43, "text": "From wearing my neuroscience hat, I'm obviously very interested in what consciousness is and this phenomenon of consciousness. And I think that this journey we're on to build general AI in this neuroscience-inspired way, I think might shed light on this consciousness question. In the future, we may have this artificial system that we can compare to the human mind in some ways, and maybe that will reveal some of the deep mysteries of the human mind, like consciousness, dreaming, creativity, and elucidate either what's special or how..."}, {"start": 4008.43, "end": 4039.31, "text": "How to mimic those behaviors. Either way, I think that would be an interesting scientific question. One very last quick question. I'm not sure where the microphones ended up. Okay. Make it very quick. Hey, I'm Jaz. I work in AI at PwC. So as AI becomes more integrated into our lives and people rely on expert-type systems, for example, which skills do you foresee becoming most essential in the future? Yeah, it's a good question, actually. I think..."}, {"start": 4039.31, "end": 4063.84, "text": "I think creative skills are going to be more and more valued. So you'll have these very powerful systems that can crunch lots of data, but you'll still need to, I think the top creative people will still be able to use those in unique ways that add much more value. I think it might be a multiplier on that. Interestingly, having self-learning systems might..."}, {"start": 4063.84, "end": 4092.24, "text": "Democratize kind of AI and engineering in a sense that maybe you don't need to know how to exactly program these things because maybe you can teach them instead. So it's interesting that that could happen. I think also a lot of the skills, human skills like empathy and emotional skills, those types of skills and those types of jobs might become much more valuable than they are today. So I think there's going to be lots of interesting sort of thought debates about that."}, {"start": 4092.37, "end": 4109.65, "text": "Fantastic. Well, thank you for your brilliant questions. Thank you to the Royal Academy of Engineering. And finally, I think we must give one last hand of applause to the superhero of AI, Dr. Demis Savis. Thank you. Fantastic. Cheers. Brilliant."}]}