[0:00:00] And welcome to the second Royal Television Society and Institution of Engineering and Technology public lecture. I'm Tim Davey, I'm chairing tonight. And, you know, when we conceived of this lecture series, our aim was to hear from some of the world's finest minds who work in the spaces that bridge cutting-edge science and technology with human creativity and media. The amazing response, and we were lucky to be there last year, those of us who saw Mike Lynch in full flow,
[0:00:30] Suggested we were onto something. And the excitement in the room tonight speaks to the enduring appeal in my mind of public lectures in sparking our imagination. In many ways, and I really believe this, we are lucky enough to be enjoying another golden age of advancement that demands public engagement and debate. Over 200 years, it was a rather precocious Humphrey Davy that was wowing crowds and causing carriage traffic jams.
[0:00:59] We sell out lectures on the nature of human progress and scientific knowledge. Noticeably, as a friend of Coleridge, he was fascinated by the intersection of the subjective and the technical and the possibilities, the endless possibilities that this throws up. So tonight, at this wonderful institution, we continue that tradition with our very special speaker, Demis Hassabis. Demis will talk for 35 minutes or so.
[0:01:28] Then he will open up for Q&A, and I'll be conducting, have the lights up, and we're really up for a good old discussion with you guys. As you know, Demis is acknowledged as a world-leading thinker in the realm of AI, which in recent years has become one of the hottest topics, dominating the media and the imagination of the public. You know, in my world, we ran a recent BBC survey which was looking at which jobs are at risk of computerization.
[0:01:55] We attracted 2.3 million page views, people viewing that and seeing how safe they were. Demis needs little introduction, but I have to say his achievements put even the highest achiever's CV deeply in the shade. Chess master at 13, World Games Championship winner five times running, successful poker player, particularly impressive, double first in computer science at Cambridge, a pioneering video games developer.
[0:02:23] Leader of numerous important pieces of research in the field of neuroscience. Notably, his landmark paper on the similarities of how we shape memory and how we imagine the future was a landmark and breakthrough piece of research. And then, of course, founder of DeepMind in 2011, which was then sold to Google in 2014, with Demis becoming VP Engineering with special responsibility for AI.
[0:02:52] DeepMind has set out a goal of solving intelligence, a humble objective. Specifically, Demis has said he is involved in building something that can expect the unexpected gracefully. I think that's probably a great brief for an audience of a public lecture. Demis, the floor is yours. Well, thanks very much, Tim, for that very generous introduction. So it's a real pleasure to be here giving this lecture. And I'm going to talk about...
[0:03:29] Artificial intelligence and its impact on the future. In fact, it could be the relatively near-term future. So AI is really the science of making machines smart. And I got into AI firstly through the medium of games. And games started for me with chess. As Tim said, I started playing chess when I was very young, at the age of four. And I think if you play chess seriously from such a young age, and you're quite an introspective kid, which I was,
[0:04:00] Then you start thinking a lot about how is it that your brain is coming up with these moves, these ideas that allow you to play this game and win these games. And I started thinking a lot about this as I got into my teenage years. And allied with that, I got into computing. I actually bought my first computer, ZX Spectrum here, 48K, with some winnings from a chess tournament when I was about eight years old. And I started teaching myself how to program.
[0:04:30] And I think very early on, and the engineers in the audience will, I think, resonate with this, I sort of realized on an intuitive level that this computer is a kind of special type of machine. You know, most machines, like cars and planes, they allow us to, they extend our physical capabilities. You know, cars allow us to move fast and then we can run. Planes allow us to fly. But I think computers do that, but in the realm of our minds. They really extend the capabilities of the brain.
[0:05:00] And this really became clear to me when I used to write my first programs and did sort of basic math calculations and other things, which it really struck me. You could set something running overnight and then go to sleep, and then you'd wake up the next morning and your computer would have solved some problem for you whilst you were asleep. So this felt like a really powerful and in some ways magical tool. So my love of computers and my love of games obviously came together in a kind of obvious way.
[0:05:30] In the designing of video games. And actually, this is one of the reasons I accepted to do this lecture is I love the idea, as Tim said, of the confluence of bringing together RTS and the creative arts and the IET and sort of engineering. And that's why I got into commercial video games. Because at the time, this is sort of like the early and mid 90s. Computer games were really pushing the cutting edge of engineering. And even the machines that were being built.
[0:05:59] to run these games. So I remember the debates in the 90s about Intel was bringing out their new Pentium processors and people were sort of saying, well, how much more power do we need to run our word processors and spreadsheets? And haven't we got all the computing power we need? And actually, one of the answers was that if we wanted more and more realistic and complex games, then we would require more and more powerful computers with larger memory and things like graphics chips. So for a long while...
[0:06:28] Games were actually driving the development of cutting-edge hardware. And furthermore, the games that I used to sort of design and program all involved AI as a core gameplay mechanic. So probably my best-known game was called Theme Park, which is some screenshots of it here. This came out in 94 and was very successful. And it was actually the first game of its type. So the idea here was that you designed your own Disney World.
[0:06:58] And thousands of little people would come in to your Disney World and kind of play on your rides. And how enjoyable they thought your theme park was would sort of have an impact on their emotions and how happy they were. And then that fed into the economics model about how much you could charge them for the hamburgers and the balloons. So the better design your theme park was, the more money it made. And then that allowed you, of course, to expand the theme park further.
[0:07:25] So this game and actually another game called SimCity were the first sort of games that had AI as a core gameplay component and really spawned a whole genre of management simulation games, as they're called. And one of the reasons these games are so popular is that the AI adapted to the way the player played the game. So that means that every single person who played this game had a unique experience.
[0:07:49] And people used to send in, I remember, into magazines, game magazines, and write in to us showing what end state they got their theme park into. And there was all these amazing designs that people had created that we had no idea could be done, even as the inventors of this game.
[0:08:07] So that really struck a chord with me when I was around 16, 17 years old when I wrote this game and thinking about, you know, maybe if I devoted my career to AI, advancing AI, what an incredible technology that could be. So then after having a career in games and running my own games companies and things, I then went back to academia to do a PhD in neuroscience, which I felt was another piece of the puzzle that I needed before launching an effort like DeepMind.
[0:08:37] I wanted to understand a bit more about how the brain solved tough problems like imagination and memory. And I specifically picked those topics to do my PhD on because those are things that, at least back into mid-2000s, we were not very good at doing in computer algorithms. So I wanted to look at the way the brain solved some of these very, very tough problems that we didn't know yet how to imbue our machines with.
[0:09:04] And I'll come back to that towards the second half of my talk. So all of these different experiences then culminated in finally in setting up DeepMind in 2010.
[0:09:15] And really, it's been a 20-year plus journey for me to get to this point and have enough of what I thought were the basic ingredients, both on an algorithmic level, but also in terms of the founding scientific team and making those contacts to actually put together something like DeepMind and plausibly go after as big a mission as solving intelligence. So another way we look at the company is as an Apollo program for AI, a sort of moonshot project.
[0:09:46] That really focuses on the very ambitious long-term goals. And we've collected together more than 100, actually nearly 150 now, of the world's top research scientists in this area. So I think DeepMind is by far now the biggest collection of machine learning experts anywhere in the world. And another thing we're experimenting with, of course, apart from trying to build AI, is actually new ways to organize scientific endeavor.
[0:10:12] So what we've tried to do with DeepMind is really combine the best from Silicon Valley startups together with the best parts that you find in the best academic institutes like MIT, UCL, Cambridge, and so on. And see if we can fuse that into a new hybrid way of doing science, which is more productive and extremely efficient, but still allows for extreme creativity. So our mission then, as Tim said, we articulate it in a kind of two-step.
[0:10:42] So firstly, we talk about solving intelligence. And we use the word solve, which is a kind of ambiguous word there because actually what we mean, what we're interested in is understanding natural intelligence, so the human mind, but also recreating that intelligence artificially. And then step two, we want to use that technology to help us solve everything else. Now, you know, that might seem a little bit far-fetched.
[0:11:10] Possibly a little bit fanciful to some of you, but we really believe actually that step two naturally follows on from step one if you can solve intelligence. And I hope by the end of this talk, you'll agree with this conjecture. So more prosaically, how are we going to solve intelligence? Well, what we're trying to do at DeepMind is to construct the world's first general purpose learning machine. And the key aspects of this are the word general and learning.
[0:11:41] So at DeepMind, we're only interested in algorithms that learn for themselves. So they learn automatically from raw experience or raw data. So they're not pre-programmed in any way. So what we're talking about here is autonomous learning systems. The second thing is this idea of generality. So what we're interested in is the same system actually being able to operate across a wide range of tasks and environments out of the box.
[0:12:10] With no reconfiguration. So, of course, we have an example of such a general learning system. It's the human mind, where we're able to apply our minds to almost endless number of different tasks. Now, I should say, most of AI today, although it's a huge buzzword right now and is very fashionable, most of it, AI is not of this type of technology. So we call most AI, actually, internally at DeepMind, narrow AI.
[0:12:36] And what we mean by that is pre-programmed AI that is being built in a bespoke way for one specific task. And actually, most of the AI we interact with every day, from Siri on your phone to self-driving cars, is actually of this pre-programmed type of AI. And what we're interested in is what we call artificial general intelligence, this idea of a general learning system.
[0:13:02] And perhaps still the most famous and clearest example I can give of this is the famous Deep Blue match against Garry Kasparov. Of course, this was a watershed moment in AI when in the late 90s, IBM's Deep Blue beat Kasparov in a six-game chess match. But the interesting thing is I came away from that match actually more impressed by Garry Kasparov's mind than the Deep Blue machine because, you know, of course, it was an impressive engineering feat.
[0:13:31] But Deep Blue was programmed by an amazing team of programmers along with a bunch of chess grandmasters trying to distill chess knowledge into an algorithmic sort of construct. And those programmers were directly programming in the sort of ideas and solutions into the machine. And of course, what that meant is that Deep Blue, although it was very good at chess, it was no use for absolutely anything else.
[0:13:56] Including strictly simpler things like, for example, playing noughts and crosses, which any chess ground master, you could trivially teach them how to play noughts and crosses. But obviously, Deep Blue, nothing that Deep Blue knew or in its code would help it with even something strictly simple like that, let alone other kinds of domains like speaking languages or driving cars or all these other things that, of course, Garry Kasparov could do effortlessly.
[0:14:22] So instead of that, we think about intelligence in the framework of what's called reinforcement learning. So I'm just going to illustrate what the main basic parts of that in this little cartoon diagram, because it's important for what I'm going to show next in terms of the videos of the algorithms working. So you start off with your agent system represented by this little humanoid character. And that agent finds itself in an environment.
[0:14:47] Which could be virtual or real world. If it's real world, the agent will probably be a robot. If it's virtual, the agent will be an avatar. And the agent has some kind of goal that it's been given, that it's trying to achieve in that environment. And the agent only interacts with the environment in two ways. One is that it gets observations through its sensory apparatus, observations about the world. And we mostly use vision at the moment, but we're also looking to use other sensory modalities soon.
[0:15:14] And those observations are always incomplete and noisy. So you never get full information about the world, unlike, say, a game of chess, where it's a perfect state information. You see everything that's in the game world. In the real world, of course, you don't get to see all the information. And one of the jobs of the agent system is to build as accurate a model as possible of the environment out there based solely on these noisy, incomplete observations.
[0:15:43] And the agent is doing this in real time. These observations are coming in every time step. And the agent is continually updating its model of the world based on this new evidence that it gets. And the second job of the agent is to then pick what action it should take, what's the best action it can take in that particular moment in time that will best get it towards its goal from the current situation that it finds itself in.
[0:16:10] And once it's decided what that action should be, it outputs that action, that action gets executed, and that then may drive a change in the environment which will then drive a new observation. And this goes around in an endless sort of cycle. Now, although this diagram is very simple to sort of explain, actually it hides an incredible amount of complexity. So we know that if you could solve all the problems behind, that underlie this diagram,
[0:16:38] This representation, then that would be enough for true artificial intelligence. And we know that because this is the way that biological systems learn, including humans and most mammals. And in fact, in humans, it's the dopamine system that implements a form of reinforcement learning. So we went on to test these kinds of systems. And actually, we chose to test the intelligence of our systems on computer games.
[0:17:10] A true thinking machine, we believe, would have to be embedded in a sensory motor data stream. You can't have true intelligence and true thinking unless you have the ability to affect the world that you're in and the ability to sense that world. And so usually, this is called embodied cognition. And usually when people subscribe to this view of AI,
[0:17:34] They normally start working on robots, real robots, based obviously in real-world environments. But robots are very tricky to use, they're very expensive, they're very slow, and they break down. So if you talk to anyone who's used or tried to develop on robots, you'll hear a lot of the work actually goes into fixing the mechanics of the robots, the motors and the sensors and so on.
[0:17:59] And actually, we didn't want to be distracted by that. We wanted to focus on the intelligence algorithms themselves. So what we decided to use was video games in the first instance. And of course, it's a little bit to do with my background, where it came in useful here in video games, and use it and repurpose the games as a platform for testing the intelligence and the capabilities of our AI algorithms.
[0:18:24] Now games are really good because obviously you can run them in the cloud, you can run them much faster than real time, you can run millions of experiments in parallel and it's very easy to measure progress because most games fortunately have game scores so you can see very conveniently if your algorithmic tweaks are gaining you an advantage and whether you're heading in the right direction based on the performance in those environments.
[0:18:53] And that's something that's very important, especially for a very long term mission like we have, and very ambitious mission, is to be able to break down an ambitious mission into smaller chunks that are very easy to measure the progress on. The other key thing about games is that obviously they were designed by other people and other engineering teams, and they weren't designed specifically for AI testing.
[0:19:19] And so what that means is that you have to deal with all kinds of interesting problems that you would never have dealt with, designed yourself as an AI designer.
[0:19:28] And I think that actually makes sure that there isn't any bias in the types of problems that you apply your AI to. So one of the big problems in AI research that has been over the last few decades is that, generally speaking, it's the AI designers that also design the problems. And subconsciously, whether you like it or not, you end up designing problems that you know your AI algorithms are well suited to. So what we started off with was actually Atari games.
[0:19:57] From the 80s, which were really the first iconic platform that had a lot of very popular, challenging games on it. And we decided to start with that. And what we did is we started with an open source emulator for Atari games. And we souped it up and made it more robust and made it run faster. And then we plugged it in our AI algorithms into this system.
[0:20:19] Now I'm going to show you a couple of videos of the AI system working. But before I do that, I just wanted to explain to you what it is you're going to see. So the AI system here only gets the raw pixels as inputs. So it's almost as if we'd set up a video camera observing the screen, and the only information that it gets is the raw pixels.
[0:20:42] So it doesn't know anything about what it's controlling. It doesn't know what the aim of the game is. It doesn't know how to get points. All it's being told is that its goal is to maximize the score. Everything else is learned from scratch. And then there's the sort of generality component comes in again where we require a single system to play all the different games out of the box. And there's obviously dozens and dozens of very, very different Atari games.
[0:21:13] So the first video I'm going to show you is Space Invaders, probably the most iconic game that there's ever been. And I'm going to show you sort of two parts of this video. So in the beginning, as I roll the video now, you'll see what the agent looks like when it first encounters this environment. Now, it's controlling the rocket at the bottom of the screen. Obviously, it's trying actions randomly because it has no idea what it's supposed to be doing, and it loses its three lives almost immediately.
[0:21:41] Now, if you leave the machine training overnight and you come back the next day, the machine now is superhuman at the game. So every single shot it fires hits something. It can't be killed anymore. It's worked out that the pink mothership at the top of the screen is...
[0:22:02] Worth a lot of points. It does these amazingly accurate shots. And you can see that the model it's built of the world is extremely accurate. So those of you who played Space Invaders back in the 80s will remember that as there's less of them, they get faster. If you just watch the last shot, you'll see that it's sort of predicted where that Space Invader is going to end up and fires the shortest shot ahead of time. So I'm going to show you a second video now, which is the game of Breakout.
[0:22:30] It's my favorite video where I'm going to show a few more gradations of the agent getting better and more capable. So in this game, the agent is controlling the pink bat and ball. And the aim of the game is to break through this rainbow-colored brick wall brick by brick.
[0:22:46] So to start off with, after 100 games, the AI system is not very good. You can see it's missing the ball most of the time, but you can see, maybe you can convince yourself, it's starting to get the hang of the idea that it should be moving the bat towards the ball. Now, after 300 games, you can see that the system has now got pretty much as good as any human can play this, and almost always gets the ball back, even when it's coming back at very vertical angles. So we thought that was pretty cool.
[0:23:15] But we thought, well, what would happen if we left the agent running for longer and played another 200 games? And then this unexpected thing happened. It discovered the optimal strategy was to dig a tunnel around the side and send the ball around the back of the wall. And it's doing that, again, with superhuman accuracy in terms of the motor control there and that strategy.
[0:23:38] And one of the funny things is that although the researchers on that are amazing programmers and engineers, they're not so good at playing Atari games. So they didn't actually know about that strategy. So it's, I think, an example of a system that you've created actually teaching you something, which is quite a watershed moment for us.
[0:23:59] So if you're interested in that work, this was then fully published in Nature on the front cover earlier this year, and we actually even released the code as well, so you can have a look at that and play with that yourselves. So now we're moving on to 3D games, Go, robot simulators, and of course we are interested in robots, but as a sort of application rather than as a development platform.
[0:24:27] Now I'll just show you one thing on the 3D stuff. So we'll have a lot more announcements, new announcements to make in the next year. But I'll show this sort of, run this little video of the same agent that you saw playing the Atari games actually now driving a racing car around a track in a 3D game. So again, the only inputs here are the pixels, the raw pixels.
[0:24:51] And the steering wheel controls. And it's learned just from raw experience driving the car around how to drive and even do things like overtaking the other cars at sort of 200 kilometers an hour. And again, just from the raw pixel data. So we're now moving towards much more advanced 3D environments where we're looking at maze problems and all kinds of much more complex pathfinding problems.
[0:25:19] Now, I spoke about neuroscience at the beginning of the talk, and I just want to come back and touch on that now. So we talk a lot about the AI that we build at DeepMind as being neuroscience-inspired. And in fact, many of the research areas that we're looking at now, we are looking to neuroscience very closely for inspiration for new types of algorithms as to how the brain works. So we're looking at memory, attention, concepts, planning, navigation.
[0:25:48] and Imagination. Now, because each one of those areas would probably need a whole talk to sort of get into, so I'm just going to focus on imagination because I think that's most relevant for the audience here and is also what I did for my PhD. Now, it turns out that imagination is quite dependent on an area of the brain called the hippocampus, which is actually here, this area in pink here at the center of your brain in this diagram of the human brain.
[0:26:17] Now, the hippocampus is quite a small part of the brain, a small brain region, but it's a very critical brain region. And it's been known for more than 50 years now that if you damage the hippocampus, then you become amnesic. So it's well known that the hippocampus is vital for episodic memory.
[0:26:40] But what wasn't known was what else was the hippocampus useful for? For example, was it involved with imagination? Now, I suspected that it might be because when I started reading the literature on memory and hippocampus when I started my PhD, I sort of came across this literature that was talking about memory as being a reconstructive process rather than something like a videotape.
[0:27:06] And that's actually the way that memory works. If you remember this lecture tomorrow, it's not kind of stored like a videotape somewhere in your mind. Actually, you'll be combining it from all sorts of components of things and experiences that you've had before. Other lectures, perhaps other visits to the British Museum, as well as specific pieces of content that are to do with this evening specifically.
[0:27:28] And what your brain does and the hippocampus is involved with is reconstructing that, pulling all those parts together into a coherent whole, which then is recognized by the rest of your brain as actually an episodic memory. So I was thinking, well, if memory works as a reconstructive process, then if we think about imagination as being a similar process, but in this case, it's a constructive process. If we think of memory as trying to put your components that you have together in a way that your brain thinks, looks, and judges as familiar,
[0:27:57] Perhaps creativity is the converse of that. You're still bringing together those components, but now you're trying to create something novel that actually your brain judges as unfamiliar. So I was thinking, well, if memory is heavily dependent on the hippocampus, then maybe imagination is also very heavily dependent on the same brain structure and the same processes. So the way we decided to test this was actually by going around the country to interview
[0:28:28] Patients who had damage to the hippocampus, but only the hippocampus. And there's very rare sorts of diseases that cause that. Although things like Alzheimer's actually do attack the hippocampus, but also other brain structures. But what we needed were patients that had only specific damage only to this one brain region. And we tested those patients on their imaginative abilities rather than their episodic memory. And what we did is gave them fairly a kind of simple imaginative task.
[0:28:57] We would give them word cues like the following. Imagine you're lying on a white sandy beach in a beautiful tropical bay. Describe in as much detail as you can what you can see and hear and experience around you. And what happened was is when we compared and broke down their descriptions, we scored it in a very complex scoring system to break down how rich that description was. And we compared it to age and educated and IQ matched.
[0:29:26] Control subjects. So they were matched in every way except obviously they had intact and healthy hippocampuses. We found that on our richness measure or experiential index measure that the imagined scenes that the hippocampal patients were describing were hugely impoverished.
[0:29:45] Compared to the healthy controls. And you can see that in these two, I don't know if you can see this laser pointer in these two bar charts. So on the left hand side here is the patients, the five patients. And then here are the 10 matched controls to these patients. And you can see this is the sort of richness index, if you like, of the described scene. And you can see that the patients are massively deficient compared to the controls.
[0:30:14] So it seems indeed that the hippocampus is very important for imagining the future. And actually, the New Scientist reported on this work, and it became quite a big study that was also listed in Science as one of the big breakthroughs of 2007. And the New Scientist had quite a nice headline talking about hippocampal patients being stuck in the present. So the idea was that we know they can't remember the past.
[0:30:43] And now it turns out they can't imagine the future either. So, but yeah, if you were to talk to them, they seem, you know, for a few minutes, they would seem completely normal to you and you'd be able to converse with them completely normally because they're processing the present in the same way that a healthy person would.
[0:31:01] So we then followed this up in fMRI, and we found out, of course, the hippocampus doesn't support imagination on its own. It's a critical sort of core part of a much larger brain network that includes all sorts of other brain regions, from the retrospinal to the medial frontal cortex to the medial temporal lobe, lateral temporal cortex, and parietal cortex. So all these regions reliably come on when you scan somebody in a brain scanner and you get them to imagine scenes.
[0:31:30] And this network is the imagination network, if you like. So then much more recently, I was thinking, well, okay, so this is how humans imagine. But what about animals? Can they imagine as well? For example, can a rat imagine? Again, we know that rats have memory, very good memory, in fact. But can they do things like imagine the future?
[0:31:57] So before I just show you the study that we did to investigate that, I just need to take you through a couple of things that we know about rats and we know that rats can do. So the first thing to tell you about is play cells. Now, play cells, you can really think of them as the GPS coordinates in a rat's brain about where they are.
[0:32:20] So if we imagine a box that a rat might be in a box, so here we're looking top down on this environment, then the rat might be roaming around this box. And in these experiments, you record directly from the rat's brain while they're roaming around. And what you find is that cells in the hippocampus fire in specific places in the environment. So for example...
[0:32:46] Cell A might fire only when the rat is traversing through this particular part of the box environment. Conversely, cell B might only fire in another part of the environment. Now, the amazing person who discovered this, John O'Keefe, who discovered these place cells in the 70s just around the corner at UCL, won the Nobel Prize for this last year.
[0:33:08] And I was lucky enough to have him on my Viva committee for my PhD. So I got to know him well and the entire rat literature very well. And what I started thinking about is since that discovery of place cells, people have found that actually sequences of place cells fire in kind of sequence when a rat moves through an environment. So for example, let's take a new environment here, a linear track. So this is like a linear box.
[0:33:37] And the rats moving from left to right here. And what you find if you record from the brains of these rats is that place cells will fire in order, A, B, C, and D, depending, mimicking the way and matching the way that the rat is moving through that environment. And in the 90s, other people found that when they recorded from the brains of these rats while they were asleep,
[0:34:03] After they had walked around and navigated in one of these maze-like environments, the rats would replay the trajectories they'd experienced in their awake session before. So you could really think about this as rats dreaming. So they would replay this trajectory of A, B, C, D, but now they would be sleeping soundly.
[0:34:29] And it will be just their brains replaying this. And what's interesting is that the brains of these rats actually replay these trajectories an order of magnitude faster than they actually experienced it in real life. So if you think about dreaming as maybe helping the rats learn about the environments they're in, then they're actually learning from this much more efficiently than they can experience it when they're awake. So that shows that rats have memory and perhaps they dream.
[0:34:59] But it's not showing that they actually imagine new experiences that they haven't experienced while they're awake. So we wanted to show that unequivocally. And recently, we published a study with some colleagues of mine at UCL in eLife that I think unequivocally shows that rats do imagine. So we designed this simple but I think quite elegant design to test this hypothesis out.
[0:35:24] So what we had here is a tea maze this time. So again, we're looking top-down on the rat environment. And the tea maze has a barrier. And this barrier here stops the rat. The rat starts off in the stem of the tea maze, and it stops the rat moving to the arms of the tea maze, but it sees through the barrier. So the rat can see past to the arms and see what's on the arms. So the rat is initially, in the first session, running up and down the stem of the tea.
[0:35:55] And what we do to make the rat really interested in the arms is we put some food, a rice pellet, on one of the arms here, depicted by this yellow dot, and on the right-hand arm. And the rat can see this when it gets to the barrier, but it can't reach the rice pellet. So obviously it's very motivated to think about, to try and get the rice pellet, but it can't get past the barrier. So then, after it's experienced that environment for a while, we let the rat go to sleep.
[0:36:25] And of course, we're recording from the rat's brain while this is going on. And then we wake the rat up again. And now we put it back in the environment. But this time, we remove that barrier. So now it's free to run around the whole teammates. So it does that happily. It moves around the stem and the arms, both the left arm and the right arm. And it fully explores this whole environment. So obviously, as I've just told you with the play cells,
[0:36:54] What we can do is we find that there are play cells, let's say cell A and cell D, that fire on the arm section of the maze. Now what we can do is then go back to look at the data we collected when the rat was asleep and see if the rat was imagining about those trajectories towards the rice pellet before it ever had experienced it in reality.
[0:37:20] So don't forget, when it was sleeping, the rat had never experienced walking on this arm. It had only seen that arm. And what happens is we find, our conjecture was sort of proven, that actually you get, we find if we go back and analyze the sleep data, you get this replay or preplay, if you like, of this trajectory, A, B, C, D. And what's more is this is not just random preplay.
[0:37:47] You actually get more, significantly more preplay to the right-hand arm than to the left-hand arm, which is exactly what you would expect if it's behaviorally consequential, right? So you can, I mean, of course, this is anthropomorphizing the rat, but you could imagine the rats really wanting to get to that rice pellet and is imagining plans of how could it get there, right? Almost imagining itself walking to the rice pellet and then eating it.
[0:38:11] And then it's dreaming about its imagined experiences. And that's really what's going on here, I think. And of course, we're now going to look into this further, and we have a number of plans to look at follow-on studies from this with more complex environments. So humans imagine, rats imagine. So what about machines?
[0:38:36] So this is something that's key imagination to planning for the future, making plans, good plans about the future. So this is obviously something that we also want our machines to be able to do. So, you know, I've entitled this slide, Do Androids Dream of Electric Sheep? Which, of course, is a reference to Philip K. Dick's famous book and one of my favorite films, Blade Runner.
[0:38:57] And this is really, I'm just going to give you a very short excerpt here with this video of, which is a little bit of an insight into the mind of the machine that you saw earlier playing Space Invaders. So I told you that one of the purposes of the agent system is to build a model of the world so it can predict the future of what's going to happen in that game world. And here, what I'm going to show in this sort of 10-second video is the machine.
[0:39:21] Getting an initial input from space invaders, like this is the game position, and then freely imagining or dreaming about what might happen over the next 10 seconds. So you can see it's dreaming about moving the rocket, and it's dreaming about getting points and shooting some of the space invaders. Now, it's quite fuzzy because there's uncertainty about what might happen in the world. It's all probabilistic. So it's not as certain as seeing an actual screen.
[0:39:49] But it's the beginnings, I think, of imagination-based planning. Now, I'm just going to end by talking a little bit about, so that's imagination. So once you start thinking, well, could machines have imagination? What about creativity? Now, this is something I get asked about all the time and, of course, is very relevant to a lot of the work that people in this room do. And, you know, I think we're a long way away from machines being truly creative. But I don't think it's impossible.
[0:40:17] And I think that when we start to understand what this process is, this mysterious process of creativity is, I think it will become actually more obvious how to implement that in an algorithm. So I just want to show a couple of little hints at things that might surprise you. So let's just take a picture of the British Museum, the front of this building. And if we then say to the machine, and this is a new type of algorithm that was actually first
[0:40:46] Very recently invented at Max Planck Institute in Germany. And then we've implemented our own version of this internally. And what you can do is you can give it an artistic picture, like this Van Gogh picture, and say you want that photo redrawn in the style of Van Gogh, right? And actually, so you end up with outputs that are like this, where you can actually, it's not ready yet to be hung in the Louvre.
[0:41:15] But you can sort of start thinking, it's pretty surprising when I saw these things, like how actually coherent the output can be. And then we can look at other examples. So actually, this is a piece of concept art for a new Google campus that's being built in Silicon Valley. And we give it a Syrah painting. And then we ask it to output it. This is one of my favorite ones. And it produces a pretty good version.
[0:41:43] of the original, but in the style of Seurat. And this isn't true creativity, right? In some sense, this is a parlor trick, because what we're doing here is deconstructing the features of both the original photo and the painting, and then we're swapping those features, overwriting the photo features with the painting features, and that gives this.
[0:42:07] These kinds of outputs. But the surprising thing here is that there isn't much sort of what you would regard as creativity here, and yet you get these kind of very interesting outputs. So it may be that creativity isn't as mysterious as it seems to us when we ultimately find out what it is. Now, I'm just going to end by talking a little bit about the bigger picture and sort of the impact that AI might have in the future. And so, you know, I think some of the big problems that are facing us
[0:42:37] as a society are information overload and system complexity. So everywhere we go now, we're deluged by information. So things like obviously genomics, big data in general, but in the world of TV, entertainment. I mean, there's so many TV channels now and modes of watching things. How can you really find what it is that you're interested in?
[0:42:57] And personalization is one kind of technology that might help, but it doesn't really work because it's really based at the moment on quite primitive sort of wisdom of the crowds, collaborative filtering technology. And that doesn't give you unique recommendations that are unique to your what I would call long tail of interests.
[0:43:14] And then in terms of system complexity, the kinds of systems we would like to master, climate, disease, energy, macroeconomics, even particle physics, are becoming so complex now that even teams of the best and brightest human experts are having difficulty comprehending the implications of these systems and actually making useful predictions about them.
[0:43:37] So I think solving intelligence, solving AI is potentially a kind of meta solution to all these problems. If we can solve intelligence, then maybe we can use it to help us as human experts get a better handle on all these other systems. And my dream really, the thing I'd like to use general AI for is to build AI scientists or to make AI assisted science possible. And of course, if we have something this powerful, then obviously we need to think about the ethics of that at the use of it.
[0:44:09] And as with all new powerful technologies, and I think AI is no different from many other technologies in the past in this regard, we have to be very cognizant about using these technologies ethically and responsibly. And although human level AI, I think general AI, is many decades away, I think we should start the debate now. And indeed, that's what we're doing, both with our own internal ethics committees, but also by supporting academic work and academic conferences on these topics.
[0:44:39] And then finally, with a nod to neuroscience, I think building AI actually in this way, this neuroscience-inspired way, help us better understand the mysteries and the workings of our own minds. And I think in the future, I think part of the journey we're on is that as we try to distill intelligence into an algorithmic construct, if we then compare that with the capabilities of the human mind, I think we'll better understand about what's unique and special about our own minds, like dreaming.
[0:45:07] Creativity, and perhaps even the great consciousness question. And as Feynman said, one of my all-time scientific heroes, what I cannot build, I do not truly understand. Thanks for listening. Brilliant, well done. Let's take a pause. That was brilliant. If we get the lights up, I'm going to ask one question, but not hog the limelight here, and I'll hand over to the audience. We're going to have 20 minutes, so have a little pause, have a think about what you want to ask Demis, because that would probably provoke.
[0:45:44] So much kind of thinking. One question I've got, a very simple one actually, is what I was amazed by was that what you detailed as your 20-year plan, you know, get your chess skills sorted, you know, in terms of those virtual environments in gaming, then you needed the new. But that was the first 20 years, and you just talked about decades away, because I think people are pretty obsessed with taking that raw data in, building the picture of the environment. We're at Atari Games.
[0:46:12] By the way, I knew that breakout trick, by the way. Anyway, we're at Atari Games, and then the other end of the spectrum is the AI scientists taking raw AI. Just give us a sense in the generation you're in, what's a realistic moon landing?
[0:46:29] In your terms, what is it? I touched on some of those things in that neuroscience slide of the big things I think are important to solve beyond like the Atari games. Of course, we're sort of on the first rung of the ladder there. The Atari thing was significant because it was the first time anyone built what we call an end-to-end agent. So something that took raw data and then made decisions and did that in one big cycle. Obviously, I think the next big breakthroughs will be can it really learn abstract concepts?
[0:46:57] And go beyond just perceptual inputs and have a real underlying understanding of the semantics of the world it finds itself in. So that's, I think, you know, for us is our big kind of... I mean, is this going to be exponential, like the way computers developed? Are we all going to be in 20 years be bowled over by the speed of progression? I think it's hard to predict. Because we've only got one of you and you're not going to be here forever. I want to know how much we can get in 20 years. I think it's hard to predict because...
[0:47:25] You know, we need at least a dozen really huge breakthroughs. And I think to get all that way and research breakthroughs are notoriously hard to predict the timescales of. So I think we'll have, you know, several very surprising things over the next few years. But, you know, as to how far we'll get all the way, I think it's too hard to say from here, you know, precise timescales for that. Brilliant. I'm going to open it up because it's easy for me. I could be here all night. But why don't we start up there? We'll get a mic to you, that gentleman there with me.
[0:47:55] Then we'll come down to the front. I don't know how many mics we've got going. Tim Marshall. If the squeamish will just close their ears for a second. A few weeks ago I was at a conference where a robot was performing a prostate operation. More than just performing the operation, it could actually understand the tumor and make a decision whether to proceed or not. Today we've seen in the news the challenges of providing health care.
[0:48:24] What role do you think AI can play in diagnosis and treatment in health? Because the way we practice medicine at the moment is a 19th century paradigm. So I'd be interested in your thoughts on that. Yeah, it's great. Actually, healthcare is actually one of the main application areas we're focusing on first. I think we could probably revolutionize the sort of quality of the care and the efficiency of it. You know, as you say, we're still using kind of 19th century methods. And I think...
[0:48:53] Why did you decide to publish the code and were you ever worried or concerned from the ethics point of view that that code might get into the wrong hands?
[0:49:24] We try to be as open as possible about what we're doing. So we generally publish almost everything that we do. And where we can, we do open source things as well. So actually our neural network library is called Torch that we build our algorithms on top of. That's open source. And we felt, and there was a demand for some of the nature reviewers and editors, that it would be nice if we could release our code. So we thought about it and we decided in that case that was fine.
[0:49:53] But that may not always be the case for the stuff we do, and we'll obviously have to consider that on a case-by-case basis. But in general, where we can, we like to engage and support the general academic community, and we think it's important that knowledge is shared, and I think that's the way that humanity can advance as quickly as possible. You're one of the largest brains on the planet, and Google have now bought you.
[0:50:21] And I'm going to ask you that question, which I'm sure you're expecting, the Stephen Hawking thing, the concern about AI, that once you get the genie out of the bottle, we're all fucked.
[0:50:38] Can you make it a bit clearer? What are you doing to try and put some safeguards in there? Go for it, Dennis. This is the regular media challenge, isn't it? Yes, sure. I mean, look, I've actually spoken. I had a long chat with Stephen Hawking about this a few months ago. Who won that debate? Well, I think it was very enjoyable, and I think we spent hours together. We only spent half an hour, but he had so many questions, and I think he was...
[0:51:06] Quite reassured after we talked about how we specifically were approaching it. And I think, look, you know, there are big issues here, very big issues about autonomous learning systems. What goals should we give them? What value systems should we give them? How can we make sure that those are exactly what we want? And there are very tough pieces of research that need to be done.
[0:51:28] There hasn't been much work done in that area yet, partly because there have been no systems to really try this out on. So it's all been thought experiments. And I think if you do, you know, mostly the people thinking about this and worrying about this to that extent are not in the AI field, right? They're either philosophers or they're other very famous scientists or industrialists, but not actually working in AI themselves.
[0:51:50] And if they were working, I think they would see that the problems are much more prosaic at the moment. And it's easy, I think, to get carried away with science fiction scenarios that are many decades away. I have confidence that as we get, build more powerful systems, we'll have much better ideas about answers to these questions that I just talked about, value systems and so on. And mathematical proofs of it, empirical work that will allow us to have a much better idea of how to keep these systems under control.
[0:52:20] Well, I'm pleased to see that you've got the ethics committee there on board and you're thinking about these issues, but you have taken the Yankee dollar. And I am worried about this because you're so smart. And I hope that everything you do actually improves the society rather than kills us off. Well, so do I. But, you know, I think AI, you know, could be the greatest thing for humanity in the sense of if we build it right.
[0:52:47] We'll solve all these big issues. Corporate responsibility versus your own personal ethics, which shine through. That's a big one, isn't it? In terms of where power lies in society. Sure. I mean, I should probably say a little bit about that.
[0:53:03] Obviously, we spent a long time doing due diligence ourselves on Google, right? And we had a lot of other options, including stay independent. And we decided to join forces with them, partly because the people high up at Google agreed with things like the ethics committee and thought it was a good idea that governed the use of the technology of DeepMind's technology. We've already ruled out things, obvious things, like military or intelligence applications. So, you know, by default, DeepMind's stuff cannot be used for those things. And then, obviously, we've had our inaugural meeting.
[0:53:33] The meeting of the committee, ethics committee, there are very big luminaries on that, many of whom are some of the people that you've mentioned who are worried about this stuff, not just the people who think positively about it.
[0:53:44] And a big part of that, actually, because we are decades away, is to just start educating everyone on what the real issues are and separate sort of fact from science fiction. And I think that's the first starting point. And then we can actually get to the hub of the really core technical, difficult questions there are. And there are some, but I'm very confident if we apply enough brainpower onto it with enough time, we'll solve those problems. Great. You've been in Google now for about a year.
[0:54:13] Can you give us a couple of examples or anecdotes about how DeepMind has changed the company, taken over some processes, changed the way the company works going forward? And just to tag onto the ethics committee, why haven't you publicized or published who's on it? So first question is, I'm pleased to say almost nothing's changed.
[0:54:38] And that's the whole point of it. That was one of the main agreements. Our headquarters is still in the UK, around King's Cross. We've invested in the research team there. So the whole of DeepMind is still UK side. And we work as a kind of semi-autonomous type of unit.
[0:54:57] The plus sides are the amount of compute power obviously we have access to has really accelerated our progress and obviously the other resources that Google has. I'm sorry, I meant the other way around. How has DeepMind affected Google as a company? Oh, I see. Well, so that's harder to say. I mean, Google's very big. But I think that we have actually affected it in some senses the way that some other parts of Google research do their work. So there's actually thousands of people in Google research.
[0:55:26] And there's thousands of people working on machine learning. But we sort of have a more coherent, specific mission than the more applied machine learning that gets done elsewhere in Google. So I think we bring together a kind of longer term research focus that I think maybe Google wants more of now. And that requires quite different organizational structures and management processes, some of which is being adopted over in Mountain View in Silicon Valley now.
[0:55:56] So your second question was about the why don't we publicize stuff. Well, firstly, you know, we're very early days and there's a lot of scrutiny on this. And there's nothing at the moment. It's about simply about educating people or getting everyone up to speed with the issues. Once you start making things public, then immediately that changes, that can change the debate. And I wanted to have a period of sort of quiet behind the scenes.
[0:56:24] Calm Collector Debate before we additionally brought on ourselves this additional sort of public scrutiny. So at some point I think we will announce who these people are and also a little bit about what the issues are that are being discussed. Having said that, we already do lots of public things.
[0:56:42] So there was a big conference in Puerto Rico that was talking about AI ethics and safety. There's another one at New York University in January that we're sponsoring and I'm keynoting and I'm on a program committee of, along with Facebook, the heads of AIs there and Microsoft and some of the other companies. So I think probably the next stage next year would be to create a cross-industry panel and bring together all the big companies and...
[0:57:08] Academic labs that are working on this, in addition to our own internal committee. Let's crack through a few more. We've got about 10 minutes. So I think someone's got a mic here. Thank you. Yes, that was a brilliant presentation. Thank you very much for that. I was an engineer. I'm now a slightly aged academic. We work on, we've done general video gaming.
[0:57:31] So I appreciate that very much. But when you go the next stage up to imagination, there must be so many individual random coordinates, if you like, because I think we all imagine differently.
[0:57:45] You'll spend an infinite amount of time trying to analyse these to actually make machines, if you like, imagine in planned fashion. So how do you cope with these coordinates you can't understand? I'd build on that as well, because a lot of the audience, for the television audience here, I can see, I suspect there was a bit of a shudder around, he's kidding, the creative bit, which is, yeah, that's a bit clunky, which you admit, you know, kind of merging the mix, versus, you know, when the first director...
[0:58:11] When the first writer comes, when the points in the environment, the choices become exponential versus a Space Invaders screen. Exactly. It's like the rice on the chessboard. So I think most people's jobs in here are safe for a long time to come. The CEO was quite happy. I was so relieved. I don't think there's going to be any directors directing something, the quality of Ridley Scott or something. That's probably going to be, if ever, one of the last things that a computer will be able to do. So we are talking about very constrained things.
[0:58:40] Where, you know, that's a very difficult thing that humans, of course, do better than, way better than computers is, you know, they can have this rudimentary kind of brute force imagination, but one of the big things that humans do is they have aesthetic judgment, right? They know that not all paths are equal, and some of them are likely to be more fruitful than others. Even if you compare chess grandmaster playing chess compared to a computer.
[0:59:07] A computer might look at millions of moves to make that one decision, whereas a test ground must only look at a few hundred, but judicial ones. In some sense, our brains even filter out, our low-level part of our brain, any of the moves or trajectories that are not going to yield anything useful. When you do that filtration, are you working on that filtration process? Part of that is to do with how well you model the world that you're in.
[0:59:35] So if you're better at modeling the world, then what that means is you should make better predictions about what are going to be useful things to spend your compute time imagining or thinking about. And at the moment, we're still very early stages of that. David. Hi, David Abraham from Channel 4. You touched on the challenge of how many choices people have in entertainment.
[1:00:03] That's something that we in our industry are spending a lot of time thinking about. Are you working more specifically on the area of recommendation engines? And are you going to be, as it were, capturing the power of that algorithm on behalf of Google? Yeah, we are looking at recommendation systems.
[1:00:27] In all forms actually, all sorts of forms. And I think it's a very interesting area and it's something that our technology is quite sort of applicable to. Again, it's about can you model user journeys and trajectories through things and in a way that then delivers much more compelling content or recommendations. And I think the current systems we have are not good enough.
[1:00:52] And we're experimenting that. Again, we're quite early days with that. And we're looking at that for things both internally at Google and external partners. So you do see external. Because I think that would be deeply intriguing to a number of us in the room who are working in the media business about how to serve up stuff in a world where choices exploded. Yeah, exactly. And the general application of that is a few years ago, you think it would be...
[1:01:16] Announcing stuff, products coming in. Product might be the wrong word, forgive my ignorance. But systems by which the likes of Channel 4, the BBC, other broadcasts in the room, independent companies can serve their content up. You think that's not far away, do you? Yeah, I think in the next couple of years you'll start seeing under the hood algorithms helping these kinds of recommendation systems. And then maybe four or five years out, actual totally new systems that you might interact with in a different way to what we do now. Fascinating.
[1:01:43] Right, we're coming towards the last few, but we'll get through as many as I can. Hi, Daniel Toole from IBM. I think we're one of the other big investors in AI, and I liked your comments around the big breakthroughs. It probably takes more than one player to get us to the future space. I had a couple of questions that intrigued me in your talk. One was, go.
[1:02:06] So years ago, I think there was a big effort around going, it was the big one that was hard for computers to solve. It might be a bit esoteric for this audience. And then the second question, just to bring us to the point is, for RTS, how can AI be used not to supplant creativity, but to enhance and support it and allow us to do more creative things as humans, not as computers? Let's do a quick one on the first one, because you might want to do that afterwards if it's really...
[1:02:31] Yeah, so Go, for those who don't know, is an Oriental board game, which is probably the most complex game there is. It's what they play in China and Japan and Korea instead of chess.
[1:02:42] And one reason it's been so hard for computers to crack is that the branching factor, the number of choices you have in each move is the order of 100, whereas in chess it's more like 20. So as you start planning, that branching factor explodes. So if you're going to do it in a brute force way, there aren't enough, I think, atoms in the universe to describe how many go positions there are, for example. I bet you're good at Go, aren't you? I'm reasonably good at Go. Were you world champion at Go? No, I wasn't world champion at Go.
[1:03:12] The second problem is that in chess, because chess is a very materialistic game, so the queen is worth more than a rook and so on, it's quite easy to hand program an evaluation function to tell you whether your program is winning or losing or how well it's doing in that position. Whereas in Go, all the pieces are worth the same. There's just one piece.
[1:03:32] And so whether you're winning or not is much more about the overall pattern of the board. So it's a much more beautiful game in some sense, very aesthetically pleasing. But it's much harder to hand code an evaluation function. So yeah, we're going to have some very big announcements to make on go. I mean, it's sort of been the holy grail for the AI research communities for the last 20 years since Deep Blue actually beat Kasparov. Sorry?
[1:03:58] Yeah, sure. So then the last question was on... Helping create. So that's really the same thing I'm thinking about in science too, right? And with doctors and with... What I'm really thinking about is AI surfacing the right information for you in a much more digestible way. So you can just leverage that for whatever it is that you're trying to do. In the world of media, we talk the recommendation area. Is there anything else in your head that springs to mind in terms of...
[1:04:29] The creative process, the creation of media. I think that's a lot tougher. So I think recommendation is the obvious one. We are looking at things like music, which is a kind of more constrained domain for a computer than visuals, which is incredibly hard. And there's some very interesting work being done in music, music composition, music analysis, which I think is pretty promising. So I would imagine that would be the next place. Very good. I'm going to take...
[1:04:55] Three more, because we're really running out of time. I'm sorry, because we could just keep going all night here. Take the gentleman here. Has anyone got a microphone on them? Because we'll just get the mics. The gentleman here. Anyone at the back want to have a, because I've been very front focused. We'll take the gentleman on the end of the row right at the back there. Hi, Nicola Rosa from IBM.
[1:05:14] We've been shaped from the search of pain and pleasure during our all cultural history, and it's shaped the way that we are thinking and the way that we are behaving. How can you teach pain and pleasure to a machine? Well, one question is whether we need to, but also, or whether we should.
[1:05:34] But there is sort of, this speaks to this idea actually that we look at internally of intrinsic motivation, we call it. So, you know, there are emotions and other things that drive human behavior, not just external rewards. So, you know, and at the moment, our machines don't have anything like that. But maybe to do more complex tasks.
[1:05:54] We're at the moment working in game worlds where there is conveniently a score most of the time. But even if you start going to more complex games, open-ended things like Minecraft, now there isn't a score anymore. So how are you going to decide what you should do, what's useful, what's good, that you're making progress? And I'm talking about the agent system here. And of course, that's more like the real world for us as humans. And yet somehow we have our own internal drives.
[1:06:23] Probably that have been evolved to that help influence our behavior. So I think it's interesting to think about. And, you know, we have neuroscientists or experts in these areas who work with us as consultants. And it's something I'm very fascinated by. But, you know, we don't have a definite answer on that yet. Thank you. Right. Two more. So we'll check the gentleman at the back. I think he's someone still want to question the back. I've got a hand up there. Yes, there is right there. I just feel.
[1:06:52] And then we'll take the gentleman in the red as the last question. No pressure. My question leads on quite well from the last, actually. I was thinking, have you thought about generalizing the goals? So you talked about how you have observations and you have actions which you take, but presumably you define the goal and you tell the system how to measure its goals. And if we're thinking about AI as something which is a servant to human intelligence, then...
[1:07:23] Have you thought about AI which can derive its goals from the environment? And also as humans, we segment our goals. We might have life goals, but we focus on sub goals to get there. And also, I'm sort of imagining a human saying to a system, can you help me with this? And the AI being able to derive its goal from the...
[1:07:45] Things that it hears, but also going further than that, being able to derive goals before they're specifically instructed. So being able to anticipate goals that people might want out of AI. That's right. I mean, that's a great question. And actually, it's a fascinating research area. Can the machines learn their own goals through observation of, you know, learn what it is you like through observing you, for example, right?
[1:08:11] And then trying to, you know, maybe even be able to preemptively guess what it is that you need before you even ask for it. So I think, you know, those systems are very interesting. I mean, even there, you would still have some kind of top level goal, which is to satisfy the user, right? Although it may learn what the sub goals are. And that's another very active area of research is how do you break down a large goal into automatically into sub goals?
[1:08:36] And of course, that's something our minds do effortlessly. If you're going to plan a trip to Paris from here,
[1:08:43] Your brain is not going to plan over your muscle fiber movements all the way from here to Paris. But yet that's how robotics works at the moment. They have no defining of hierarchy. What your brain is actually going to do is you know at a high level you need to get to the Eurostar terminal and then take a train there and so on. And then only at the point where you get up off that chair does your brain then go and unpack the muscle fiber movement of get up off a chair and walk.
[1:09:11] So whereas at the moment, because we haven't solved this problem of automatically generating subgoals, a robot, for example, trying to do that task would have to plan over the primitive action movements all the way to Paris. And of course, this is not feasible and therefore not tractable because it ends up becoming, you know, speaks to the other gentleman's question about you imagine all those paths from here on muscle fiber movements, there's basically an infinite number of them.
[1:09:35] So that's one of the key things we need to solve is the sub-goal problem. The last question. Oh, no mic, sorry. Get that across. Malcolm Horrell, broadcast engineer. Following on from an earlier one and your list of memory, navigation, imagination, et cetera, you didn't have emotions. And as we're going to be dealing with and interacting with humans, how explosive is that with your conflicts of ethics and parameters?
[1:10:08] Yeah, I mean, again, this relates to the question of the gentleman down here about emotions. We currently, you know, there is no equivalent of that in our systems. But if you think of emotions, and probably this is too simplistic, but part of emotions are internal drives, give us internal drives, then that's something we do need to explore and try and work out what ones might be needed. And it might be...
[1:10:36] We need similar ones so that these systems can empathize with humans. Obviously, there's a great Channel 4 series. I think some people are in the audience of humans, which I really love. And that's interesting. They're trying to empathize with the humans that they serve. Alternatively, you might want to have systems that have very different types of drives that help them be complementary to what humans are good at. So I could imagine we might need both types of AI. So in motion, just on a visual.
[1:11:05] You think it's essential that you're going to have to cope with emotion as a driver of, if you like, the response? I think some types of emotions. So there's two reasons you might want them. One is so that these systems can empathise and work better hand-in-hand with humans. And the other thing is if it turns out the environments they're in, there aren't many external reward signals, they have to have some internal drive to get them going in the right direction. Brilliant.
[1:11:29] It was a privilege. I'm going to hand to Naomi Clymer from the IET, the president of the IET, in a second. But Demis, it's been wonderful to have you here. Thank you.
[1:11:52] Thank you, Demis. As the president of the IET, we're absolutely delighted to have co-hosted this lecture with the Royal Television Society and delighted to have someone of your extraordinary caliber, Demis. Events like today fulfilled part of the IET's charitable remit, which is to inspire, inform, and influence people. And I don't know about you, but you sure as hell have inspired, informed, and influenced me on a topic that I believe is going to change the world. I love the idea that your AI journey started with games and that machines...
[1:12:22] Thank you very much.
[1:12:51] So listening to you, it all sounds incredibly real and feasible that AI can make a positive difference to humanity. And even if it's not going to be directing any movies anytime soon. So once again, please join me in thanking Demis Hassabis, founder of DeepMind, for an absolutely fantastic lecture.
[1:13:24] I would also like to thank our feisty chair, who did an excellent job, Tim Davie, the CEO of BBC Worldwide. I think we've all been inspired, informed and influenced, so it's been very nice for me to be part of this. Thank you very much for coming. You'll be delighted to hear that drinks are now served outside. Thank you.