{"url": "https://www.youtube.com/watch?v=Qgd3OK5DZWI", "page_url": "https://www.youtube.com/watch?v=Qgd3OK5DZWI", "title": "A systems neuroscience approach to building AGI - Demis Hassabis, Singularity Summit 2010", "show_or_channel": "Google DeepMind (Singularity Summit 2010)", "date": "2010-08-14", "duration_s": 2004, "type": "talk", "speakers": ["Demis Hassabis"], "confidence": "high", "notes": "Singularity Summit 2010, San Francisco. Alt copy: https://www.youtube.com/watch?v=F5PSyu7booU (singularitysummit channel).", "people": ["demis-hassabis"], "id": "2010-08-14_google-deepmind-singular-a-systems-neuroscience-approach-to-build", "audio_duration_s": 2003.6266875, "worker": "jc-w08", "transcribe_s": 94.4, "model": "large-v3", "segments": [{"start": 0.27, "end": 32.14, "text": "Our next speaker up on the floor was the highest rated chess player in the world at the age of 13. He wrote the best-selling video game Theme Park at the age of 17, and then he got a double first in computer science from Cambridge."}, {"start": 32.34, "end": 59.84, "text": "He put all those childish things aside and founded the computer game company called Elixir Studios. And once he was done with all that, he got his PhD in cognitive neuroscience and then became known on the world scene for noticing some connections between memory and imagination. Science Magazine called his breakthroughs connecting memory and imagination one of the scientific breakthroughs of the year in 2007. So..."}, {"start": 59.84, "end": 97.78, "text": "This guy's been busy, and he still looks pretty young to me. So, please welcome Demis Hassabis to the stage. So today I'm going to be talking about different approaches to building AGI. Specifically, I'm going to be advocating a systems neuroscience approach as potentially the best way forward to making progress. Before I go into details about the unique advantages I think the systems neuroscience approach has, I'm going to set the context by doing a quick overview of"}, {"start": 97.78, "end": 128.78, "text": "The current state in the art in AGI projects, past and present. And see if we can categorize them in some kind of meaningful way. So at the top level, the kind of most fundamental split, I guess, is between those approaches that take a biological and a non-biological approach to building AGI. So let's talk about the non-biological approach first. Chronologically, in the history of AGI, these were the first attempts."}, {"start": 128.78, "end": 159.65, "text": "Basically, the umbrella term we can put these under is symbolic AI. And this includes formal logic systems, logic networks, lambda calculus, and expert systems. And there were many projects involving these systems in the 80s and 90s, run by very smart people. Now, the issue was, what they found when they pushed these projects and they scaled them up to try and approach AGI, is they tended to suffer from one or more of these flaws. They tended to be brittle."}, {"start": 159.65, "end": 181.26, "text": "Not dealing with ambiguity and uncertainty very well. They were time consuming to train. And they were generally poor at generalizing. Also, a specific problem to symbolic AI systems is there are some known classical philosophical problems, really. For example, how does a symbolic system acquire new symbols in which to describe things?"}, {"start": 181.26, "end": 211.42, "text": "And also the classic symbol grounding problem, probably one of the biggest issues with symbolic AI systems. How do you actually create meaning when you're only describing symbols in terms of other symbols? So probably the prime example and probably the most ambitious project under this umbrella is Doug Leonard's site project, which has been going on for about 25 years now. And for those of you that don't know, basically this project was about trying to encapsulate all of the world's common sense knowledge."}, {"start": 211.42, "end": 241.23, "text": "In a massive database of the order of about a million predicate rules. Now the problem with this is apart from the time it's taken to put those rules into the database, as I said, 25 years and counting, and around about estimated 600 man years, they come across big problems where now when you add one new rule into the database, this can cause a cascade of inconsistencies throughout the rest of the database that can take weeks to resolve if they can be resolved at all."}, {"start": 242.83, "end": 273.9, "text": "So on the other side of this divide are the biological approaches. So these are biologically inspired approaches to AGI that use the brain to a certain level of abstraction, and I'll come back to that, as a blueprint for what AGIs should contain. Although I put this as one camp, it really covers a massive range of different approaches. Before we move on into the biologically inspired and splitting that down further,"}, {"start": 273.9, "end": 300.46, "text": "Let's just think about really what it is that the underlying belief systems that would lead one to conclude which is the best approach between these two. And I think fundamentally if you strip away all the procedures and the various systems that you might use, it comes down to one question. What you believe the search space of possible AGI solutions looks like. So I've done a little cartoon to demonstrate that. So let's consider the search space of possible AGI solutions."}, {"start": 300.46, "end": 328.98, "text": "I think really we can break this down into two possible regimes that we are in. So let's consider regime one, where we have the search base of solutions is basically small and dense. So this is a little cartoon of it. So here we have a relatively small search base. The blue stars indicate possible AGI solutions within that search base. And there are relatively many of them compared to the size of the search base."}, {"start": 328.98, "end": 358.7, "text": "If we're in this kind of regime, the human brain, of course, is another example of an AGI, but it's maybe not worth the effort if you thought you were in this kind of regime to spend the time to investigate the human brain and use that as an example. On the other hand, regime two, we have a large and sparse search space. So here we've got the relatively large search space, and now we have relatively sparse numbers of solutions. So if this is the..."}, {"start": 358.7, "end": 388.99, "text": "Regime we find ourselves in, then clearly it makes sense to look at the human brain and spend the time and effort to see what we can extract out of it. So then, just to finish this line of thinking, which regime are we in? Now, regime 1, small and dense. Regime 2, large and sparse. Now I would say that, of course, we can't categorically prove this one way or the other yet, but the current evidence points towards the fact that strongly I would say that we're in regime 2."}, {"start": 388.99, "end": 420.32, "text": "The large and sparse regime. And I've got two arguments for that. One is a natural argument. Evolution, which is itself an optimization and search algorithm through this kind of search space, has only produced human-level intelligence once, after hundreds of millions of years of trying. And then really going back to these projects in the 80s and 90s, these serious projects and efforts to make AGI that largely failed to make progress towards the overall goal, although produced interesting results on the way. That's another indication that maybe that..."}, {"start": 420.32, "end": 449.7, "text": "The optimistic views of the 60s and 70s of how easy AGI was to build was potentially wrong. And it's actually a lot harder, and the search base is a lot bigger, and there's many fewer solutions in that search base. So let's say we buy this argument, and that using a biologically inspired approach is the right way forward. But this itself can be broken down into a number of different clusters. So I basically created a spectrum here. So let's say on the left is..."}, {"start": 449.7, "end": 475.44, "text": "The most loosely based on the brain, and I've determined that abstract. And on the right is the closest approaches to the biology itself. So let's think about the left. So over here, I'm going to put cognitive science architectures. Now, I could have picked dozens of examples of these architectures, but these are three of the more well-known ones. There's SORF, that's Alan Newell's and John Laird's SORF, ACTAR, John Anderson's ACTAR, and more recently, Ben Goertzel's OpenCOG."}, {"start": 476.24, "end": 505.54, "text": "And here, basically, what these systems are is they've been inspired by behavioral psychology to try and extract from the brain what modules and what functions the brain maybe implements. And then what you get out of all these systems is effectively a module circuit diagram. Here, for example, is the one for ACT-R. And each of these boxes basically refer to one function that the advocates of these systems believe the brain is doing, and then there's some wiring between those."}, {"start": 505.54, "end": 531.15, "text": "And the problem I have with these systems is that they're fairly unprincipled. And these issues come up when, for example, psychology throws up a new function that we suddenly realize is important to thinking and higher cognition, let's say episodic memory. And then now these kinds of systems have to shoehorn post hoc these new modules into these kinds of architectures and then just hope that doesn't ruin everything."}, {"start": 531.63, "end": 554.61, "text": "And I think another reason that shows the problem with these unprincipled approaches and why they're a bit unsatisfactory is the very fact that there are so many, such a proliferation of these kinds of architectures. And I believe the reason there is such a proliferation is it's very hard to prove your architecture is better than someone else's because there are inherently no principles you can point at that prove that."}, {"start": 554.61, "end": 586.46, "text": "Like all AGI researchers, we all believe we have the answer. And I think that the standing joke is that there are as many architectures as there are AGI researchers in the field. So on the other end of the scale, on the very biological scale, we have the whole brain emulation advocates. So again, there are many examples of this, but the two most famous ones are the Blue Brain Project by Henry Macrom and Moda's IBM Synapse program. And of course, these are very impressive technical feats. They push supercomputers to the limit, and they produce beautiful,"}, {"start": 586.46, "end": 612.85, "text": "Diagrams like this. This is taken from Moda's recent PNAS paper that shows the wiring in the macaque monkey brain. But the question we have to ask ourselves in terms of building AGI is, is this level of detail, this level of implementation detail, useful towards building AGI? And I would say the answer is maybe not. So we may know some of the wiring, but what is this telling us about the function? So there's a long way to go here. And in fact..."}, {"start": 612.85, "end": 642.11, "text": "Kind of serious whole brain emulation advocates go further than this, and they want to do one-to-one mapping and correspondence between a brain, a living brain, and then copy that one-to-one into an artificial substrate. But we're really 50 years away plus from creating that kind of imaging technology. So there's way in the future, especially if you want to keep the original copy or the living brain alive. So what we're advocating is this middle way."}, {"start": 642.11, "end": 671.94, "text": "What we're calling systems neuroscience. And here, what we're interested in is the brain's algorithms. So let's just unpack that a little bit more and look at it in a slightly different way. And hopefully this will show you the real difference between these three different approaches, biologically inspired approaches. So David Marr is, as many of you know, the father of computational neuroscience. And in the 70s, he wrote a number of seminal papers. And in one of those papers, he suggested..."}, {"start": 671.94, "end": 701.73, "text": "What he was saying was that to understand any complex biological system fully, it's no good to just understand it at one level of abstraction. So just say, for example, the neuronal level in the brain. You need to understand it at what he proposed, three different levels of abstraction all at the same time. So let's look at what the levels of abstraction that he proposed were useful for this kind of analysis. So he termed them computational, algorithmic, and implementational."}, {"start": 701.73, "end": 721.46, "text": "So by computational, what he meant was defining what the goals of the system are. Algorithmic is the how, what are the representations and the algorithms the system uses to achieve those goals. And implementation is the medium, the physical realization, if you like, of the system, the actual substrate and how that works."}, {"start": 722.45, "end": 752.22, "text": "So now, using this level of analyses, we can go think back to the spectrum of different approaches on the last slide and look where they fall into this level of analyses. So it's quite clear from this, we can see that the whole brain emulation camp is basically focused on the implementation level and then building out from there would be the hope. On the other hand, the cognitive science architecture camp is really focused on the very, very high level of computation of the goals of the system and then just taking that."}, {"start": 752.22, "end": 783.38, "text": "And what I'm suggesting is really we should be focusing on the algorithmic level of the brain and especially extracting what kinds of representations and algorithms the brain uses to solve the kinds of problems we'd like to solve with AGI. So that's all very well, that argument, but it would be a moot point if there weren't rapid advances in neuroscience. Literally, in the last decade, neuroscience has advanced enough that we can start answering some of these questions for AGI usefully."}, {"start": 783.5, "end": 807.98, "text": "So in the last decade, there's been a revolution in cognitive neuroscience. And this has mostly been driven by new experimental techniques. There's a whole raft of them. There's new ones coming every few months. So obviously, of course, from imaging, multicell recording, optogenetics, and transmagnetic stimulation, to photomicroscopy, and a whole range of different techniques for answering different questions."}, {"start": 808.3, "end": 837.68, "text": "There's also been a revolution in the analysis tools used. So, for example, using multivariate pattern classifiers like we saw in the last talk to classify brain activity. So as this whole Singularity Summit is very fond of exponential themes, you can see here on this graph, if this laser pointer works, yeah. So here, the green is the fMRI machine, here's a PET machine, and here's an MEG machine. Now, if we look at this graph,"}, {"start": 837.68, "end": 860.4, "text": "This is a graph of on the y-axis is number of citations of papers per year, and down on the x-axis is the year. And we can see this huge exponential growth here happening from around 1995 and continuing to this day. And not only is this exponential in total, you can see that the later developments like fMRI, which is in green, are exponential themselves."}, {"start": 860.4, "end": 894.5, "text": "So you can see here in this graph here shows the proportion of neuroscience papers using each of these techniques. And you can see the green area as we go further along in time is taking up more and more proportion of the number of papers. So that too is all very well. So there's tons of neuroscience out there, but most of it is not relevant for AGI. So how do we find the nuggets in all of that wealth of information? How do we keep abreast of this sea of information?"}, {"start": 894.5, "end": 922.61, "text": "As we just looked at, there's this exponential growth in neuroscience. There's 50,000 neuroscience papers were published just in 2008. What this means, though, is if you last seriously looked at neuroscience circa 2005 or before that, you're way out of date now. You're somewhere here on the graph. So you're somewhere here on the graph. So there's this whole load more things that have been happening even in the last five years."}, {"start": 923.79, "end": 955.34, "text": "So then the question is, how do we sift through it all to find the parts that are relevant to AGI? What level of implementation should we be looking at? And furthermore, really, we shouldn't just be passively observing this neuroscience work. We should be actively conducting and directing neuroscience, cutting-edge neuroscience research, in ways that are useful for the questions that we want to answer for AGI. So there's no easy answer to this. I don't have an easy answer to this. You basically need a ton of expertise and full immersion in both areas. It's the only way."}, {"start": 955.63, "end": 985.68, "text": "And we found with our colleagues on this project that this takes about five years of intensive learning and research and doing intensive research from very motivated smart people to gain the necessary knowledge to be able to cope to do the rest of these things. Okay, so what uses systems neuroscience for AGI? I've been banging on about neuroscience being useful, so what are some examples?"}, {"start": 985.68, "end": 1016.46, "text": "Firstly, I think neuroscience provides two things, if you boil it down. It provides direction and what I'm calling validation testing, and I'm going to unpack those. So let's look at provides direction first. What I mean by this is it will provide inspirations for new algorithms and architectures. So there are, again, a ton of examples I could choose, but I'm going to talk through two. The first one is a classic example for computer vision that many of you will know, and then a much more recent example from navigation."}, {"start": 1016.46, "end": 1048.78, "text": "New navigation systems. So let's talk about the visual system. So Hubel and Wiesel in 1959 did some seminal work on cat cortex, visual cortex, and they discovered two types of cell for which they won the Nobel Prize in 1981. First type of cell they found was simple cells tuned to preferred input stimulus. And here are some examples of stimuli that these kinds of cells, in this case primary visual cortex, V1 cells, were tuned to."}, {"start": 1048.78, "end": 1073.9, "text": "So we can see here that these are basically just straight lines, and they're different orientations. And if a cell saw one of these straight lines in its receptor field, it would basically fire. So that's what a simple cell does. Okay, then there's a second type of cell, complex cells. And what these do, this is slightly more complicated, but they take a bunch of simple cell inputs."}, {"start": 1073.9, "end": 1099.2, "text": "We can see here an image that's been deconstructed into its simple cell inputs. And then here's a complex cell, the first layer of complex cells, which is taking a bunch of these simple cells. And the key is that they're all tuned to the same preferred stimuli. So in this case, they're tuned to the line pointing in this direction. So all of these ones. But the key is that it's taking a bunch of simple cells with slightly different positions in the retina."}, {"start": 1099.2, "end": 1119.78, "text": "And different scales. And what this turns out, this introduces, is invariance, which is exactly what you want for a visual system. Tolerance to scale and translation across the visual field. And then it turns out you can actually start building hierarchies of these simple and complex cell layers. And this is exactly what Tommaso Poggio at MIT has been doing."}, {"start": 1119.78, "end": 1149.73, "text": "And he's created a state-of-the-art vision system by basically mimicking a primate visual system. So here on the left, we have a macaque monkey brain's visual, the wiring diagram of their visual cortex. You can ignore most of it, but just notice the colored ones here. This is to do with the object recognition pathway, the ventral visual stream. And then on the right here, you can see Poggio's model, HMAC's model, which basically mimics the same layers in the same places to do object recognition. And this is state-of-the-art."}, {"start": 1149.73, "end": 1179.34, "text": "So what we can see is the system's neuroscience procedure or approach extracts the principles behind an algorithm brain uses, creatively implement that in the computational model, and the result is you get a state-of-the-art technique and possibly an AGI component. Let's look at one other example, place cells. So place cells are neurons that fire when a rat is a specific location and environment. These were discovered by John O'Keefe in 1971. And you can see here, this is a rat."}, {"start": 1179.34, "end": 1208.1, "text": "In his home pen. And he's free to roam around here. And what they do in these experiments is they stick electrodes into the rat hippocampus. So you can see this is the rat brain. And the blue bit is the hippocampus. And here's an electrode stuck into the hippocampus. And that's recording a single cell from the hippocampus. And then what they do is they let the rat run around this pen. And you can see here, this is a top-down view of the pen. And the black lines are basically the route the rat has taken through that pen."}, {"start": 1208.1, "end": 1236.8, "text": "And what you see here is the red dots, or the red clusters, are where this cell fired during this journey through the pen. And you can see, when you do a heat map, they're all clustered around what is called the place field of that place cell, around this top right corner in this case. Now, much more recently, what was found by the Moses in Norway was a bunch of cells called grid cells. And these are..."}, {"start": 1236.8, "end": 1264.85, "text": "These were found in the entorhinal cortex, which is next to the hippocampus and provide input into the hippocampus. And these are neurons with the spectacular regular periodicity, and they form hexagonal firing patterns. So I just showed you place cells that had a single firing pattern, but here you've got hexagonal firing patterns. So let's just look at this final column on the right here. It's the only one that's relevant for this. These are three different cells, and in this case, the rat is in a circular environment rather than a square one, and you can see..."}, {"start": 1264.85, "end": 1291.71, "text": "That this particular single cell is firing at these regular hexagonal places. So it's incredible. When this was first found in 2005, no one believed it because it looks non-biological. It looks like someone's coded this. And this is what the brain uses to mark out space. So it seems like this is what the brain is using to define an intrinsic measure for space. So what you could think of it as is like the graph paper of the mind."}, {"start": 1291.71, "end": 1324.27, "text": "It's something that's automatically tessellating space. We're doing this right now. We're looking at this room. And several new navigational systems, including several current DARPA projects, are using these two types of cells to build new kinds of navigation techniques. Okay, so it provides direction. And we've seen that, inspires new algorithms. And so what about validation testing? What do I mean by that? So what this is, is really we'd like to answer the questions of this form."}, {"start": 1324.27, "end": 1355.41, "text": "Does an algorithm that we may have invented through machine learning or some other way or through maths constitute a viable component of AGI system? Does this algorithm that I have sitting here on my machine that does some interesting things, is that a useful component of an AGI system? Well, so to show this as an example, I'm going to use the most classic example of all, which is reinforcement learning. Now, for those of you who don't know, the reinforcement learning problem is a whole class of problems where the only teaching signal is the reward gained from the environment."}, {"start": 1355.98, "end": 1382.58, "text": "Now, temporal difference learning, or TD learning, is a method for solving part of the reinforcement learning problem. That's what I'm just going to talk through here. And the TD learning algorithm works by minimizing the error in predicted future reward. So this was an algorithm pioneered in the 70s and 80s by Richard Sutton and his colleagues, and it's been applied very successfully to a wide range of control and prediction problems."}, {"start": 1383.63, "end": 1409.6, "text": "So the question then to reform what I stated just in the last slide is, what we'd like to answer is, should reinforcement learning algorithms be considered for an AGI system? And we've had many arguments with engineers that say, yes, it should be, or no, it shouldn't. And we advocate, why don't we just look it to the brain for an answer to this question? And in fact, when we do that, we find the answer. So here in this, one of the most famous neuroscience papers of the last..."}, {"start": 1409.6, "end": 1439.28, "text": "This is a single cell recording of single neurons in a monkey brain. And what they found is that temporal difference learning is implemented precisely like the algorithms created by Richard Sutton, ubiquitously across the brain. So let's have a quick look at this. So here, this is a raster plot of neurons being recorded, a single neuron. So you can see here, these are all the different trials, and these are the aggregated firing rates across those trials. So we can just look at this top line."}, {"start": 1439.28, "end": 1464.4, "text": "Now this panel here occurs before the monkey has had any training. So what happens here in this system is that the monkey is just going along doing its business and it gets a reward. So this is indicated by R. And the reward in this case is a drop of juice, fruit juice. And you can see here, in response to getting the fruit juice unexpectedly, there's an uptick in the firing rate of this dopamine neuron."}, {"start": 1465.01, "end": 1491.68, "text": "Now, after a little while later, they train the monkey to associate a light coming on with the delivery of the fruit juice a short while later. So the light coming on is denoted by this CS symbol, Condition Stimulus, and the reward is still being delivered here, where the R is. What we can see is that this neuron firing has travelled backwards in time to be associated now with the light coming on, which reliably predicts the future reward."}, {"start": 1491.68, "end": 1516.69, "text": "Rather than when the reward is presented. So you see here, there's no extra firing. And then in the third manipulation, the final manipulation of this study, in this final panel, they now remove the reward. So the light comes on, the neuron fires as before up here, in anticipation of a reward, but now there's no reward. So what happens? The dopamine neuron reduces its firing rate."}, {"start": 1516.69, "end": 1546.11, "text": "This is exactly the behavior. This dopamine neuron is exhibiting the exact behavior you'd want to signal prediction error that a TD learning algorithm would need. And it turns out, of course, this is not just one neuron. This is a whole massive system, the dopamine system within the brain. So here is the dopamine system. And it's pretty ubiquitous across the brain. So one other thing then is to look at the architectures then. So in the brain, on the left-hand side, we have the brain implementing TD learning."}, {"start": 1546.11, "end": 1576.3, "text": "And this is a box diagram showing which brain areas implement which parts of the function of TD learning, the TD learning algorithm. And on the right, we have the engineering approach with just math functions implementing these supporting functions. And you can see, just by looking at the architecture, they're identical. So what can we conclude from this? Well, I think we can safely conclude from this that implementing reinforcement learning algorithms as part of an AGI system is a compelling approach to take. It doesn't mean it has to have this, but it means that if you..."}, {"start": 1576.3, "end": 1608.59, "text": "Try and build one with it in that may be not such a crazy thing to do. And of course, I haven't got time to go through these, but there are many, many other systems that we know about now. So we just talked about dopamine for reward. There's serotonin system that regulates mood and emotion. There's acetylcholine that deals with expected variability. And then there's norepinephrine that deals with unexpected variability. And there's others. So but just to be clear, although I'm making the case for systems neuroscience, what I'm really advocating is a hybrid approach, which is that"}, {"start": 1608.59, "end": 1638.08, "text": "We want to combine the best that machine learning has to offer and systems neuroscience. So I think I can make this most clear by saying where we know how to build a component for an AGI system, let's use the state-of-the-art algorithms. Let's just take those, the best of breed. So whether that's reinforced learning or hierarchical neural networks. Where we don't know how to build a component, then we should continue to push machine learning algorithms as far as we can. But also, it makes sense in parallel."}, {"start": 1638.08, "end": 1664.13, "text": "to look at systems neuroscience for ideas for potential solutions. So to do this in parallel, not one or the other. In fact, my colleague Shane Legg will be talking about some of our specific research using this approach tomorrow in his talk. So, example systems. So I've been talking about what other systems are there out there. We've seen some examples that have been used in the past. What are there left to be used? Well, actually, there's dozens."}, {"start": 1664.13, "end": 1686.43, "text": "And each one of these things I've just put up, flicked up here, which I've just chosen, would take a whole lecture to describe. But these are all potential candidate systems that might be useful for investigating in terms of AGI. So mirror neurons, model-based versus model-free learning, theory of mind, working memory, top-down attention, conflict resolution, and construction system for mental simulation, my particular neuroscience work."}, {"start": 1686.43, "end": 1717.07, "text": "But I haven't got time to talk about all that. I'm just going to talk about one thing, which I think is probably the most important of all, which is conceptual knowledge acquisition. How do we acquire conceptual knowledge? So we believe that concepts are a key thing we're going to need to solve in order to make some strides towards AGI. So let's have a look at what I'm meaning by concepts. So knowledge in the brain can really be split into very coarsely, and there's no hard boundary here, between three different levels of knowledge. So at the bottom level, you have perceptual."}, {"start": 1717.07, "end": 1745.52, "text": "Which is your sensory stream. And then this then goes into conceptual. So now we're talking about more abstract things. So perceptual may be what a dog looks like. Conceptual may be what a city concept is. The concept of a city. And then symbolic finally is labeling those concepts. So calling that concept of a city, a city. And let's look at the analogs of this on the machine learning or AGI side. So what we have is we have plenty of systems."}, {"start": 1745.52, "end": 1762.38, "text": "All these symbolic AI attempts, for example, based on logic and logic networks that deal with the symbolic level of information that the brain deals with. Also, in the last decade, we've had many advances in some very cool techniques that deal with sensory..."}, {"start": 1762.38, "end": 1791.31, "text": "Learning and Sensory Processing, including Geoffrey Hinton's Deep Belief Networks, Podjo's HMAC, which I just showed, and Jeff Hawkins' HTMs, Itamar Erol's Destin, and so on. There's a lot of good systems now that deal with this. But what we don't know how to do is really concepts, this missing bit in between. And if we could solve this problem, then we would be able to deal simultaneously with the classic simple grounding problem. How do we ground these symbols in terms of meaning?"}, {"start": 1791.31, "end": 1819.82, "text": "And also, how do we go from perceptual knowledge at the top of these hierarchical neural networks to conceptual knowledge? Okay, so that's the problem space. So how does the brain acquire conceptual knowledge? Well, we have some very good ideas now, and in fact, I think we're only a few years away from fully understanding this. And what seems to be the candidate system, if you like, is the hippocampal neocortical consolidation system. Now, for those of you who don't know, the hippocampus..."}, {"start": 1819.82, "end": 1850.75, "text": "It sits at the apex of the sensory cortex. You can see it here in blue in the middle of your brain. And then when I'm talking about neocortical, in this context, I'm meaning high-level neocortex, which means prefrontal cortex on the front here and association cortex on the side, as opposed to sensory cortices, which are around the back and top. And how does hippocampal neocortical consolidation work? Well, hippocampus stores the memories of recent experiences or episodes. It then replays those memories during slow-wave sleep at..."}, {"start": 1850.75, "end": 1879.54, "text": "Fast-speeded rates. I think we saw that in the last talk. It orders a magnitude faster than you experience them in real life. And what this does is gives high-level neocortex a tremendous number of samples in which to learn from, even if you only experience that one important thing once. Furthermore, memories are selected stochastically for replay, and rewarded memories, emotional and salient memories, are biased to be replayed more often."}, {"start": 1880.78, "end": 1910.48, "text": "So suddenly now, what this means is we have a system that can circumvent the statistics of the external environment. So the sensory cortices are limited to learning the structure of the external world, the statistical regularities of the external world. But that may not be what we want for our behavior. We may want to fiddle around with the statistics so we bias the things that are more important to our survival or progress. And this system has the ability to confer that. So this really is the first step that leads towards abstraction and semantic knowledge."}, {"start": 1910.86, "end": 1938.62, "text": "And as an aside, also I think it explains the contents of dreams. But I haven't got time to go into that. But for those of you who are Freudian advocates, I've got bad news for you. I think the evidence shows that dreams are epiphenomenal. So you can talk to me about that afterwards. So to summarize, the brain is a useful proof of concept for AGI. We now have a tool to meaningfully investigate it. And systems neuroscience can inspire new algorithms and validate existing algorithms."}, {"start": 1938.62, "end": 1964.34, "text": "And what I'm advocating is combining the best of machine learning with the best from systems neuroscience. And this system neuroscience approach has one final advantage, which is that building an AGI system in this way, at this level of abstraction, I think may also be the best way to understand our own minds and natural intelligence. So effectively what we're doing when we try to build AGI systems is distill intelligence to an algorithmic construct."}, {"start": 1965.23, "end": 1995.41, "text": "And by comparing this algorithmic construct to the mind, this should shed light on long-standing enigma-like consciousness. And then finally, what I envisage is a kind of research program that combines machine learning and neuroscience at a very fundamental level in a cross-disciplinary group of people, and this will help each other, so in some kind of virtuous cycle. So I'm just going to finish by leaving you with a quote from the great Richard Feynman that I think sums up this whole approach best, which is, what I cannot build, I cannot understand. Thanks for listening."}]}