{"url": "https://www.youtube.com/watch?v=EiStan9i8vA", "page_url": "https://www.youtube.com/watch?v=EiStan9i8vA", "title": "ICML DL Workshop 2015 - Panel Discussion", "show_or_channel": "KyungHyun Cho (ICML 2015 Deep Learning Workshop)", "date": "2015-07-10", "duration_s": 4143, "type": "panel", "speakers": ["Demis Hassabis", "Yoshua Bengio", "Yann LeCun", "Juergen Schmidhuber", "Neil Lawrence", "Kevin Murphy"], "confidence": "medium", "notes": "ICML 2015 Deep Learning Workshop panel, Lille; panel composition from memory, verify.", "people": ["demis-hassabis"], "id": "2015-07-10_kyunghyun-cho-icml-2015-icml-dl-workshop-2015-panel-discussion", "audio_duration_s": 4142.6605, "worker": "jc-w15", "transcribe_s": 56.0, "model": "large-v3", "segments": [{"start": 0.43, "end": 30.16, "text": "So I think we should probably start. Welcome, everybody, for the final event, the panel discussion. Let me introduce our distinguished panelists here. On the left, there's Juergen Schmidhuber from ITSE Switzerland. Then we have Neil Lawrence from the University of Sheffield. Next to that is Kevin Murphy from Google. Then we have Joshua Bengio from the University of Montreal, Jan LeCun from Facebook and NYU."}, {"start": 30.16, "end": 59.95, "text": "And Demis Hassabis from Google DeepMind. Hassabis, sorry. So let's welcome our speakers today. So there's one rule of engagement for today, which is we'll be speaking about the future. And we're going to leave the ghost of deep learning past for another panel discussion."}, {"start": 62.06, "end": 91.98, "text": "So the questions are projected up here in case you forgot what the questions are. So I actually sent the questions to them yesterday evening so they had some time to think about it. So we'll spend about 15 minutes on each question. So there's four questions. And if there's time left over after each 15 minutes, then I'm going to ask the audience to think up some questions."}, {"start": 91.98, "end": 121.87, "text": "You know when you have a question is when your heart is starting to pound really fast. That's the moment that you know that you want to ask a question. Okay, so the first question for the panelists. I'm also going to randomize the order in who is going to answer the question first in order to remove any bias. So the first question is, so deep learning has transformed fields such as speech recognition and image analysis and natural language processing."}, {"start": 122.03, "end": 148.4, "text": "So the question is, which fields and applications are next in line to be revolutionized by deep learning? And reversely, on what problems or data does deep learning not work so well? So the first to answer that question is Joshua Bengio, since he's looking at me. Not quite random, but you know. Thank you, Max. Hello? Hello? Yes, thanks. Thank you."}, {"start": 148.4, "end": 178.02, "text": "I don't think that natural language processing has been revolutionized in the way that speech and computer vision have been. But I think that it's very likely that we'll see some of the biggest changes in that area. In particular, there's a lot of effort these days in the general direction of language understanding. And we've seen a lot of progress, for example, in machine translation."}, {"start": 178.02, "end": 197.17, "text": "But I wouldn't call that a revolution. Let's say we've reached a state of the art where we were very far from it just one year ago. But I think the best is still to come. So that's one area where I see more big, big changes. Besides..."}, {"start": 197.42, "end": 226.05, "text": "The second question about what remains really challenging, and I would say we haven't succeeded, something I keep saying over and over again, and other people as well, is unsupervised learning. So we're spending a lot of energy on this, and there's a good reason. It's because if we are able to make serious progress on this, I believe there could be very big payoffs for AI, so really,"}, {"start": 226.05, "end": 255.54, "text": "Machines that understand a lot more about our world and are able to take advantage of the vast quantities of unlabeled data. Maybe other people want to add other things. I'm sure there are other directions that people in the panel want to go ahead. Yeah, let's move to the right in cyclic order. Jan. Okay. Well, I was going to say about the same thing. I don't think NLP has been revolutionized yet. I mean, it's pretty clear there's fast progress. The derivative is pretty steep, so we're going to..."}, {"start": 255.6, "end": 284.98, "text": "We're probably going to see this kind of an announced revolution for NLP, but it hasn't quite happened. And there is a lot of pockets of applications in computer vision which have not really been explored very much. So things like medical image analysis, I think it's going to be a huge practical set of applications of deep learning that people haven't quite explored yet. And there's enormous societal consequences to that. And otherwise, in things that"}, {"start": 284.98, "end": 313.54, "text": "Machines cannot do yet with deep learning. I think we don't really have a good handle, of course, on how to do unsupervised learning, but that's not really an application. It's more like a class of technique we're gonna apply to something else, like say video. So video is probably one area where we think unsupervised learning is really indispensable. And that might be a way to teach machines to have common sense in a way. The other thing that we don't know how to do very well is"}, {"start": 313.54, "end": 340.99, "text": "Of course, there is a lot of very interesting work going on in this area, in particular at DeepMind and Facebook and various other places, is reasoning and being able to work with working memory, so accumulating knowledge or facts in an episodic memory kind of system, and then being able to do long chains of reasoning and then planning on top of this. So all the things that we expect."}, {"start": 340.99, "end": 371.65, "text": "Robots or animals and humans to do and we don't yet do a particularly good job at this. So for the first part of the question, I think there are just so many areas and markets that are going to be revolutionized by deep learning in the next few years and other machine learning technologies. So I know there's lots of people in the room working on things like drug discovery. We're really interested in healthcare applications, medical imaging being one of those."}, {"start": 371.65, "end": 399.71, "text": "I think, and I know of friends of mine who are doing, using these techniques in finance and hedge funds. I think there's, if you're so inclined, there's another, I think it's going to cause a revolution there. There's been a lot of Q&A, dialogue work, both with us and Facebook and other places. I think that's another interesting application area. Robotics and continuous control. I think Ian mentioned earlier in his talk cybersecurity. I think that's another interesting one, and fraud detection."}, {"start": 399.71, "end": 426.43, "text": "And really anywhere where there's, you know, if you combine it with RL, kind of sequential decision making to be done. And in the second part of the question, in terms of what we can't do at the moment, I mean, obviously I agree with, you know, unsupervised learning. Also, the idea of having something like a hippocampal system or function to bring in episodic memory and working memory into deep learning. That's something we're working very hard on with neural Turing machines and Facebook are working on as well."}, {"start": 426.43, "end": 447.57, "text": "I think that's going to be huge. The other things that we think are really important that haven't been cracked yet are things like continual learning. So that's the idea of doing one task after another task, but then not forgetting and still being able to perform well on the original tasks. So even within the domain of Atari games, that's pretty tricky. And that's a hard problem."}, {"start": 447.57, "end": 465.42, "text": "And then moving on even beyond that, things like transfer learning, where not only don't you overwrite what you already know when you learn a new task, you actually bring to bear your previous knowledge in some useful, appropriate way to the new task. So I think there's huge challenges ahead. Thank you. Juergen?"}, {"start": 465.97, "end": 479.74, "text": "So I think I agree with much what my colleagues have said before. There are the kind of low-hanging fruit, which are passive pattern recognition, where you just do the progress in computation."}, {"start": 479.74, "end": 509.55, "text": "Per dollar, which is still increasing by a factor of 100 per decade, where the stuff that already works well is going to work 100 times as well, because at the moment it seems like it scales linearly, and so not only vocabulary is of that size, but 100 times the size, video understanding, understanding how video relates to text that explains it, and so I would call that low-hanging fruit, because it's kind of clear how that will be done, although faster computers,"}, {"start": 509.55, "end": 530.99, "text": "And cheaper computers will be necessary to spread it widely. And then there's the really interesting stuff, which is about reinforcement learning, where you really try to maximize your future expected reward. But then where you do the thing that little capuchin monkeys do already, namely plan hierarchically."}, {"start": 530.99, "end": 559.57, "text": "We use subprograms, previously learned subprograms. For example, up there in the tree, there's this fruit. And you see how this little capuchin monkey knows that he first has to go to the base of the tree and then invoke this subprogram for climbing up the tree and do all these little things that we do so easily, but our machines don't. And there are these excellent pattern recognizers which retrospectively decompose observed patterns into..."}, {"start": 559.57, "end": 588.5, "text": "I think within the next 10 years or so."}, {"start": 589.36, "end": 609.57, "text": "Not much longer, maybe just a few years. We will have something like an artificial animal-like intelligence, not necessarily connected to a real body because the mechatronics of real bodies, we cannot really copy those, and so I'm much more impressed by the..."}, {"start": 609.57, "end": 639.22, "text": "By the physical hardware of monkeys than by their intellectual skills. But once we have one of these little guys in a simulation which is sufficiently complex, a simulated world with which you can interact, you can talk to the guy, he sees you, knows what you're doing, understands what you're doing, and plans ahead to achieve its goals, whatever it is. We will have that, I think, in the next not so many years. And once we have a little cappuccino,"}, {"start": 639.22, "end": 668.35, "text": "Capuchin monkey-like intelligence. Capuchin monkey-like intelligence, then the next step to human-level intelligence may not be so great, because look at evolution. It took billions of years to evolve a Capuchin monkey, but just a few millions of years on top of that to have something like human-level intelligence."}, {"start": 668.35, "end": 694.03, "text": "Yeah, so I thought it was interesting that the three subjects that are mentioned, broadly speaking, are three sort of sister fields. So sort of vision, speech, and natural language processing. So they're in some sense already quite well studied fields in themselves, and there's a lot of data. There are also things that humans are really good at, and computers traditionally bad at. So I think it's interesting that the three have been identified as ones where deep learning is making a lot of progress. With regard to NLP..."}, {"start": 694.03, "end": 723.06, "text": "Maybe it's not revolutionized it yet, but NLP is kind of like a rabbit in the headlights waiting to be flattened by the deep learning steam train. It's very interesting. Everything's mentioned. I thought that was very relevant and interesting, particularly with respect to reinforcement learning. It makes me think of an example Carl Rasmussen uses. He shows the sort of chess board and sort of says, when can computers play chess? And, of course, you can work out how to move the pieces far earlier than you can work out, move the pieces what you should be doing."}, {"start": 723.06, "end": 742.24, "text": "Far earlier than you can actually work out the motor planning and control to move the pieces. And I think it's fascinating that we're so far behind on what should be a relatively simple task, such as motor control. You said we'll have the cappuccino monkey working away, but it won't be able to go anywhere. So no climbing up trees. I mean, it's incredible when you look at what animals can do."}, {"start": 742.24, "end": 769.95, "text": "The domain I'm really excited about is health, as Dennis mentioned. I think that that's going to be a really, really important one. One thing I'm excited about there is moving into the domain where we're doing things that humans aren't good at. So we can do all these tasks, but we do it by looking at a lot less data. So we need to be able to do things like look at entire populations' health statistics and draw conclusions about how we should deploy treatments. And this is something humans, we don't have systems at all."}, {"start": 769.95, "end": 793.49, "text": "Demonstrating systems that can do that, whereas here we have demonstrator systems. And then I think a final thing that's really interesting is how we interface the results of the deep learning to humans. I think we're in a difficult area where people sort of when they look at what a computer says to them, it outputs something, they just assume it's right. They don't have the same sense with the computer of sort of..."}, {"start": 793.49, "end": 822.06, "text": "You know, understanding its uncertainty, how it expresses itself, and why it's coming to a particular conclusion. And I'm kind of really interested in this intermediate stage before we get very good AI, where we're just using AI as an extension of ourself, how we ensure that the conclusions are appropriately presented to human operators. Do you feel that you understand how I came to my conclusions? Your own conclusions, even. No. That's what I thought."}, {"start": 822.35, "end": 853.1, "text": "But I want the deep learning thing that's more like, I don't know, I was going to say more like my wife, but that's probably a bad example. More like someone I've known for a long time. Right, Kevin. It's hard being last, everything's been said. So I would certainly echo that I think health is waiting to be revolutionized. Medical image analysis and also reading all of these articles that appear in Nature and New England Journal of Medicine, there's a ridiculous rate of..."}, {"start": 853.1, "end": 879.39, "text": "Publications coming out that humans can't consume. So this is in line with what Neil was saying. So, and then you get into this interesting territory. No one's mentioned, you know, combining all of this, right? So you want to interpret your image in the context of the surrounding texts and maybe clinical markers and, of course, background knowledge that you've acquired. So in all of these domains, but especially in medicine, you need to combine different modalities. And then you start getting into when you..."}, {"start": 879.39, "end": 906.46, "text": "In sort of echoing the RL theme, when the system can actually collect its own data and maybe create hypotheses and then go test them in the lab or do different tests on patients within the realms of ethics, then it's a closed-loop system, right? And things can really take off. I mean, especially if it's done in simulation or an automated laboratory. I think science is going to be revolutionized by automated machines carrying out high-throughput hypotheses. But to make that happen..."}, {"start": 906.46, "end": 932.08, "text": "We need algorithms that can come up with interesting hypotheses, preferably causal and mechanistic, and then the system can say, ah, you know, is A inducing B or inhibiting B, and then design the right experiment. And right now, I think we're picking up on a lot of correlation, and Leon said this in his talk about the car in the pool, right? We learn from context it kind of looks like it doesn't look like a car because it's not on the road, but it clearly is a car. So we haven't really got the essence of car."}, {"start": 932.08, "end": 948.85, "text": "And just a couple of other examples I was thinking of. So earlier today in this workshop, Ariel Vinyals gave a great talk about using RNNs to learn the traveling salesman problem and the convex hull of a set of points. But it didn't quite get the concept, right? The traveling salesman solutions that it created"}, {"start": 948.85, "end": 973.01, "text": "I don't want to pick on Oriole in particular, but just as an example, they would loop back on themselves. So it didn't get the concept of a tour, which is that you visit every city once and only once. And then, subject to that constraint, you want to minimize the distance. So that's sort of the basic essence of the problem that it didn't capture. It's sort of a logical constraint. Similarly, in the car, in the swimming pool setting, there's sort of an essence of car, which it still maintains, even though it's out of context. So I think..."}, {"start": 973.01, "end": 1003.36, "text": "We're missing, in some ways, the fundamentalism. We're picking the low-hanging fruit where the color or the texture is correlated with the object, but we're missing the fact that the shape or its function is really what defines it. So these are some of the problems that I think we haven't solved. Yeah, we're running a bit late, but if you want to have a short sort of reply, Juergen. But keep it short, please. I'm trying to keep it short. I think it reflects on both what Joshua said and what you just said. I think we do have a lot of good ideas, how to build this little Capuchin monkey."}, {"start": 1003.36, "end": 1018.3, "text": "You need a recurrent network which is a world model, a predictive world model. You need a recurrent network which is generating the action sequences leading to success. And part of the success is that you want to motivate the controller to improve the..."}, {"start": 1018.3, "end": 1048.26, "text": "World model, which means that you have to motivate it to construct experiments that improve the understanding of the world in the world model. And the way to do it is you measure the learning progress of the world model and give that as a reward to the guy, to the current network, which is coming up with the experiments that lead to the data that has the property that you can still learn a new insight about the world. And although the basic ideas are quarter century old when we first published that,"}, {"start": 1048.26, "end": 1077.74, "text": "With the increased computational power, it's now becoming a possibility to really go forward there. So I think we are further along these lines, and you may be pessimistically estimated. Okay, so we probably should stop here. I was just in time. Yeah, there's a lot of things to be said, I'm sure. So question two. So companies have access to more data and compute power than university labs."}, {"start": 1077.74, "end": 1104.99, "text": "If data and compute power will dominate the performance of deep learning models, will there be a danger that companies will completely dominate this field? And if companies may not be able to disclose all details of their research and share their code, what is then the impact on reproducibility of research? So Rodrigo Benison wrote in a blog post, so does some imagine a future where Google, Facebook, Amazon, et cetera, would sell pre..."}, {"start": 1104.99, "end": 1131.31, "text": "Trained models to be consumed and fine-tuned by smaller entities. What is the panelist's opinion of such worries and possible future scenarios? Is there anybody who wants to first respond to this? All right, Jan, great. Right. Well, I'm kind of in the middle of this, having one and a half feet in industry and half a foot in academia."}, {"start": 1131.5, "end": 1159.44, "text": "So I think it takes a village to make science progress, and there are different types of ideas that come out of academia and industry, because people work under different assumptions with different objective functions they're trying to maximize. So there is a very important role to play for academia, certainly. I still see most of the interesting basic ideas coming out of academia, not coming necessarily out of industry, although there's a lot of interesting things coming out of industry."}, {"start": 1159.44, "end": 1178.02, "text": "There's a very bright future for academic research in this area. Now, there are certain things, particularly on the applied side, where indeed you need a big infrastructure to be able to do it. But the advantage you have with this infrastructure only lasts a very short time. So the kind of infrastructure that we have at..."}, {"start": 1178.02, "end": 1193.71, "text": "Facebook or Google or similar companies, universities will have this two years from now, maybe. And that means that a lot of those things kind of disseminate in the wider research community."}, {"start": 1193.71, "end": 1217.97, "text": "Are we going to do research in secret? So it's certainly not the case. I can't speak for the other companies. Certainly, there are companies that are known to be very secret, like Apple and Amazon. But at Facebook, we pretty much publish everything we do. We distribute most of our code in open source. In fact, we even share open source code with DeepMind, because we both work on the Torch platform."}, {"start": 1217.97, "end": 1244.59, "text": "The reason we are open is for at least three different reasons, three main reasons. One is our research is better quality if we publish it. When you have a research lab that's isolated and works in secret, the quality of the research goes down by a lot, which is why military research sucks. And so it has to be done in the open if you want it to be high quality. Also, if you want to attract the best people in an industrial lab,"}, {"start": 1244.59, "end": 1273.3, "text": "You have to offer the possibility of publishing. Otherwise, the good people won't want to come. Third, research. Companies like Facebook and Google view research as kind of a way to show themselves as kind of leader in a technology field or in science. And it's good for hiring people even outside of research in their company. So this is an important role to play. And thirdly, we want people to work on the stuff that"}, {"start": 1273.3, "end": 1298.75, "text": "Students, people in academia, in other industries to build on top of things that we come up with, which is why we give our tools and our open source codes so that others can improve on it. And then, you know, these are perfect targets for hiring later or people to collaborate with and give money to in academia. There is this kind of interesting dynamics, for example, between Facebook and DeepMind where, you know, we're kind of half a generation away from each other."}, {"start": 1298.75, "end": 1322.34, "text": "Publishes a paper. Then another group publishes a paper just a month later on the same topic with kind of a slight improvement. And then the other one publishes again the same thing. Or two days later. Sometimes two days later, yes. That's happened. So I think it's very vibrant. It makes the whole field progress faster. We don't care, actually, so much if Google uses the same techniques that we use."}, {"start": 1322.34, "end": 1332.11, "text": "We do. In fact, it's great. What we're interested in is making the whole field progress faster. And if progress is faster, the community is larger. That's why we do open research."}, {"start": 1332.21, "end": 1361.57, "text": "So just to follow up on what Jan said, so first of all, just like with Facebook, at DeepMind and at Google in general, we publish pretty much everything we do. So I think I was counting, we published like 40 papers, I think, in the last 18 months. So it's a lot of work out there. Also, we try and, where we can, release code and platforms to open source, like the Atari stuff and DQN code. Obviously, we work on Torch as well, like the Facebook guys. So we share that in your network library."}, {"start": 1361.57, "end": 1384.14, "text": "And I think in general, that's for all the reasons that Jan said. It's important that we do that as well and engage with the wider machine learning community. And in fact, we have tons of collaborations with various academics in neuroscience and in machine learning. One thing I would say, though, is that maybe I can give some insight into those of you who work for small companies or in smaller labs."}, {"start": 1384.14, "end": 1413.42, "text": "When we were independent as DeepMind, one of the reasons we chose to work on games was, apart from a lot of us had backgrounds in games, so we knew what games to choose and how to convert them into useful platforms. Also, we chose them because we can create as much synthetic data as we'd like. So that's a very important point. Obviously, if you're not at a big company, you can't access tons of customer data and things like that, which actually is harder than you might think even in a big company, in fact, due to privacy and other things that are taken very seriously, I'm sure, at Facebook, and they are at Google."}, {"start": 1413.42, "end": 1434.11, "text": "So if you create sort of synthetic data generators, then you can get around a lot of the sort of problem of not having enough data. The other thing I would say is that I believe that there's going to be a lot of big algorithmic breakthroughs are going to be needed before we solve AI. I don't think it's going to be a matter of just..."}, {"start": 1434.11, "end": 1443.76, "text": "Purely scaling in terms of computation and data. So I think if you are in academia or in a smaller company, I think it's important to pick the right"}, {"start": 1443.76, "end": 1473.04, "text": "There are many, many open problems. We just discussed that on the first question. And pick the right ones that aren't necessarily just a question of getting large amounts of proprietary data or compute power. Of course, having said that, compute power does help. And that's one of the reasons we decided to join forces with Google so that we could access their incredible cloud compute. And that has, no doubt, accelerated our overall research program."}, {"start": 1473.04, "end": 1499.22, "text": "There are many complementary things that academia can do that industry isn't doing. Thank you so much. Juergen? It's my turn. Yeah. So I think in the long term, I'm not at all worried about a couple of monopolies taking over or something. I believe that in hindsight, the source code of AI"}, {"start": 1499.22, "end": 1530.03, "text": "Yeah, so,"}, {"start": 1530.03, "end": 1559.14, "text": "So in the future, I think that it's not a big problem. People sense these things are problems at the moment. I would characterize the companies mentioned there as quite fat companies in terms of they have a lot of money. They're actually extremely atypical. Most companies have struggled with their data. And I'd say that these three companies are what I would call data from the ground up. So they're companies that understood the value of data from the start and built all their systems around it."}, {"start": 1559.14, "end": 1587.15, "text": "There's masses of opportunity and money in going into companies that don't have that philosophy and trying to sort things out there. And actually, there's many, many challenges there. I also think a long time ago, like when Microsoft Research first started up, people moved there. I was there for a year and some people told me the reason they moved there was because they wanted the chance to work on real problems that made a difference. And Microsoft was the place to do that because they had access to desktops and systems. And I think that made a lot of sense."}, {"start": 1587.15, "end": 1616.7, "text": "I kind of sometimes worry, like, if I were to go to Google now and sort of say, hey, I'm going to improve your web search, that, like, I'd be competing with Kevin and everyone at DeepMind. And, you know, there's, like, more people in Google trying to solve these problems than there are outside. And actually the number of problems that are considered serious is fewer than there are problems in the entire world. And the availability of data via the Internet means that you can almost, I would say, be more creative about how you're going to influence directions outside the company."}, {"start": 1616.7, "end": 1646.29, "text": "Also this issue of privacy. I suspect, certainly for the moment, people would be more trusting of universities looking at certain types of data than companies, whatever the intentions of the companies are. So there's a lot of freedom inside companies. And, you know, I have to give a shout out to Amazon. They gave us a large grant to work on the Amazon Cloud. Grants like that and support like that is a wonderful help for academic groups to get the compute resources. So I think this worry is definitely over."}, {"start": 1646.29, "end": 1670.38, "text": "So it's true, you know, at Google we have a large compute cluster, but as researchers we share it with the advertising group and they get priority. So the edge that we have is very small and diminishing over time, like Jan said. You know, Amazon AWS is an amazing resource that's available to you all. It's extremely cheap for academics because it's subsidized. Microsoft and Google also have, you know, cloud compute efforts. And deep learning is just going to be commoditized."}, {"start": 1670.38, "end": 1697.22, "text": "You know, scikit-learn, you know, anyone can download it and run it on the laptop and then just deploy it on Amazon with a button press and it scales beautifully. Deep learning will be like that, you know, within a year or so. It's just a software engineering exercise, at least for the supervised problems. I mean, it's already like that. So this last comment about selling pre-trained models, yeah, it probably will happen. But, you know, how many people in the audience have downloaded a pre-trained ImageNet model and fine-tuned it on their data set of interest? I mean, this is what people do right now."}, {"start": 1697.22, "end": 1725.46, "text": "And you can either get that pre-trained model from Berkeley or you can get it from Google or wherever. So I don't see that's a problem. So the edge in computing, I think, is smaller than believed. And then the edge in data, I think, is also smaller than believed because there is a lot of data, but a lot of it we can't touch because it's personal or private. And then secondly, it's not necessarily that diverse. I mean, there's, what, six billion people on the planet. But there is a long tail of unique behaviors that you see."}, {"start": 1725.55, "end": 1748.72, "text": "But if you have access to massive data, it sort of sometimes biases the way you think, and you worry about, well, you fit very large models, which of course is good, but then you forget problems like zero-shot learning, and learning from very small sample sizes, and learning very quickly in a new domain, or a new language, or whatever, which humans and other animals are excellent at, and babies just learn from,"}, {"start": 1748.72, "end": 1778.74, "text": "Tiny amounts of data compared to what we feed into our machines. So you can be distracted by having too much data. So I think being in academia, it's kind of like Russia in the space age, right? They didn't have access to as much electronic power as the U.S., so they were very creative in squeezing their assembly code down and coming up maybe with better algorithms. So sometimes paucity is an advantage. Everything that has been said, I agree with. I like what Juergen said."}, {"start": 1778.74, "end": 1803.6, "text": "One thing I'd like to add is that for academics, there are lots of opportunities with the data that is available out there. I mean, the Internet is an amazingly rich source of data, and especially if you care about unsupervised learning, well, of course, there's an infinite amount of data available to you. For supervised learning, the problem is a bit more difficult, but nonetheless..."}, {"start": 1803.6, "end": 1829.66, "text": "Many of the most interesting problems from a research point of view can be tackled, can be problems not in the sense of applications but like research challenges can be addressed by looking at not such a large scale of problems. If you're trying to look at optimization issues, if you're trying to look at inference problems and so on. There's a lot of progress we've made in recent years has been done with not so huge models."}, {"start": 1829.66, "end": 1850.53, "text": "So I'd say we shouldn't worry too much. We can pick the problems for which we can find the data, and they're really exciting and interesting. All right. Thanks. So we just steam on with question three. So deep learning may have become overhyped and created a bubble where the expectations are higher than what can be delivered."}, {"start": 1850.53, "end": 1880.3, "text": "Recent discussions on social media on renaming practices of existing models and cheating on competitions may be the early signs that the stakes have become very high. Are we over-hyping, and how can we avoid that this will fire back on us like it did during the previous neural network hype cycle? Is there anybody who would like to start? We already started last time. Yeah, so you're, unfortunately. I've been actually writing quite extensively about this."}, {"start": 1880.3, "end": 1904.35, "text": "On Facebook, obviously, a little bit on Google Plus as well. So hype is very dangerous because hype killed AI multiple times in the past. The AI winters that we've been observing for different subsets, subfields of AI have been due to unmet expectations, essentially. And it's mostly due to three kinds of people or three kinds of activities that sort of drive..."}, {"start": 1904.35, "end": 1929.42, "text": "Drive people who would otherwise be completely reasonable to kind of make unreasonable claims. The first one is self-deluded academics who promise things that they obviously cannot deliver, but somehow convince funding agencies that they can. The second one is founders of startups who have to promise the world to get enough funding for their startups."}, {"start": 1929.42, "end": 1956.88, "text": "And then the third one is actually program managers in funding agencies promising to their management things that on behalf of the scientists that the scientists cannot deliver and then are kind of on the line to do it. So all those things kind of create hype. And then the last aspect is the attention of the press. So AI has attracted a huge amount of attention from the press recently. Sometimes when"}, {"start": 1956.88, "end": 1981.95, "text": "There's kind of a good piece of work that we're very proud of. DeepMind Google does the same. We make a press release, but that's relatively rare. What's more frequent is that we put a paper on archive, and it's picked up by Bloomberg, Wired Magazine, whatever, without us doing anything. It's just they write about it. And sometimes they write that it's a breakthrough, and we don't think it's a breakthrough, but they somehow present it as a breakthrough. So this is the kind of hype that's great."}, {"start": 1981.95, "end": 2010.9, "text": "Very dangerous. There's another kind of hype that's very dangerous, and it's the one where the press is so clueless that they present as kind of AI revolution, things that are pretty pedestrian by our standards. And it's very difficult to explain then to people in the press that it's not what you think. There's machines that do dialogues or there's systems that generate images that are not that intelligent after all."}, {"start": 2010.9, "end": 2040.58, "text": "We've been thinking about this, some of us have been thinking about this collectively, how to fix the problem. And the fact is we need to put our paper on archive as fast as possible because that's the way the field progresses quickly. I'm sure most of you have had read many of the papers that were presented at this conference before coming here because they were on archive many months ago. But the journalists have no way, and even the wider public has no way of knowing whether a particular paper is good or not. So what we need is an open review system such that"}, {"start": 2040.58, "end": 2066.32, "text": "People in the community, et cetera, can write reviews for papers, make comments. And all of those reviews are possibly anonymous, but are seen by the wider community, the research community as well as journalists. And that will sort of help stem down the hype in some instances. OK."}, {"start": 2066.32, "end": 2090.77, "text": "So, you know, I agree with what Jan said, and I think that we have to be careful about overhyping the field, especially as it's such a hot topic right now, obviously not just in academia, but also in venture capitalists and industry sort of circles. So, you know, it's very important that we keep grounded with the actual rigorous results that have been produced, and don't try and over-claim."}, {"start": 2090.77, "end": 2109.26, "text": "I would say, though, that just the converse of it is, of course, that there are all these AI winters."}, {"start": 2109.26, "end": 2141.39, "text": "The two AI winters that have happened. And I think we understand the causes of those better now. Like why those algorithms didn't work or why they didn't scale. And obviously a lot of it was not having enough compute power. But I think there's nothing wrong with aiming high. But making sure you're not claiming success that isn't there. Or over promising. So I think aiming high is fine. But making sure you stay rigorous with and not delude yourself in terms of what you're actually achieving."}, {"start": 2142.58, "end": 2152.18, "text": "Well, I'm happy to say something. Yes, go ahead. Yeah, I think that this is important. I'm sorry, I was getting confused. I missed the end of Denise's thing because my microphone went."}, {"start": 2152.27, "end": 2176.69, "text": "So if I cover things you said again. So one thing I'd like to mention is Talking Machines, Katie Gorman and Ryan Adams podcast on machine learning. Absolutely fantastic stuff. And really explaining to people what's going on, what the technologies are. I think this sort of engagement with the press from academics rather than journalists. I mean, to be honest, the cheating thing."}, {"start": 2176.69, "end": 2206.37, "text": "I kind of really think the MIT Technology Review were well out of order in terms of the first reporting of that and then the follow-up reporting. It was like the same journalists that hyped the thing in the first place. Now, these are journalists. They're supposed to do it. If we can't trust journalists, then we have to do it ourselves and step in. So one of the things I've been trying to do is write posts which I think reflect a bit more what's going on. I mean, we are probably overhyped, but it's difficult to control. And we just have to be very, very aware of it, I think, and react."}, {"start": 2206.37, "end": 2233.26, "text": "And perhaps prepare store coal for the next AI winter. The other thing that I think we also have to do, it makes me feel a pressure to get on with these algorithms that I've been working on that I think will address some of these things, I hope will address or encourage other people, work together as a community to make sure we deliver as best we can."}, {"start": 2234.26, "end": 2259.57, "text": "Yeah, so since you're self-organizing now, just go ahead, whoever wants to talk. I completely agree with what Jan said, that in the past we often had a source of hype through people seeking funding. I remember in the 60s there were these... I don't remember writing one of those, because I was a baby back then, but there were these grants where you said in the first half..."}, {"start": 2259.57, "end": 2288.1, "text": "Half year we will do a speech recognition, and the second half year we will do the video recognition, and then we will combine these two in the integration phase. And we will do all of this with a tiny fraction of our human brain power, which was really ambitious. And then, of course, from today's perspective, we are in a better position, because now we have computers that seem to approach human brain power, or at least, see it as animals' brain power."}, {"start": 2288.1, "end": 2310.19, "text": "Why am I convinced that we are not going to see another NN winter? Because at the moment it looks like the future of hardware will be very much NN-like. What is the future of hardware? It will be a compact volume of matter where there are many, many different processors connected through..."}, {"start": 2310.19, "end": 2339.2, "text": "Many short wires and few long wires, so essentially something like a recurrent neural network, which obeys these constraints that physics dictates, which is there are certain speed limits, light speed in the extreme case, lower speed limits in the brain, and you want to minimize energy costs, and so you will end up with something like a recurrent neural network of a certain 3D structure, which means that all of what we are doing now with recurrent networks is going..."}, {"start": 2339.2, "end": 2369.71, "text": "To be more and more important just because that's the future of hardware in general and every single decade we are going to win another factor of 100 per dollar just in hardware terms and lots of what we can do already now is just going to scale linearly. So it would be, for me, very difficult to see another NN winter coming. So I actually had to ask Joe what he meant by the renaming practice, so I guess it was referring to"}, {"start": 2370.29, "end": 2398.61, "text": "You say your technique is called X, and it's really equivalent to Y, which was published a year ago, but it's with a slight epsilon, and you gave it a radically new name. There was a specific paper he had in mind, which I don't remember probably for the better. But, you know, this is not new. I mean, I remember when Conditional Random Fields came out, you know, 10 years ago, whatever, Pereira, Al, and McCallum. And then a year later, a paper came out called Discrimitive Random Fields. And the only difference was one was a 2D lattice and one was a 1D chain. It's exactly the same model other than that. Now, of course, that makes a big..."}, {"start": 2398.61, "end": 2428.29, "text": "Algorithmic difference in terms of computational complexity, but arguably it didn't need a new name. So people will be doing this all the time to try to get ahead. So I don't think that's new. And in cheating, okay, so Baidu did it, but students cheat all the time, right? We know that. So that's also not new. It's just human nature, I'm afraid. I think the difference is, sure, there's money on the line now and the press is watching. So that does raise the stakes. In terms of hype, so one advantage of being at a company is that you're expected to make stuff that actually works."}, {"start": 2428.29, "end": 2455.23, "text": "And then you're supposed to launch it in front of, you know, millions, hundreds of millions of users. And if it doesn't work, you'll find out. And so it really stress tests, right? There's a lot of academic results where they test on one data set and it works great on that one data set and it fails miserably on another. There's a great paper by Antonio Tarraba and Alyosha Efros-Khoda, Unbiased Look at Data Set Bias, where they show that you can get a model that works great on ImageNet and works terribly on Pascal VOC and vice versa."}, {"start": 2455.23, "end": 2486.0, "text": "And so people are not doing a lot of this cross-dataset testing. And in the wild, if you launch a product in front of users, you don't get to control how it's used. And people will really stress test your model. So I think that's keeping us honest, which is a good thing. And many academics who are working on these problems also want to see their systems being used in the real world because it's fun and impactful, but it's also a way of ensuring scientific honesty. Joshua? I think a lot of good things have been said."}, {"start": 2486.0, "end": 2515.87, "text": "I'd like us to return a little bit to some of the suggestions of, I would say, social engineering that Neil and Jan talked about. And I think we should really put a bit of our minds on this. How do we get involved in sending the message and self-discipline in changing the way publications are put out and commented on and reviewed?"}, {"start": 2515.87, "end": 2540.1, "text": "to encourage a more open, rigorous way of doing science. And I don't have the answers, but I think it's worth all of us thinking about it and making different experiments because we'll need to experiment. So I very much like Jan's idea, and I'm trying to promote it, of an open review system."}, {"start": 2540.1, "end": 2564.67, "text": "Where very quickly we can get feedback from researchers after papers have been posted. And it can play a role that's similar to the traditional reviewing system, but it could be much faster, much more open. And one thing that would be very different is that I'd like to see people put comments that are not anonymous."}, {"start": 2564.67, "end": 2583.79, "text": "You won't have the choice of anonymous and non-anonymous comments. But the non-anonymous comments are interesting because when I write something on the web with my name on it, I have to be very careful about what I say because my reputation is at stake. So at the same time, I'll be more rigorous in what I write, and I think that's missing in a lot of reviews."}, {"start": 2583.79, "end": 2613.1, "text": "And also, I'm going to have an incentive not only to do a good job, but to actually do that job because I feel that it's like a little mini paper, this review or this comment I've made on the paper. I've thought about it. I have some maybe constructive criticism to make. So this can inform others and make it much faster for us to know what are the interesting ideas floating around and increase not only the progress of science, but also reduce some of the effects of hype that we're seeing today."}, {"start": 2614.61, "end": 2643.17, "text": "I'd like to say one more thing. I forgot to say it earlier. Reproducible research. So I know this is a key topic. I know Neil's a fan of it. Yoshua's a fan of it. So I mentioned if you're in a company, you've got consumers basically keeping you honest. But the other avenue which is available to everyone is to put your code on the web and have a script that you can run. And maybe it takes a week, but it's running on the cloud or it's a pre-trained model. And someone can actually validate your results and reproduce it. And it is, okay, it's a pain. I had to..."}, {"start": 2643.17, "end": 2671.63, "text": "Persuade students to do this in the past. But it's very helpful even to the person doing the research because then they can reproduce their own results and months later when they've forgotten exactly how they did it. So it encourages a level of code, health, cleanliness, and just making things transparent. And then it also enables other people to rerun it on new data. And, you know, of course there's no universal bullet. If your method just fails in another data set, it doesn't mean you were dishonest or that you did a bad job. But showing failure cases"}, {"start": 2671.63, "end": 2700.74, "text": "It's very enlightening, right? So in computer vision community, it seems to be a trend these days where people show a few successes and they also show where it fails. And I've seen that in a few talks at ICML, but not as many as I would like. So I think revealing the failure cases of your model is the way to promote progress and to avoid overhype. People will realize, okay, it's not going to work in scenario X because it was never trained. It's never seen data like that. So reproducible research. Put your code on the web. Document it right."}, {"start": 2700.74, "end": 2729.25, "text": "Maybe use CodaLab, all of that. So I want to remind all of us something that Leon Boutou talked about in his plenary talk the first day of the conference, where he suggested that maybe the editors of journals and the chairs of conferences request from their viewers that they request from the authors that the papers contain"}, {"start": 2729.25, "end": 2756.26, "text": "A description of the limitations of the method that's fair and honest so that it becomes a habit to do exactly what Kevin is talking about. Whereas right now, of course, there's a race to hide the things that we don't want the reviewers to see in case they might reject our paper. And this is very, very bad for the whole community. Great. So this is my last question. And since we're running a little bit out of time, let's not maybe dwell on it too long."}, {"start": 2756.26, "end": 2785.33, "text": "Because it would be really nice if there's some questions from the audience as well, unless we can go over time. And I'm sure you'll be pretty tired after an hour of talking. So I'll just read it. So famous scientists such as Stephen Hawking and captains of industry like Bill Gates and Elon Musk have repeatedly warned for the fact that AI may become dangerous for humanity within a few decades. Do you agree that the singularity is near? And if so, how would it look like? Will AI indeed outsmart us and become dangerous for humanity?"}, {"start": 2785.33, "end": 2800.67, "text": "What other, perhaps more realistic, dangers do you see coming from AI? I'll start here, I think. So I definitely think these fears are overblown. There's a quote going around, it might be due to Andrew Ng originally, that worrying about"}, {"start": 2800.67, "end": 2823.9, "text": "Smart AI overtaking humanity and killing, or maybe not necessarily killing, but taking over the world is like worrying about overpopulation on Mars. Okay, so maybe that's a possibility, but it's so distant, it's a distraction. And the problem is not just that it's sort of a ridiculous thing to think about, but it's taking up bandwidth and mental cycles, which could be spent on more realistic worries. So I'm much more concerned about climate change."}, {"start": 2823.9, "end": 2849.14, "text": "And inequality and unemployment induced from technology, which seem much more likely to happen than these sort of very, very speculative things. So I think it's a distraction and we should discourage discussion of it. So I should stop talking about it. So I think there's a worry that comes much before this. The amount of power we give. I've written about this."}, {"start": 2849.14, "end": 2875.79, "text": "Giving people technologies that can process an enormous amount of data. And if the data, there's a sort of natural data monopoly, which we've talked a little bit about. That leads to what I'd call a digital oligarchy, where you've got people who have power over data. They didn't even develop the algorithms themselves, but they make large-scale decisions. And I think that that's extremely dangerous, and it will happen way before this. So if you look in history, I think you can find very few occasions when machines started killing."}, {"start": 2875.79, "end": 2896.37, "text": "You can find very many occasions when, given a technological change, people killed many people or affected many people's lives for the worse. There are many, many hard-form-worn rights that date back, as I like to say, 250 years, written down by a bunch of Englishmen living near Boston."}, {"start": 2898.42, "end": 2924.34, "text": "And these rights are somehow, they're not being revisited in the digital age. And I think there's privacy issues. I think we should be looking at algorithms that are more privacy aware, starting work on that, and worrying about those sort of things, because those things will come to us before these problems. That would be my thing. Yeah, I just want to say I agree with Kevin. Great. I agree with Kevin."}, {"start": 2925.36, "end": 2946.82, "text": "So, yeah, so I mean, I think broadly speaking, these things are massively overblown. And I mean, I know Elon Musk quite well because he was an investor in DeepMind. And actually, I was the first person to sort of introduce him to the idea of AI. So maybe it's kind of partially my fault. But, you know, I think he, you know, the thing with..."}, {"start": 2946.82, "end": 2976.18, "text": "With Elon and also Stephen Hawking, who I recently had the pleasure of meeting, invited me up to Cambridge to talk about AI. And we ended up, he had like three hours of questions for me. And it was a very interesting chat. And I think at the end of it, he was more reassured. Because the thing about these guys that are mostly talking in the press, like Elon and Stephen Hawking, I think Steve Wozniak, they are amazing, obviously, in their fields and incredible people. But they don't work on AI. And they don't work close to the engineering side of it. So they don't see the everyday struggles."}, {"start": 2976.18, "end": 2992.93, "text": "I think they tend to read books like Nick Bostrom's book on superintelligence and kind of get carried away with the sort of almost the sci-fi scenarios in there that are many, many decades away. So I think, having said that, though, I do think there are some legitimate concerns about..."}, {"start": 2992.93, "end": 3020.27, "text": "Optimization systems in general that when they get hugely powerful, you know, how do we specify things, for example, like their value systems and their goals robustly and correctly specify them so they do the things that we want them to do. And I think that those kinds of questions, while the impact of that won't be for many decades, they're very tough questions. You know, it's tough or maybe tougher than getting and then actually building AI. So I think that it is worth"}, {"start": 3020.27, "end": 3049.58, "text": "Thinking about that at least and possibly doing some, having some academic work sponsored by, you know, it's what Elon's doing. I think there's no harm in doing that even though it's taking some CPU cycles, human cycles away from other research. I think there's plenty of things, plenty of people working directly on AI research and machine learning. So I think having a few people, philosophers, mathematicians and other things, other people, other disciplines thinking about these issues is worthwhile so that"}, {"start": 3049.58, "end": 3077.54, "text": "We're ready in the decades to come for any challenges that come up. I get asked this question from the press all the time. And one thing I have to explain to people is that the only example of intelligent entity that we face is basically humans. And so we have a very hard time dissociating human intelligence from human..."}, {"start": 3077.54, "end": 3095.82, "text": "Qualities that machines don't necessarily have. A lot of the bad things that humans do to each other are due to very sort of ingrained behaviors that were built into us by evolution so that we survive. Things like the survival instinct and the"}, {"start": 3095.82, "end": 3122.78, "text": "Desire to have access to resources so we don't starve and desire to have some control over other people, etc. And jealousy and things of that type. There's no reason that machines will have that. And so there's no reason for them, unless we build those behaviors in them, that they will take over the world. And of course, to people like us, this might sound kind of trivial, but the public has a hard time."}, {"start": 3122.78, "end": 3153.82, "text": "Visualizing this given everything we see in science fiction movies and stuff like that. Yeah, I agree with that. That's what I was going to say. I think we have a psychological bias that makes us see machines as if they were humans. And it's just not reality. Juergen, you want to say something? Yeah, I guess so. I don't like to call the singularity a singularity. I prefer the name Omega."}, {"start": 3153.82, "end": 3184.91, "text": "Because that's what Teilhard de Jardin called it 100 years ago. And also because it sounds much better than Singularity. It sounds a little bit like, oh my God. Having said that, it seems still clear to me, and so maybe it's a little bit of a contrarian view here, that within this century, we will have..."}, {"start": 3184.91, "end": 3210.51, "text": "AI is much smarter than myself. I'm very unimpressed by my way of thinking. Most of the time I'm spending avoiding obstacles, always doing the same thing more or less every single day, sometimes rarely having a halfway reasonable idea. But I'm very unimpressed by my own intellectual capacities, and I don't see how the much faster computers of the future won't be at least."}, {"start": 3210.51, "end": 3236.66, "text": "A match for that and actually go far beyond that. So I'm a little bit less pessimistic or optimistic as my colleagues here. It depends on the viewpoint. However, to me it's also clear that we should not base our predictions on Arnold Schwarzenegger movies, but instead take into account that there's a certain..."}, {"start": 3236.66, "end": 3257.63, "text": "Far future protection from these super smart AIs of the future, if there indeed will be any, through their lack of interest in us. Because look at yourself. You are mostly interested in other beings with whom you can either collaborate with or compete with. Politicians are mostly interested in politicians."}, {"start": 3257.63, "end": 3280.94, "text": "10-year-old girls are mostly interested in other 10-year-old girls. Frogs are mostly interested in frogs. And super smart AIs of the future will be mostly interested in other super smart AIs of the future. What about cappuccino monkeys? Won't we be their cappuccino monkeys? Cappuccino monkeys are mostly interested in other cappuccino monkeys."}, {"start": 3281.33, "end": 3309.12, "text": "And of course, as soon as we have one artificial capuchin monkey sitting in a high-definition physical simulation, we will have a friend for him, and this will be a whole little society of capuchin monkeys, and they will learn their social context and will know to hide their fruits from the other guys and so on. So you will quickly have an ecology of smart and not-so-smart guys and so on."}, {"start": 3309.12, "end": 3329.07, "text": "In many ways, it will be a very distributed system and very soon we will have a situation which will maybe be much closer to the ecology, the biological ecology that we have today, where there's not one super mind or whatever which is controlling everything. No, instead we have..."}, {"start": 3329.07, "end": 3363.86, "text": "Millions of different niches, which are filled by quite different little beings, and the future of robotics and AIs will be just like that, I think. So did you already say something, Neil, or can we move to the questions? Because I think we're running out of time, and I think if some people have burning questions, then it would be really nice to listen to them. So is there people who can hand out microphones? Okay, so you want to ask a question?"}, {"start": 3364.14, "end": 3391.66, "text": "Oh, you want to hand out microphones? Okay. There's two microphones at the top they can hand out. The guy had two microphones. Oh, okay. I'm out of the game now. No, you can just take another microphone. Your neighbor has a microphone. No problem. Okay, but now we still need the questions. We have the microphone. There's a question right there. Hey, guys. I'm just wondering if you think we should be exploring the idea of..."}, {"start": 3391.66, "end": 3419.26, "text": "Artificial curiosity and artificial creativity a bit more or is that not really an area of significant interest? That may be a question to myself because this artificial curiosity thing that is something that really has occupied me a lot in the past 25 years and of course we want that and so in that sense we will also have these little"}, {"start": 3419.26, "end": 3440.99, "text": "Curious artificial scientists who are going to devise their own little goals which will just lead them to better understand the world. This is what you want to see in an intelligent being. So if you have a reasonably intelligent being, then it's reasonably intelligent because it is curious, because it devises its own active experiments to understand how the world works."}, {"start": 3440.99, "end": 3457.65, "text": "That's what little babies do. That's what 20 years later, when they are not babies anymore, scientists do, when they are just creating bigger experiments like CERN to find some particle and whatever. But this basic drive, this curiosity, which..."}, {"start": 3458.99, "end": 3488.59, "text": "In my point of view, it goes back at least 25 years. This will have to be in there. And so the future of unsupervised learning that Joshua mentioned will actually be this active type of unsupervised learning where you not only find regularities in the data, in the passively observed data that is now stored as your history. No, you will actively shape the data that is coming in, just like a physicist actively shapes the data through his experiments."}, {"start": 3488.59, "end": 3500.02, "text": "And so artificial curiosity means just give a reward to the guy who is providing the experiments, who is inventing the experiments, such that you can learn something about the world which you didn't know yet."}, {"start": 3500.02, "end": 3524.7, "text": "So you find regularities in the world, which you didn't know yet. And you can measure the insights, the depths of your insights, by just looking at the compression progress, which is then the reward for the guy who is inventing new experiments. Of course you want to have that in the future. It will be the future, I would say. Without that, no true intelligence. Thank you."}, {"start": 3524.7, "end": 3548.78, "text": "I'm not going to say disagree, but can I give a different answer that doesn't have the same conclusion? I think understanding these things as aspects of human intelligence is very, very interesting because I think it's very important, as I said before, for the computer to communicate correctly with the human."}, {"start": 3548.78, "end": 3565.78, "text": "In which ways we're curiosity, just from how we use it, I think is great. But I would like to think that the curiosity that you're talking about emerges just from doing it the right way. In our case, if you say artificial curiosity, it says, oh, well, let's put a subroutine in here which says, now be curious."}, {"start": 3565.78, "end": 3589.9, "text": "And then the computer is curious, which sounds like what people used to do. Whereas I think what we really want is just algorithms where it's obvious that this is emergent, that they have natural explore-exploit trade-offs, it's principled, and the curiosity is the explore. So I think Zubin has said quite often that artificial intelligence is a dumb term because it's just intelligence. How can it be artificial?"}, {"start": 3589.9, "end": 3617.94, "text": "What's it like? Artificial flying. If it's a plane, that's artificial flying, I think is what Zubin says. It's just intelligence. And the same thing, curiosity may be emergent. I think then there's another aspect to that, which is understanding the ways in which humans are curious. That could also be important for the interface. Thank you. So there's a professor of psychology at UC Berkeley called Alison Gopnik, who's written a great book called Scientists in the Crypt that elaborates on the thing that J\u00fcrgen mentioned, that scientists are just..."}, {"start": 3617.94, "end": 3647.49, "text": "Doing what they were doing as babies and exploring and testing hypotheses. She uses a very pithy phrase to sort of justify why people do this. It's called Explanation as Orgasm. It's a title of a paper she published and the editor wanted to bolderize the title because they thought it was inappropriate for an academic journal. But it really is pithy. The point is that you get reward for having explained the world. So you don't need to have an external reward. Understanding is its own reward. If that's part of your objective function, you'll get model building."}, {"start": 3647.49, "end": 3675.55, "text": "And exploratory behavior for free. The other thing I wanted to say is, going back to the earlier point of massive data, a lot of the data we have is extremely redundant. And we do our SGD algorithms, and you just keep saying the same thing again and again. And you beat the computer upside the head with like a million images of a car. And a kid will learn it with one or two shots, but they're interesting views. They'll look from the side, and maybe the light will change or something."}, {"start": 3675.55, "end": 3695.34, "text": "They're looking at variations, either self-induced or just naturally, but they're not just getting lots of very redundant samples. So I think we can improve statistical and data efficiency by building exploration as well. Okay, so I think there's another question there. Timon? Thank you."}, {"start": 3695.34, "end": 3726.24, "text": "I think this question follows a very interesting discussion. There seems to be some consensus that we should be more afraid of people misusing this technology than of the computers going off autonomously and misusing it. Still, it's a real danger, people misusing it. Do we as a research community have a responsibility to prevent that or at least try to? I'm sure somebody is dying to answer that. I think we don't have the power."}, {"start": 3726.24, "end": 3752.29, "text": "The responsibility we have is to explain the science, the technology, and the consequences to people, the public at large, the decision makers, so that societies organize themselves to deal with it. So that's the duty we have, to explain to people really what this is about and what the potential benefits and dangers are, etc."}, {"start": 3752.29, "end": 3779.47, "text": "We as a community don't have the power to tell societies what to do with our technology. It's never been the case. Well, we could sort of refuse DARPA funding. That would be a clear statement. Well, okay. No, there's a clear statement, which is that as long as the funding you get does not come with restrictions on publications, there's nothing wrong with it. Because you communicate what you..."}, {"start": 3779.47, "end": 3808.69, "text": "The progress you made to the rest of the world and the rest of the world profits from it. It's very rare that there are things that are done for an inferior purpose where you're allowed to publish. So that's where you put the limit. If someone gives you money but gives you restrictions on publications, then you should be worried. I think there's two answers to your question. One is yes, we do have a responsibility, and the other is no, we don't have a responsibility. I'm pretty certain that's all there are. But there's a bit more to it than that. I agree with Jan."}, {"start": 3808.69, "end": 3836.4, "text": "To an extent. We can only persuade society to do what society wants. But on the other side, we are the tool makers. So if we don't provide the tools that allow these things to be implemented safely, that protect individuals' privacy, then we're not allowing society to make the choice to go in those directions. And when you look at the sort of dystopia, utopia type perspective, people will say, look, there's all these problems with health, we can save all this money, let's use it, let's do it."}, {"start": 3836.4, "end": 3867.14, "text": "But it will go in without protections. So we're supposed to be the far-sighted academics community of these things. So I think we should be worrying about that and building that tool set. At the same time, we can't fully control. We can just persuade and provide the tools, the environment. All right. So I want to mention that we are five minutes over time. So I think it's wonderful, great, and very interesting. But you will have to decide whether it's going to be too much. So I'm going to do one more question."}, {"start": 3867.14, "end": 3895.92, "text": "And then I think we're done. So is there one? I'll ask myself if it's okay. So actually, it's going to be a bit directed to Demis, Jan, and Kevin. For instance, in DeepMind, DeepMind is working heavily on the reinforcement learning. And then let's say one day in several years, your agent suddenly started talking to you with certain level of artificial intelligence or the human level intelligence. Would you make it public or would you keep it to yourself?"}, {"start": 3896.98, "end": 3922.45, "text": "So it goes to three of you, actually, all of you, yes. Well, I mean, it would partly depend on how it's doing that, right? So we'd need to understand why that happened. Presumably it wouldn't be a surprise to us. So I think there is a question of certainly until for the next foreseeable future we'll be publishing and releasing everything we build. As the systems become more integrated..."}, {"start": 3922.45, "end": 3935.62, "text": "There's sort of a large, big system, has many, many components. Maybe it's being used commercially. And the capabilities get more and more advanced. Then there are other considerations that have to come in, like"}, {"start": 3935.62, "end": 3965.23, "text": "Potentially people using, you know, maybe we're using those technologies in a constrained way, in an ethical way, but what happens if someone else uses it in a different way? So I think some of those issues will come in. I'm not sure if you're also getting at the question, like, would there be some rights of this, you know, a certain level of intelligence? I'm not sure if you're implying that as well, then would there be... Right, so in some sense, I wanted to ask, like, who is to decide? So let's say if the DeepMind builds something, would it be..."}, {"start": 3965.23, "end": 3994.45, "text": "Up to DeepMind to decide whether this one goes public or is going to stay here. If Microsoft came up with, let's say, Skype system that suddenly starts talking to you without thinking or without commenting, would it stay inside Microsoft? And will that decision be up to Microsoft, for instance? It's just not the way science works. You cannot have a secret that is, you know, when someone somewhere produces something"}, {"start": 3994.45, "end": 4023.65, "text": "Immediately there's 100 people who know that it's possible and they try to reproduce it and usually they reproduce it within a few months. So you cannot keep something like this secret for very long. Second of all, it's not like companies have a complete control over their employees. Employees circulate, they have interns, they have people who are interns are the pollinating bees of technology these days. So a technology like this cannot stay in a box for very long."}, {"start": 4023.65, "end": 4049.06, "text": "And conversely, it's not like this kind of event can happen by accident, or that anyone at Microsoft, DeepMind, or Facebook, or the rest of Google, can go underground for five years and work on something that's completely different from everything else in the world, and all of a sudden come up with something revolutionary. That just never, ever happens, except in Hollywood movies. Some guy in Alaska comes up with..."}, {"start": 4049.06, "end": 4076.78, "text": "They will build the robots and hardware and everything that takes over the world. That just doesn't happen in real life. If it turns out something like this pops up in DeepMind, it will pop up at Facebook within two months or something, or vice versa. If you do it first, they'll do it next, or one of these guys will do it. It's a community, and we all build each other's work, and I don't see how that could possibly happen."}, {"start": 4076.98, "end": 4102.48, "text": "Okay, so maybe we should... I don't really have much to add, except the Hollywood reference in Alaska, there's actually, I watched a movie on the plane on the way over called Ex Machina, and it's exactly that scenario, and it's a terrible, terrible movie. That's what I was referring to. It's actually very nicely filmed and entertaining, but the background... Artistically beautiful, but a very bad plot and a bad premise. Well, thank you. I'm more relieved now. Thanks."}, {"start": 4102.61, "end": 4141.23, "text": "Okay, so this will then end the panel discussion. I want to thank two groups of people. Let's first thank our panelists. And then I also wanted to thank Diederik Kingma, if you want to come up here, quickly show your face, and Kyung-Hyung Cho, who did awesome work in organizing this workshop, two-day workshop. It was a lot of work. So thank you. All right, and safe travels home."}]}