{"url": "https://www.youtube.com/watch?v=hOWYHHLU1Hs", "page_url": "https://www.youtube.com/watch?v=hOWYHHLU1Hs", "title": "Hope vs. Hype", "show_or_channel": "New York Times Events", "date": "2018-02-20", "duration_s": 1119, "type": "panel", "speakers": ["Greg Brockman"], "confidence": "medium", "notes": "NYT New Work Summit 2018 \"Hope vs. Hype\" session.", "people": ["greg-brockman"], "id": "2018-02-20_new-york-times-events-hope-vs-hype", "audio_duration_s": 1118.7141875, "worker": "jc-w13", "transcribe_s": 27.4, "model": "large-v3", "segments": [{"start": 0.72, "end": 29.12, "text": "Okay, I've got two people who work for organizations that basically deal with the concerns we have and many have around AI, so we're going to talk about that very thing. I thought a good way to start is a concrete example. My colleague Steve Lohr wrote a great piece just over the past week about bias in AI when it comes to identifying faces in photos."}, {"start": 29.12, "end": 55.04, "text": "So I was hoping, Tara, maybe you could explain what the issue is here and why this is so problematic. Sure. Yeah, thanks, Cade. I think the potentiality of AI to broaden social inequality has pretty widely been acknowledged. And the question of bias, I think, is a really complicated one. It has a lot to do with several different aspects of artificial intelligence, including the..."}, {"start": 55.04, "end": 81.28, "text": "The population of developers that actually make the technology, I think certainly the technology itself, the platforms that are being developed, the algorithms that are being used is another aspect that contributes to bias. And I think a third aspect is the potentiality of data that is biased itself on which we train these systems or which we use in these systems. So I think broadly, we've seen"}, {"start": 81.28, "end": 111.28, "text": "We've seen a lot of public conversation happening around applications where artificial intelligence directly implicates human life, whether in criminal justice applications or in health care, so on and so forth. So I think a lot of public discourse now is surrounding some of those different circumstances where bias comes into play along any of those three vectors or more that I described. And I think that it's generally generating a lot of attention right now. It was mentioned earlier in the day that one of the perceived issues, at least, is that"}, {"start": 111.28, "end": 125.97, "text": "And this is true. So many AI researchers are white men. Does the diversity of the researchers play into this? And is it more so the case here than, say, other types of coding?"}, {"start": 125.97, "end": 148.78, "text": "Yeah, that's a great point. I think diversity and inclusion is a huge issue for this industry. I mean, in computer science writ large, it's a problem. Certainly in the AI sort of subfield of computer science, I think it's even worse a problem. You know, the statistics glaringly show this. I think 11% of all CS engineers in most tech companies today are female."}, {"start": 148.78, "end": 175.36, "text": "The stats on racial diversity are even worse, actually. So I think 11% of the tech industry generally is African-American and Hispanics combined, those two demographic groups. And that's to their 28%, 27% representation in the population generally. So it's pretty dismal. And I think that is a huge contributing problem to the bias issue, whether because engineering teams aren't cultured to think."}, {"start": 175.36, "end": 202.64, "text": "In the diverse ways that they might need to in order to create platforms that serve a broad swath of diverse populations or just because of the problem of the lack of diverse thought in general. When you're building anything, you need to have representative teams working on these things. Another concern, which I would like you to explain, Greg, is this idea of an adversarial example. Explain what that is and why that's such a concern today."}, {"start": 202.64, "end": 229.92, "text": "Well, so the state of machine learning today with respect to security is similar to what computer security was in the early 90s, where we kind of built these systems, they seem to work, but if you have a smart adversary come and try to attack them, that there's all sorts of attack vectors. And so one thing that researchers have found is that it's actually pretty easy to craft an image that fools an image classifier into thinking..."}, {"start": 229.92, "end": 257.41, "text": "That it's whatever the attacker wants them to believe. And so people have taken this from kind of just the pure conceptual phases to actually going and printing out these patches that you could attach to real-world objects. And so, for example, imagine if you could take a patch, put it on a stop sign, and make your self-driving car think that it's a green light. That that kind of technology exists right now, and that is what an adversarial example is. It's worth underlining that, right? This is not theoretical. The research has shown that this is true, right? And, of course, we've got..."}, {"start": 257.41, "end": 283.01, "text": "Driverless cars using these types of systems that identify images in that way on the roads. So this is not like a concern in the future. This is a concern today. Is that fair? That's definitely true. Absolutely true. So given these two things we talk about, and this is just two of many things, some of which we've talked about over the course of the day, these problems, these flaws in the way AI operates, why at the same time..."}, {"start": 283.01, "end": 311.23, "text": "Over the course of this same day, are we so concerned about AI being so proficient that it steals all our jobs? Let's start with you, Tara. That seems like an irony there that needs to be addressed. Yeah, absolutely. I don't want to be redundant to previous sessions. Unfortunately, I wasn't able to make this morning. But I think it's worthwhile recognizing that uncertainty plays a huge role in a lot of the discussion happening right now on the AI-based automation and its impacts on the labor market question."}, {"start": 311.23, "end": 338.72, "text": "And the future of work question in general. And I think most statistics, a lot of studies that have been run recently, which a lot of people point to, have a very wide range of potential future outcomes. And I think that that range lends people to be concerned about the future state that we might find ourselves in, obviously. So I think between a recent McKinsey study ran determined that between"}, {"start": 338.72, "end": 359.09, "text": "Zero and 30% of all jobs will be automated by 2030. And further, there's some really interesting work being done around the different types of shifts and movements that will happen in the labor market between different types of jobs instead of just the binary question of whether people will have work or whether they won't. There's a lot of uncertainty around the question of..."}, {"start": 359.09, "end": 382.35, "text": "Whether there will be work that we just won't be able to fill because we won't have people properly trained or capable of accessing that work. So I think these are all questions that implicate companies at the ground floor level. I think they implicate macroeconomic policy structures and geopolitical interests. So I think there are some very good reasons for the discussions of the sort that have happened over the course of today."}, {"start": 382.42, "end": 411.33, "text": "And what I want your opinion on, Greg, because you're on the front lines here. One of the things I think about, and we haven't discussed as much here today at the conference, is these AI models are progressing so quickly and have over the past five years because we have the computing power needed to train them, to analyze the data. But that computing power continues to increase. We're on the cusp, actually, of a serious increase. And that may be one of the reasons for greater concern."}, {"start": 411.33, "end": 426.9, "text": "That's right. Yeah, and I think the thing about exponentials is that at any given point in the exponential, it looks like a pretty linear curve. And if you just extrapolate it out, things don't seem that different from what they are today. And the thing about exponentials is that if you just..."}, {"start": 426.9, "end": 450.64, "text": "Keep going forward, then suddenly everything looks very, very different, a lot faster than you expect. And so when it comes to the hardware, so we spend a lot of time talking to different hardware manufacturers and tracking all the different hardware startups. And the thing that we see, you look at next generation hardware like Google's TPU 2.0. These are chips specifically built to train these networks, right? That's right. That's right."}, {"start": 450.64, "end": 469.9, "text": "So you look at chips like that, like Google TPU 2.0. You look at chips like Graphcore that is kind of a startup that's still in stealth and a number of others. And you're seeing this increase of 10x each year for the next five years in terms of the total amount of compute that we have available to run our neural net experiments."}, {"start": 469.9, "end": 498.67, "text": "And so that means 10 to the fifth compute is a lot of compute. That's basically 25 years worth of Moore's Law of Progress. What was 25 years ago? We're talking like 1993. You take a computer from then versus a computer from today. Complete stark contrast. And that's what we're going to have in the future. And so the question of what is that going to be capable of? What are we going to be able to do with it? Well, we need that computer in order to know for certain. And so the future is kind of covered in this mist. And we can kind of..."}, {"start": 498.67, "end": 526.0, "text": "Peer through the mist today by running things at much larger scale than people typically do, by really trying to get a sense of how our different algorithms scale, by having a list of ideas that we want to try and try them out and improve them out at the smaller scale, so that as the faster computers come online, that we can test them out. So the one thing that you can say for certain is that we're just going to be in a very qualitatively different regime with respect to these algorithms, and that things can transition very quickly from not working at all to working surprisingly well."}, {"start": 526.0, "end": 551.17, "text": "I think it's a key point that needs to be made. But I still wonder if we're overblowing the possibilities here, especially when we start talking about superintelligence, the idea that these systems are going to somehow spin out of control and do real harm in various ways. Is it worth thinking about this? Do you think about this as the partnership for AI, Tara? Or is it too early?"}, {"start": 551.17, "end": 575.44, "text": "Yeah, I mean, certainly. So the partnership on AI, just for those in the audience who are not familiar, it's a nonprofit organization that was founded last year by some of the largest tech companies on the market, Google, Apple, Amazon, Facebook, IBM, Microsoft all sit on our board, so does OpenAI. We have an equal representation of nonprofit and for-profit board directors and are supported by the ACLU and the MacArthur Foundation as well as some other organizations."}, {"start": 575.44, "end": 592.58, "text": "We have 54 members right now, so we're a membership model community. They range in representation from civil society and nonprofit organizations to academic institutions, for-profit tech companies, civil society organizations, et cetera."}, {"start": 592.58, "end": 621.94, "text": "We're basically trying to eat the whole zoo. I don't even want to say elephant because I don't think it's an exaggeration to say that we're really trying. We are trying to grapple as a multi-stakeholder community with all these diverse interests and voices as a part of the conversation with a lot of the toughest issues at hand in the space right now. And this is one of them. And it's interesting to acknowledge also that there's a tension, I think, in the field right now, which a lot of people in the research community and otherwise are grappling with regard to."}, {"start": 621.94, "end": 643.86, "text": "This notion of an existential risk category of fears and then sort of everything else which has been bucketed, I think, fairly binarily into the category of near-term issues to deal with, which include bias and inclusion, diversity, fairness, accountability, and transparency, explainability of algorithms, a lot of the labor market issues that we've talked about, and so on and so forth."}, {"start": 643.86, "end": 665.71, "text": "And I think our interest is in bridging the gap between those communities on the one hand and also in making sure that we are addressing the full spread of potential concerns and futures that we might face together and to ensure that equal voices are given to those in a community of constituents that might not otherwise be brought together at the same table together."}, {"start": 665.71, "end": 693.42, "text": "All right, Greg, but you're the one who's really got to explain this. Elon Musk, who created your lab, he's by a sizable margin the leading voice raising concerns over the future of these ideas and the harm that they could bring to us. Meanwhile, he goes out and builds his own AI lab in addition to building Tesla and so many other companies that are using AI. You could argue he's pushing AI forward."}, {"start": 693.42, "end": 723.36, "text": "And the lab is a prime example of that. How do you explain that irony and that even your lab was founded with the idea of protecting us from the very systems it is building? So what OpenAI is really about is about ensuring that the post-AGI future, so that if you actually are able to build a machine. Explain to them what that means. Yep. And if you're actually able to build a machine that's as smart as a human on basically all the axes that you care about,"}, {"start": 723.36, "end": 753.31, "text": "If you can actually build a machine like that, making sure that the world afterwards is good for us, for us all, non-digital life forms, good old classic life. And it's not a guaranteed thing, right? If you're actually able to build something like that, you can kind of imagine all sorts of sci-fi dystopias, and you can imagine all sorts of crazy sci-fi utopias. And that OpenAI is really about trying to steer both the short-term and also the long-term."}, {"start": 753.31, "end": 779.49, "text": "Towards making sure that when that event actually happens, on whatever timeframe it occurs, that there's a good outcome for humanity. But is it worth talking about at this point? I mean, what concrete evidence do we have now? I mean, you're at the heart of this research. What concrete evidence do we have now that this could spin out of control? That worrying about robots not allowing us to press their off button."}, {"start": 779.49, "end": 809.18, "text": "Is there anything that indicates that we should be concerned about that today? Yeah, and so this is, I think, kind of a good thing to build off the previous point of how the way that AI works is it's a very different technology qualitatively, where you start with these very toy systems, and you start out with these things that are just totally in simulation, and then suddenly they start working in a much more general and broad way than you thought. You apply more computation power, you scale them up, you apply them in a bunch of different domains, and the point of these algorithms is very, very general."}, {"start": 809.18, "end": 820.62, "text": "And so the things that we see today, the concrete things that we see today, they're kind of funny to look at. So for example, we train a simulated robot to push this puck towards a target on a table."}, {"start": 820.62, "end": 849.87, "text": "And it learned that it could just move the table to get the puck under the target rather than actually learn how to play this game. And you look at it, and it's like, okay, it's a funny little robot. What does it really know? But then you think if this is a real control system that has control over a power grid and that it's starting to do these crazy unintended behaviors like that, is that the kind of thing that we're going to be happy with? You can also watch things like Black Mirror, and you can really think about how hard is it to go and build systems like that, even just..."}, {"start": 849.87, "end": 879.49, "text": "Take today's technology, extrapolate forward. And so I think that what we're really seeing is that we're seeing the leading edges of, OK, adversarial examples show you that we can build these powerful AI systems. You can almost build a self-driving car today. And it's going to be so subvertible. If you just deploy technologies without any additional fixes, which, of course, people are working on, then it's going to be totally at the whim of anyone who's being malicious. And then let's say that we take systems that are 10x more powerful, 100x more powerful."}, {"start": 879.49, "end": 908.62, "text": "And they can still be subverted in the same ways. That's going to be bad. And let's say that you just design them poorly. Today, again, we see these AIs that discover these very surprising behaviors. Another example is we train a lot of AIs in video games because it's a great test bed, that a lot of humans have gone and put in a lot of effort to make sure that there's these cognitive skills that you have to learn and that you can run these things at very large scale. And we often find that these AIs find bugs in the games."}, {"start": 908.72, "end": 937.26, "text": "I'm not sure that I really want to be solving these sorts of problems in the real world for the first time when these systems are deployed. And so another way of thinking about it is it's really a question of risk allocation. What kind of risk portfolio do we want for humanity? And if we knew, let's even say a very pessimistic version of, let's say that there's a 5% chance that an asteroid's going to hit the Earth and destroy us all in 200 years."}, {"start": 937.26, "end": 975.73, "text": "Would we allocate 0% of our budget towards trying to avert that disaster? It's hard to argue with that argument. Any questions? And the fact that, if I remember well, there is some people working towards a ban on Killer Roberts with a previous Nobel Prize. She's really working very hard on that. And up to now, we haven't talked about that."}, {"start": 975.73, "end": 1001.89, "text": "But it exists. Yep. So there's a lot of people talking about kind of lethal autonomous weapon systems and the ill effects that they can have. I think it's a really important debate. I think these issues are always much more nuanced than just simply this is something that we need to, you know, it's a nuanced conversation. And so I think that the..."}, {"start": 1001.89, "end": 1030.93, "text": "The important thing is to really engage with these issues and to have open conversations. And I think that I'm actually pretty encouraged to see the kind of activity that's been going on in that space. Yeah, I would agree. I think that there has been a huge uptake in interest, I think, in the global governance community on some of these topics, and especially those concerned with lethal autonomous weapon systems and appropriate use scenarios. And I think that the more we discuss these things in a global forum, the better off we'll all be."}, {"start": 1031.41, "end": 1062.22, "text": "We have one more. Hi. Last year, I think the government announced that every autonomous vehicle should have a black box in it. What are you guys doing from the perspective of validating the control systems in autonomous cars? So if there's an accident, for example, how are you going to control the biases in that scenario without all the different vendors revealing how they're programming the algorithm? Is that under your..."}, {"start": 1062.22, "end": 1071.65, "text": "Kind of remit or not? So the partnership identifies safety critical AI as one of the six principle."}, {"start": 1071.65, "end": 1100.54, "text": "Areas of interest for the organization. To be honest, I've only been executive director for about three months, so we're still growing out our program's portfolio. But certainly I think something of that ilk could fall underneath a category of work that we would end up taking up, working in collaboration with our partner community. So I don't want that to be a pun, but that's the honest answer I have to give. Yeah, so we don't work on self-driving cars, but I think that the general issue of..."}, {"start": 1100.54, "end": 1118.71, "text": "You have systems that are deployed in the world. Who's at fault? How do you make sure that we're able to actually go and diagnose failures? I think that these are all really important issues, and I'm excited to see what partnership in AI, and we're happy to contribute to that conversation as we can. Greg and Tara, thanks for coming."}]}