[0:00:01] Hello, I'm Jack with OpenAI and I believe some slides are going to materialize shortly. There they are. So I'm here to talk about a couple of issues which sort of relate actually to the previous talk. And one is dual use and how the lack of a sort of notion of dual use in the AI community as it stands today is exactly the sort of thing which will lead to the late stage regulation which we kind of want to avoid because it will have a bad effect. And I want to talk about measurement which is the thing that...
[0:00:30] We as a community are starting to do, but part of the reason I'm here is I think it's a thing that many people need to be involved in, which will allow us to do the sort of better thing, which is earlier regulation that's sort of governed and predicated on knowledge about where the technology is going. So, I mean, let's start with dual use, and let's talk about the kind of basic machine learning technologies we have today. So on the left there on that slide is a synthetic face generated from a paper in 2014, or an array of them.
[0:01:00] These are quite obviously not real. The ones on the right are also synthetic faces, which came out last week. This was a paper by NVIDIA. It's a good paper. I advise any of you who really like AI to go and read it because it has some interesting ideas in it. And it was released along with the code. Do I think this is dangerous? Not quite yet. But I think that it gives you a flavor of this rate of progression of this technology and the way in which change can come from sort of surprising areas.
[0:01:29] And as a community, currently, the default setting is to publish everything. And I don't think this is necessarily the right thing to do in the long run, nor do I think it's the thing which will lead to a sort of happy and relaxing world for all of us. I prefer to be relaxed. So I'll show you where we're currently thinking the areas of notions of dual use need to enter. And here you have examples of reinforcement learning. We've spoken about them earlier. Again, on the left, you have
[0:01:58] This is work by DeepMind in 2013 and 2014. They show that you can beat Atari games with reinforcement learning. In 2015, they have the first AlphaGo result. And then in 2017, we had this result on Dota, which is a partially observable game. Similarly, other people are working on games like StarCraft, and there's broad interest in taking actions in very, very complex environments.
[0:02:26] I'd sort of challenge you to think of what another two or three years of progress looks like in terms of these algorithms, and what happens to us if we don't develop a notion of dual use as a community, and if we were to willingly sort of publish everything. I'd note that for the latter two, AlphaGo and Dota, not all of the details have been published yet, and I doubt that they, in the fullness of time, will be, and that may provide a kind of template for how we think about dual use going forward.
[0:02:54] What I'd sort of challenge you to think about is how we should work out what technologies we need to think about as being dangerous to publish or not. And I want to motivate you by showing you a selection of papers which I picked up over the last week or so, week or two weeks, from just the public internet. You'll see papers on vehicle identification from aerial imagery. You'll see papers on pedestrian identification and re-identification.
[0:03:23] Many of these papers come along with techniques to implement them and code. They're published by a variety of actors, ranging from universities to government-linked research labs. I believe one of these is a US Army research paper, and also papers from other actors in other geographies. Now, the fact that this is happening now, I think, is broadly benign, but I don't think it's going to be benign for long. And so I've been thinking about possible solutions to this and ways in which we can avoid
[0:03:53] Two more years of progress leading to papers being published that could lead to credible harm happening, which would obviously lead to regulation, which would happen rapidly in regulation, which we may not ultimately want to occur. So the big idea I've got is that we, not just AI people, but the national security community and other parts of the government, need to indulge in measurement. And I'm going to show you why this will be extremely painful and difficult now.
[0:04:20] Here is a selection of graphs about AI where we have done measurement. This is not useful at all. And the reason why I've put all of this together for you is to show you that reasoning about AI is difficult. Measuring different aspects of it and different slices of it doesn't really give you the kind of holistic view that you'd need to create regulations. And the reason why this is the sort of best thing I can do now is that these are the sorts of measurements being conducted by...
[0:04:46] Individual communities of individual scientists, usually without any kind of dialogue with policymakers about what they'd like to see measured, nor with any kind of push from the government side on raising conversations about what they think should and shouldn't be sort of something that we consider. And I'd sort of ask you to think about what the coming generation of AI looks like and what we should do to measure it. You saw these reinforcement learning systems which can take actions.
[0:05:15] In about a year or two, it's going to be possible to train drones in simulators and explore transfer learning technologies and transfer them into the real world to do things. You saw previously a paper on reinforcement learning for quadcopter landing. That's clearly something that people in this community in particular would like to get a read on ahead of time. And it's not going to come solely from the scientific community that produces these papers. It's going to come from a big collaboration.
[0:05:43] And so what this collaboration might look like would be, in my mind, a wonderful community of not only scientists, but also people from across the agencies and government actually collaboratively measuring this. So that rather than perceiving the kind of individual slices of the subject area, you end up thinking about the actual whole. This won't be remotely easy. It's actually going to challenge a lot of our notions about
[0:06:10] How scientists should and shouldn't operate, you know, what we should and shouldn't publish, but also where government's place is to mandate what sort of should and shouldn't be being explored. Because the tendency is going to be towards being open. You don't want to kind of restrict the size or quantity of eggs that the kind of golden goose can lay. So obviously you don't want to do premature regulation to damage the sort of position of leadership the US has attained in AI.
[0:06:37] But you do definitely want to know the shape of the sorts of systems which are emerging. And I realize that this is kind of an elephant here. I think that the sorts of systems we're going to produce are going to be of such power and magnitude that we need to develop new ways to think about measuring their impact. It won't be just a single readout on the ability of a quadcopter to transfer to the real world. It'll be a readout on...
[0:07:03] It's ability to transfer to the real world and take orders and the amount of compute it costs to train it, the amount of power it costs to train it. We need to generate the necessary telemetry so that good regulations can be made because otherwise at some point something sort of negative will occur and then we'll make regulations in the absence of any true data. Another example I'd give you here is we've talked about automation a lot today and we've talked about the issues of how
[0:07:31] Automation may affect the workforce. It may not surprise some of you to know that if you look at the annual survey of manufacturers, which goes out to American manufacturers, we aren't asking them what level of automation is present in the robots you're buying for your factories. We aren't asking them the necessary information that we could do to give us the data we need as a community of people who want this to benefit America for a good outcome to occur.
[0:08:02] Thank you very much.
[0:08:23] More proactive measurement of exactly how AI can benefit things like medical diagnosis. I think there'd be less wariness from a part of people about it making its way into society because you'd learn of just how it can benefit you as well as how it poses certain unique challenges which we need to be aware of. And so in closing, what I'd like us to think about is sort of what do governments want to measure in this area? Without that dialogue, it's very difficult for us as a scientific community to know
[0:08:53] What our norms should be. We're going to try and define them among ourselves, but without more of a public conversation and dialogue, stuff will sort of invariably go wrong, which we obviously want to avoid. We also want to make sure that we can do this kind of top-level analysis of where progress isn't happening to guide eventual policy or eventual possible investments on the part of governments or the private sector into certain high-value research areas. We've talked a lot about China today.
[0:09:23] China does do aspects of this already. It does do reasonable amounts of measurements and some of its national plan was informed by what I describe as a kind of greater gut level intuition about AI's progress at the government level. And I think that for people in China, that's probably seen as a good thing. They have a more coherent strategy as a consequence. And I think that we can do the same thing and it actually doesn't take much. It just takes a few steps in this direction.
[0:09:52] In closing, we are going to think about dual use. Publication and the nature of open publication will change over time, but that has to happen in the form of a two-way conversation here so that we avoid surprise and so we can be proactive. Thank you very much.