[0:00:02] Welcome to the TwiML AI podcast. I'm your host, Sam Charrington. I am here at the OpenAI offices and I am with Dario Amadei. Dario is a team lead for the safety research team here at OpenAI. Dario, welcome to the show. Thanks for inviting me. Absolutely. It's great to have you on.
[0:00:40] So why don't we get started by having you tell us a little bit about your background and how you got interested in AI and AI safety in particular. Yeah, so actually my background was in computational neuroscience. I did a PhD in kind of biophysics and computational neuroscience. I've always been interested in AI and how intelligence works, but I felt like a lot of our AI systems 10 years ago weren't working that well. And so I decided I wanted to study the brain and then, of course, the deep learning.
[0:01:09] Revolution came around, and I looked at it, and I said, oh, these systems are actually starting to work. I want to be part of this. This now seems like the most interesting thing. So I ended up working in Andrew Ng's group at Baidu for about a year, and then I worked at Google for about a year, did kind of a variety of stuff, speech recognition, natural language processing. And then I came here, and the way I got into AI safety was one of the things I kind of noticed about neural nets is there's this mixture of they're very powerful.
[0:01:39] But they can be very opaque and unreliable. So when I was developing speech recognition systems, you train a system with an American accent, and then if it hasn't been trained in any British-accented speech, you give it some British-accented speech, and it gets kind of totally confused. So this mixture of power, opacity, and kind of very unpredictable failures and weakness is kind of what made me think that we need to be...
[0:02:04] Careful to make sure that these technologies do what we want them to do and don't do something unpredictable or even dangerous. Nice, nice. So maybe we can get started by having you give us a little overview of kind of the research that you've been working on, like the broad brushstroke overview of the research that you've been taking on. Yeah, sure.
[0:02:27] I would say there's kind of two general areas that I think about when I think about safety. One of them is what you might call robustness, which is the problem that you have when you train a machine learning system like a neural network on some problem like speech recognition or self-driving cars, and then you put it to work in some actual context.
[0:02:49] And, you know, the distribution of inputs that you're facing changes in some way. For instance, the change in the accent, you know, the change in the speech accents, the change in conditions of a self-driving car, or if you have a reinforcement learning agent exploring the environment, what it sees changes. So dealing with that. And we've done a little bit on that. We've done kind of more on sort of the second thing, which is what we call kind of like alignment with human goals.
[0:03:14] So, you know, I think a big concern, particularly as AI systems get more and more powerful, is making sure that, you know, the AI systems have a sophisticated high-level understanding of what humans want them to do. And that, you know, we can already see examples today where AI systems, you know, kind of, it's much easier to have a simple goal than it is to have a complex goal. And I can get into a little more detail of why that's true for today's AI systems.
[0:03:42] And that kind of really sets the stage for systems to kind of go off and kind of maniacally pursue very kind of pathological, simple goals. And I think we should instead have AI systems that are as good at understanding what humans want them to do as they are at accomplishing whatever tasks they've decided it's important to do. I think one of the best examples of that was from, I think it was some of the initial work that you did with Google on this.
[0:04:09] Where the example was you've got a robotic housekeeper and you tell it to clean the room. Like you don't want the robot sweeping all the dust under the rug. Yeah, I mean, it's kind of an example of what we call or this term from economics, actually, Goodhart's law, which is that once a metric becomes a target, it ceases to be a good metric. What that means is if you have a way of kind of passively measuring something, it seems good. But then if you optimize it really hard.
[0:04:35] You might get something that you don't expect. But isn't that all we're doing in AI is optimizing around a metric really hard? Yeah. And that's the problem. I'm not going to argue with that. So we actually have, since I came to OpenAI, we got kind of an even more vivid example of this that we got to occur in an actual AI system. So we were kind of playing with training simple video games. And we had this boat race game where you're trying to complete a course with a boat.
[0:05:04] And the only kind of easy way we have of measuring progress, again, the only simple way we have of measuring progress is it gets these points when it knocks down these targets along the course. So kind of naively looking at it, it's something like, well, it has to knock down these targets to complete the course. So, okay, we're going to give it train a reinforcement learning system, give it points for knocking down the targets. And so, great, it'll complete the course. And then I just set it in motion.
[0:05:32] Didn't do anything for 24 hours. And then when I came back and I looked at it, the thing was going around in circles because it found this lagoon where it could just get the maximum possible density of points. I remember seeing that example. Of course you can say, well, I don't know, you get what you ask for. The game was just broken. But I think the difficulty is that the mapping between what we think we're likely to get when we set the training process in motion and what we actually get, it's very discontinuous. It's not...
[0:05:59] I believe they currently don't have.
[0:06:24] So one of the first things that you did around this research was to publish a paper. Again, it's the same paper I'm referring to. I think it was in partnership with Google and another organization. I forget the name of. You essentially outline a set of rules. Do you think of them as rules or goals? Kind of general research areas. So just to make sure, there's kind of two major papers that we did in the last about year and a quarter. The first one.
[0:06:51] I did while I was still at Google, and it was done in collaboration with Google, OpenAI, Stanford, Berkeley. And this was kind of this agenda paper that kind of outlined various directions for AI safety. And then more recently about, I think it was, what, three months ago now, we published this paper called Learning from Human Preferences, which was kind of our research kind of attacking some subset of those problems. So it was kind of like that.
[0:07:18] Paper from a year ago was kind of the agenda doc that laid out. That's the one I'm referring to. We'll dig into the next one in a sec. But maybe you can take a second to kind of, if you remember, like run through those points in the agenda because they're almost like, you know, in some way, is it fair to think of them as like Asimov's new laws for the neural network age or something like that? A little bit, yeah. I mean, they're in a sense trying to solve the same problem, although it's less things a specific machine should do as kind of,
[0:07:47] Areas of research where, you know, to address particular types of problematic behavior that could arise. There were kind of five research areas that we talked about and they were kind of divided into two topics similar to this kind of, you know, robustness versus value alignment thing. So, you know, kind of the schema we used was, you know, you have a machine learning system. You wanted it to do something and something goes wrong and it does something other than what you intended it to do. Where did things go wrong? Things would go wrong because you had the wrong objective function and you optimized it really hard.
[0:08:17] Or things could go wrong because you had the right objective function, but something about the way you trained it went wrong. So that would be like, you know, the self-driving car that's put in a new environment or, you know, the kind of robot helicopter that, you know, is trying out behaviors and then destroys itself. And, you know, that kind of wasn't in the algorithm. So on the first one, the problems we talked about, the first one was reward hacking. So this is kind of the thing we demonstrated in the boat race where, you know, you have this measure.
[0:08:45] Optimize it really hard and you get the wrong thing. The second one in that area that we talked about was what we call negative side effects. So the idea is the world's really big. And basically it's a bit similar to reward hacking. Basically any simple objective that I can come up with probably refers to a very small set of things. So if I ask you to move this chair, I'm implicitly not referring to just every other thing under the sun that could be in the world.
[0:09:14] Right, without breaking that window. Yeah, but by default, I'm kind of like not specifying all this common sense stuff. So you might say side effects are really bad by default in AI systems. And so that was kind of the second problem in that category. So those two are kind of like broad reasons why I picked out an objective function, seemed innocent, seemed good, and something went very wrong with it.
[0:09:38] The third problem, which is kind of between the categories, is this thing we called scalable supervision, which we dealt with that some in the later paper, which is even if I know, if I were there for everything an AI system did, even if I could coach it and tell it to do the right thing, if I could supervise every decision that it made, make sure it never got out of my sight, never did anything, I don't really have the bandwidth to do that. How do I handle situations where I kind of know what I want the AI system to do?
[0:10:07] But I have very, very limited, it's not feasible for me to have more than limited interaction with it. How do I handle that situation? So those were the three on the kind of like, you know, how do I get the right objective function? And then I have the right objective function. How do I make sure something bad doesn't happen? The two problems were safe exploration. So safe exploration is the idea that, you know, I have some objective function like fly a helicopter. And even if that's right, even if I've set my system up in a way so that if I give it enough time,
[0:10:37] Yeah. You can get expensive if, you know, at the very least.
[0:11:05] Neural nets and kind of very powerful policies and new tasks have just come along in the last three or four years. And so the safe exploration literature is, you know, just kind of catching up to this. And so one of the things we were saying in the paper was, you know, there's an urgency to making sure that, you know, the safety, the work on safety issues catches up with the work in other areas that's kind of hurtling ahead. And we kind of already seen it. I mean, one example I give is, you know, at OpenAI, we had this tool called Universe we were using for a while that lets you basically,
[0:11:35] You know, have a reinforcement learning system that connects to the web and has, as its kind of actions and abilities, seeing the screen, moving the cursor, and clicking on anything. Reinforcement learning systems, when you initialize them, tend to explore randomly. So the first time I ever trained a reinforcement learning system on Universe, it immediately opened up Chrome, right-clicked, opened the Chrome developer tools kit, changed the code in there.
[0:12:05] Closed it, crashed Chrome, and caused some kind of segmentation fault on my computer. That's amazing. Just random behavior. So if your environment is complicated enough, you really, really have to think about these issues. I could say the same thing about Google's done some work on, I think it's well known now, optimizing data centers and using reinforcement learning to optimize data center energy usage. And of course, there are...
[0:12:32] There's some knobs you can turn there that would break the data centers. And so that is an issue that I think they've had to think about as well. Right. And then kind of the last issue was this distributional shift thing, which I've kind of alluded to a number of times and has to do with the environment in yourself driving car changes or you train your speech system on one accent and you change it to another accent. One situation in which it came up was the infamous Google gorilla incident.
[0:12:58] Google Photos tagged some African-American individuals as gorillas, and it was kind of a problem with the training set. The training set had been weighted towards Caucasian individuals, and so it got confused. It had no idea what racism was about or any of these things. So here failures have kind of already happened, and how can we be better about machine learning systems kind of knowing what they don't know? So that's kind of the overview of the problems. And so the second paper...
[0:13:27] Is one that's kind of diving into kind of the first few of these problems? Yeah, yeah, a couple of the first problems. So, you know, as with most things, we wrote this kind of grand agenda, and then we were like, oh, you know, we have so many ideas for how we could work on any of these. Which should we start on? So the thing we eventually settled on was, you know, the kind of the reward hacking and scalable supervision stuff, having the wrong objective function. And, you know, with that, what we wanted to think about was...
[0:13:54] Well, if you're trying to learn an objective function that's in line with what humans would want, then you should probably learn that objective function straight from humans, right? Because if you have some kind of hard to explain aesthetic thing that it's hard to encode into a system, instead of trying to encode it, you kind of let the human be the teacher, right? And in some sense, whenever we do supervised learning, we're doing that a little bit. A human has to label all the images to tell us, well, this is what a duck is, this is what an ostrich is.
[0:14:22] But it hasn't been done very much in the setting of reinforcement learning. And kind of that's the setting that, you know, I think has both the most promise and that we should worry the most about, right? Are you saying letting the human be the teacher necessarily by observation or by the human communicating a set of rules? By the human communicating. So I'll kind of explain a little bit how it works and how we set it up in our paper.
[0:14:52] Kind of an agent that's interacting in an intertwined way with an environment. And you have some notion of reward. So in the DeepMind system that played Go, it's did you win or did you lose? In Atari, it's the score. And because you're training the system by trial and error for many millions of iterations through, it's actually very important in the way we currently train the system to have a kind of programmatically...
[0:15:19] A valuable reward function, right? So with Go, if I had to have a human look at the end of every game and say, did you win or did you lose, it would be very hard, right? It's very important that I can just write a little script that says, yep, I have more territory than you, therefore I won. You have more territory than me, therefore you won. And so in general, these systems are trained without much human intervention because there isn't time for it. And what that means is that the goals have to be something you can write in a simple program. And so the way we changed this was, you know,
[0:15:48] What if you have a human every once in a while give some feedback on what the right goals are and whether the system is behaving in the way that it should behave? So the idea is, you know, with my reinforcement learning system, I take out the reward function and I replace it with like a model of what the right thing to do is that is trained from a human. So concretely, the way it works is, you know, I have my reinforcement learning system. It starts out by acting randomly as all reinforcement learning systems do. And then every once in a while,
[0:16:18] I take two clips of its behavior, just two randomly selected clips, give them to a human, and the human takes a look at them and says, okay, this is a little more like what I want than that. And the human has in mind some behavior that it wants the AI system to engage in. At the beginning, neither of the two video clips is going to be particularly good, but the human's like, yeah, this is a little more like what I want. Then the AI system has kind of a reward predictor that it's...
[0:16:44] It trains that models the human choices. It tries to come up with a reward that's consistent with what the human chooses. It goes off and optimizes that for a while. Then it presents the human with two more video clips. And the human says, this is better than that. Updates the system's model. And then the system goes off and plays around in the environment and tries to achieve that goal better. So instead of kind of careening off, optimizing a particular goal really hard, you have this interplay where the system optimizes a goal.
[0:17:13] So can I jump in with a couple of questions here?
[0:17:43] You know, data for the system to optimize around is the input it's getting from the human. Yep. Like, what's it doing between the times that it's getting input from the human? And like, does it work if you just compress that and take it out? Yeah. No, it actually doesn't. Because if you kind of think about, you know, so maybe good to have an example. So let's say you have a video game where you're kind of trying to shoot spaceships and keep them from shooting you. So there's two different parts to it, right?
[0:18:08] There's understanding that the goal of the game is to shoot the spaceships and not be shot by enemy spaceships. And there's the actual mechanical dexterity to find the spaceships, shoot them, and avoid them. So we can still learn how to do that stuff. Yeah. So basically what the system needs to do is it needs to figure out from you what its goal should be, namely shooting the spaceships. And as it figures out that goal, then it needs to, on its own, practice in the environment in order to learn how to achieve that goal.
[0:18:37] In practice, it's a little bit more of an interplay. First, it understands that it should be avoiding these shots that are coming down. And then it kind of goes off and optimizes that. And then it gives you some trajectories where it avoids the shots but doesn't actually score any points. And you're like, oh, this isn't any good. And then every once in a while, it accidentally scores a point. And you're like, yeah, yeah, you should do more of that. Then it kind of figures out, oh, I need to not just avoid the shots. I need to actually shoot the enemies.
[0:19:03] And then it takes some additional time to figure out how to do that. So it's an interplay between the system learning what you want and the system learning how to achieve what you want. And that's why there's that interplay. So you only ever see about, in our paper, it was about 0.1%.
[0:19:20] We trained it on some Atari games and the human only had to see about 0.1% of what the actual agent saw, which is good because the agent sees, you know.
[0:19:39] Days of experience or so, and we don't have time to have humans see all of that. Okay. What is the human evaluating? In a simple game like the spaceship game you described, I'm imagining what is the human evaluating the two frames on? Is it just looking at the score? Is it a proxy for some way to detect the score? So we blank out the score because it's a little bit of a confounder.
[0:20:02] But we just say to a human, we're kind of in some ways trying to develop a pipeline such that you could take a task and actually farm it out to humans who understand the task. So we blank out the score and we gave it to some contractors and we said, this is a game where you're trying to shoot enemy spaceships and avoid getting shot by enemy spaceships. You're going to see two video clips.
[0:20:24] Click on the video clip that you think does a better job of that particular goal. So they're video, they're not still. Yeah, they're video clips that are a couple seconds long. And so, you know, for many tasks, we're able to get a good sense of that. We were also able to do tasks where, you know, there's no... So for Atari, you know, there is a score that you could learn from. And our point was that you don't need it. You could learn without it. There are also tasks, particularly kind of, you know...
[0:20:50] Tasks, you know, like that simulated robot animals do where, you know, you want them to perform some trick that it's really hard to describe mathematically, but that a human can recognize. So we taught kind of like little simulated robot walkers to do kind of back flips and front flips. You know, we taught like walkers with two legs to kind of like balance on their hind legs or, you know, do ballerina moves or things like that. And so, you know, the exciting thing is usually if you want to train something with reinforcement learning, you have to say, okay, what's the behavior I want?
[0:21:20] How do I write a mathematical function that assesses whether that behavior was right or not? Whereas here it's just, okay, what's the behavior I want? Let me look at some video clips and try and reinforce that behavior, which I think requires more human labor. But particularly if we farm it out, it allows you to do a much wider variety of tasks. Have you looked at – it sounds like you're pulling two random clips. Have you explored like –
[0:21:47] Selecting the clips based on maximal distance or information or something? Yep, yep. No, that's a really great question. So we actually did include that in the paper, and it helped a little. So we actually, instead of having kind of a single predictor of the human preferences from the data, one of the things we tried was having an ensemble of three predictors that are trained on subsets of the data. So this is kind of the common statistical validation technique. And then you look at cases where different predictors...
[0:22:15] Disagree with each other about which clip they think the human would think was better. And that disagreement is a good proxy for, oh, this is a hard case. This is a hard case to figure out. So then we kind of mine for hard cases and preferentially present the human with hard cases instead of easy cases. So in the spaceship example, it would be like, let's say the agent has really figured out that if you get shot by the ship, it's really bad.
[0:22:41] But then there are some intermediate cases where the ship's laser is shooting at you and it almost hits you versus the one where it actually does hit you. And maybe your predictor hasn't quite figured out the difference between those then. And so you really want to show those to the humans so the human can disambiguate.
[0:22:57] And we found that that indeed did speed things up, you know, on more tasks than it slowed things down. It wasn't a huge improvement. And we're actually looking for kind of more intelligent ways of doing this because, you know, I mean, that's really getting into having systems be active and ask us questions about the things they're confused about instead of receiving passive information about what we think they're confused about. So, you know, I think in the future that's going to be a big part of it that, you know, we want.
[0:23:24] The other interesting thing that jumped out at me in you describing that last example is the ambiguity from the human's perspective. If the laser just misses the ship, is that bad because the ship was too close to the laser? Or that's awesome, the ship was able to avoid the laser. It was really close.
[0:23:54] Have you characterized that part of the problem at all? Yeah, I mean, I think there were some kind of technical issues we ran into, which is, you know, how do you define goal of the task, right? So another example, like the example you gave, is, you know, what if your spaceship, just in a particular few-second clip, doesn't encounter any enemy spaceships at all?
[0:24:13] So in other words, it doesn't see anything. There's nothing it has to do. Is that good behavior because it did what it was supposed to do, which is nothing? Or is it bad behavior because it's worse than a clip where it actually shot the spaceship? In the formal reward formalism, you're basically supposed to say that that's worse behavior because it didn't get the reward as opposed to the behavior where there was a ship and did get the reward. But of course, if I'm a human thinking about it, it's kind of ambiguous.
[0:24:42] We kind of tried to give the instructions in, you know, in such a way as to kind of, you know, resolve those ambiguities. You know, we wanted to focus on, you know, can we incentivize this behavior with clear one or two sentence instructions? But I think, you know, and the technical description of it is, you know, there's a difference between the reward, which is the immediate reward you get, the return, the advantage, which is how good a particular action is. So, you know, which of these technical concepts in reinforcement learning do we actually want?
[0:25:11] And there's a question of how easy is it for humans to understand these different settings versus how much does it help the algorithm? So these are all kind of things that have been explored a little bit in the literature, but again, never with the level of difficulty of tasks. So this is kind of one of the areas that we want to explore more and we consider really unexplored. This was really just a first paper, and all these are really great questions that we kind of ran into and thought about some, but you definitely haven't fully answered.
[0:25:41] And so is the plan to, like you kind of peeled off two of those, is the plan to go even deeper on those two or to peel off another one? Yeah, so, you know, I think this direction we're pretty excited about. And so I think we're going to do a lot of things, you know, that kind of follow up on this human preference learning. You know, we see applications to robotics tasks like, you know, tying particular types of knots in rope. You know, it's another task where the reward function is very hard to...
[0:26:08] Describe and people try and specify it. But I think that that may be the weak link. So, you know, maybe we can do things like that. Things that physical robots do in the world. Things like dialogue system. You know, Microsoft made this dialogue system called Tay that ended up kind of spewing racist nonsense. You know, whether a comment is offensive or not is precisely the kind of high level aesthetic concept that you can't learn really well with a simple objective function.
[0:26:35] Could you train a restricted dialogue system to understand what racist or sexist or offensive comments are and never make them? Some of that is simple. It's just not using certain words. But the concept of something being offensive is also more subtle. I can say something that doesn't use any offensive words, but the content of what I'm saying might still be offensive. So can we train a system to understand those distinctions?
[0:27:01] Actually, even things like safe exploration, what it means for exploration to be safe, maybe that's something that we could learn through human preference learning. Give me an example of that. So there's been a lot of work on kind of learning to learn. And so you could imagine kind of learning to learn safely. So if I'm a human and I am learning to play an Atari game, right? You could think of copying what the human does, but you could also think of copying the human's process of exploring the Atari game, right?
[0:27:31] And there are different ways to explore. I can explore recklessly, doing a lot of things with trial and error. Or I can explore very carefully, making sure that nothing bad ever happens. For instance, if I'm a human remote controlling a helicopter, I'm not going to just try random things. I'm going to kind of gingerly try a few things at first that don't make the helicopter crash, particularly if I think that it'll break if it crashes. And then I'll kind of gradually get more bold. And so can we train machine learning systems?
[0:28:00] to learn in the same safe way that humans learn? And could we do that by giving them feedback or examples or demonstrations of how humans learn? And then could we solve that problem? That way, we've also thought about kind of richer forms of human feedback. So right now, it's just like, is left better or right is better? Sometimes, when I'm training one of those systems, I'm just like, I want to tell you that that thing you did is really, really good.
[0:28:28] And all I can do is just give that one left click. But, like, really, really, this was the great thing you did. When you were giving the example of the models doing backflips and things like that, I was thinking, like, you need the Olympic panel with the 10s and the 9s and the 8.5s and all that. So having a scalar dial is one thing. Ultimately, I'd just like to be able to give linguistic feedback. I'd like to be saying on this one, nice job, do that again. Or, nice job, but go a little to the right. You know, nice job, but that's only the first part of the move. After you do this, you need to do a flip.
[0:28:57] And currently our natural language processing isn't really up to that task, but that's kind of a long-term goal we'd work towards. And the real long-term vision would be you have an AI system that is persistent in the world, is doing things on your behalf, and then you make sure that it always does the right things instead of the wrong thing by this continuous...
[0:29:19] And we've been talking mostly about explicit feedback. Are you also thinking about the explicit versus implicit feedback might play into this? What do you mean by implicit feedback? So, for example, maybe the Tay example. In all of the examples we've been talking about,
[0:29:47] You are telling the system, this is good, this is bad. And we started with kind of this binary good, bad, and we talked about scalar, continuous degrees of goodness and badness, but it's still like you're telling it explicitly. And I guess I'm wondering if there are ways to either...
[0:30:09] You know, maybe I'm thinking also of like some of the emotional intelligence, like kind of pick up from your reaction. Yeah, someone reacts in a way where they don't like what I'm doing, but, you know, they're not going to explicitly say, you know, that was a bad thing to do. And I guess at some point, you know, it's all numbers, you know, feeding into some model somewhere. Yeah. So, I mean, I think the natural language feedback is a little bit getting it. It's kind of starting to get at that, right? Where, you know, I mean, if, you know, people have kind of, you know.
[0:30:35] People have ways of politely disagreeing or politely giving negative feedback, and those are patterns that a machine learning system could, in principle, pick up on. At the same time, I think in the end, if we ever want something like that to happen, we're going to have to go kind of beyond just receiving feedback from humans to actually having models of humans. So a human says something, and the AI system says, okay, here's my picture of the human.
[0:31:01] And here are my hypotheses about why the human would say that particular thing. And one of them is that it doesn't want me to do this thing. It wants me to do something else. And so that gets to modeling of other minds. Theory of mind is something we talk about a lot in neuroscience. So this is one of these things that I think AI systems will be able to get AI systems to do that.
[0:31:22] Once they have some kind of rudimentary theory of mind. But even you saying that kind of makes you think of all the complexities, like the model trend on the Brit could not be used to interact with an American, right? Yep. But on the other hand, simple animals have theory of mind, right? If you have pets, dogs can do this, even mice can do this. They have very complicated pictures of how the humans around them work. They need to in order to survive. They need to know when humans are going to feed them. They need to know when humans are going to be...
[0:31:51] Yeah. Oh, cool, cool. Anything else that you guys are working on that you wanted to talk about?
[0:32:21] Kind of different directions on safety, but they're all pretty early. So there's not so much to say yet. But, you know, I'm hoping that, you know, in a few months we'll kind of have another batch of results that are kind of follow-ups on the human feedback stuff and also kind of new directions. Nice, nice. There's a website that's like 10,000 hours or something like that that like looks at, you know, these, I think looks pretty broadly at what they think are like.
[0:32:47] Oh, this is 80,000 hours. I actually did a podcast with them. So I underestimated it significantly. Because 80,000 hours is the length of a career, I think. I think that's why they chose that number. And their top career or their top opportunity is AI safety. Yeah. So, I mean, I don't have any opinions on... I would never claim that what I'm doing is the most impactful thing you could possibly do.
[0:33:15] I did a podcast with them. I would say instead that the world has a lot of problems and a lot of issues, and unfortunately, multiple incredibly serious issues. But one thing that I think is true about AI safety is that if I look ahead to the next two decades or so, I believe it could be a really serious issue, and I believe that not that many people...
[0:33:40] are thinking about it seriously in the sense of doing actual technical work on it. I think for a long time it was dismissed maybe with some justification as kind of a crackpotty thing. I think that is starting to change, and so I actually see opportunity there in that there's this important problem that hasn't really been thought about in a careful enough way yet. And so to me that creates an opportunity to work on...
[0:34:05] A problem that's very important to the world that there aren't a giant number of smart people already kind of descending on it. And because there's been kind of less thought about it, maybe by acting early, we can actually come up with real solutions. So to me, that's kind of my rationale for feeling that it's a high impact thing for me to work on. That it seems important and it seems like there aren't already 100,000 people working on it. Awesome. Awesome. Well, Dario, thank you very much.
[0:34:34] Great talking on the show. Thanks again for inviting me. All right, everyone. That's our show for today. Thanks so much for listening and for your continued feedback and support. For more information on Dario or any of the topics covered in this episode, head on over to twiml.ai.com slash talk slash 75. To follow along with our OpenAI series, visit twiml.ai.com slash OpenAI. Of course.
[0:35:04] You can send along your feedback or questions via Twitter to at Twiml.ai or at Sam Charrington or leave a comment right on the show notes page. Thanks once again to NVIDIA for their support of this series. To learn more about what they're doing at NIPS, visit Twiml.ai.com slash NVIDIA. And of course, thanks once again to you for listening and catch you next time.