[0:00:03] Thank you for joining us. I'm Zannie Minton-Beddoes. I'm the editor of The Economist, and I'm thrilled that so many of you are here in this, I must say, very, very nice surroundings, very Swiss. We are going to do something. We are going to find clarity amid chaos, as it says up there.
[0:00:22] I don't know how long you have all been in Davos, but I think finding clarity amidst what's going on in the world right now is a torn order, but it is what we try and do at The Economist. We try and join the dots and we try and find clarity. And we are lucky in the next 90 minutes to be able to have the chance to hear from some of the smartest thinkers in the world on some of the most important issues, but particularly in this first conversation, to Daria Amodei, who I...
[0:00:50] I'm really not exaggerating when I say that you are one of a very small handful of people who is actually going to shape the world that we're all going to be living in. Dario, as you know, CEO and founder of Anthropic, one of the, maybe one of the very small number of foundational models that will determine when we get to full AGI, super intelligence, how we get there, what it looks like. So the future
[0:01:19] that we are all going to live in is disproportionately going to be shaped by this man. So that's why he's here. That's why I'm really excited to have this conversation. And I wanted to start, Dario, but before I do that, I must thank Zurich, of course, for making this possible. You have fantastic surroundings. Thank you. Thank you for having us. We're delighted to be here. But Dario, two things have happened in the last few weeks and months that I think collectively
[0:01:47] might have a bigger impact on the development of this technology, even than ChatGPT. And remember that moment two years ago when everyone thought ChatGPT, wow. And I'm not sure that this is really sunk in, but one of them is the development of what are called reasoning models, these new front sort of top-end models by some of your competitors, not yet by you, but by some of your competitors. And the other...
[0:02:15] It's kind of seemingly from nowhere, the development of really good, cheap models from China. DeepSeek, if you haven't heard of it, is an incredibly powerful model that has kind of come from nowhere. And I think these two, my thought is that these two events together are going to take this whole technology in a slightly different direction than we might have thought six months ago. So I want to start by exploring those. And firstly, let's start with the US cutting edge models.
[0:02:44] How big a deal is it that we now have these reasoning models which learn by thinking? They're kind of like what you tell your kids, spend a little time on it and you'll get a better answer. Isn't that sort of what they're doing? And what can they do? And what will we see in the next year or so from these models? Yes. So first of all, thanks for having me and very kind words. So on the reasoning models in particular, we have a little bit of a different perspective on it, which is
[0:03:12] For the longest time, most of the training that was done in models was what was called pre-training. So this was you train a model on a bunch of data on the internet. And that plus a tiny bit of reinforcement learning or reinforcement learning from human feedback is what language models were until very recently.
[0:03:30] There was another paradigm, if we think back to AlphaGo in 2016, done by DeepMind, I believe Demis Hassabis is here as well, was one of the pioneers in that area. So this other paradigm of train via reinforcement learning, learn via trial and error, has also been developed and has been around for a long time. What has happened is that we have started to marry these two paradigms together.
[0:03:59] Where you do the pre-training and then you do large scale RL. And our perspective is that there aren't separate reasoning models and normal models. That's an artifact of the early RL heavy models being trained on a narrow set of tasks. What is going to happen is that first we pre-train the model and it learns about the world in general.
[0:04:23] And then we do large-scale reinforcement learning, where the model does very large-scale tasks. It learns to do coding tasks. It learns to do math. It learns to solve scientific problems. It learns to solve puzzles. It even learns to do things like tell comedy routines. At some point, we'll get to that. And so we are thinking about, over the next year or two, how to develop models to do that task.
[0:04:51] Much better than has been possible before by models. We have seen with some of the early reasoning models, and we will see similar things from our models, starting to get to professional level performance at things like math and coding. And that's the striking thing, right? That they are now getting professional level performance is a nice way of putting it. These models are getting as good as humans, better than humans across a very wide range of thought tasks.
[0:05:18] So that's the excitement about it. But the interesting thing, I think, is also how this changes the economics of these models. Because, and I'm going to, forgive me, I'm going to really simplify, but the old style models, you spent a ton of money training them. Ton of compute, ton of capital to train them. But then once they were trained, it was pretty cheap to use them. Now, because they're thinking as they're doing, you're going to use compute while they're actually working. That means they're more expensive to run. What does that mean for you? Because it's going to cost you more.
[0:05:47] For your models to be used. Yeah, so I actually, on the cost side, I actually don't think of it as different. I think of it as you're buying compute time on a cluster, and what you're buying is thinking time in a mind. So in all previous models, it was just that the mind wouldn't think for very long, right? We would ask it one question, and it would generate an answer in 10 seconds.
[0:06:13] The way I've always thought about the economics is you're basically, you're renting someone's brain. You're renting a virtual brain for a certain amount of time. Now what's happening is
[0:06:24] All that's changed is the questions we're asking the virtual brain are different. And so now sometimes you ask the virtual brain a question instead of like, you know, what sports team, you know, run one, you know, this contest in some year, you know, you're asking, hey, can you prove this mathematical theorem? Can you implement this coding feature? And of course, just like a human, the model says, okay, it's going to take me 10 or 15 minutes to do that. So you're paying for 10 or 15 minutes of its thinking time.
[0:06:53] Instead of one or two minutes. Now, I do think that has implications on the output side, on the price side relative to the cost side. But there are implications we've always been thinking in terms of. We have thought of these models, and it relates to our picture of where things going, as more like labor, more like effort than like software.
[0:07:16] As these models get smarter, they're going to do things like, and I've talked about in 2025, we're going to release probably in pieces parts of a vision we call a virtual collaborator. So it'll be a model that you give it a task. It works on that task for several hours. Maybe it checks in with you every once in a while. And that could easily generate thousands of dollars of value, tens of thousands of dollars of value. If you think of, I give this model to a medical chemist.
[0:07:45] It finds a better drug candidate than it would have found otherwise, or it reduces the number of animal trials it has to run, or phase one trials it has to run. It's quite possible the words that come from that model are worth thousands, tens of thousands of dollars. And so I do imagine a world in which, and we're already starting to see it in some of the high tier pricing plans, enterprises, and to some extent,
[0:08:13] Power user consumers, which Claude is actually very popular with, may actually in some ways dominate, are the main source of revenue because they're paying thousands, tens of thousands of dollars a year because the quality of the model's thinking is worth that. The model is being paid like a coworker, at least like an assistant. But let me continue your analogy because it's a very interesting one, that what you're paying for is you're kind of, you're paying for the labor of the AI.
[0:08:42] Right, along comes, right at the same time, cheap Chinese version. That's pretty much nearly as good, frankly. DeepSeek is, you know, not far off Claude 3.5 Sonnet. In fact, as good in some ways. They tell me, don't ask me, but that's what they say. You know, the cheap Chinese version comes along. Why is anyone going to pay thousands of dollars for yours? And why aren't they just going to go to the cheap Chinese version?
[0:09:08] Yeah, so well, I mean, first of all, it's not currently as good. But we're actually living in an interesting moment in time. And it actually relates to the first question you asked, which is this switch to reasoning models. So if you imagine there's a lot of compute, right? We're increasing the amount of compute. At all times in the past, the 99.9%
[0:09:35] We're now executing a switchover where we figured out how to put small amounts of compute into this second stage, this reinforcement learning stage. And because none of it was being done before, there are big gains to that stage. The amount of compute in that stage is increasing to the point where it will even become dominant. Right now, we're in the switchover region.
[0:10:05] We're with a little bit of RL training. You can kind of catch up with the current situation. Whenever the paradigm switches over, the landscape scrambles a bit. But then it kind of reestablishes itself. So later this year, we and probably others will have hundreds of thousands of chips. I think probably there will be millions of chips from various companies in 2026 and more in 2027.
[0:10:32] DeepSeek, the Chinese company, it's marketed as cheap, but it's at least been reported that DeepSeek has 50,000 H100s, which for reference is about half of what Elon's Colossus cluster has. I won't say exactly how many of these Anthropic has right now. So I actually think we have an opportunity, and this relates to China policy. Right now, both sides are roughly in the...
[0:11:02] Mid tens of thousands, low hundreds of thousands of chips. That's why we're kind of close to parity now. We're at the crossover point. But as we go to hundreds of thousands and millions of chips, there's two possible futures. In one of those futures, the US and its allies are able to provision that many chips fast enough. And because of the export controls on chips to China, and because Chinese Huawei chips are inferior,
[0:11:32] China cannot get to that scale. There's another world where both sides get to that scale. Let's dwell on that a little bit more because I think given, and this is not a discussion about geopolitics, but given the geopolitical backdrop, I think this is really interesting because for four years now, I've heard from American officials, the single most important thing is that the US retains its lead in AI.
[0:11:56] And U.S. policy, particularly towards China, has been focused on that, and this entire export control regime has been built up. And for a long time, you know, Chinese were way behind. We just used to write it, Chinese are way behind. They are not way behind. Maybe it's not quite as good as Claude saw it, 3.5 saw it, but it's not far off. And were you expecting it to be that good, honestly? Yeah, so I think it's a, again, I think it's an expected consequence of the switchover in period times. Really? You think?
[0:12:23] I think DeepSeek in particular has a higher quality than some of the other companies in China. Actually, we were aware of DeepSeek for quite a while and they kind of identified it as more likely to succeed than some of the others like Alibaba. So it's a particularly talented team. But the general dynamic that we've seen is, I think, to be expected, particularly in the tens of thousands of chips.
[0:12:51] Smuggling is very possible. I obviously can't speak for either the last administration or the current one. I think this is a bipartisan thing. But I think the purpose of the export controls is to prevent China and other adversaries from getting to the hundreds of thousands of chips, the millions of chips, because that's really hard to smuggle. And that's really hard to replace with Huawei chips. So there's this bifurcation in front of us. And we can have a big lead.
[0:13:21] Or as you're suggesting, one of these other routes, things can be parity. And those are two very different futures for the world. I guess my argument is, my question is, can you really have such a big lead? When maybe you expected them to be this good, but I think a lot of people didn't. A lot of people, even in your industry, have been pretty blindsided by how good these new Chinese models are.
[0:13:44] And it leads me to wonder whether actually the half-life of export controls is not a lot shorter than you think. And even if you have export controls, the Chinese innovate around them, and they come up with other ways to fix things. So the premise that you can, through export controls, ensure that the U.S. is dominant in cutting-edge chip provision, do you want to base your entire policy on that? So I actually believe this is possible. So first of all...
[0:14:13] This may or may not be something that works. This may or may not be something that's well implemented. But I do think that if it is well implemented, it still has a fairly good chance of working. And it is something that we should try. Because of this critical period, right? I think I maybe didn't come earlier in this interview, but in other interviews I've said, 2026, 2027 is the critical window. And if you're ahead then,
[0:14:40] The models start getting better than humans at everything, including AI design, including using AI to make better AI, including using AI to make all kinds of intelligence and defense technologies. So I think this is pretty important. And again, I don't believe the export controls were designed to fundamentally change the chip allocation in 2024
[0:15:10] Or even in the first half of 2025. Like, again, DeepSeq's cluster, same order of magnitude as the cluster of the frontier companies in the US. I don't think that could ever have been prevented. Interesting. So 2026, 2027 is when you effectively get to AGI across the board. And it's the threshold moment. Whoever's ahead then is ahead forever. Is that what you're saying? Potentially. I mean, we don't know these things, but there's a risk of this. That's happening at a particularly...
[0:15:39] Uncertain time geopolitically, but we'll get to that. I've heard those same years floated as, no, I'm aware of how frightening this is. So you've made the case for export controls. We'll come to that in a minute. You've made the case for export controls. What else does the U.S. need to do to ensure that it is ahead in 26, 27? Yes, so energy provision within the U.S., I think that's important. Making sure that we're able to provide enough to the U.S. and to its allies in Europe as well. Making sure that we have a coalition.
[0:16:08] of democracies that are able to provide enough compute. There's another issue that's kind of the mirror image of this, which is for all our concern about authoritarians, and I'm very worried about it because when AI is so powerful, there's the opportunity for authoritarians to abuse them. Many limits, constraints on the power of authoritarians comes from the fact that they need the consent of humans, of followers to do what they want. If we start to automate this, then very dark things become possible.
[0:16:38] So that's very important. But at the same time, if we have these models that are better than humans at everything, that are like a country of geniuses in the data center, we also need to worry about the dangers of the models that we ourselves are building. Are they dangerous in an autonomous sense? Can people misuse them? And so that's why we've been advocates of testing and measurement for AI systems.
[0:17:04] So this was done by the US AISI, the UK AISI. It's all been voluntary so far, which I think is the right approach because we don't know for sure how to run these tests. And the tests have focused on national security risks like bioweapon risks, cyber risks, emergent autonomy of the AI systems. And these tests have actually been helpful to us. In some cases, they've even helped us do a better job of deploying our commercial product because we understand better what the models are capable of, for example, in areas like cyber.
[0:17:33] So we've been an advocate of keeping this kind of testing around. And actually, we found some bipartisan support for this. For example, when we talk in Congress, the idea of measuring models for national security risks make a lot of sense to people. I think people have the impression that this kind of testing, maybe it's the name Safety Institute, is kind of related to DEI issues or bias. That's a misconception.
[0:18:00] That's not the kind of testing that's being done at the AISI. Is this administration keen on it? I mean, the president, and correct me if I'm wrong, has rescinded the Biden administration's executive order on AI. Correct. One, what impact does that have? And isn't this part, isn't there some element of the executive order that has this? The AISI is separate from the executive order. The executive order focused on reporting requirements for models that were above a certain size.
[0:18:25] I think those reporting requirements were good, but they were really small potatoes. So there's a lot of discussion of the executive order. It has this kind of totemic significance, right, as something Biden did and Trump is ripping up. But actually, there wasn't that much to it. There wasn't that much in it. I don't think it's actually the important thing. And generally, I kind of advise that we focus on the important things and kind of not the things that kind of gather this polarized totemic significance.
[0:18:52] So we're going to come to safety a bit later because this is something you're known for. But just before we come to that, can we talk a bit about where you are seeing the fastest adoption? What kind of industries, what kind of companies? Because the impact that all of this will have for all of us depends on how quickly this is adopted, how quickly it diffuses throughout economies. And what are you seeing there? Yeah, hands down, the fastest area of adoption has been coding.
[0:19:19] Multiple hundred million dollars of revenue have materialized overnight, over the last three months. Over the last three months. Over the last three months or so. Anthropic's revenue in general has grown from roughly 100 million to roughly a billion over the last 12 months. And coding is not the majority of it, but it's the largest single area.
[0:19:45] And there's many companies. There's Cursor, there's Codium, GitHub Copilot, actually, which is GitHub is owned by Microsoft. They were actually willing to go through the step of using Amazon's cloud in order to get access to our model. There are others like Vercel, Augment, Cognition, several others. And so this is happening very fast. And coders across the industry are using it.
[0:20:15] And the quality of coding is only going to get better. So I think that explosion is only going to inflect upwards. And with things like the virtual collaborator, it's going to extend to other areas. I think the growth we're seeing in coding, that growth is going to happen in every area. And that's the way that this industry is going to get relatively soon to $10 billion, $100 billion revenue amounts.
[0:20:39] You have to imagine, if you take serious the country of geniuses in a data center better than humans at everything in two or three years, well...
[0:20:47] What are the commercial consequences of that? Somehow, you know, that doesn't make any sense unless the revenue amounts get up. And I think they will for multiple companies within two or three years. That sounds like a crazy prediction, but I don't know. Everything else I've said is maybe even crazy. Presumably also for the NVIDIAs of this world, too. Yeah, I mean, you know, I think there's multiple problems with me telling you whether to sell or buy a stock. But, you know.
[0:21:15] I don't think it's a secret. Your hand is giving away the direction there. No intended to none. But it doesn't take a genius to see that probably NVIDIA has a bright future. The only way in which they might not have a bright future is, of course, there are other chips being developed by other players. And tell me about what kinds of companies are going to...
[0:21:40] Kind of reinvent themselves fast enough to be able to take advantage of all of this. Because I was in a really interesting conversation yesterday, actually. And one argument is big enterprises have the capital, the scale, they can kind of afford to spend all of this money. And then there's another argument was actually startups may come from nowhere because they'll reinvent the whole nature of what a firm is.
[0:22:05] You may not in this world need an HR department. You may not need an accounting department. You may completely reinvent what the firm is. And so startups will be much faster. What do you think? So we are seeing the following. The dynamics are very interesting. Spendings from startups has grown much faster than spending from enterprises. There are startups that spend $50 million annualized or even more of that. We've seen that.
[0:22:32] It's very hard for enterprise, and they get to that level over a few months. It's very hard for enterprises to get to that level that quickly. However, the potential for enterprises to spend, given the value of the technology, is much higher. So you have this smaller pool that's spiking very quickly, and this larger pool that's starting to spike but is doing so slowly. I think the enterprises will ultimately overtake the startups and will spend much more than they do. But that's not the only possible outcome.
[0:23:02] There's another world in which the enterprises can't adapt fast enough. And we have a kind of massive industrial disruption of something we've never seen before, where across industries, just all the incumbents are knocked down one by one by these innovative startups that can adopt AI much faster. So either is possible. They kind of both lead to the same world, but they look different while they're happening. And what kind of time frame do you expect this in?
[0:23:32] Geniuses in a data center by 26, 27. Over what kind of time horizon is every single industry completely sort of overhauled by this? Yeah, so one of the things I dealt with, you know, I wrote this essay, Machines of Loving Grace. And one of the key questions it asked, I mean, it was focused on what are all the positive applications of AI, right? I've talked about a lot of, you know, risks and dangers here, but what are the positive applications? And the key question it asked is, look, the positive applications are going to be bounded by...
[0:24:00] By the human side of things, I use this term called marginal returns to intelligence. Economists talk about marginal returns to labor, to land, to capital. You don't usually think of intelligence as, I mean, it's connected to labor, but you don't think of it as an asset. So what happens when you max out intelligence, but there's still the physical world, there's still people, there's still laws, there's still logistics. What goes fast and what goes slow? It's related to this enterprise versus startup question, actually.
[0:24:29] Because enterprises are human institutions that do things a certain way, have existing customer bases. And the general thing that I concluded is kind of not surprisingly, it sounds almost obvious, but things will go fast in areas where markets function well and in areas where things can be done digitally rather than physically. I'm extremely optimistic, like insanely optimistic about biology and health. This is an area.
[0:24:58] So I used to be a biologist. I did my PhD in biophysics and computational neuroscience. I worked on understanding real neural networks in brains as opposed to the artificial ones that I build now. And then I did a postdoc in computational proteomics where I looked for cancer biomarkers. And knowing what I know about these areas, I think AI is just getting to the point where we're going to see some really transformative stuff.
[0:25:28] I mean, you said, sorry, I didn't mean to butt in, but in your essay, which I really recommend you read, it's a really interesting essay, you said, and this stuck with me, that you thought AI could increase the rate of progress in biology by 10x. So we could get 50 to 100 years of progress in 5 to 10 years. Yes, that was my view. And just to be clear, this 10x sounds like one of these kind of hypey things that like, what does it even mean?
[0:25:55] I actually think that intrinsically, AI systems can do things very, very fast. There's very few limits to it. The question I'm asking is like total factor productivity, as an economist would say. Like, you know, look at all the drugs we develop over the next, you know, that we were going to develop over the next hundred years. How quickly can we get them? So include everything. Include the clinical trial system, which will get faster in some ways because we can replace.
[0:26:20] Late stage trials with early stage trials and early stage trials with animal trials and animal trials with experiments on cells and cells with simulations. But it'll still take a while. It'll take a number of iterations. The AIs will, as time goes on and there are more of them and they get smarter, they'll find ways around it. They'll say, actually, you don't need to run that experiment. I can figure it out by thinking in my head, that other experiment, nope, too complex. You still got to run it.
[0:26:46] What does it all add up to? Where do we get to at the end of it? And my view was that something like 10x is feasible. Something like 100x is not outside the realm of possibility, but I'm skeptical. I think 2x is possible, but would be, in my opinion, a pessimistic assessment of how fast things are going. And then if I think about, well, what would we have? So let's say 10x is right. What would we have done in 100 years?
[0:27:13] I think in 100 years without AI, humans have a lot of ingenuity. I have a lot of respect for the basic biologists, the folks who work in pharma. I think in 100 years, humans can live to maybe 140 instead of 70 or 80. And so the prediction that makes is maybe that will be possible if we build our powerful AI in 2027. Then in 10 years, by 2037, will humans be living to 140, 150?
[0:27:39] And, you know, I bet someone even nods at that. So I want to open up to questions, but just before I do, I just want to kind of take stock of what you've heard here. Which thing I said was most crazy? Well, no, the first time I had these conversations with people like you, I just thought, like, what planet are these people on? I'm very self-conscious about it. But actually, the last couple of years,
[0:28:08] The pace at which this technology has advanced has just been mind-blowing. And so, you know, when you start saying, you know, we're going to have 10x biological innovation by 2027, 2027 we're going to have, you know, models that are better than humans at everything, I'm kind of inclined to believe it actually now. And so I just listen to this. And 2027 just is two, three years away. This is what you've just heard here. So with that in mind...
[0:28:36] And I've got more questions for you in a minute, but I want to give a chance to other people to ask questions. I promised you that this was one of the most important and interesting people in this field. Dario absolutely is. So who has a question for him? Martin, go ahead. Okay. Why the hell do we want humans to live to 140? It's unbearable already. But anyway, the serious question is this. Let's suppose you were standing in front of a...
[0:29:04] of a group of parents in a school. Their children are 10 to 18. My daughter's in this situation now. And they ask the question, OK, this is the world in which all the thinking is being done by machine. What on earth should we try to educate these pieces of wetware? How? And your first answer, obviously, is, well, you've just made a big mistake. You're making a colossal investment in a completely outmoded piece of kit. Scrap it.
[0:29:34] And then they say, well, that's true, I'm sure, but we love our children. And at that stage, what do you say? Yes, so I talk a little bit about this in the essay. And this is one of the risks I didn't touch on earlier, but I think it's going to be an increasingly important one. We've often worked on these safety and security risks and this issue of competition with China. But I think just as these powerful AI bring power into the world,
[0:30:03] They also bring economic power into the world as compared to the economic power that we all have as individuals. And so this issue of labor displacement and ultimately meaning for humans is going to be really important. I think there's a short-term and a long-term component of that. In the short term, we're going to have labor displacement that is large in magnitude but looks like the kind of labor displacement we've had with previous technologies, like the internet, like much of industrialization.
[0:30:32] I think that always has some pain to it, but we are thinking a lot about how to design our systems so that they're as complementary as possible, so that it's easiest for people to adapt to using AI. Again, this is within the paradigm of AI can do some things humans can do, and humans need to adapt and do different things. I think comparative advantage is very powerful. Even if the AI does 95% of what you do,
[0:30:57] Then the other 5% really expands to be useful. And so I think the short term will be a little longer than people expect. Then there's the long term. What is a little longer? Given that we're going to have the world six months? It could be four years instead of two. I don't know. I guess short term and long term have taken on weird meanings in my world. But then there's the long term, which I think you were maybe a little more asking about. And there, I think we are facing a world where...
[0:31:27] We're building systems that are as or more intelligent than we are along almost all axes. We still have a physical presence, although I think in the long run, AI will propagate to robotics as well. So we have to confront this kind of deep philosophical question of our intelligence is the thing that we value so much. I think most of the people in this room.
[0:31:56] You know, put a lot of their self-worth and value in their intelligence. And, you know, this is something that we're learning to create and copy freely. And so we're just going to have to confront what that means, right? We've had our kind of socioeconomic contract for the last few hundred years that, you know, there's been problems with it, but it's worked relatively well, which is, you know, we all contribute value with, you know.
[0:32:22] Mentally and physically, we're able to grow the economy. We get compensated for it. We feel a sense of self-worth for it. All of those assumptions are going to break down. And it's almost like we're going to need a new social contract, almost a new culture that kind of explains why are humans...
[0:32:43] Why are humans valuable? What meaning does our lives have? Can I just say, are we still, I mean, you said in a sort of throwaway way, we're going to have a physical presence. I just want to make sure that is still true. We are going to. I mean, if you read science fiction novels and all this discussion of uploading people into the cloud and things like, that's a little bit far out even for me. I don't think it's necessarily impossible, but I do think probably we'll be in the physical world for the.
[0:33:11] For the foreseeable future. But we'll be like the, what the kind of, you know, pretty things around there for the intelligence to look at. There's several answers here, right? Like one is that we can, you know, we can use AI to kind of enhance ourselves and keep up. That's one possible path. Another possible path is that human meaning is, you know, it's suddenly, it's not about generating economic value or like making something unique. I actually think.
[0:33:39] Human meaning, I mean, you could talk about this question for hours, days. I think it's connected to two things. I think it's connected to relationships to other humans. And I think it's connected to projects and goals over a long period of time. So striving for something, right? Having some kind of long-term project where it's not just a game. The stakes are real. You're trying to do something important, live for something important.
[0:34:08] I think that's still possible in a world where the AIs are better at everything. I think there's nothing about that fact, nothing about either of those two things that AIs detract from. In fact, I think they could even enhance it. What that looks like, I don't think any of us know. And my main anxiety is like, I'm probably not the best person to give a full answer to that question. It has to come from all of us.
[0:34:35] Maybe the only place that people like me can help is to prompt us to start thinking along these lines. Because right now, there are very few people thinking along these lines. And the technology is coming at us like a speeding train. And that makes me nervous. Because if we have zero time, if when we react, we have zero time to react, our reaction is guaranteed not to be good. It's guaranteed to be reactive, just knee jerk.
[0:34:59] Crazy, just the worst impulses you could possibly imagine. And so we need to start thinking about this now so that we have a chance of rising to the occasion, right? Of reacting in a way that reflects our best selves.
[0:35:14] I understand with everything going on in the world that may seem idealistic. We've got a couple of years, it sounds like. Look, I'm just the one telling you what I think is going to happen. You're blowing certainly me away. Yes, gentleman at the back, and then lady in the front middle, and then I think that would probably be all we have time for. Thank you so much for this fascinating talk. The trajectory of progress in the physical sciences seems extremely clear.
[0:35:43] In the social sciences or in complex human systems, for example, in policy design, could you please give us your assessment of the trajectory of progress in those domains, domains involving complex human choices in interaction? And relatedly, Demis has recently argued that it may be the case, as a hypothesis, that these so-called unique advantages of quantum computing are not necessary for progress in such domains. Thanks. Yes. So I don't know if this is a question about how much AI will drive the progress, but I guess I'll answer it in that assumption.
[0:36:12] Definitely, I think the physical sciences will go faster. Anything that is objective, anything that there's a skill, you can do that. If I think about political or geopolitical or social science expertise, on which I don't have any expertise, these things seem like they often have long cycle times. They're also based on intuition. I think AI will gain increasing intuition and probably will surpass humans in the intuition.
[0:36:42] But also just gaining the experience to gather that intuition will take time. And many of these decisions are very kind of, I don't want to say one-off, but it's like, if I think about, go back to the last seven recessions, right? Try to make a macroeconomic theory about what caused them. So many things were different in those recessions. You can't make a...
[0:37:07] You can't do a controlled experiment, right? It was like stagflation in the 70s versus the inflation that happened two years ago. So many things were different about the situations. And yet what humans do, and what I think AIs will do as well, they look at the commonalities, they dive into the data, they use their common sense to come up with, ah, here's a simple model, right? Here's the Keynesian theory of recessions or whatever, or here's the other thing.
[0:37:37] And so I think AIs, they will get as good and eventually better at us than doing that. But there are limits to what knowledge is possible because we're missing some data. This is the marginal returns to intelligence. At some point, there's only so much you can predict. I suspect that ceiling is well above humans, right? Our economic theories have gotten better for all that.
[0:38:04] Economists are often wrong about things. The question is, how high above humans is it? And I don't know. My guess is that there will be a lot that's just fundamentally unpredictable. And we look at it and we're like, the economy did this crazy thing over the last year. We don't understand. It's some emergent phenomenon. Maybe we'll never understand. I think there's going to be these things that are lost to complexity and mystery of the universe. Even to the super intelligent AIs. Lady there, yes, go for it.
[0:38:34] Thanks, Danny, for the opportunity. And I agree with you, Dario, on how the AI can revolutionize the medicine field. Amazon also is trying to create an AI for its pharmaceutical company, and they actually have collaborations right now. But I want to pivot to a different question.
[0:38:52] I have a little question. Yeah. So like people used to go to, trying to go to Ivy League schools, like you went to Stanford and Caltech, but OpenAI and also like Claude AI is a really good teacher for students. Do you think that will break the polarizations for the education in the future where people no longer has to go to Ivy League to become a good entrepreneur and be successful in their career? Yeah. I mean, I think education is one of these interesting areas, right? And, you know, the model's getting, again, there's this short term and the long term for it. You know, the model's getting so much smarter.
[0:39:22] We maybe should start thinking of education something as more as a thing of self-achievement, of trying to become a human being who understands things better, who has more capabilities, who's able to do things that you couldn't do before. And for that, I think AI will certainly help us a lot. I do think learning is a social activity. And ultimately, the social
[0:39:48] Every aspect of it and component of it will always be important. So I think there'll be an AI aspect of it and a social aspect of it. How those two come together, whether it's at a university or some other structure, I don't know. So I know there's tons more questions, but Dario, I know you have to go. I'm going to end, but there's just two specific questions I want to ask you. And one is, you're known as someone who has focused more than many of your fellow leaders in this field on the risks. You've mentioned it a fair amount today.
[0:40:14] But in your essay, Machines of Love and Grace, as you say, you focused on the upside. Have you become, over the last year, more confident in your certainty about the upside or about the risks? I think I've increased my confidence in both because I've increased my assessment that the technology is going to happen. And that kind of pushes on both. I don't think the balance between them, for me,
[0:40:44] In a factual, objective sense, has changed. Mostly, I don't know. I mean, there's some world where none of these risks really happen. They're kind of theoretical. In theory, someone could do X. In theory, someone could do Y, but maybe it doesn't work out that way. There's another world where the benefits are slower than we think because these regulatory barriers, like I think when I wrote my essay, Tyler Cowen said, I don't think it'll be 10 years. I believe you that I've been in this timeline, but I think.
[0:41:11] You know, regulatory barriers are so slow that it'll take 100 years, right? So he has this libertarian perspective that like regulation... Just remind everyone what you think the biggest risks are, because I think we didn't delve into that in detail. Well, first of all, let me finish this thing, which is factually my view has not changed. Like characterologically and attitudinally, it has changed a lot. I used to think that markets provide benefits, and so it's important to think about risks. I now actually think that like...
[0:41:41] We actually need to inspire people, like truthfully, you know, we shouldn't make things up, but we need to inspire people with the benefits that are possible because the benefits are so awesome. And if we can actually find a way to navigate all of the problems, including, you know, the economics and the human meaning, as well as the security and geopolitics risk, if we can somehow find our way through the obstacle course, the thing on the other side is really beautiful. It truly is.
[0:42:11] And actually maintaining that vision, I think, is extremely important to finding our way through, to getting the good outcome. And so I think I've had a big change there, not in terms of the facts on the ground, but just in terms of the attitude with which you have to approach these things. And lastly, is this now inevitable? Is what you've laid out definitely going to happen? Can anything derail this?
[0:42:40] Yeah, so I still think there's uncertainty, less than there was before. There's maybe two kinds. One is a fundamental technical uncertainty. So a thing I'll say is I would give a 5%, maybe 10% chance that we get to 2100. And none of these things that I've described happen. What's going on there? Well, I don't know. You can just be wrong about things. You can just be seeing the situation completely differently from how it is.
[0:43:09] I think that's still possible. I have a list of what could block it. And I've been crossing items off that list. It's getting shorter and shorter. But it's not zero. And you can always be wrong. And maybe in 2050, when I'm old and gray, I can laugh along with everyone else about all this crazy stuff. What's still on the list? What could block this? I think we could be wrong about the last part of scaling. I think...
[0:43:37] In reinforcement learning, there could be a lot of things that are harder to get right than we think. There could be something we're missing algorithmically, particularly on tasks that are hard to evaluate. Again, not likely, but I think it's possible. The other thing, and this wouldn't take us to 2100, but if you're like, what's the most likely reason this doesn't happen in 2026, 2027, but gets moved out a few years? Some kind of geopolitical crisis.
[0:44:04] The chips that power all this are located in maybe not the most stable region in the world. So that's one thing that could really derail things. And we may not be in the most geopolitically stable time right now. Yeah, the intersection, it's not my world, but the intersection between that and all that we're seeing here gives me great concern. But we are going to go on to some... You said it, not me. It is a perfect storm. We have...
[0:44:33] The fastest and most dramatic technological change, I think, in human history, at the same time as we have a geopolitical shift that we haven't seen in at least 100 years. You said it was a perfect storm, not me. I mean, I hope it's not, but this is what we have to fight against. All of these things converge in a bad way and somehow flip them around and make them converge in a good way. Well, I hope you focus on that. Dario Amadei, thank you very, very much. Thank you for having me.