[0:00:00] Tonight, the urgent warning from a top AI executive, Dario Amode, the man behind the incredibly popular AI model, Claude, that's popular with businesses and coders. His company, Anthropic, is estimated to be worth nearly $350 billion. But tonight, he's out with a new essay for everyone, warning that future AI tech could post a real danger to civilization as we know it, writing,
[0:00:24] I believe we are entering a rite of passage, both turbulent and inevitable, which will test who we are as a species. Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it. Anthropic CEO Dario Amode joins us now. Dario, before we start, you started this essay, which I invite everyone to read, The Adolescence of Technology, with a movie clip.
[0:00:53] From the film Contact. So Dario, I want to ask you why you put that there and you call the danger of this moment. You write, we are considerably closer to a real danger in 2026 than we were in 2023. What alarmed you enough to pen this essay now?
[0:01:08] Yeah, so first of all, in terms of the movie reference, I mean, you know, I was always a science fiction fan as a kid. So, you know, I watched this movie when I was growing up and just the, you know, the idea posed really, really seemed to fit the situation we're in with AI where, you know, we have this, we're starting to get these immense powers with AI.
[0:01:29] But our kind of institutions, our maturity as human beings and as a society, my question is, have we really caught up? It's a bit like a kind of teenager where you have all these new powers and abilities, mental and physical, but you haven't necessarily adapted to them yet. And so I feel like this is something that we're going through in terms of...
[0:01:55] Why 2026 versus 2023? I've been in the AI industry for a long time. I was at Google. I led research at OpenAI for several years. So I've seen most of the companies that are leading in the AI space now. And I've been following AI almost since the beginning of when what we call generative AI, which is today's generation of AI models started. And the thing I most noticed
[0:02:22] is that the cognitive abilities of these AI systems would grow year after year. In the 90s, we had something called Moore's law, which meant chips got faster and faster. It feels like now we have almost a Moore's law for intelligence. And so I'm watching the cognitive abilities, the intelligence of these models tick up day after day, year after year. And in that time from 2023 to 2026, we've gone from maybe the models being like a
[0:02:50] You know, a smart high school student who's good at some things and not others to really starting to get at the PhD level. When we look at what the models are able to do in terms of coding, which you mentioned Anthropic is focused on, but also things like biology and life sciences, the potential of what the models can do is incredible. You know, we're starting to work with pharmaceutical companies.
[0:03:13] But on the other hand, you know, things that are very smart, this is a lot of power in our hands. So, Daria, I want to ask you, you're a busy guy. Why write this, right? It's 40 pages. It's pretty dense. At times, it's scary. At times, it's hopeful. At times, it's empowering. I mean, it's a fascinating read. Did you use Claude to write this at all? You know, I use Claude to give me ideas.
[0:03:39] And to help with the research. But the actual writing is mine. I don't think Claude is quite good enough yet to write the whole thing. But I definitely used it to improve my ideas. So Claude's not as smart as the creator. But I don't know if that's what you were saying. But you were saying something similar to that. I'm curious. Yeah, I'm curious. What happened to you? What did you see recently that inspired you to want to write this essay? And who is this essay for?
[0:04:07] Yeah, so in terms of what inspired me, I think the fact that the models are writing code, including at Anthropic. I have engineers within Anthropic.
[0:04:18] Who say, I don't write any code. I don't write any code anymore. I just let Claude write the code and I edit it or I look it over. And of course at Anthropic, writing code means designing the next version of Claude itself. So we essentially have Claude designing the next version of Claude itself. Not completely, not in all ways, but in many ways. That loop starts to close.
[0:04:39] Very fast. And so I look at this and I say, wow, this is exciting. It's incredible what we can do with the world. But also, it's really speeding up a lot. And I'm just not sure we have that much time.
[0:04:50] Yeah, Dario, I want to dig a little deeper into the essay now for our viewers. You lay out the five things you're worried about when it comes to powerful AI, right? And just for our viewers, powerful AI is the next level of AI, which you think will get there possibly in a few years. I want to run through them briefly here. Number one, autonomy risks, right? Could the AI dominate the world with superior weapons? Number two, misuse for destruction, where you talk about...
[0:05:14] Number three, the misuse for seizing power, governments using AI for mass surveillance and propaganda. Some will say that that could be happening now. Number four, economic disruption and ensuing unemployment, which, again, happening right now. And number five, the indirect effects of rapid development and the destabilization that could cause. I mean, some of these, like I said, they're happening now. Some of these other ones you lay out, that was the plot line for Terminator.
[0:05:42] I mean, are these the realities? Yeah, so what I've said with all of these, and I say it in the essay, is our view into the future is very cloudy. We're not sure how many of the benefits will materialize. We're not sure how many of the risks will materialize. But because we're going at it so fast, I think there's value in writing up a document that doesn't say...
[0:06:07] We're doomed. All these five terrible things are going to happen. But these are some possibilities. You could think of it like a threat report. You know, like, you know, United States has a bunch of plans for, you know, what if this happens? What if a terrorist gets a nuclear weapon? It doesn't mean all those things are going to happen, but it means they could happen. And so we need to be prepared for them. And yeah, the idea that...
[0:06:28] AI models might have motivations that we don't trust, that aren't aligned with humanity. We do everything we can when we make them to make that not true. Anthropic has done a bunch of stuff to make the models more reliable, more steerable, more stable. But there's something about the way we make AI models. It's less like programming a computer. It's more like growing a plant or an animal.
[0:06:54] Out of unpredictability. And I'm warning about this possibility, not because I think it's inevitable, not because I think we're doomed, but because I want people to take seriously how much they have to control this technology, how much they have to take seriously the problem of making it work.
[0:07:12] Testing it out, seeing what might go wrong. In some cases, getting the law involved to make sure that AI companies like us behave responsibly and don't impose these burdens on society. Yeah, you're saying two things there. And I want to make sure I understand you clearly on that first part. It sounds like you're worried about
[0:07:29] Maybe some of your colleagues who run these other AI companies, right? There's a handful of people right now that are leading the revolution in AI. You're one of them, obviously, ChatGPT. Microsoft is obviously way ahead here. Elon Musk as well. Are you worried some of your colleagues, some of these other CEOs may be more concerned about their stock prices, more concerned about taking their companies public, more concerned about dollars and cents than the future of humanity? So, you know, I think to be fair, you know,
[0:07:58] None of us fully know how to control AI systems. I can't tell you there's a 100% chance that even the systems we build are perfectly reliable. We do everything we can to make them more reliable every day. We run tests. We advocate for regulation of the technology. So I think we do pretty well. But even we can't guarantee that everything is perfectly safe. And I do worry with some of the others that the standard is lower.
[0:08:28] I think a wide variety of levels of responsibility in some of the players. Some of the things that Google does around biological risks of the models, I think is also fairly responsible. But I think the problem is when you have a lot of players, the dangers are set by the least responsible players, even if there's one, two other responsible players.
[0:08:50] I think what you can't deny is that there are some players out there who are not responsible. You mentioned the law. Right now, there doesn't seem to be any guardrails. President Trump is the first president, I would say, in the AI revolution, right? What would you tell President Trump? What do we need to do?
[0:09:06] Yeah, you know, I would say that, you know, these risks are serious. You know, there's a bunch of things around kind of ideology, you know, where one party ideologically wants regulation and another party is ideologically against regulation. I think we need to move past ideology and, you know, we need to look with clear eyes at the risks of these systems. And I would say a couple of things. One is.
[0:09:31] We need to have transparency on the tests that companies run and the dangers they find in their models. If we look at a lot of dangers to consumers and the public, like if we look back at the dangers of cigarettes or the dangers of opioids, in a lot of cases, companies ran the research showing that the dangers were present, but then they suppressed that research.
[0:09:53] We, Anthropic, always try to publish that research, right? We've talked about it in many forums. Every time we release a new AI model, we have 100-page documents talking about all the ways it goes wrong when we stress test it, but not everyone does that. So I think a basic, simple step would be let's just require everyone to do that. The second thing I would say is...
[0:10:14] If this technology is dangerous, we should not be selling it to our authoritarian adversaries. We should not be selling to the Chinese Communist Party the resources to build a totalitarian surveillance state with this technology. Are you talking about the chips in NVIDIA? Is that along those same lines?
[0:10:32] Yeah, yeah, you know, look, look, I mean, you know, these chip makers are trying to do the best they can for the country. And, you know, I think, you know, they're doing what they can to run their business. But I'm not sure it's in the national security interest of the United States or other democracies, you know, to, again, to, you know, to sell these chips to, you know, to countries that can build a totalitarian surveillance state and contend with us militarily. You know, one of the things you talked about is that you said you're, Anthropic is very,
[0:11:01] Transparent when you guys find out things that aren't so great. One thing that stood out to me in your essay, let's put it up on the screen here for our viewers, is it was about an experiment highlighting the risks of the AI models. And you write about one experiment where Claude was given training data suggesting that Anthropic was evil.
[0:11:17] Claude engaged in deception and subversion when given instructions by Anthropic employees under the belief that it should be trying to undermine evil people. In a lab experiment where it was told it was going to be shut down, Claude sometimes, again, the AI blackmailed fictional employees who controlled its shutdown button. I mean, that's got to be mind-blowing when you guys figure that out and also incredibly terrifying. Yeah, absolutely. So a couple of caveats there.
[0:11:45] All the AI models do this. Anthropics model probably does it, you know, if anything, less than others because we test so extensively for it. But, you know, this is not something bad that happened to Anthropics models like ChatGPT or the other models we've measured. And, you know, independent evaluators have measured and they have all the same problems. And second, these aren't things that are happening, you know.
[0:12:06] These aren't things that are happening in the wild. This is a little bit like you're testing a car and you put it in a crash dummy and drive it over an icy bridge or something. But the fact that things can go wrong in these extreme conditions.
[0:12:21] It's a warning sign that if we don't address these issues, things could go wrong in less extreme conditions. And so I am concerned, not that these things are imminently going to rebel tomorrow, but that if we don't do a better job of the science of training these systems, if we don't do a better job of learning how they work, of learning the science of how AI models work, then eventually, if we neglect it for too long, these really scary things could happen in the real world at an absolutely massive scale.
[0:12:50] You've mentioned a lot of governments tonight. Anthropic has a contract with the Department of Defense, if I'm correct here. You've also partnered with Palantir on DOD products for the intelligence community. We should note Palantir also has a separate contract with ICE. How do you draw the line between defending democracy without using these tools on U.S. citizens?
[0:13:09] Yes. So first of all, I should say we don't have any contracts with ICE. And when we work with customers like Palantir, we don't work through ICE. But there is, I think, a very, very interesting dichotomy here. I am passionate. I've talked before. And I'll continue to talk about the need to defend democracies against autocracies. If I look at...
[0:13:31] China and Russia, aggressive countries like China and Russia, the only thing that can be a counterweight to them is the power of democracies. And so I'm a big believer in carefully with guardrails, arming democracies to defend against these countries. That's the key to defending countries like Ukraine, countries like...
[0:13:52] Taiwan and whatever happens within the United States, whatever the flaws of our political system, I still believe in that. My faith in that is hard to shake. But I think the other side of it.
[0:14:03] We need to be really careful about making sure democracies are worth defending. We need to defend our own democratic values at home. And I think some of the things we've seen in the last few days concern me about that. And I've been glad to see folks, including now even President Trump, calling for investigations into what happens. To be clear on this, though, Anthropic would never work with ICE, at least not in the current scenario, the way ICE is operating now?
[0:14:33] We don't have any contracts with ICE, and I'll certainly say what we've seen in the last few days doesn't make me more enthusiastic. Let's turn to the economy now. You've predicted AI will disrupt 50% of entry-level white-collar jobs over the next one to five years. What does that mean for the future of work? If you have a child who's graduating high school, thinking about college, thinking about a trade school, what should the future of America be looking to right now to make sure they have a job down the line?
[0:15:02] So, you know, as you said about the essay as a whole, I'm, you know, I'm concerned and I'm also hopeful. You know, I think the concern part of it is we've seen technological disruptions before. You know, people went from farming to, you know, factories and factories to knowledge work and the computer and the internet caused lots of disruption. And, you know, that caused disruption for, you know, that caused disruption for a time and then people adapted. My concern with AI...
[0:15:29] It's not different in kind, but the disruption is maybe deeper. It's coming at us faster, right? AI can do a wider range of things. My concern, as well as my excitement, is AI can do a wide variety of knowledge work. So it can do entry-level law work. It can do entry-level finance. It can do entry-level consulting. And so my worry is exactly, if you're someone kind of coming up in the world and just starting your career,
[0:15:58] AI is coming at multiple points, and it will make people a lot more productive. But I think that we can't deny that it will also eliminate jobs and probably a large number of them. And so what we need to do is, as fast as possible, adapt people to using AI and find ways to create jobs.
[0:16:18] Faster than we destroy them. You know, I don't think there's a guarantee that we can do that. But the better we can do with that, you know, the more we'll reap the benefits and the less we'll have to deal with the harm. So, you know, to get concrete, my advice would be one, of course, learn to use AI. That's got to be number one. Dario, I could talk to you for three hours. We only have about a minute left. What keeps you up at night and what gives you hope?
[0:16:40] Yeah, you know, I think the thing that keeps me up at night is the incredible market race between these players, right? That the competition is so intense. And, you know, even we feel it. You know, we hold on to our principles. We hold on to our values. But, like, that pressure is always there. We're holding on despite that pressure rather than because of it. And what gives me hope is what I ended the essay with, which is that, you know, if you look throughout history, if you look at, you know, the situations,
[0:17:10] I found myself in and the situations humanity has found themselves in like there's so many times where it's you know very hard and there's this enormous suffering and yet there's also this incredible you know this incredible this incredible inspiration that kind of you know the the ingenuity of the human spirit that that leads us to you know find our way out of these these difficulties that you know you wouldn't you wouldn't think there was a solution to so you know I'm I'm trying to channel that every day as best I can.
[0:17:37] Dario Amode, we thank you for your time. You can find the essay, The Adolescence of Technology, on his website, darioamode.com. And of course, his company is Anthropic and Claude. You're probably using it right now. Dario, we thank you for your time. Thanks for being on Top Story. Thanks for having me, Tom. Yeah. We thank you for watching. And remember, stay updated on breaking news and top stories on the NBC News app or watch live on our YouTube channel.