[0:00:00] This is the Reason interview with Nick Gillespie. My guest today is Jack Clark. He's the co-founder of Anthropic, and he's one of the most influential figures in AI policy. We are going to talk about the government clampdown on Anthropic in particular and various things going on with that. We're going to talk about the power of AI, the perils, the pitfalls, pretty much the whole ball of wax. So, Jack Clark, thanks for talking to Reason. Thanks for having me.
[0:00:26] Let's start in a broader sense. Talk about what you see as the main promise of AI. Right now, we are definitely in a technological dark cloud moment, where people are freaking out over AI. And you guys at Anthropic are freaking out on a certain level that we'll talk about. But just broadly, in a general way,
[0:00:54] What's the biggest selling point of AI? I'd split it to three different layers. There's stuff for the individual, stuff for what you might think of as the economy, and stuff for science. For the individual, today, we have a universal teacher, which is something that people fantasized about building for decades and decades, and it just exists. You can go and spend $0 a month to talk to an expert teacher, and if you spend $20 a month, you can talk to a really good teacher. It's incredible. That's for the individual.
[0:01:24] For the economy, the economy, especially in the Western world, is defined by back office work and bureaucracy, or what David Graeber called bullshit jobs. We now have a system that can deal with, navigate the bureaucracy and back office stuff, do it better, in some cases get rid of it, and allow people to get back to the parts of the jobs which either use their skills or have more meaning. Put a specific example on that. Yeah, I'll give you an example. We worked with...
[0:01:51] Closely with the makers of Azempic to help them do work on the back end for how they processed and formatted results from their clinical trials. And we took it down from something on the order of two months of work to a week of work. I don't think anyone misses that. No one there is crying at their desk because they can't do that. I was the best back office paper shuffler. Damn it, that was me. That was what I did. And examples like that abound everywhere. There's someone I work with at Anthropic who got so annoyed at...
[0:02:21] Processing a certain type of invoice, but they just fully automated that. They're not sad about it. Everyone's happy, but that's automated. And then for science, we now have systems that, if you read the literature, are co-creating science at the frontier of biology, mathematics, physics, medicine. You are seeing the world's expert scientists co-author papers with Gemini, Claude, ChatGPT.
[0:02:48] This is a new era for scientific discovery. Not Grok, right? Nobody serious is using Grok. I may have seen it somewhere, but there are other names that come to mind as well. Right. And this is also one of the examples that gets talked a lot about, which seems amazing, is that when you think of something like x-rays for...
[0:03:07] Tumors that might be cancers or whatever, that suddenly the entire world becomes a database and you have computers training and looking at all of this and just coming up with better diagnostics and prognoses and things like that. Much better diagnostics, much better prognoses. Also, on an individual level, we've now replaced WebMD, which always tells you you've got cancer and are going to die, with AI systems, which often tell you something a lot more reasonable, factually accurate, and also say,
[0:03:35] Probably call the advice nurse and tell them these things. Right. I'm forgetting the name of the disease that somebody made up that has gone through, like it's in every AI model now, Baxillism or something. I guess, just quickly before we move on to more policy issues, how does AI work to filter out?
[0:03:58] Kind of bullshit or mass delusion because this is always the problem, right? So there's a couple of things here. Years ago, AI systems used to hallucinate. They used to make stuff up a lot. And it's basically because we train AI systems to be like a contestant on Jeopardy. They want to hit the buzzer. The moment you say something, they want to answer. It took a lot of work to say to them, it's totally reasonable to hit the buzzer and say, I don't know.
[0:04:25] Thank you so much for having me.
[0:04:54] Like a friend, something that will actually push back on you a little. Now, this introduces many, many rich and interesting questions about what's appropriate or inappropriate there, but we sort of know it when we see it. We know it when we experience obsequiousness or sycophancy in our own life. Talk a little bit about military applications of AI, technology, historically, certainly going back to people like Galileo and whatnot, technology, technological innovation, and defense.
[0:05:23] Broadly, kind of loosely defined has always been a big part. This was part of your early contentious intersection with the Trump administration. How is AI going to change kind of military and defense applications? Well, there's one way where it's exactly like how it changes everything else. And there's another way that is unique to this kind of genre of work.
[0:05:46] Often a joke about military organizations is they are logistics companies that have a sharp bit at the very end. But logistics stuff is the same back office stuff I've talked about for everything else. AI can just be wildly helpful there and help automate and improve decision making and back office schlep, of which the military and other organizations famously do tons of. I mean, this is catch 22. The military is the bureaucratic system that exists to kind of
[0:06:14] Punish or torment the people who it supposedly is serving. My brother was in the British Army as well, so it sounds familiar. Yeah. At the same time, AI can introduce new capabilities at the frontier. You know, the advancement of science creates new things that can be used by the military. An example outside of AI is, you know, hypersonic missiles. So we invented missiles.
[0:06:38] Now we've invented a much faster missile using advances in material science, which is much harder to shoot down. Also, there's the example of things like drones and other stuff. I think that we can just generally expect AI to create new capabilities at the frontier of the military. And those capabilities, like previous technological creations ranging from nuclear weapons to hypersonics to cluster munitions to biological weapons.
[0:07:03] Are going to be subjects of deep debate among militaries, among governments, and among citizens. Is there a, not first the market per se, and obviously AI has already kind of permeated the major powers and many minor powers, but is it going to be something like if we can get a nuclear bomb first, whoever we is, then we can stop the creation of further...
[0:07:29] It's most helpful to think of AI as being almost equivalent to maybe electricity or oil in the sense that it is both the thing and the production and management system around it. And that delivers these huge range of commercial, civil benefits.
[0:07:56] But it has profound uses in conflict as well. And it also introduces profound weaknesses in conflict. Additionally, the most refined forms of AI, really, really, really smart systems, may have properties that are akin to handle with care technologies like nuclear weapons, where you need to find a way to proliferate there.
[0:08:18] And this was basically the crux of Anthropic's confrontation or conflict with the Trump administration recently, right? Can you explain that? Because this seems to be the essential backdrop to the current limitation on Anthropic's ability to kind of...
[0:08:44] We've been longtime supporters of working with the intelligence and defense communities, and Anthropic has also always sought to talk about the full range of issues that lie ahead of us with this technology. At the moment, what we're grappling with is we have a technology, these models called Fable, that is intensely useful in a bunch of commercial ways, and it has some potentially
[0:09:11] National security or defense relevant properties in cyber and bio. We've worked to find ways to make it hard for those capabilities to proliferate. And we're now in sort of daily discussion, and I obviously can't get into the specifics, with the government about what a sensible policy framework is for how you can both proliferate and export these models without proliferating and exporting the thing that seems more profoundly useful for national security.
[0:09:39] Take it back a step. I mean, it was, what, a few months ago when Pete Hegseth, the Secretary of Defense, I keep meaning, I don't want to call it whatever he wants to be, the Secretary of War, Secretary of Defense, but he was saying that Anthropic wouldn't allow for the way he wanted, or the Trump administration wanted to deploy certain types of AI.
[0:10:02] And you guys in the public narrative were insisting, no, that certain kinds of kill decisions cannot be relegated to AI. That contradicts your corporate philosophy. And the Trump administration threw a shit fit about that, right? We outlined two red lines. One is about the potential use of AI for domestic mass surveillance of Americans.
[0:10:26] And another is the use of AI for fully automated weapons. The reason we outlined this is not because we think that we are the ones that can define what's appropriate there. That's absurd. But because it's clearly an area where there needs to be a wider societal debate, there needs to be some notion of what we think is right as a set of individuals, citizens of this country.
[0:10:49] And we remain standing and ready to work with the U.S. government in all ways that can be helpful, while also we believe that that discussion about those red lines is something that is beneficial to have. Where does that argument stand right now? Because the government wanted to put you guys on a list of bad corporate citizens, right? Is there any discussion still going on related to defense?
[0:11:19] We're in regular discussion with the US government, and it's important to note that we are constantly finding ways to support the government and work with them. And I think that the broader thing that's useful to focus on here is that these models from us are not special. There are other models from other providers that are going to exhibit the same qualities. And our role, now that we've sort of outlined our views,
[0:11:48] We believe it's for there to be a broader discussion and debate had in the public sphere about this and what is appropriate and what makes sense. Right. I mean, it's hard kind of sitting, reading the news. And it's like, OK, so you guys get into a fight with the Defense Department over the parameters of a contract that you guys had signed.
[0:12:09] The Secretary of Defense and the President says, no, you have to give us this. We don't care about whatever contract there was. And then you release Mythos 5, or you're bringing that to market, very powerful. And you say, hey, we're going to pull that back, and we're going to release a slightly slimmed-down version for general release, Fable.
[0:12:33] And then the government says, yeah, you know what, you can't release that outside of the country, which is effectively saying that you really can't release it. How is that not just obviously payback from an administration that acts kind of like a criminal gang, where you pissed in the punch bowl, and now this is what you get? I can't talk too much about the specifics here. It's obviously a live discussion that we're having. But I would say,
[0:13:02] This is a new technology. It has new capabilities. I actually think that these capabilities are deeply surprising, and they would be surprising to any administration. And here's my view of, and where I have huge empathy for people that work in the administration is, if you've spent years with a national security community that is developing and fielding capabilities around, say, cyber.
[0:13:27] These capabilities come from a whole bunch of contractors you work with. They come from a system you understand. And then here are these companies that are building something called AI.
[0:13:36] AI is a civilian technology. And one day the AI systems that have been sold as a civilian technology are used by civilians and are wildly successful within the American economy. Start popping out things that your colleagues in the intelligence agency say, hang on, these things look like the things that we work on over here. How do you deal with that? I actually think this is a really challenging question. And what we're doing right now is the messy way of dealing with it, which is you're saying basically...
[0:14:03] This is a lot. We need to figure out how to slow this down and get our arms around it. But it is a debate that would be had under any admin. And I think what we're going to need is some kind of policy framework for how you bring these systems out without bringing out national security capabilities or controlling those capabilities such that we can be comfortable with their proliferation.
[0:14:33] Encryption, and particularly something like Phil Zimmerman's pretty good privacy or PGP, when that got disseminated and suddenly everybody could encrypt everything, people freaked out. The Clinton administration at the time freaked out and was like, this is ammunition, we're going to regulate it as if it's ammunition.
[0:14:54] We still have these conversations about encryption, where it's like, the only people who want to encrypt things are criminals, obviously. Are there lessons from the 90s that we should be reminding ourselves about? There are also lessons from the late 90s or early 2000s, where Sony made a console called the PlayStation with something called the Cell Processor. I believe that we briefly wanted to control that because it was an incredibly powerful supercomputer inside a games machine.
[0:15:23] I think the lesson I take from it is your first contact with something which blurs the line between civilian and national security borders is always messy. And when things emanate from the non-defense or national security world, your initial response is, let's take a minute and figure this out and let's control it. And then your longer term response is, okay, we now need to balance.
[0:15:45] Like individual sovereignty and the ability to sort of like protect and secure the nation. And in both the case of computing and encryption, we've ended up in places where people have broad access as individuals to private encryption and broad access as individuals to private computation. And we've also figured out ways to kind of secure or manage some of the more national security relevant aspects of this. So I think that's...
[0:16:11] Roughly, what's going to happen in the longer term? So if we talk about Anthropic, you guys were about to release Mythos. And please correct me if I'm getting the names wrong and things like that. But you're going to release Mythos. You kind of do internal security checks on it. And your main kind of hacker in residence says, holy shit, this is too powerful.
[0:16:36] You kind of pull it back, and then you release Fable, which has many of the capacities, but it stops short of the full range of mythos. You, as a private company, made the decision to do that. Isn't that the way this should be governed? Why should you then have, whether it's Pete Hegseth or Howard Lutnick or Donald Trump, who doesn't even use email?
[0:17:04] Why are they the people who get to say what you can and cannot release? It's never going to be only companies making the judgment calls about a technology this powerful. It is going to be a shared exercise between not just companies and government, but also other parts of society like the broader scientific community. We're in the messy part right now, but I think that the job of companies, as we do,
[0:17:29] It's to communicate about what they see and run basically different forms of release experiments. And each release experiment generates data that we as a community can use to figure out what correct is. I think the role of government often has been to look at what industry does and then figure out what the de facto norms and standards are. And if any of those norms or standards are sufficiently sensible and good for society that you should enforce them. Seatbelts might be a good example here.
[0:17:57] We should probably ensure that people don't put CFCs in fridges because that seems dangerous. That's sort of the role. That's where I think that we'll end up. And our role, as I see it, is the benefit of being on the frontier is we get to see this stuff early and we get to talk to the world about it. And our responsibility is to share what we see and then we'll figure out some system.
[0:18:21] Mark Andreessen, venture capitalist, creator of Netscape, the internet poster boy of 1995, because Netscape, I guess, was generally considered the first pure internet IPO. He talked about the reason why he and many people in Silicon Valley supported Donald Trump this past election was because the Biden administration had said, independent of defense stuff, it's like, hey, we're putting a lot of
[0:18:50] Borders or boundaries on AI. Trump said he wasn't going to. First off, was that accurate in your experience? Was the Biden administration generally less receptive to kind of the proliferation and development of AI than the Trump administration? This wasn't really my experience. I'd say that there was any...
[0:19:15] Any different administration has a different set of interests and priorities. And I'd say for the Biden administration was just sort of like somewhat more focused on aspects of maybe safety and security challenges of AI systems. The previous Trump administration had been focused on some of the security aspects of cloud computing, for instance, and that had been carried through by the Biden admin.
[0:19:38] The Trump admin has been focused, I think, rightfully on, okay, economic security is national security. How do we both push the sector forward, but also sort out things like the power stuff, and also find ways to ensure that we actually successfully build semiconductors in America, which is something that started under Biden. And then there was perhaps, at least initially, a bit less emphasis on some of the security stuff. And then as the security stuff has arrived, well, they've raised the emphasis of it.
[0:20:07] I, for context, before founding Anthropic, I also worked at OpenAI. So, I've worked on AI policy under the end of the Obama admin, the whole of the first Trump admin, the Biden admin, and then the Trump admin. And my experience is there's actually tons of continuity between all of these admins around cloud computing, supply chain, and this question of how you test and evaluate models. And then there are just different forms of emphasis along the way.
[0:20:32] But we are now in an era where the government seems to have moved to a new level of flexing regulatory muscle over AI. We're in the era which many people working in AI have long assumed would arrive, which is the AI systems got not just scientifically interesting at domains that you might care about, but practically interesting.
[0:21:02] are going to have cyber properties, but are genuinely useful. That just presents a new challenge that we have to deal with. And it makes total sense to me that the government changes its attitude in response to that, because you've gone from the theoretical or academic to the practical in here and now. So one thing that Anthropic has been very clear about over the course of its existence is that it kind of believes in government regulation, right? I mean, you guys are not...
[0:21:30] The wired masthead circa 1996 or something, saying, you rotting hulks of old governments, go fuck yourselves. What's wrong right now? There's somebody like Bernie Sanders who says, these are very nice AI companies you have here, and we want to take a huge chunk out of them and spend it the way that we see fit and regulate you. You have a Trump administration in the name of
[0:21:59] It's certainly not free markets, but national security saying, OK, we want to control this, and we're telling you now you cannot export these sorts of things. What is wrong with this? Isn't this the world that you dreamed of having, where the government was taking a very strong interest in regulating what you do and how you distribute it? We have this joke at Anthropic where, when I was a teenager, I worked in a bookshop, and I thought to myself,
[0:22:27] I have to get a real job because I'm extremely bored. And boy, did the monkey's paw curl on that one. And now I'm really seeing the fruits of that. What do you think the response of the world is to probably the most powerful technology ever built by the species? It's not going to be nothing. It's going to be a highly politicized response. It's actually appropriate that the way you govern the most powerful technology that I believe may ever be built by our species,
[0:22:57] Can't be nothing. It can't be we use only the existing rules and norms that we have and we apply no new governance technology to it. I just can't reconcile that. I mean, there's a lot packed into the idea that this is more transformative than fire or the wheel or technologies that developed without.
[0:23:17] You know, kind of top-down regulation, or pre-civilization. The most powerful technology developed in the modern industrial era. No, no, no. I'm not trying to be pedantic or anything. But there's a lot riding on that question. But then, okay, let's assume that. Then, what's wrong with the way the Trump administration is treating Anthropic right now?
[0:23:42] Well, the way I'd put it is, like, there are all of these specifics to this situation, which I can't really talk about. It's a live situation. But the broader thing is, the government has said, oh, these systems have national security properties and we need to figure out how to deal with them. That's good. That's a good thing. You know, Bernie Sanders and others on the left are saying, if the AI companies are right, they might create, like, vast, vast amounts of wealth and also massively centralize that wealth into a small number of entities. What are we going to do about that?
[0:24:11] That's a good thing. It's interesting, though, that he was saying that 10 years ago and 100 years ago when he was in his teens. Maybe now the evidence has arrived more. But my broader point is, I think if you look at the history of policy debates and how change happens, it always starts off with a load of people having different ideas, radically different ideas of what you should do, getting very interested in it. There being forms of disagreement and debate.
[0:24:38] And then, you know, there's this phrase in policy of I think it's called the art of muddling through. And what you actually end up doing is typically you end up muddling through. You don't do any of the things that are like wildly extreme. You come up with regimes. Are there any elected officials now? Because it's also, you know, the Trump administration is one part of the federal government. Congress, you know, and obviously has constituent parts are others.
[0:25:04] Are there elected officials that you think are articulating the best way of having these discussions? You talked about the public sphere. That includes government, but it also includes media and things like that. But are there elected officials? Because it seems to me the Sanders model is insane. The Trump administration, where it's saying, if Sanders is saying...
[0:25:28] We want to take a 50% one-time tax on whatever that means, and then spend that money. But Trump has already shown that he's willing to say to companies, you have to pay me some kind of vig or some kind of protection money in order to export or import things. Neither of those seem very good ideas. Are there people out there in Congress who you think are...
[0:25:53] Talking more sensibly about what this is. There's an AI policy bill working its way through Congress now from Obernolte and Trajan that has a bunch of sensible ideas in it around transparency, third-party testing and verification. It's a long bill, thicket of stuff. We don't agree with everything in it, but it's got a broad set of things where you look at it and you're like, this seems very reasonable. There are bills working their way through Congress around export controls, preserving...
[0:26:18] National security for not just us, but Western countries by working on the control of compute that we might otherwise ship to China. Broadly sensible. The Trump administration put out an executive order a few weeks ago on how it might approach some of the national security questions of AI systems. The architecture described in that executive order of testing and validation, coming up with shared frameworks, broadly sensible. I think that eventually we'll actually get to some system like that outlined there.
[0:26:47] Yes, there are situations that happen that have a lot of heat and light attached to them. But I think if you look beyond them, there's the policy machine across both the admin and Congress and the intelligence community is processing this technology and coming up with relatively sensible, implementable ideas here.
[0:27:08] And if you look at the states, many states have implemented basic what you might think of as transparency bills that will be applied to AI systems, which functionally mean that my AI systems now get regulated as much as like a packet of potato chips.
[0:27:22] Which is better than nothing. It means I have to put some ingredient details on it and talk about how I made the chips. They're also not saying you can sell them or they're not regulating the number of calories or the percentage of fat. Yeah, they aren't doing that. A lot of this, our philosophy is you need to start with mandated transparency from the companies about how they've tested their systems and what they've done. And then you need some form of third-party validation that someone checks the homework of the person.
[0:27:48] And through that, you're going to build this giant base of evidence for society of are there any things that we actually need to mandate? It seems likely to me that at some point on cyber or bio, you might develop some very clear thresholds that become mandated, similar to how if you want to go and buy explosives in the world today, you can buy certain forms of explosives as a civilian like you or I. Others, you need to be a construction worker with a license, and others, you need to be a member of the military.
[0:28:15] Broadly reasonable. Has some liberty elements, but we found our way to something sensible there. But you don't want to have to deal with 50 state laws, right? You want there to be a federal approach. That's why we're excited about some of the bills working their way through Congress, because otherwise, you will end up with a patchwork, which no one wants. Or California. I mean, this happened for auto emissions and things like that. One state gets to de facto set an actual standard, and it might be more
[0:28:44] Ideologically extreme one way or the other than it would be from the national government. The whole way I'd put this is, AI is an incredibly powerful technology. Pressure is building up in the system about what to do about it.
[0:28:56] And the longer that the answer is nothing, the higher the chance that different parts of the policy system say, nothing ain't good enough, we're going to do something. So, you know, one of the things to think about is, you know, this is something where libertarians and socialists, and I'm thinking particularly of the socialist historian Gabriel Kolko.
[0:29:17] He wrote a famous book-length treatment of the progressive era and railroads. The progressive story about railroads is that railroads were these dominant, super-powerful, politically-connected monopolies or cartels. Finally, progressives gained enough power where they regulated the railroads, and everything was good.
[0:29:41] Socialists said, no, that's actually wrong. It was the railroads when they reached a certain point of power, and they also realized that...
[0:29:49] The market, it's going to get tougher and tougher to have profits. They went to the government and said, hey, we need to be regulated. There was a moment with social media, or the tech sector more generally, in 2018, where Mark Zuckerberg said exactly that to Congress, where Tim Cook of Apple said, you know what? The tech sector isn't working anymore. We need regulation.
[0:30:11] From a libertarian or a coccolian socialist perspective, how can you assure me that this isn't what's happened? There's a half-dozen big AI companies who are suddenly saying, you know what, it would be a damn good thing for the citizens of America if we had national regulation.
[0:30:36] That preempted all of these problems. How is that not just locking in your market position right now? Well, I've said the same thing for 10 years. I've testified in Congress for 10 years. I've said the same thing for 10 years. We started saying this the moment we started Anthropic, when we were a company with no product, no revenue, nothing. Generally, it's usually put around a billion-dollar valuation, right? I mean, you're private.
[0:31:04] I think it's reported as a trillion dollar. A trillion, excuse me. Yeah, that's a minor difference. I'm a lit major. I'm not a math guy. These things happen. Thank you. Yeah. So, look, absolutely, I totally understand this frame. I'd say, for us, look at just our consistency of action statements and beliefs over time. You know, Dario, I've worked with Dario, the CEO of Anthropic, for many years. We were both at OpenAI together. He's said the same things for 10 years as well. And the approach which we've always had is,
[0:31:33] We believed and continue to believe this will be the most powerful and consequential technology that gets built in this century for sure. And the answer to how you deal with it must be that we need to create new tools for us to govern and manage it. And we need to integrate it with our existing laws. And we need to integrate it with some kind of system by which...
[0:31:55] It can't only be companies making decisions about it, because the technology will be too powerful and too influential. Can you outlay my worries that, and this is something that Zuckerberg said to Congress, that I will help you write the regulations to control social media, but you realize that means that there will never be another Facebook. But is that the price you're willing to pay? And obviously, it got a little bit squirrely for a lot of different reasons.
[0:32:22] If Anthropic and OpenAI and a couple of other companies get to write the rules,
[0:32:30] Does that mean, okay, these are the five firms or whatever that are just going to dominate forever? We've shared our ideas for regulations, but I really welcome and encourage people like you and others to write your own take on it. I think you've seen people like Dean Ball, who was briefly a misadmin. He does a huge amount of public writing on what he thinks a sensible policy approach is. Oren Kass has done similar work.
[0:32:55] Not a free market. The point I'm making is, the more people you have outside the companies articulating a view here, the better it is for everyone. Because you want a big pile of different ideas around how to approach this. And in some of the ideas, there might be commonality. Do those ones. If you can build me a table that has companies, libertarians.
[0:33:20] And there are some ideas where everyone agrees. Maybe those are sensible ideas to do. And if there are ideas that only the companies propose, be really, really skeptical of them. Because there's probably a reason the others didn't propose it. Talk about when we're dealing with issues of governance. Okay, so it's the U.S. is one thing. The E.U. is something else. And I think generally speaking, certainly most people in America would believe this.
[0:33:47] I think many Europeans would. Their approach to technological governance has been bad. It's tamped it down, and Europe is further behind than it was. They've done regulations ahead of their own capacity to have the thing itself. Then you have places like China, which are like, we don't care. How does U.S. governance end up ...
[0:34:11] How does this play out in the best possible scenario? The best possible scenario is something that happens millions of times a day, which is cars get exported between all of those markets, maybe with the exception of the U.S. and the car case to one another. Planes take off from airports and all of those places and land in the other ones.
[0:34:38] And you somehow have confidence. Oh, food gets exported. You have confidence in the cars, planes, and food. Why is that? It's because you have some notion of shared standards, shared testing, shared means by which you can validate that the thing has the safety properties needed for it to be exported and imported. And rather than having some kind of global governance regime, you instead have a bunch of standards bodies in different countries which basically have reciprocality. And I think that this has just worked.
[0:35:06] Amazingly well for commerce. I buy a lot of baby toys now as I have a three-year-old and a seven-month-old. It's great to have certain labeling standards because my wife likes to buy organic food and also doesn't want toys with lead in them. I understand, but there's actually labeling schemes that let me do that. A lot of those are voluntary, right? Or you know if you buy something at Ikea, a great Swedish company, it's less likely to...
[0:35:35] Many of these are voluntary, like the Energy Star thing from the utility industry. But then some of these get codified as just hard standards. There are certain hard standards for automotive.
[0:35:48] airline security, which just baked in. And then there's a lot of stuff where it's voluntary and we compete. But that's just how the world figures this out. It figures out, what is the core set of things which just has to be true? I am worried in the sense of, when you were talking about the commerce, and one of the great success stories of the post-World War II era was a general reduction in tariffs and trade exemptions, as well as capital requirements.
[0:36:16] If it's not coming to an end, it is very much under attack by this administration, by a Republican administration, as well as Democrats like Elizabeth Warren or Bernie Sanders types. That worries me, because Trump has made it clear he doesn't want particular vegetables or people from certain countries coming here, and there seems to be broad support for that.
[0:36:41] I worry about that. But if we may, let's talk. I want to ask you about recursive self-improvement, which is a big focus of material coming out of Anthropic. Can you explain what that is and why this is the kind of the big thing that we should be talking more about? So AI moves forward by people gathering resources like computing data and using relatively well understood algorithms to train.
[0:37:11] Well, these systems have started to get really good at also writing code for training AI systems. They've started to get good at proposing ideas for how you might train AI systems. There may come a point where...
[0:37:32] We as people can step back entirely from their forward development, and it's sort of a case of wind them up and watch them go. We give them the resources and we say, okay, Claude 10, build Claude 11. And Claude builds the architecture, does the research, de-risks it, does the training run. Claude 11 comes out. It's better in every way than Claude 10. We don't see that today, but I think that there's real potential for this to arrive.
[0:37:58] Certainly this decade, but if I had to bet the year, it would be 2028 towards the end. And that's based on reading a lot of the scientific literature and also we published information. 2028 is like, I mean, you said you have a seven-month-old. I have a six-month-old. They'll be like a couple of years older. They'll be a couple of years older. I mean, 2028 isn't even two years away. It's quite close. Yeah. AI moves quickly. And what the implication is of that is we should expect AI systems to...
[0:38:24] Move even further in the future, grow even more powerful than they have so far. We should expect the progress we've felt in the last five or six years to actually maybe be compressed into the next two or three years and then compressed again. Are there other examples of that at the surface level where a technology has
[0:38:45] Just the improvements of the technology have accelerated that much. There's basically no example. The only good example is evolution of biological creatures, and it takes millions of years. But you can make better dogs. It just takes you many generations. Probably the best analogy I have is, imagine a 3D printer that could print a finer resolution printhead than the one it had.
[0:39:10] That would be amazing, and it would yield an amazing burst in productivity of what you could manufacture and make. That's kind of what we're talking about here. How do you put hedges on that? How do you build in breaks? Because this is part of the doomerism and the apocalypticism, is that AI realizes, it's not just that it doesn't need humans or shareholders, but they're in the way.
[0:39:36] It's a harder problem, but of similar form to what we're currently dealing with things like cyber and buyer, which is you have some property of the system that you're worried about. Well, you need to know how to measure it. You need to know how to potentially control it, make it so that the system can sometimes not be capable of AI R&D. And you need to probably demand transparency out of the places which are building this, because you as society might say,
[0:40:02] This is great, and it's going to drive loads of strategic and important stuff in our economy and national security. We probably don't need it to go thousands and thousands of times faster than any human can hope to govern it. You need some measurement property of it. A lot of the work that I'm doing at the Anthropic Institute, which is a think tank with a supercomputer inside Anthropic, is studying...
[0:40:25] AI research and development within Anthropic and figuring out the measures that we might use to spot recursive self-improvement if it were happening. And eventually, you want to share those measures publicly and socialize them with policymakers similar to bio or cyber and say, hey, we're going to need to figure out a framework and a set of thresholds around this because this is how we can make science go orders of magnitude faster than today. It's how we can intentionally...
[0:40:53] Bring benefits to today that will seem like miracles. Novel therapies, incredible advances in material science. And the same would be true of national security-relevant capabilities. And you're going to need to figure out what the control layer is on that. Is there broad buy-in from other major AI companies to kind of this type of sharing of standards and transparency? Because it's also, you know, like, I've got my...
[0:41:20] I've got my black box up here. I'm not giving you the Coca-Cola formula. We said recently in a post about recursive self-improvement, we think it would be really good for the world to have the option to potentially slow down the pace of progress. If you hit something like RSI, you might want to be able to intentionally slow that down to give us time to adapt. Who is we in that? I'm not trying to be a libertarian, although I fail at that on an hourly basis.
[0:41:50] To be like, who's we, Kimisabe? Anthropic, so far, has acted like a good corporate citizen. You delay the launch of something that you think is too powerful. OK, so that's good, but then what about other companies or part of a government that's like, no, we've got to go for broke right now?
[0:42:13] All we can do is put these ideas out there, socialize them, and I believe that if it's truly important, the world will find ways to come together to coordinate around that data. I think that the world has done this in the past, and it may need to do it in the future. What are the best examples? I think CFCs is one of the best examples. We were like, oh, okay, all right, no one actually wants the hole in the ozone layer. Manufacturers, scientists, governments, let's just stop this. It's great. That was like a great, amazing example of global coordination. And it's interesting that very few...
[0:42:42] People talk about that. The big loss was that McDonald's started wrapping things in wax-coated paper rather than using their clamshell foam.
[0:42:52] Container. The world cried a single tear, but came through it. Can we talk a bit about your background? The work that you're doing at the think tank and policy tank and whatnot at Anthropic. You went to university at East Anglia, England, which is famous for its creative writing program. You studied creative writing and literature. You worked as a tech journalist at Bloomberg and other places.
[0:43:21] Oh, God, is it The Register? It's like a libertarian tech rag. It's tech language biting the hand that feeds IT. It's just wonderful. God, I've been reading that forever. One of the things that people say a lot, or I hear a lot, is that Silicon Valley needs more philosophers.
[0:43:48] I think if you're a journalist and you have a background in reading widely, you recognize that all technologies have political and social elements. All technologies ultimately come along and they change how people behave and they change the structure of the world. And I think probably Silicon Valley spends a lot of time rightfully focused on building stuff because it's really...
[0:44:12] It's really fucking hard to build stuff. You put a lot of effort in. We kind of take it for granted, really. It's really hard. Over the past 50 years, it's pretty amazing. And some of the lessons of the last 15 years is in that total focus on building, they sometimes forget that they're also shipping a social and political technology into the world. Social media is a great example where...
[0:44:31] Ran an uncontrolled experiment on hundreds of millions, billions of people around the world. Is the world better off, though, because of social media? In many ways, it is. But I feel like we could have better information, and we could have dealt with some of the issues like, say, teen anorexia or suicide better if we just shared a bit more information as it was going through. Now, who knows? But I'm just saying that's a frame that you can use. You don't stop the technology, but maybe there are ways we could have been better prepared to deal with it. Smartphones, we're just getting our hands around them similarly. So, that background...
[0:44:59] I think caused me to see technology as fundamentally something that would change the kind of political economy of society. I read an article in 2012 called Facebook, Google, Rise of the New Feudalism. And it was just by studying and touring data centers from large companies, studying the types of software that were being used to organize and analyze that data and early machine learning and realizing, hang on a second.
[0:45:25] This is like a system by which you can gather huge amounts of data around the world. You can have people working on your platforms. You can analyze the people. You can exercise unusual control. This looks a lot like something that we used to do called feudalism. And it's not a specific critique of these companies. It's an observation that technology has political qualities. So, prior to, I think, the start of this conversation, you mentioned you...
[0:45:51] I read a lot of the beats. That kind of formulation of technology, in particular, makes me think of William Burroughs. William Burroughs, broadly throughout his work, saw systems of power or systems of control that enslaved people. The paradigmatic example that's in his first book, Junkie, everything is kind of like heroin, where it takes over everything that you do, and your entire world is structured around that.
[0:46:18] Kind of how you see technology or what about the liberating aspects of this? Because, you know, when people, people who make arguments about the new feudalism, the things that they kind of, you know, just kind of wave away are the fact not just that ultra poverty, like super extreme poverty has basically vanished over the past 20 years, but that the rise of a global middle class, people who have disposable income.
[0:46:44] Relative to the countries they're in, until Homie Karras, the Brookings Institution demographer, says it was around 2017, for the first time in human history, the majority of people were living at the middle class or higher level. That has political implications, right? Because middle class people are less likely to put up with as much shit as surf.
[0:47:08] When we talk about the new feudalism, is that a way of smuggling in an understanding of technology as ultra-deterministic, that we should not accept at face value? No. It's an observation that the technology can have these properties, and it's sort of the job of the world to closely look at, I think, what the distribution is and figure out if there are ways you spin the dial. Smartphones are a great example, where smartphones...
[0:47:34] They allow you to start businesses. They allow you to communicate. They allow you to... Things like photography has had myriad amazing benefits, not least of all how many photos I take of my kids, which actually fills me with joy. It's great. And I sometimes wake up in the middle of the night and I look at Instagram reels and I waste 45 minutes of my life. And I'm like, whoa, what's going on here? Why did I do that? The world gives me the tool and it lets me use it however I like.
[0:48:01] And also we know that the tool within itself contains multiple different forms of political reality, ranging from sovereignty to
[0:48:08] Do you believe in progress, either with a small p or a capital P? How does that factor into the way that you conceptualize technology and AI? Define progress here. I'm asking, do you believe in progress, or how do you define progress? Is it technology good because it makes everybody wealthier, or is technology good because it makes individuals freer?
[0:48:36] I fundamentally think that technology is a tool by which humans can greater enhance our understanding of the world and our understanding of ourselves and each other. And I think that the human species needs a lot of powerful technology to get through this century. We need to improve our energy systems. We need to improve government. We need to improve all kinds of stuff around us. And digital technology has been especially helpful for some of this. And AI is going to help it expand into the physical world and make things like
[0:49:06] Medicine, energy, material science go incredibly fast. And I think that will lead to more material abundance for everyone. And I think abundance allows you to create liberty because it allows you to create more time. But then everybody, you know, when you have more liberty, you have more choices, and then you have more responsibility, and we've kind of massified existential angst, right? In some sense, but in other senses, I think that technology can be...
[0:49:31] Very empowering for community. I lived in Oakland from 2013 onwards. There was this period in Oakland, California, where there was an amazingly vibrant and diverse DIY punk scene. Permissionless art communities creating an amazing sense of community. How did it work? Flyers on Instagram.
[0:49:53] Which said, like, there's a show at X. And it used to be flyers on the street poll. But now it had just transitioned to this new technology and was using it to not atomize people, but to de-atomize people and to bring people together. So, it always contains both. And hasn't that continued to pace? Or rather, you know, it seems to me a lot of times when people say, well, you know, the problem with capitalism broadly or technology is that it atomizes people.
[0:50:17] Well, you like this particular DIY punk scene. I might have liked it, too. Or I like Burning Man. But everything that allows you to pursue your bliss or create that community allows people the right of exit to be like, yeah, thanks, but no thanks. I'm going to hang out over here. And we often mistake the ability of other people to live how they want to live with the end of the world. Yeah. I mean, I think that one of the challenges of AI systems is you want to
[0:50:44] And in setting that border.
[0:51:02] It's basically the grand political policy discussion of our current era. That's what this whole thing is about, and it's why it's so messy. The nice thing about AI is you work on problems which humans have never had satisfying answers to. This is one of them. As we move to a close of our conversation, talk about the employment effects. One of the things that gets talked a lot about, which seems distinct in AI, is that in the past,
[0:51:31] It was always automation, or technology was going to wipe out the crap jobs. In many ways, it has. I said this as somebody in high school. I worked in factories. Factory jobs have disappeared not because of free trade, but mostly because of automation. If you've ever worked in a factory, you're like, thank God! Who wouldn't rather have a robot do something better and in a painless way?
[0:51:56] But what people talk about AI is, oh, you know, the problem with this is it's coming for upper middle class educated people. Is that accurate? There are two contradictory things to hold in mind here. Number one, the data today doesn't show this. But the data today, the starting point of the current AI revolution is also the starting point of...
[0:52:18] Right. Right. We can see...
[0:52:35] Some weakness in early graduate hiring in some sectors. That is the only thing that the evidence actually shows you. I mean, it's kind of a bad time historically to be graduating as a computer science major right now, as opposed to 10 years ago. Yeah, it may be the case. And when we look within Anthropic, we're hiring more people.
[0:52:57] With lots and lots of experience than we did before, because the returns on intuition are much greater than before, because now you don't do the schlep work to run your experiments. What do you mean by returns on intuition? It's like expertise. Yeah, if you're an experienced researcher and you have a ton of ideas for experiments to run, well, previously we needed to also give you an engineering team so that you and the engineers could run the experiments. Now Claude runs the experiments. So actually, let's hire way more people with senior intuition.
[0:53:26] So, do you worry about a kind of reserve army of unemployed recent college grads? Or is it that, as you were saying, the data so far, there seems to be a softening of hiring recent?
[0:53:43] College grads or younger people in certain positions. Overall, the sector is growing. Overall, the sector is growing, but the confounding factor here is that the software engineering sector is part of the white-hot center of the American economy that's been growing more than anything else. So, it's all confounded. But this question of pedagogy and skill formation is really important.
[0:54:04] How people get good at stuff is they start working on stuff. The implication of what I've talked about is we may have slightly fewer people starting work, and we may be taking more for people who've been working. So we recently announced something for Anthropic called Claude Corps, where we're taking 1,000 early college graduates around America.
[0:54:24] And we're basically paying to embed them in nonprofits and other organizations and teaching them to help use AI skills to help those things improve their systems. And the idea here is twofold. One, proliferate the benefits of AI to organizations that might otherwise not access them. And two, give these people some experience. And I think from that, we will be able to run this natural experiment and see what happens in the economy. Now, the second thing, which I mentioned is almost contradictory. When I talk to many of my colleagues, they say,
[0:54:54] I cannot reconcile the future economy with the normalcy of today's economy. Everything I'm working on and the pace of the technology advancement seems to imply to me, a non-economist, machine learning researcher, the potential for my own unemployment or my own redundancy. And it's almost a belief that many of my colleagues hold. Either you could say this is just a belief they hold and it's wrong, or you could say...
[0:55:19] These people have correctly called for advance of a very, very profound technology. They have correctly called how far it would advance, how much more quickly than anyone expected for multiple years. So maybe there's something to that as well. And so something I'm starting to do with governments is basically saying, how are we prepared for unprecedented things to happen into the economy that are induced?
[0:55:41] Not by a recession, but by actually the arrival of a new, very productive technology. Because it's something we haven't dealt with so much. Yeah, although this is where, and only time will tell, I remember reading an economic analysis of Elizabethan England.
[0:56:00] Early emanations or premonitions of the Industrial Revolution were happening. You didn't need as many workers. People were also aggregating in cities for the first time. Queen Elizabeth and King James did this, too, where they were like, OK, we're going to limit the amount of work that anybody can do. You have to be an apprentice. If you're an apprentice, you can only sew to a tailor. You can only sew this many buttons, etc.
[0:56:27] I am acutely conscious that everyone who predicts specific things about the shape of the future economy tends to be horribly embarrassed a few years down the line. All I'm saying, and what I say to governments, is you should plan out for a world of many different scenarios. And I sort of expect that AI might yield more extreme scenarios than ones we've had in the past. Like, it might yield...
[0:56:52] Far above trend GDP growth. And that GDP growth might be accompanied by a spike in unemployment that you typically only see during a recession. That would be weird. And I actually just think governments generally maybe could do more work to invest in the scenario planning, including for the weird cases, so we know what to do. And obviously, it's partly what your job is right now, is to be talking to governments. And also, Anthropic is embroiled with governments in a very profound way.
[0:57:20] And I'm not saying this just out of pure libertarian cant. I say this as somebody who goes to the DMV and who deals with government bureaucracies and things like that in a place like New York, which has a mediocre, at best, government. I mean, what kind of faith do you have that governments of all entities are going to be able to be the backstop for any kind of social problems that come out of this?
[0:57:48] I'm an optimist and I don't think it's going to be every government, but I've interacted with government systems in America which have been good. I paid my student loans off in England using a government system that was actually fine to use. To my surprise, I was like, oh, this is just like doing good online banking. Good job, guys. I've been to Estonia, which has a highly digitized government but works extremely well. Governments can do this. Not all of them are going to.
[0:58:14] Some can, and we will work to try and find ways to help them do it. All right. I think we will end it there. Matt Clark of Anthropic, thanks so much for talking to Reason. Thanks very much.