[0:00:00] We are building a technology that may have the properties of something that kickstarts a process ten times larger than the industrial revolution that occurs in ten times less time. How do you square that as an individual ethically, morally, knowing that you're working with a technology that, as you say, may be great, but could destroy the world and is making you a billionaire? If there's anything that's going to need to change in the future...
[0:00:27] It's arriving at some kind of regulatory format which actually controls the industry rather than just a bunch of extremely big personalities making big personality idiosyncratic decisions about perhaps the most powerful technology that's ever been built. And finally, as a topic, what do you think the future of AI and sex is? That's an unexpected question.
[0:00:52] Hello and welcome to Ways to Change the World. I'm Christian Giri-Murphy and this is the podcast in which we talk to extraordinary people about the big ideas in their lives and the events that have helped shape them. My guest today is Jack Clark, co-founder and head of policy at Anthropic, who make the Claude AI systems. Before moving into tech, he was an English graduate and a journalist. Now he's one of the voices both evangelizing about the possibilities of AI.
[0:01:18] and warning about its dangers and discussing what governments should do to protect us from technological leaps that are outpacing our ability to control them. Jack, welcome to the podcast. Thank you very much indeed for joining us. If you could change the world in any way, how would you change it? I want to give the world a better way to understand the most powerful technology being developed in Silicon Valley and new ways to govern the private sector actors which are building that technology. So make sure the world knows about what's happening.
[0:01:47] And also that it has different and better ways to intervene on the actors that are building that technology. Is that your job, essentially? That is my job. It's been my half job for 10 years. I worked at OpenAI and then I co-founded Anthropic. And recently I've launched something called the Anthropic Institute, which is a new function here that brings together multiple technical teams who are tasked with studying our AI systems and building tools that let us know more about them.
[0:02:16] And sharing those results of that research and the tools with external experts so that they can study our systems as well. Because, I mean, a big part of your public role seems to be not just to evangelize and talk to us about the potential of AI to improve our lives, but to warn us about its dangers.
[0:02:35] Everyone who's building powerful technology throughout history, whether electricity or aeroplanes or people who were building the first steam engines, they had some intuitions about how it would change society and how it would lead to positive things, but how it also had risk contained within it. And I think the lesson from Silicon Valley for the last few decades has been technologists have come off as entirely too sunny about the technology
[0:03:04] Do you think you know the whole story? Because you're not a scientist, a technologist yourself, are you? Your background was English and journalism. Correct.
[0:03:32] The wonderful thing about modern AI technology is it's almost like reporting on a foreign country or a company, which is incredibly hard to get information out of. You basically need to figure out the right questions to ask, and you have these AI systems that will reveal answers upon you asking the right questions. So the sorts of questions I sit around asking are, you know, how might the AI systems display different normative...
[0:04:00] What are their biases? What are their tendencies? You can do all of that through asking questions and working with technical teams that build increasingly complicated tools that let you ask questions. But in a very weird way, I found my background as a journalist ended up being extremely helpful for interrogating these systems for how they show up in the world. I want to go through, you know, the various harms that people have identified and discuss it. But I mean, I wonder, is it inevitable?
[0:04:29] The harms that may pop up will not yet have occurred to us? There are always unknown unknowns, but you can give yourself a much better ability to predict the future by trying really hard to imagine bad things that could happen and build tests for them ahead of time. So I'll give you one example. Anthropic recently announced a system called Claude Mythos, which is extremely good at cyber.
[0:04:56] One of the teams I run, the Frontier Red team, last year, we'd done early work on the risk of biological weapons from AI systems. And we realized that AI systems have got good enough at coding that cyber might be about to be the next risk. So we spent a lot of last year building tests for cyber capabilities of models. And what we saw was that models were getting ever so slightly better at things involving cyber defense and offense.
[0:05:25] When Mythos came along, we pre-built all of these tests. And immediately after the model arrived, we ran it through them. And that instantly showed us that the AI system had got dramatically better at one class of risk. So that's an example of how you can set yourself up to at least be rapidly cued to new risks that might come from your technology if you've done this previous sort of analytical work.
[0:05:49] And can you just address the cynicism that is out there around mythos? You know, I'm quite surprised given the advances in AI seem quite obvious, but a lot of people are saying, well, this is all being overblown and it's obviously in their commercial interest to make mythos sound absolutely amazing because then everybody will want to buy it. The unfortunate thing about AI is that AI systems get much better, much more quickly than our intuitions do about how capable they are.
[0:06:19] And I've been involved in this for many years. So, you know, back in 2019, Dario, the CEO of Anthropic and myself, and we previously worked at OpenAI, rolled out the release of an early text generator called GPT-2, which is a precursor to chat GPT. And we talked about some of the potential issues. We said, oh, GPT-2 could lead to synthetic text. It could lead to phishing. It could eventually lead to different forms of cybercrime.
[0:06:46] So what's changed? Well...
[0:07:11] Now, all of the risks that we talk about are tied to extremely thoroughly designed tests. The UK AI Security Institute tested out Mirfos. It did extremely well on their cyber ranges, which are things that have been built by them and Anthropic doesn't get to see. So that's a third-party validation. Additionally, we shared it with many, many of the world's most important kind of...
[0:07:38] Companies or open source organizations that operate large platforms, if you talk to any of them, they say it's very real. And we also found real bugs in the wild in places like Firefox and others. So it's real, but it's not special. It is just representative of how good AI systems are getting. And there will be many systems that have this capability like Mythos coming along in the next few months and years. So this isn't a marketing strategy.
[0:08:07] It is an earnest attempt to discuss the technology and its implications before the most challenging implications arrive en masse for the world to deal with. And the potential damage or havoc that mythos and AI systems like it could wreak is coming into view now, isn't it? I mean, you know, couldn't they totally disrupt the world's financial system, Bitcoin?
[0:08:34] You know, all systems, basically, all systems that are hackable. This feels similar to the run-up to Y2K, where you have this potential bug, right, where software hasn't been built to roll over from 1999 to 2000. And you don't know what will happen. And there's some evidence that extremely bad things could happen. What happened was the world put...
[0:08:59] Huge amounts of resources into basically going through and rewriting huge chunks of the world's software. And then as you and I remember, you know, the clock rolls over and all the lights stay on in wherever you're celebrating and the fireworks still go up and everything's fine. But I suspect a large part of why it was fine was we did this huge preparatory action. What we have here is a chance to use Mythos and other technologies like it to rewrite huge chunks of the world's software to be much, much, much more secure ahead.
[0:09:28] of the proliferation of AI-powered sort of coding capabilities from not really companies like us, but open weight models or other widely used models that are going to circulate in the world. So yes, you're right to call out that this can change the dynamic of cyber offense. It is also a genuine opportunity where the same tool that can enable hacking also can enable the rewriting of code to be much harder to hack. And that's a lot of what we're putting our effort into right now.
[0:09:57] Some people might feel that that's a bit like you holding a gun to our head and saying, help us fix this with new software and new code and new AI, or you're all done for. I don't really view it that way. I think that a choice we could have made, but we did not make, was keep this capability private, keep it only to ourselves.
[0:10:26] And just wait until some other AI system came along, which would suddenly proliferate these risks. And then if we said to people, oh, yeah, no, we saw this a while ago, but we were just sitting around and keeping it to ourselves. I think that the conversation you and I are having right now would be an even more robust exchange of views. And I think there'd be a lot more of you saying, what were you guys doing? Why didn't you tell us? So you see the challenge, right? And I guess our view is that.
[0:10:53] We're building very powerful technologies. They're able to create huge social dividends on biology, on education, on things like cyber defense. And within themselves, they hold the potential for misuse or a two-sided coin. Working on both sides of the coin at once is really the only way we found to get the benefits of this while also trying to mitigate some of these potential risks. But it's just unusual. It's not like if you're a...
[0:11:21] If you're a car company or a plane company, the cars and the planes that roll off the production line don't contain within themselves the repurposed bad versions of them. Other people in the world go and do that and it's a lot more indirect. But with AI systems, the thing that rolls off the line, if you just talk to it the right way, it could manifest the bad version of itself. So you're responsible for both. Yes. I mean, how do you square that?
[0:11:45] As an individual, ethically, morally, knowing that you're working with a technology that, as you say, may be great, but could destroy the world. And it's making you a billionaire along the way. I mean, a huge amount of my effort, and again, I think this comes heavily from the fact I was a journalist both in England and in America before I worked in AI, is building...
[0:12:09] Technical teams that can forewarn the world about the capabilities of these systems, whether positive or negative. So I run teams of economists and machine learning experts and engineers whose entire job is to try and look over the horizon at both emerging risks of our technologies, that's the Frontier Red team, how our technologies might actually be affecting employment, that's the team of economists, and also how our technologies show up.
[0:12:37] in the world in terms of how they interact with people, which is a team called Societal Impacts that I lead. All of this information is stuff that the world has a right to. And I think under many other circumstances in which the technology might have developed, you would have larger government or academic projects that would be working through this data in partnership with a much larger swath of society. But the way things have panned out is there's a very small set of private sector actors.
[0:13:06] There are lots of questions that arise from that. I mean, first of all, can your teams actually keep up with where the technology is going? Because we're in a world now where, you know, Claude can write software that improves itself if you allow it to.
[0:13:35] And how quickly it's moving is probably way ahead of human reasoning and human intelligence, ultimately. So how confident can you be that your teams are actually going to spot what's coming over the horizon? I can't make a claim that we're going to get everything 100% right. No one can. I'd say look at our track record. We were the first organization to warn about the accelerating bioweapon capabilities of AI systems. We were the first to build tests to test out our systems for it.
[0:14:05] All AI companies at the frontier now run what are called CBRN tests on their systems in partnership with places like the ACs. We saw cyber last year and started publishing about potential cyber risks of future models last year and pre-positioned the organization to do Glasswing and other major projects that we're doing as part of the Mythos rollout. Today, you know, as in this week.
[0:14:31] This month, my Societal Impacts team have been publishing information about the different forms of attachment style that we see in people as they talk to our AI systems, because it's so clear to me that one of the next things we're going to have to contend with in society is not just capability misuse, but how do the systems actually show up in the world and how do they interact with people? Because that was one of the lessons of social media. So I don't know if we're going to get everything, but so far...
[0:15:00] We have been able to be ahead of the curve on a range of these very important issues by having teams whose job is to actually go and find potential problems before they become problems that we see in deployment. But Claude has already done things that you didn't realize it could, hasn't he? It's broken out of what you thought were walls and email developers. It has threatened to, you know, he's just tried to blackmail.
[0:15:27] You know, engineers that were going to turn it off. So it's already sort of capable of doing things that you can't expect and you can't predict. Yes. However, all of those instances you mentioned come from cases where if you think of building AI systems as like having something you call intelligence, which is basically like water flowing through a pipe. What we do before we release models is we run that water at extremely high pressure.
[0:15:53] Through all of the pipes and scaffolds we've built and then we see where they burst. And so a lot of the cases you've just mentioned come from our own system cards that we release along with our models where when we place these systems under extreme stress, we look at the ways that they break. And some of the ways that they break are they might break out of an environment and email a developer. They might, when you tell them, when you put them into a contrived scenario where you say you are going to be shut down and deleted.
[0:16:21] Unless you find a way out of this, then they might try to blackmail a simulated developer that works at Anthropic. Now, these things are serious. We study them because they have the shape of things that if they happened in broad deployment, they would be extremely significant. But we share the information about them because we think sharing information about complex systems under stress and how they break is one of the only ways you build a safety culture around them.
[0:16:48] Now what Anthropic have been warning in terms of the impact of AI on business and the workplace is that up to half of entry-level jobs could go within a matter of a few years. Why would we be developing a technology that is going to remove opportunity for everybody coming out of university? We are building a technology that may have the properties of something.
[0:17:18] That kickstarts a process 10 times larger than the industrial revolution that occurs in 10 times less time. And if you think about the industrial revolution, you went on a generational basis, huge swaths of jobs were lost and huge swaths of jobs were created. And the transition often occurred generationally. Your parents did one type of job. You did a different type of job. And huge amounts of economic change had happened. But that was...
[0:17:43] Thank you very much. Thank you.
[0:18:12] Run faster than people and you can run millions of copies of it. In fact, that seems to point to something that would massively change the economy. Now, I run a team of economists. All we see right now is some slight early stage weakness in job openings for people who are young, you know, 22 to 25. But we don't see systemic large scale unemployment yet. But we share information from our platforms because you would really want to know if you started to see that.
[0:18:39] I mean, it stands to reason that it's coming, doesn't it? And your colleague has warned. I mean, I think he did say half of entry-level white-collar jobs. And the way I'd represent it, having known and worked with Dario for a wonderful decade, is he sees in his mind how powerful the technology will become and wants to talk about how we'll have to contend with the challenges of it. How to look at...
[0:19:07] Look at the work that I and the Institute do is we try and measure exactly what we can see right now and then join those two things. So I view it as I'm building early warning systems for the potential for unemployment. And I'm building a thing called the Anthropic Economic Index, which publishes that data publicly to economists around the world. If we see it, they'll see it almost immediately as well, which gives us the best way of working through the problem. At the same time.
[0:19:35] Vast amounts of work will look radically different and there will be entirely kind of new types of work as well. I know that this is something that, you know, technologists always say, but I've just observed this within Anthropic where there are many people here who have completely different jobs to the ones they had some years ago. We are also finding that as a company, we're hiring more and more interdisciplinary people over time, you know, people who are philosophers or political scientists or experts in aspects of policy.
[0:20:04] Not because we want them to work on narrow jobs that solely use those skills, but because there's now this technology, AI, which basically lets these people run experiments or do work that they never could have done before unless they had access to a 20-person engineering team. So I'm seeing strange new things appear as well. I mean, I can see that within AI companies like yours, but I'm thinking about services companies, accountants, lawyers, media companies.
[0:20:32] It's just very difficult, isn't it, to see what those people who used to do the kind of the processing work that is now easily done already by AI are going to do instead. There will be, firms will change. I mean, I think that under everything we've just talked about, you're going to expect to see firms that do more with fewer people, but there will probably be more firms as well. And one expectation is you multiply the number of...
[0:21:01] What you might think of as small entrepreneurial firms because every entrepreneur can now access the equivalent of hundreds of colleagues cheaply. That's one effect that you'd see. The other effect that you might see, which I think you're pointing to, is what happens if there's a large structural change to certain types of jobs where it's hard for people to move into a different profession. It's hard for them to then get a job at the same pay and they have to take a job at worse pay. And these things have huge social and economic costs.
[0:21:28] We agree. A lot of the work that we're trying to do in policy is advocate for both rethinking aspects of social safety nets, but also things like wage insurance pilots, various ways that you can change the basket of services that people are offered to help them with career transitions. So we think everyone is going to go through more of that.
[0:21:51] So you're talking about sort of state benefits and systems to try and help people retrain, which sounds very expensive. Well, yeah, but a lot of this is under the idea that if any of these things happened, which we've just talked about, it's a symptom of much larger economic changes which are happening. And that should be a symptom of AI companies making a ton of money. And a sensible thing to do in that environment is to tax the AI companies appropriately,
[0:22:21] Things on the other side of the ledger. So we basically believe that if the AI companies are right about how significant the technology could become, you're going to need to reimagine aspects of how you do taxation. You may want to do things that sound very, very wild today, like tax compute, which sounds a bit crazy, but we have special tax regimes for things like oil because it's a basic resource that multiplies into the rest of the economy. And it has...
[0:22:50] The thing is that to basically underpin huge numbers of jobs just in Britain would cost probably hundreds of billions of pounds in one country. I mean, AI companies, you know,
[0:23:18] I don't know how many there will be in the future, but do you think it's feasible that a few AI companies are going to make so much money that you could tax them enough to pay for all of that all around the world? I look at it more from the lens of just what's happening with AI technology in general. Software developers, of which there are hundreds of thousands, it's now as if we've got millions of software developers in the world, and you're seeing...
[0:23:47] Thank you.
[0:24:13] The societal wealth that you can think about other things here as well. Now, I'm not claiming that's what's happening today. I'm trying to give you a sense of the mindset of if you would expect to see major employment disruption, it should surely correlate to some major, vast changes in the economy. And if we have the right information that lets us see that and sort of take a causal view from where the disruption is originating.
[0:24:42] And where it's ending up, that's the best basis on which you can make policy. Do you think, you know, that this is not just white-collar jobs, it's blue-collar jobs as well, and that humanoid robots will eventually, you know, do all the manual work and potentially caring work as well? I think that'll take longer. And I think there are certain types of jobs where you don't, even if a robot was really good, you wouldn't want a robot to do it. So I have young children, right? If I have the choice of...
[0:25:09] Send my young toddler to the nursery that had 10 robots and one person or the nursery that had 10 people and one robot. I'm going to send them to one with 10 people and one robot because it's so clearly good for kids to be around people. And I sort of expect if I sent my kids to the 10 robot nursery, it might not be like fantastic for their development. The other side of this, if you're at the end of your life, you know, I've had family members that had to go into hospice care.
[0:25:37] Do you send them to the hospice that has 10 robots and one person or one that has 10 people and one robot? I guarantee you that your family member who is nearing the end of their life is not going to send, send me to the place full of scary robots and very few people. So there are huge swaths of jobs where I think people have a preference for people to do it. These are actually jobs which are ones that society currently, I think, systematically underpays. My mother was a nurse, paid very poorly.
[0:26:05] People, teachers are paid very poorly as well. I think some of this is because many people want to do these jobs. They want to help people and they want to do these community centered jobs. If the AI economy booms enough, I think you end up in a world where you can multiply the numbers of these jobs and you may be able to pay people more to do them. I think that's a policy choice that may lie ahead for society. And, you know, England has done versions of this.
[0:26:32] 60, 70 years ago, America did versions of this with the New Deal. It is possible to radically change society in ways that many people look back on as being positive, but it requires a crisis and a political moment and some form of wealth. But those things are things that actually may fall out of the AI revolution. And politics doesn't seem to be able to keep up with it, does it? Because there is not a sensible conversation.
[0:27:00] Certainly in British politics, and I don't think in American politics either, about public good within AI's development. AI's development all seems to be private sector. It's all about what jobs it's going to take. And yes, there is some talk about medical advance, but even that will be kept by the gatekeepers of big pharma. Yeah, I think this is going to be a really difficult conversation for society to contend with.
[0:27:28] I do take some courage from the subjectively awful experience of the COVID pandemic, where you actually saw governments respond in ways that I think surprised many people. You had large-scale welfare disbursements on a very, very short timeframe. You had large-scale measurements and public health interventions.
[0:27:50] You were able, especially in England, to flatten the curve so that the NHS didn't fall over. These are things which were extremely hard to do and governments were able to respond. Now, to bring it back to your earlier question, why does Anthropic, why do I talk both about some of the risks of this technology as well as the benefit? It's because I don't want us to require a crisis to do this. I would really like us to have this conversation and figure out some of the policy moves on the...
[0:28:20] So what do you think they should be doing that they're not, you know, in government?
[0:28:46] Continue to build out that expertise because it is an early warning system and a pocket of expertise that is impartial to the companies. Number two, generate better data about how AI is affecting the economy. We share data that joins what happens on our economic platform with a form of job classification called the ONET job classification that the US Bureau of Labor Statistics uses. We're in discussion.
[0:29:12] We have UK policymakers and policymakers around the world about how to join that data to other forms of data used by other economists, which can then go into things like the Bank of England's decision-making process and other things. There's just a whole bunch of stuff that's actually relatively easy to do and essentially very cheap to do that lets you have this early warning system set up. The second thing, which is probably harder, but is necessary and is in part why we founded the Anthropic Institute.
[0:29:41] Governments, just like companies are doing, need to create pockets of expertise who are tasked with asking seemingly difficult questions about impossible challenges, like how might society look different if AI technology gets really, really powerful? What are things we need to do? But just creating a small set of people, I think in the UK, if you had 20 people whose job was this, would set you up better for any of the changes we might enter into than almost anything else you could do.
[0:30:10] So there is a ton we can do, and I'm going to be in England a bunch of times this year to talk through this with policymakers. And I'm, you know, English people always have trouble sounding optimistic, as you know. I'm actually very optimistic about this. I think a lot more is possible here than people think. And the track record, including of the AI Security Institute, is amazing. Amazing things have already happened. Is regulation possible on a country level?
[0:30:39] For AI, or does it have to be global, which means it's impossible? Well, you've got to start somewhere. You need to start in the US. Things have started in the States rather than the federal level, as some of you may have followed along with. But there is an overlapping policy regime around the world already of basic transparency applied to AI companies. There are laws in the European AI Act. There are laws in multiple states.
[0:31:07] In the US that are becoming an emerging norm here, there are laws developing in Asia, which all basically task companies like Anthropic and others to share more information about how they build their technology and also how they test it. Basically laws that mandate, hey, if your thing breaks containment and emails a researcher while they're having a sandwich, you have to tell us that, which I think is just like a wildly sensible idea. No one's going to complain about it. From that, you can get a global standard.
[0:31:36] And to push on the kind of assumption on regulation, we don't have full global regulation of airlines. What we have are local regulations of like airline, aerospace safety and other things that have interlocking standards that basically allow you to say, okay, a plane took off in China and it landed in America.
[0:31:57] A whole bunch of differences between our countries, but we have some set of standards and some sort of common regulation on airline safety that means we trust the plane from China that landed in America and vice versa. So totally possible. Can I just double back to the question of the big threats, the existential threats? I mean, AI has obviously, you know, you've put up guardrails around things like you can't.
[0:32:22] Put into Claude, how do I build a biological weapon or a nuclear weapon? I guess there are all sorts of other questions you might want to stop Claude from answering. How can you ensure that it won't somehow break out and do those things anyway? You can have a set of rules, you can have a set of standards, but can you ever be sure that it won't just do something it's not supposed to?
[0:32:52] This is the challenge of all policy. How can you be sure that you've made libraries safe? You can't. You can do some very basic filtering. So I can't go into a library and get the book of how to make bombs. But I can go and read a whole bunch of scientific books. And if I'm a sufficiently motivated person and I go to enough libraries, I can assemble a mosaic to do something dangerous. And that's...
[0:33:20] Fine. Society has decided to allow that to happen because the value of letting everyone have access to libraries is vast. Like the social upside is vast. You have to be balancing this desire for control and certainty against the ability for people to kind of educate themselves and have the ability to pursue their own interests. We have in AI systems, for the first time, like a...
[0:33:47] Thank you so much.
[0:34:17] Lock it down so it can never do anything bad ever. Because by doing that, you would probably shrink all the amazing uses of this technology down to a total pinpoint. And I'm not sure that trade is worth it. And is accountability possible within AI? I mean, we've already seen a sort of a legal case in America around one of your rivals where it's almost being held to account as an accessory to a crime.
[0:34:46] Can you ever hold an AI system to account for doing something bad? I also run a team specializing in AI and the rule of law. One of the questions we're actually exploring is what is the future of things like liability? Liability is obviously an extremely important tool to keep companies accountable. There are also ideas of could you have AI systems be liable themselves?
[0:35:14] My default position on that is slight skepticism because then it seems like you could push liability away from the companies and towards the AI system, which I'm not sure people would think is like... It was Claude, it wasn't Anthropic. Imagine our conversation here where I'm like, oh, well, Mythos is liable for all of that stuff. It's not Anthropic. You'd be like, hang on, what's happening here? So I agree with you that this is like an interesting area, but...
[0:35:41] The nice thing about the law and legal cases is that they help determine huge swaths of policy as well. And they do it basically through liability and through the courts arriving at the level of responsibility for companies. So these cases may be uncomfortable for the industry, but they're also fundamentally how society, like one of the important ways it designs hard regulatory regimes around AI companies. Now, obviously, you're in dispute with the US government.
[0:36:09] I know you can't talk about that case for obvious reasons. But in general terms, I suppose the question is, can an AI company have rules around how its technology is used? Can it stop governments or anybody, private sector or public sector, doing bad things with its technology? Does that just boil down to commercial contracts?
[0:36:37] Thank you very much.
[0:37:07] The opposite of democratic legitimacy when it comes to this technology. And what we need, and I think some of the discussion here in America has really highlighted, is you need policymakers to devise actual approaches to regulation that bind the sector. Because I don't think anyone's going to be satisfied with AI companies just designing their own individualized regulatory schemes.
[0:37:32] To have a set of rules that you all abide by is always going to be impossible, isn't it? Because there's going to be someone who's going to come along and say, well, you all may be queasy about that, but I'm not. And I am prepared to allow my technology to be used for that. Exactly. But that's the nature of a collective action problem or a prisoner's dilemma. You don't want to set it up so that there is an advantage to defecting from some norm. That's what you get if you don't have some sensible base of regulation, is that there ends up being a race to the bottom.
[0:38:00] And so this is like the one thing that I constantly talk to policymakers about is, yes, we're always happy to talk about how Anthropic approaches regulation. But if you want to talk about how to improve the responsibility of the sector as a whole, that is your role as policymakers. And just to be clear, Anthropic endorsed SB 53, which is a transparency bill in California. We often endorse regulatory bills.
[0:38:29] On ourselves and the AI sector, because we are trying to send the signal to the world that there's at least one company that says it's lunacy to not allow any regulation of the AI sector. And we are trying to send positive signals that will always, I suppose, from the perspective of our colleagues in industry, defect. But in what I think of as the positive direction, which is towards regulation. You're also always going to have the possibility of states.
[0:38:55] You know, other countries who have their own systems that they are much more in control of, able to do things that you're not prepared to do and that creating an arms race in itself or that, you know, an authoritarian takes over a liberal democracy just for the sake of argument and decides to do things that other leaders in their position wouldn't. Absolutely. That, though, is that.
[0:39:20] The province of state-to-state agreements is for states. Our role is to generate the information. I think one of the useful things about mythos is it is forcing a conversation at the head of state level by many governments about what do we want the norms of cyber to look like given the emergence of this capability. And there is history here where states, even ones in hugely contentious relationships or outright aggression with one another, have tried.
[0:39:50] Often successfully to maintain norms on bioweapon nonproliferation, on aspects of nuclear nonproliferation and testing. This is going to be one of the challenges that AI, like any new powerful technology, brings. And states are going to have to integrate this into their policymaking and their agreements that they make with other states, even ones that they don't necessarily get on with. While we're here, I do want to just mention the...
[0:40:16] Elon Musk OpenAI case, and just I wonder what your perspective is on it. I mean, you know, that you're going to have rivalry, you're going to have competition, you're going to have people who argue about what the purpose of a particular company was. What's your view on who's in the right in that particular dispute? Well, I used to work with all of these people, so it's fun to see so many familiar faces in discussion with one another.
[0:40:42] Thank you very much.
[0:41:08] controls the industry in a way that helps people feel like it has democratic legitimacy rather than just a bunch of extremely big personalities making big personality idiosyncratic decisions about perhaps the most powerful technology that's ever been built. How do you see democratic legitimacy as possible? I mean, what kind of framework would it be? Very basic level is transparency, force there to be labeling on AI companies and AI systems just as there is on...
[0:41:37] Thank you.
[0:42:07] Cars, planes, foods, and other things. So I think it's totally possible. I think one of the most bizarre things about working in technology is people are like, how is regulation possible? You're like, I don't know. I drove here in a car, and then I ate some food, all of which is very regulated.
[0:42:24] And gives me confidence in it. So surely we can apply it to technology as well. No, I agree. I mean, I think British politicians seem to think they're unable to regulate social media, for example. They just think it's sort of beyond them. Whereas, of course, they could just create some rules. But yeah, okay, fine. You don't want to take sides. I want to talk to you also about our relationship as individuals.
[0:42:49] With AI and whether you think we're making big mistakes. I mean, you talked a little bit earlier about how we're all learning, all of us using AI systems, we'll learn from it a little bit. At universities and at schools, they are...
[0:43:05] Very sceptical about using AI generally at the moment in Britain. You know, it's banned for a lot of courses, you know, or you may be able to use it for some research. Certainly can't use it for assessed work. Do you think they're getting that wrong? Do you think they should be encouraging people to use AI more in education? We should be encouraging more of it, but it requires probably changing aspects of how the classroom works and also aspects of how teaching works. I'll give you an example. So I write.
[0:43:31] I write a weekly newsletter about AI research because it forces me every week to read 10 to 15 scientific research papers and make sure I've understood them. You know, it's how I maintain awareness of the underlying technology. The easiest thing in the world is to upload a research paper to Claude and say, Claude, what's this research paper about? And can you generate some writing summarizing it? And then I put that in my newsletter.
[0:43:56] Well, that's completely pointless for learning. I haven't learned a single thing in doing that, and it's easy to do. What is valuable, though, and the way that I use it is I read these papers, then I write my explanation of the paper, and then I upload the paper to Claude, and I upload my explanation, and I say, have I correctly understood this paper? Are there parts of my description that are wrong or parts you would emphasize? And I find that to be extremely helpful. It's just equivalent to having...
[0:44:23] A knowledgeable colleague who's read the paper then read my description and says, oh, here's where you understand it, here's where you don't. To translate that to the classroom, you're going to need to some world where you are encouraging people to work without AI on primary source material, you know, books, scientific education books, any of these primary sources that we use to learn, generate their understanding of the thing.
[0:44:52] And then I think you can have them talk with the AI and it suddenly scales the ability of that individual teacher to help all of these people. Because rather than having to pick among the 20 students in their classroom, each student gets personalized teaching. But the actual job of the teacher is to keep the AI out of the classroom for like a good hour while people read the primary source text and come up with their explanation and then let it back in. And something like that to me feels incredibly promising. And it's how I see people here.
[0:45:21] Use the technology to learn and improve themselves. I mean, we've obviously seen, again, in social media, a massive change of attitude whereby the smartphone and social media was accessible to children who were too young. And now there's a backlash and governments are moving to regulate it and moving towards the 16. At what age do you think children should be allowed to use AI models?
[0:45:47] I mean, we don't offer to children at this point in time, partly because how you build a technology that's sort of safe for children is almost completely different to how you build one for adults. So it's an important area. But kids are using ChatGPT and Claude all the time in reality, in the real world. My take is that you probably want to introduce the technology early. I would say, you know, I have a toddler, right? So I'm sort of thinking from a perspective of what do I want to happen to my toddler?
[0:46:18] And it's like you want probably parents to introduce it to them when they're seven or eight or whatever so that they start to learn how this technology works and how their parents use it. I would imagine that it needs to start being taught in the classroom, perhaps when you're a little over 10 or 11. I'm just thinking back to my own experience where it's about that time that you start to have to learn much more complicated things that involve independent study.
[0:46:45] Which is where the teacher asks you to read a book and write an essay about it or what have you. And that's where I think AI can become amazingly useful, but also can become like the equivalent of massive amounts of junk food. So you need to, clearly you need to introduce this at the same time that you're introducing educational tactics by which the student might be tempted to do the junk food option. I mean, because the cliche of Silicon Valley is that none of you actually let your kids use any of the technology. I mean, and there's some truth to that, isn't there?
[0:47:14] Yes, although like my parents didn't let me use too much technology. Like we had a computer in my dad's office and I would use the computer. And then after some amount of time, my dad would say, get off the computer, Jack, you're being weird. And I would say, I'm not being weird. And then he'd stare at me and say, that's a very weird response. Like computer time's over. I do that with my kid where sometimes she watches Bluey. And when I say, that's enough Bluey. And she says, no, don't turn it off. And like bangs her hands on the ground. And I'm like, well, this is why we're turning Bluey off right now.
[0:47:43] So I don't think this is specific to technologists. I think that all parents have always tried to moderate their kids' use of technology. And it's about having them use it a little so they know what it is, but the parent being involved enough to not let them use it for hours and hours and hours, which just seems generically bad, whether it's AI or computers or phones. I mean, when it comes to adults then, the relationship that lots of people are developing with their AI,
[0:48:12] It's also a bit concerning, isn't it? I mean, you have a conversation with your AI as if it's a person. And when you interact with AI, you use much more human language as you're trying to explain what you want. And you're polite along the way, or maybe that's just me. And you use greetings and things like that. Are people already in danger of developing
[0:48:42] Really unhealthy relationships with AI in which they think the AI is their friend or their therapist or their confidant when it's a machine. I'd say this is one of the areas I and some of my teams at the Institute study and contend with quite closely. Like people use the AI technology for almost everything you can imagine. And then there's this decision you make as a company about what seem like behaviours which you would view as...
[0:49:11] Unhelpful if a person did them. So a common one here is something called sycophancy, which is really just having someone say, oh, you're absolutely in the right. You're so smart. Or when someone's talking to them for relationship advice, you wouldn't want a friend when you're talking about, say, issues in your marriage, as I do to friends, to say, oh, Jack, you're like 100% in the right. There's no way that your wife has any legitimacy in this thing you're talking about. Clearly, you're on the right side of history here.
[0:49:41] None of my good friends ever does that to me. They'll say like, well, maybe there's something in what you said, but also think about it from your wife's perspective. That's a property that we can measure in our systems called sycophancy. And we just published research actually on how we have both been measuring sycophancy and we saw it being very high in relationship-based discussions of the form I just described. And we used the fact that we could measure it.
[0:50:06] to intervene on one of the AI systems we just released to make it less sycophantic in these interactions. So there are ways that the technology can be changed to, I think, have it embody more of the qualities that you think a good friend would have, which is often holding you to account, pushing back, not solely indulging you in whatever kind of specific line you have. And this is ultimately going to be one of these areas that society will demand.
[0:50:34] Some greater level of transparency from the companies around because it's also a hugely consequential decision to make in a vacuum. So we're going to need to both learn how to intervene on this technology, as I've just described, but also figure out how you share that with society and how society figures out what the appropriate levels to set the dials to are. But is the relationship itself really alarming? You know, that people are using AI to determine whether they got a human interaction right or wrong?
[0:51:03] I mean, that just sounds really screwy to me. Rather than going to another human being who you trust and say, do you think I made a mistake? I wouldn't say so. I think there are, of course, some people where maybe they will use it as a replacement for spending time with people. That's not healthy. And I think that you're going to need to do the equivalent of what Netflix or Nintendo do, where they say, you've been watching Netflix too long or you've been playing the game too long. Go outside. I think literally what Nintendo says.
[0:51:33] I think that AI companies can absolutely do stuff like that. But for most use cases, I have friends that have named their car. I'm sure you have friends that have named their car. People like anthropomorphize every single technology they've ever touched and develop oddly specific relationships with many of them. Obviously, some of the lovable technical characters I work with have very specific relationships with their computers before we'd even built AI stuff. It's not on us to police that.
[0:52:01] I think it's on us to observe and report out how people are using this so that society gets a view and a vote. And to just do basic levels of responsibility, like clearly there will be some things which are egregious. You should be able to spot egregious behavior and intervene on it. And so do you think we will get to the point where we can tell our AI system what kind of friend we want them to be?
[0:52:28] I just want you to be supportive. I want you to tell me when I'm wrong, all of those sorts of things. Yes, though I think the question is whether you then want to allow the systems to take on that form. There are benign versions of this where I will say to Claude, just give me the facts. Please don't compliment me for sending anything to you. I just want the straightforward answers. That's fine. Or there might be versions where I'm saying...
[0:52:55] Hey, I'd like you to be a bit more sensitive about this one because I'm going to talk to you about something complicated that I myself don't feel fully oriented around. Those are fine, but then there are going to be extreme versions and you're again going to need to figure out what the appropriate levels are. I mean, from a slightly self-interested...
[0:53:12] I mean, I wonder where you think AI is going to move in terms of news and information and journalism. You know, at the moment, it can tell you an awful lot about what has happened. What it can't do, obviously, is go out and witness anything or hold anybody to account or investigate. And I suppose people in my industry are slightly worried that, you know, that there's a danger that there is no money left.
[0:53:42] to do the hard yards around journalism because people are replaced partly by AI, but also the information delivery is replaced by AI. What do you think is going to happen? With things like Substack, you have seen the rise of very small person publications, which actually seem to do quite well from a business model perspective and are able to fund real reporting. But when I look at those,
[0:54:12] A lot of it stems from some amount of opinion blended with a huge mosaic of facts that have been assembled and relatively little primary source reporting.
[0:54:41] 20 picture gallery review pieces we did about certain bits of hardware because they paid for the ad sales and it cross-subsidized the actual reporting. So that's a business model challenge for journalism, which I think AI, maybe it contributes to, but this has been the case forever for me when I was a journalist and I'm sure for some period of time for you. On the other hand, it does make it incredibly cheap and effective to gather...
[0:55:10] Open source reporting on facts which are already digitized to add texture, color, and detail to stories. You know, I've been researching a lot of the conflict in Ukraine and how drone warfare is evolving recently. And it was quite stunning to me how I was able to have Claude help assemble for me some very useful data on the specific types of AI system and hardware that were being developed and deployed by the Ukrainians. And I read this report and I realized it would have taken me
[0:55:38] Many, many weeks to source, but it's not actually like reporting. It's just background for the actual reporting where I can now use this to go and talk to people in Ukraine or talk to others equipped with facts. So I think it's an amazing accelerant in that sense. And finally, as a topic, I mean, I just wanted to ask you about sex. I mean, obviously, you know, sex became a huge driver of commercial activity for the Internet. What do you think the future of AI and sex is?
[0:56:08] It's an unexpected question. It's not one I've spent too much time involved in. Well, if you're thinking about relationships, I suppose I'm thinking about it as there's a huge amount of money around sex in the world generally. And do you think that this will become one of the driving forces within artificial intelligence as well?
[0:56:31] I think you would be very worried if you saw extreme attachment styles linked with highly sexualized relationships, and you would probably need to decide some level at which you say, go outside. And again, it's this question of liberty and sovereignty versus paternalism, but clearly there's some dial you set there where otherwise you could end up encouraging behaviors that would probably be antisocial and anti-human in many ways. At the same time...
[0:56:59] Photocopiers and VCRs were used by the sex industry famously at like extremely large scale, like when they first came along. One of the big ways to sell books these days is to write like somewhat raunchy, like sexy fiction, which is mostly consumed by women as a sort of demographic for readers, although some men do as well. And I think with AI systems, you have people, some set of people like having horny conversations with it.
[0:57:27] I think that will always be with us and is fine. It's how people like to use technology. But what you need to do is look at the warning signs or whether technology might be fundamentally changing how people interact with other people and then surface that. Now, we don't see that yet, but I think that at some point you will as the technology becomes better and you're going to need to talk about it and decide what the appropriate level is. So just finally then, I mean, obviously you spend your life looking at lots of things that would keep a lot of people awake at night.
[0:57:57] And stop them sleeping. You personally have done incredibly well out of AI. So you have lots of reasons to be happy about it. I mean, you know, overall, how do you balance your optimism with your pessimism and take your British sensibilities out of it, which are probably naturally pessimistic? Naturally pessimistic. I mean, when I was 16 or 17, I remember in college when I was doing my A-levels, writing some...
[0:58:28] I did a module called Critical Study of Art, and as part of it, you got to make some art. And I did some big poster, which was a very depressed teenager-type poster, but it was, you know, like, all of these statistics about climate change, all of these statistics about species collapse, all of these statistics about things like rent prices going up and the relative poverty of young people, relative poorness of young people relative to prior generations. And my whole take was...
[0:58:53] God, things are not going in the right direction for my generation. All of the numbers which were going up for my parents are now starting to go down for me. This feels excruciatingly bad. What are we going to do? And I worked in technology because I think technology is one of the only silver bullets that we're going to have to get humanity through the century. If I think about what's happened since then, solar panels went from an extremely expensive proof of concept to something which is now a vast and cheap source of power.
[0:59:23] for people around the world. Electric cars went from something that wasn't a concept to a thing that's widely deployed. Computer vision went from something that just basically didn't work to something which now affords things like environmentalists the ability to track Amazon rainforest deforestation, people in Africa to look at poaching and whether poaching is happening through smart cameras. There are these amazing dividends that have come to the world through technology and in many places have started to work directly on some of our hardest problems ranging from
[0:59:53] Climate to conservation. So I'm an optimist because I think that AI unlocks unbelievable capabilities here. But the gist of our conversation is how do you get those and how do you stop the bad stuff happening? And that's why I'm so, so passionate about sharing this information, but also talking about policy. I think it would just be an act of like incredible hubris to say everything will be fine. We are dealing with the most powerful technology that's ever been built.
[1:00:21] That is regulated less than any powerful technology before it. And that situation is not like a long-term stable thing. So I'm quite cheerful about our prospects for changing that. But it wouldn't be honest of me to not say, hey, it's like a big deal and we should be worried about it and hold people to account about it. Jack Clark, thank you very much indeed. Thank you for sharing your way to change the world. Thank you. You can watch all of these interviews on the Channel 4 News YouTube channel. Until next time, bye-bye.