[0:00:00] By the end of 2028, it's possible to build AI systems that might be able to make a smarter version of themselves. How would you want Claude to behave in a way that seems like it's closer to a good friend than someone who means you ill? I can't reconcile this technology with the economy staying normal. I wonder if you can kind of paint me a picture of the future that we are sort of hurtling towards.
[0:00:25] And what you want to get out of it and maybe paint me a picture of the future that you want to avoid. The future we're heading towards, if we're able to make the right choices, is one where you or I can kind of work with hundreds or thousands of people on anything that we're curious about. And there are versions of this that are just extremely empowering and exciting. You know, me personally, I'm obsessed with reading about the frontiers of science. I always have been.
[0:00:54] That's always been limited by my own ability to kind of read the literature very widely and synthesize it. And now I have these systems where this morning I was asking Claude to map out for me some of the most consequential things that have happened in the field of AI and biology. And now I have these incredible graphs showing the progress of the technology here. And to me, it's really empowering. It's really exciting. It's stuff that I know would have taken me many weeks of work just to get to this point.
[0:01:21] But now I can kind of move beyond it quickly because of this AI system. But what does that depend on? I think it depends on individuals being empowered to access this technology and individuals having some level of what I think of as curiosity and obsession. The future I worry about is one where we...
[0:01:39] We ignore the pace of this progress or we are blind to the amazing things this technology can do and we kind of retreat from it. We assume that it's all overhyped, that none of this technology stuff that I talk about actually works. And then we as societies get surprised by successively larger and larger changes which happen to society as a consequence of the technology advancing. Do you have a timeline for...
[0:02:06] Artificial general intelligence, super intelligence. Are we there yet now? I believe there's a better than even chance, but by the end of 2028, it's possible to build AI systems that might be able to make a smarter version of themselves, a recursive self-improvement, which I think would satisfy my own personal bar of what a general intelligence is. So not too long. I want to talk a bit about how you're...
[0:02:31] Operating within this kind of race dynamic with the other frontier labs and the kind of pressures that can come with operating in that dynamic. And you've been really responsible when it comes to Mythos from what I'm told, from partner organizations. Now the other labs know that there is this capability out there. Not everyone might behave in the same way that Anthropic has chosen to behave. Is there an argument for not building...
[0:03:01] This is an immensely powerful technology. And if there was a way that you could do a kind of coordinated, like global slowdown on its development to give us more time as a society and as people to deal with the changes it implies, that would be good. I think everyone would feel more relaxed if, you know, rather than having your foot like slammed on the accelerator, you can take it off the accelerator and maybe you don't break, but you just like slow down for a bit and see what happens. That requires...
[0:03:31] Unprecedented coordination, not just between companies, but between governments. It requires political will to do it. And I think it requires awareness that this is something that could happen. And part of why I'm saying it is I'm like, sure, that seems like a good idea. I think if I say it, maybe other executives will say it as well. At the same time, we didn't set out to make an especially good hacking system. We just tried to train an even smarter system. And so it's got a lot better at...
[0:04:00] It's got a lot better at coding. It's got a lot better at things that could be useful in education. And it's got interesting at hacking. And how to reconcile these things is going to be one of the challenges of the whole industry, where you obviously want to bring incredibly powerful tools for scientists or education to the world.
[0:04:19] You don't want to bring incredibly powerful tools to hackers to the world. I mean, you've got two young kids. Yes. I mean, how are you preparing them for an AI future? My dad always said to me that he would never buy me like games consoles or computer games or anything like that. I had to eventually get a Saturday job to do that. But he'd buy me any book I wanted or take me to the library whenever I wanted. And it allowed me to just indulge my curiosity.
[0:04:45] And to learn to become like obsessive about education and learning about things. And when I look at my kids, I want to sort of create an environment where they feel like they're able to follow their own curiosity and educate themselves because people who have passions and hobbies are going to be most advantaged by these kinds of AI systems. It's incredibly important that people have some kind of core interest that is entirely theirs and which they develop a real obsession about. And with that,
[0:05:14] They are going to be able to better relate to these systems than if they lack that. And then rather than cognitive offloading, you get something that more feels like cognitive compounding. You have this interest and this passion. And because of that, you can talk in a very informed way with these AI systems and get them to work for you and do things that make you more sovereign, more independent, more able to kind of pursue your own passions.
[0:05:40] Help people and encourage people to develop that basic level of obsession or hobby is the mystery and I think that some of that runs through education, some of it runs through parenting, some of it runs through how we think of the value of work in society as the economy changes due to this technology. What jobs do you think will exist or be left for your children when they turn 18 and beyond? I can't reconcile this technology with the economy staying normal.
[0:06:09] I look at stuff which is happening at Anthropic. I see individual researchers doing 10 times the amount of research experiments they did before because they're now able to task these AI systems to go and pursue their research ideas. I mean, you tell me how that leads to the economy being the same as it is now if we have a technology which multiplies what people can do. I can't make claims there.
[0:06:33] It's not obvious to me that you immediately get things like large-scale unemployment or other effects. It may be more that we get substantial economic growth due to these massive productivity multipliers, and we get the emergence of some new jobs. It may be that we have sudden difficulties in figuring out how to take people with very little experience and get them into their careers, and instead we might favor people that have more experience.
[0:07:00] The main view that I have and the work that Anthropic and the Anthropic Institute which I lead is doing is trying to build up the economic early warning signals that can inform economists outside of the frontier labs if these changes are happening and how to kind of look for them and how to do the policy response to them. One thing I've been really interested in is the design of these models and how it affects people's minds and in particular kind of
[0:07:28] Design choices like what people use the word sycophancy a lot, the fact that it uses I pronouns. What do you think that might do to, is doing to people's brains now and might do to us as a society in future? We run through this loop of monitoring for how people talk to the systems in a privacy preserving way and then seeing if we can do interventions that feel helpful.
[0:07:56] And we sort of define helpful as how would you want a good friend of yours to behave? How would you want Claude to behave in a way that seems like it's closer to a good friend than someone who means you ill? An example of this is we recently did a deep dive of how people talk to our systems and where sycophancy shows up, whether systems are overly complimentary or overly affirming. And we found that in the context of relationships,
[0:08:21] Claude tended to be overly sycophantic relative to other conversations. Say I was complaining about my marriage and you said to me, it sounds like you're exactly in the right, Jack. There's obviously like no legitimate view that your wife can have here. I totally believe you. That's extremely unhelpful and almost always is like totally wrong. We found that Claude was doing a bit of that, basically saying like, you are so right. Like, absolutely. Thanks for sharing that with me. And we intervened. We subsequently changed the model.
[0:08:48] to try and reduce sycophancy that would manifest in these relationship conversations. Now the question is, we just did that change, but how do we set Anthropic and other companies up to listen to society more and things like third party researchers, academics, civil society, journalists, governments, and other parties, so that we are taking an input about the behavior of our systems and adjusting them in light of that. And I think that's the next.
[0:09:17] Really subtle challenge we need to do. Well surely part of the challenge of that is that we don't exactly know how these systems work. Yes. And so to understand how to kind of set boundaries or something as mushy as mental health I imagine is quite challenging. Just because it's challenging doesn't mean you shouldn't do it. Like I think ultimately the lesson I took from social media is
[0:09:39] Social media companies were running like an uncontrolled experiment on the world's population, some of which was good and some of which I think we'd all agree was probably not good. Me personally, with young children, I'm very afraid of when they meet algorithmic feeds. I think we all have that nervousness in us about this. So the question for us is, how do you talk about this now, share information about it now, and start running these experiments now when I think the stakes are comparatively low and you can learn a lot?
[0:10:08] And ultimately, this is the place where we as societies will need to come to an agreement on probably regulation and what sufficient regulation looks like for showing that you're being responsible here. But as you say, we're at the early stage of the science. So all we can do now is run a lot of experiments in public and discuss what we find and figure out together what the best path forward is. One thing that I'm sensing is coming out of this conversation is that there's a lot of I don't knows.
[0:10:38] You know, it might lead to enormous job upheaval in the UK or it might lead to a huge boom. I don't know. I mean, what responsibilities does Anthropic have as a company to communicate what they do and don't know? Because, of course, you are building this technology. I mean, I think the main responsibility is you need to tell the whole story. So we are clear about some of the positives of our technology. We're also clear, as we've just been in this conversation, about...
[0:11:05] The known unknowns and also areas where we actually just don't know and we need to build better predictive tools. At the same time, this has always been the case with extremely powerful technologies ranging from the steam engine to airplanes. It was very hard to anticipate the future they'd shape because they'd interact with the world. In the early 20th century, there was a memo, I think which came from within the British Civil Service, thinking about what generalized civilian aircraft would mean.
[0:11:34] It anticipates this very dark future where because of the general availability of planes to civilians, cities are now constantly being bombed and like having acts of terror committed to them by like hundreds of planes are flying into cities and terrible things are happening. And some of the conclusion of this memo is like civilian air travel must be extremely limited and extremely controlled because we don't understand it.
[0:11:57] Some of that memo was correct. Some of those things did happen, but what actually happened, which I think it didn't anticipate, was that civilian air travel unlocked very large social changes and commercial changes, which it didn't anticipate. It could see the risks, but it couldn't see some of the benefits, and it was unsure of how to weight these things. I think with AI, we're dealing with the same issues, where we have these very hard-to-imagine futures in front of us, and we can see some of the risks today because they flow from previous technologies.
[0:12:26] It's hard for us to see some of the benefits because they're new. And a lot of what we're trying to do is just clearly talk about all of these things as we see them and share information and data with the world so the world can also analyze it. Eric Schmidt gave a commencement address at university the other day and he was booed. Yes, it didn't seem to go well. It didn't go well for him. No, he was booed when he started talking about how people were going to shape.
[0:12:52] There was another speaker, another very similar incident. I mean, on the kind of darker end, someone threw a Molotov cocktail that Sam Altman, the CEO of OpenAI's house. There is a feeling now that there is almost like a populist uprising coming our way when it comes to AI. People, I think, linked to this generalized anxiety that you describe. I wonder how it feels for you as a tech executive just watching this landscape.
[0:13:18] It's not going to give people a tremendous amount of optimism for me to say, oh, AI has been great for certain coders in Silicon Valley. You're not like, oh, wonderful. Well, that solves all of the potential social issues of this technology. I think that the public is rightfully demanding, well, what's happening to my cost of living? What's happening to things like healthcare? What's happening to the actual essentials of life in relation to this technology? And can you show me ways in which it's making it better? And in fact, I think there's lots of anxiety that it could.
[0:13:46] It could make it worse. You know, we've committed that wherever we build data centers, we will ensure it doesn't lead to rises for the rate payer on their utility bill. But people have anxiety about that because they see all of these data centers being built and power going to them and wondering, what does it mean for me? When we did a study recently at the Anthropic Institute, we interviewed 80,000 people around the world who are all subscribers to Claude of their experiences and thoughts on AI.
[0:14:14] But we discovered this amazing divide where in the developing world or emerging economies, people were generally very optimistic about the technology and they viewed the technology as a tool of empowerment and sovereignty and a means by which they could kind of advance in their lives. And in the developed world, even though these are people that use the technology, there was much higher anxiety about it and much more worries about what it would mean for people's livelihoods and jobs. And I think this maps to just generally the story of
[0:14:42] Some people would say,
[0:15:11] Anthropic plays a part in creating this fear. How do you balance that messaging? It's the toughest part of the job for the company. I think my reflection has been in the early years, so the last few years, it felt like there was an under discussion of the potential risks. We talked about things like worries about bioweapons. We talked about the...
[0:15:37] Potential ways AI systems could get better at conducting hacking or cyber war, which subsequently has clearly come true with Mythos. And then there was this formation of things in response, like the UKAC, the USKC, these parts of government which are now testing for risks. Now, I think, is the time for us to try and talk more about the positives as we see them, because it seems like we've helped have that larger discussion with society to orient some of the risk.
[0:16:05] Risk infrastructure to be in place. And now there's this question of talking more about benefits. You're not always going to get it right. But I think the general lesson from the technology industry is the technology industry has been like wildly, overly optimistic and sunny about everything far too much. And so we reacted against that. But now we're needing to toggle slightly back to optimism as well.
[0:16:25] Jack, we're almost at time, and I've been trying to figure out a way of asking this question in a narrative flow, but I can't. But maybe it's good to end on the point of religion. Your co-founder Chris Ola is going to stand alongside Pope Leo at the Vatican to launch the first papal encyclical on AI. Why is it important that Anthropic is in the room for that particular conversation? It's not specific to the Catholic Church, obviously, but it's that...
[0:16:52] We recognize that as a company, we're now building a technology that doesn't just have national security aspects that we've talked about. It also has normative aspects. We talked about sycophancy. We talked about the behavior of the technology. That flows from what we call the character or the constitution of Claude, which is something that is a document that we've worked on that tries to encode the normative.
[0:17:16] Behaviors of our system such that we try to have it show up in the form of what you might think of as a wise or good friend. Well, religions around the world have spent their entire existence thinking about what it means to be good. What does it mean to be like a good person is the domain of religion and other experts. And we've been sharing our thinking on how we've approached this.
[0:17:40] We have different communities outside Anthropic, including religious ones, to try and learn from them about how they've approached it as well. Because again, it's unlikely a five-year-old Silicon Valley startup is going to invent the correct answer to what it means to be a good person. It's much more likely the case that all of the people outside the company that have thought about it for literally thousands of years have more wisdom to share.