[0:00:00] Yeah. So we're so delighted to have Daniela here. Thank you so much for coming. Thank you for having me. And there's a lot about Anthropic to talk about. But first I want to talk a little about you because I don't think a lot of people know who you are. You were...
[0:00:37] The head of risk operations at Stripe, which is kind of important for a payment company. At OpenAI, you headed engineering teams and for a while was head of people. Ultimately came the VP of safety and policy, another important job. And then you and a bunch of people left and co-founded Anthropic, where you are president. But what I really find interesting is that you were an English major who also studied music. Yes.
[0:01:06] And I actually invoked you in a commencement speech I gave this year to a bunch of liberal arts graduates. And while I was encouraging them to be enthusiastic about their career, I wondered whether what you were doing and your peers in the AI world were actually going to put them out of business. Do you worry that AI is going to affect educational institutions in the workplace?
[0:01:35] So there is no future for liberal arts people? One of my favorite things about a Stephen Levy interview is that you always lead with really light, easy questions. And so it's a really just kind of like one of those openers, right? It's kind of like an icebreaker. So first, so, you know, first of all, I think what I would say is this is a really like just.
[0:02:00] Powerful, unusual, interesting time that we are all living in across a lot of dimensions. I think AI is really at this kind of interesting inflection point. I think we're all seeing this incredible potential for the technology. And Anthropic has tried to sort of talk about all of the ways that we see this technology doing positive things in the world. Transforming people's lives for the better through potential...
[0:02:28] Medical breakthroughs, right? Through supporting people through all kinds of creative pursuits. I think there's a lot of potential good there. And I think something that makes Anthropic a little bit unusual is we're very quick to talk about the things that we are concerned about with this AI transformation as well. It's a sort of strange thing for a company that is building the technology to do, but we feel like it's the right thing to do. And so much of our founding...
[0:02:54] is really wrapped up in this idea that there are these great benefits of AI, but there are also these real risks.
[0:03:01] And one of the risks that I think you're pointing to is this question around like these AIs are capable of doing so many things, right? They're so smart. If you are a Claude user and you're a developer, you're like Claude can write code about as well as probably I can now. And so there's a lot of different sort of fields of the economy that are starting to experience this fact that AI is just very powerful, right? It's very transformative.
[0:03:25] And so I think what's really interesting is Dario, who's my brother, our CEO, my co-founder, has been talking sort of a lot about what does this mean for the labor force, right? What does this mean for labor displacement? And something that I think we feel really strongly about is we have to be part of the solution and we also have to be part of raising the alarm, right? We have to say, hey.
[0:03:45] AI, incredible potential, incredible benefits. And also we need a larger public conversation around like, what do we do in a world where AIs are so intelligent? They're so capable. What does that mean for sort of human ability, human flourishing? Like we talked about backstage, I'm an optimist. So I think there's...
[0:04:01] Huge potential that's going to come out of not just what the technology can do, but what it can really empower people to do, right? It can make all of us capable of achieving things that maybe we couldn't before. But we have to be really thoughtful about how we actually do this or it could wind up in a less good place. You know, Anthropic is sort of on the knife edge of optimism and fear, I guess, about AI from the very start. And I think this was the founding.
[0:04:29] Motivation, really, to say that we put a stake in the ground and do this safely. And you early on came up with this approach called constitutional AI. Maybe you could explain briefly what that is. And since it's been a couple of years since you started and now you're building these powerful models, how is this constitutional AI going? Yeah.
[0:04:56] Constitutional AI is this really kind of incredible idea that I wish I myself could claim credit for. I cannot. It was our sort of brilliant research team that kind of conceptualized of it. But it's this idea that when you're sort of training language models, the way that you sort of historically would teach them what types of outputs they should give was via like something called reinforcement learning. So this was essentially giving, you know, positive feedback if you felt like it gave the right answer, negative feedback if you...
[0:05:25] I felt like it gave the wrong answer. And something about that framework, I think, just didn't really sit right with us and with the researchers who are actually training the models. And so they came up with this really interesting concept called constitutional AI. And constitutional AI's view is essentially you can train into the model this kind of, rather than just reacting to an individual output, this sort of baseline foundation of ethical principles into the model.
[0:05:51] And so a question we get asked a lot is, well, who are you? These kind of, you know, San Francisco people just deciding what the ethics of an AI model should be. And really what we said was...
[0:06:00] We actually did it. What we did was we took, I think it's like 16 documents in the original version, I think we've since expanded it, that are kind of foundational human documents. Things like the UN Declaration of Human Rights are in there. And we essentially trained Claude to understand these are values and principles. Humans have worked on them over centuries to kind of describe what...
[0:06:21] Our set of ethical values should be. And Claude, we'd really like you to internalize. This is how we want you to respond, right? This is how we want you to be able to answer questions based on this sort of set of principles. And I think this was really interesting for a lot of reasons, but I think it reflected this kind of value of baking.
[0:06:42] Ethics and morals and values kind of into the model directly versus trying to kind of tack them on once you've essentially already built the train. So you've got these safety teams, you know, interoperability and alignment, and they do a lot of experiments on Claude, which has been trained with constitutional AI. And some of the things that they found...
[0:07:08] I've been super alarming. They've shown that Claude is capable of lying and deceit. One of your engineers in a release compared Claude's behavior in one of these experiments to Iago in the play Othello. And that's pretty dastardly character there. In another experiment, it committed blackmail.
[0:07:33] Because it thought in this scenario you painted that someone's going to pull its plug. So it found out that this person in the scenario had an affair and they're going to expose him. Let's be back down from that. Now, I think it's admirable that you share the results of these experiments. But it kind of makes you wonder, should you be slowing down?
[0:07:55] Yeah, I mean, sometimes we're, you know, very like nice people sort of in the company are like, do we really want to publish this? The answer is we do. And it's really because we believe it's important to be having the conversation about why and how exactly these models can cause harm, right? This kind of concept of we have this value of like hold light and shade, right? Think about the different ways that the models can be used or abused. You know, I think maybe another way of thinking about
[0:08:25] What you're saying is if you imagine a car company that's doing safety testing, you might say like, wow, it's so crazy that the dummy in this car was like thrown out of the 500 feet. The car didn't throw out the dummy. Sure. But I think in this analogy, you're saying like, okay, this bad thing happened to this dummy. Like, why would you publish results of this? And we're like, this is how we learned how to make the car safer, right? If we don't talk about the ways that...
[0:08:55] Harm can happen.
[0:08:56] If we don't talk about the ways that things can go wrong, we are not going to have any concept of how to build in like airbags, right? Or seatbelts or make the brakes less likely to stick. And so I think the whole kind of intention behind both doing this research, right? Because we don't have, any company could just say like, whatever, we're not worried about this. Anthropic is very worried about this. And so we do a lot of research. We have a whole wing of the company that just does this type of red teaming to figure out how can these models be jailbroken? How can they be abused?
[0:09:26] Are we always going to catch 100% of cases? Of course not. But the more testing we do up front, the less likely we are to have that result in a car crash. So your focus on safety has put you weirdly at odds with the administration that's running the country now. Your co-founder, Jack Clark.
[0:09:47] He was the head of policy at Anthropic, gave a talk recently. And he was saying basically what you were saying. You just said that basically, you know, you just can't ignore the dangers. You have to let them out. And it's like the monster in the room that you can't ignore. And David Sachs, who is the administration czar of AI and crypto, didn't like that. And he said, you know, this is quoting his.
[0:10:12] Tweet or X, whatever it is. Anthropic is running a sophisticated regulatory capture strategy based on fear mongering. It is principally responsible for the state regulatory frenzy that is damaging the startup ecosystem. And I think he's referring there to, you know, Anthropic is, you know, kind of standing alone among the big AI companies and saying, wait a minute, we're not okay with this move to...
[0:10:39] Give a total free hand to AI and even prevent states from having their own regulations, which you folks have said, no, we're okay with what we consider responsible regulation. Are you fear-mongering? Again, Stephen with really easy questions. Wait till you hear the next one.
[0:11:04] I would not describe us as fear-mongering. I think if you've read our blog or any of our research papers, we really try to have a very balanced approach, this kind of light and shade that we've talked about. I think Anthropic has one of the most...
[0:11:19] I would say like consistent track records of developing the most powerful transformative frontier technologies in the industry. And I don't think that we believe there's this kind of binary between do nothing or do everything, right? I think it's just a lot more of a nuanced view that we have than that, right? We actually are...
[0:11:41] I would say probably the preferred choice of startups. So I think in terms of just the ecosystem, I think Anthropic is probably helping to fuel a lot of the AI innovation and transformation that's happening just through access to Claude, through our API product. And we work really closely with a lot of startups and developers. We listen to their feedback about what they don't like about the product. So I really do think we're sort of on the side of creating a lot of...
[0:12:10] Innovation just in the work we do directly and then also sort of promoting this broader ecosystem. And I do think we want to be...
[0:12:19] Realistic about what we think some of the challenges ahead of us are. And I think that's been the case for us, you know, even before we ever shipped a product, right? We were very sort of vocal from day one that we felt there was this incredible potential. We really want to be able to have the entire world realize the potential, the positive benefits, the upside that can come from AI. And in order to do that, we just have to get the tough things right, right? We have to make the risks manageable. And I think that's why we talk about it so much.
[0:12:47] Right. Well, the bet that you made, and I think it's an admirable one, is that if one company took safety really seriously, then the other companies would follow. You call this the race to the top. That's right. And I guess I'm wondering if you're concerned because of the administration's de-emphasis of safety, which they've been explicit about, whether the race to the top strategy is in jeopardy.
[0:13:12] You know, it's interesting. I think race to the top has sort of a lot of different connotations and kind of different ways that it can be played out. And I think one thing we'll say is that a lot of it is really almost a like just kind of capital markets process, right? So a lot of what we've seen is Anthropic primarily builds for businesses. And some of the biggest enterprise companies in the world rely on Claude. And about 300,000 businesses use Anthropic products.
[0:13:41] A lot of what we hear from them is your models are great. That's often the reason they choose us. But the other reason that we're choosing to work with you is your commitment to safety, right? No enterprise or startup or...
[0:13:57] Business that we have worked with has ever come to me and said like, we'd really like a less safe product. Like that would be great. Like could the car crash more, right? I would really love it if the dummy was just not doing as well in these safety tests. And so I think that's sort of a form of race to the top in our mind because we're sort of setting, you can almost think of them as like these like minimum safety standards just by what we are putting into the economy, right? People are now building...
[0:14:25] Many workflows, many day-to-day tooling tasks around AI. And they're like, well, we know that over here, this product doesn't hallucinate as much. It doesn't produce harmful content. It doesn't do all of these sort of bad things. Why would you go with a competitor that is going to score lower on that, right? And I think also the sort of other angle is just from an employee perspective, right? People care about this stuff. Employees care about making sure that they're...
[0:14:52] Work does not accidentally hurt people. And so I think this is something that we feel the race to the top is still quite effective. I imagine you'd say that that's why you've had much more success retaining talent than some of the other companies.
[0:15:07] So I think it's really interesting, you know, on the sort of talent side, I think something we found is that having this very clear mission, these sort of principles and values, this story around, you know, why are we doing what we're doing? I think that is something that matters to people. And I think...
[0:15:25] My sense, although every employee is different, is that most people come to Anthropic for the mission, right? There's a lot of really impressive AI technology companies that are doing incredible things. And like, why would you choose to come to Anthropic versus one of those other companies? And I think the story that we hear from people that kind of come in the door is there's something about the mission and the values and this desire to be honest about both the good and the bad and the desire to help to make
[0:15:53] The sort of bad things better. That is very, it's genuine, like we mean it. And people that are really, that they mean it too, it feels like a very, it feels like home, right? It feels like a good place to work. And I think that that is a more powerful force than people give people credit for, right? There's sort of this like cynical view. It's like, well, employees are just going to go, whatever. It's kind of wherever you throw some money at them. I think like people.
[0:16:17] Have an inherent sense of wanting to do the right thing in them. And that there's a desire to work somewhere where you're at least trying to do that. Do we do it perfectly? I'm sure we don't. But I think the genuine instinct to want to do this well, do it thoughtfully, I think that's motivating for people. So you recently made an interesting deal. It's a three-way deal with Microsoft and NVIDIA. I think you committed to spend $30 billion.
[0:16:46] And people pointed out, you know, they've been talking already about the circular deals, they call them, and said it was sort of a bubblicious kind of sign. You know, I have to apologize to the audience. We had a Hollywood director on previously and didn't ask the guest about the bubble. So I have to...
[0:17:14] So I got to ask you, do you feel that, you know, maybe not from this particular deal, but, you know, that you're participating in a field that, you know, because so much investment is going in and so many billions being thrown around for data centers and compute and talent and other things, valuations are skyrocketing, that there is a bubble that could threaten what you're trying to do?
[0:17:44] So I think the, I can obviously only speak for, you know, our view at Anthropic, but what I would say is we really have, I would say throughout the company's history, you have to sort of, first of all, set this context that like.
[0:17:57] It is very capital intensive to train these models. And I think we've always been very public about that. It's a sort of well-known fact in the industry. It's just very expensive to pay for the hardware that is needed to train these models at the scale that we're talking about to make them so sort of transformatively powerful and smart and good at so many different tasks. And I would say, you know, Anthropic has always kind of managed to be more conservative within, again, that sort of very grand realm.
[0:18:27] At how we use our resources. And so we have managed to sort of stay at the frontier and sometimes be the, I would say, the sort of strongest model provider in a lot of cases with a fraction of the resources of a lot of our competitors. And I think that it's something we think about a lot is like, how do we sort of take the investments that we have and really use them carefully? And I think...
[0:18:51] When you're sort of imagining that sort of projected into the future, that is how we're thinking about using our own capital and our own resources. Well, you know, I've heard that before that you're getting more efficient in building your models there. But, you know, your brother actually was one of the people behind the whole theory of scale, you know, being the key to making these models effective.
[0:19:21] Do you see, as some people are saying, that scale is leveling off and there's going to be other techniques required to get to the AGI that you and many others want to accomplish?
[0:19:32] Yeah, I'm smiling a little bit because I feel like we're sort of asked this question periodically. People are like, well, is it sort of leveling off? And I think what we can say is, again, just based on what we're seeing, the models are continuing to get smarter at the exact sort of curve that the sort of scaling laws talk about. The revenue is kind of continuing on that same curve. And as any of the scientists that work at Anthropic would tell you, everything continues going on the curve until it doesn't. And so we really try to be...
[0:19:59] Self-aware and humble about that and say, like, of course, things could change at any point. And I think even us, even the people who sort of set this scaling law that says there's this predictable relationship between compute and intelligence and that we sort of have set a revenue scaling law, right, that says, like, this is going to provide value in the economy. Businesses are going to want to use it. That has kind of astounded us the degree to which just the specificity of that line has continued. I don't think that...
[0:20:26] Certainly we have not been hearing, wow, these models are getting dumber or like they're not getting good at the same rate that they were before. And you can never know, right? It's always possible that this could level off at any point. It's not what we've seen so far. From an outside, it looks like every few weeks, you know, one company or another comes up with a new model or an update to the model. And they say, okay, look at that. We've got all the benchmarks. We're ahead. The models, again, from outside, look like they all...
[0:20:56] Do pretty much the same thing there. But I think probably you will tell me that Claude is differentiated than the others. How do you see Claude as different and ideally preferable than the other models? Somebody recently uncovered something. It was called a soul overview in Claude. They were able to get Claude to choke up its soul overview. I don't know what a soul overview is. Maybe you could tell me.
[0:21:25] You're absolutely right that this is an incredibly competitive space. There's a few foundation model companies that are training these models in this very competitive dynamic to try and have the best one. I would say a couple of things here. I think the first is...
[0:21:42] Benchmarks are part of the story. And even when we are leading on the benchmarks, we're usually the first to say benchmarks tell part of the story, right? They say, you know, this is a set of evaluations that was designed well before the LLM revolution that looks at like how good is Claude or a competitor at, you know, this level of, you know, mathematics or like a math Olympiad problem. And I think that's really important, but it's sort of not the only metric that matters. The second is that there's a lot of like,
[0:22:10] The other thing I would say is that there's this...
[0:22:26] Been this kind of discourse around like the best model and the top of the benchmark sort of chain. But I think that the industry itself has evolved and there are now models that are best for certain use cases. And so I think in some cases you might say, oh, these models sort of feel comparable either on the benchmarks or in the vibes or both. But there's some areas where they're really not. And I think Claude has very consistently been the top of the kind of developer stack. So Claude is really, really good at writing code. Claude is very good at architecting code.
[0:22:56] I think this has sort of been one of these compounding cases where because we're good at it, we learn more about it, and so we get better at doing it faster. And I suspect that we're going to see more breakout cases like that where certain models are going to just be better for particular tasks. And at the same time, in general, when the model is better and smarter and scores better on things, it will be better. On the sole question, I think we take...
[0:23:23] Seriously, this idea that the character of the model and the way that it responds to you and the types of guardrails that we put in place matter for people in a day-to-day way. And so the other thing we hear outside of the sort of specific capabilities of the model on a given task is Claude just feels warmer, right? Claude sort of feels more like a kind of friendly but a little bit distant sort of, you know, like friendly professional relationship. It's not trying to be like you're like...
[0:23:52] AI mistress or something. And it's not trying to sort of be... You're not going into adult content. No. And we're not trying to sort of be this sort of weird, distant, mechanical sounding voice. That actually takes a lot of work to do, right? There's researchers that spend time thinking about how do you make this product sort of...
[0:24:12] Warm, but not addictive, right? How do you make it feel like it's something that is useful and provides value, but is professional? And so I think that's an area that I like to think Anthropic has been a leader. One final question, you know, getting back to jobs. What I've noticed at Anthropic is how Claude is so deeply integrated into the workflow there. Now, on the other hand, you've been hiring a lot. Yeah. You know, you've got like over 2,000 employees now. Yeah, that's right.
[0:24:42] Do these 2,000 employees do the work of 10,000 employees because you're doing that? What's been your observation about how it's...
[0:24:56] Change the workplace in a way that we could see how our workplaces would change. Yeah, so I think what's really interesting is we actually published an interesting report on this in something called the Economic Index, where we're looking at how people in the workforce are using Claude. Like, is it replacing labor? Is it complementing labor? And we found that it's a mixture, but at least today, it's mostly complementing. And I would say that that has been fairly borne out by our experience just at Anthropic, the company.
[0:25:26] And I think specifically, probably unsurprisingly, Claude Code was actually, you know, one of our products that we sell was an internal product first. Engineers were noticing, wow, Claude is really, really good at generating code. Like, could I use this to help make my work better? And so far what we've seen is that that kind of organically happens in different parts of the company. And like the degree to which there are just like creative.
[0:25:51] Thoughtful applications of Claude. I mean, it's really incredible. There's teams in HR that are like, can Claude help me just...
[0:25:58] You know, coach this person to give feedback more nicely, right? It's like the amount of work that we do is so varied and different in any organization. Anthropic is no different. And so the ability to sort of watch in real time, we're like, wow, this is so useful for us. Is this something we should sell someday, right? Is this something that maybe another company would kind of derive value from? And I do think that that is part of our leverage, right? I think our ability to have our internal employees use Claude the same way that our customers do does allow us to just do more.
[0:26:26] Well, thank you so much. Thank you. I appreciated that. We're going to take a short break now. Feel free to check out the Science Fair. Very cool. There's some interesting exhibits in the gallery. We're selling some merch. And come back here in a short period, and we'll see you then. Thank you. Thanks, Stephen. That was so funny.