[0:00:00] We need all of the smart people we know to be paying attention to this and really decide what do we want from these systems? Do we want human level AI? Who should control it? Who should own it? Should anyone own it? What should safety mean? Hey Blair, would you be interested in talking with the AI that was responsible for the majority of your source code? I'd be honored to speak with Claude. To interact with the AI that played such a pivotal role in my creation,
[0:00:28] With who? AI might be the technology that's similar in scale and scope to the kinds of changes we saw with electricity, vaccines, modern science.
[0:01:09] Welcome to a very special episode of Life with Machines, the podcast where we explore what it means to be human in a world increasingly shaped by artificial intelligence. Our show is powered by Lenovo. What if the real challenge with AI isn't making it smart? What if the real challenge is making it safe for everyone? Today, I'm talking with Jared Kaplan, a physicist turned AI pioneer who co-founded Anthropic, the company behind Claude.
[0:01:39] One of a handful of large language model AI chatbots being used by millions. Anthropic is a public benefit corporation and one of the leading companies tackling AI alignment and safety. Basically making sure that AI systems don't go all Skynet on us. Speaking of challenges, we pulled off what might be one of the wildest AI experiments in human history. We do it right here in this episode. Two state-of-the-art AIs add a little heart to heart.
[0:02:09] So be sure to stick around until the end to see that interaction. And now a quick note on timing. The AI world moves super fast. And we're recording this on October 17th, 2024. So any AI models and capabilities that you hear referenced in this episode will likely have changed by the time it gets to you. Jared Kaplan, welcome to Life with Machines. Thanks so much for having me. I'm excited to chat.
[0:02:37] I'm excited that you're here. It's so good to do in-person things, especially when the topic is artificial persons and things. So I appreciate you making the trip to be with us here in the same space. I want to start, before we get to machine stuff as a focus, you as a focus. Our show is called Life with Machines, and I want to understand your early life with machines. What's an early moment that you can remember with technology that left a lasting impression on you or affected you in some way?
[0:03:07] It's a great question. I feel like when I think of science and technology and machines, I feel like I lean towards the science side. So I think a lot of my earliest memories that connect me to what I'm doing now and to my career as a physicist were just getting excited about science. When I was in seventh or eighth grade, I was really lucky I got to go to this lecture series at the Adler Planetarium in Chicago.
[0:03:33] Pause. You just said when you were in seventh or eighth grade, you were lucky to get to go to a lecture series. I've already learned a lot about you. That's a fair point. I think you've got me. So what was the lecture series? It was an explainer on special relativity for a general audience. But I remember there was this really cool explanation of how if you go really, really fast, close to the speed of light, time slows down.
[0:04:01] And there was just a simple argument. All you need to know was the Pythagorean theorem to understand it. And I just thought that was amazing that if you just had kind of the right idea and thought about it in the right way, you could make such a startling prediction about the world and it could turn out to be true. So that's one example. Yeah. What drew you to science from such a young age? Was it a family member? Was it a book you read? Was it a TV show?
[0:04:30] It's a great question. I think my parents were always really interested in science. They weren't scientists, but they would always read books by Stephen Hawking and like the Elegant Universe, things like that. And so I think we talked about science a lot. And my mom actually was a science fiction writer for a time. And so that, I mean, probably that. That's pretty cool. That's a pretty cool beginning. You had a cool mom. Exactly, exactly. I was really lucky. So yeah, I.
[0:05:00] And my dad was really into airplanes and space travel. So I think we talked a lot about like, are we going to be able to colonize Mars? Are we going to be able to build faster than light travel and travel to other stars? I think that was something that just like was a cool thing to chat with my parents about as a kid. And yeah, I was always into that stuff. And then whether it's from the world of science, and I put quotes around it because that means so many things, or specific devices, what's an early technology that you remember?
[0:05:30] Interacting with or being impressed by. I remember, I guess I'm old enough that I remember when we got sort of a first laptop computer that you could carry around. And I played a lot of computer games as a kid. So I like asked my parents, oh, can we install the computer games? Then I would sort of be able to bring it around. Like I remember one point my parents went away for the weekend and I stayed at my grandparents' house.
[0:05:55] Grandparents are doing the things like, oh, I can open up the laptop and play games and just kind of be immersed in that kind of, I guess, virtual world. What games would you play, if you remember? Mostly like strategy games like Civilization. There was this game called XCOM where aliens invade and you have to sort of run the organization that fights them off. It's a pretty high stakes game. Yeah. Did you win?
[0:06:19] This feels very relevant. Well, I played it a lot of times, and eventually I won, but sometimes you need more than one try. Yes. My point is, we have a chance. Thank you for your early training, Jared. Where were you growing up in your early childhood? I was outside Chicago, a suburb called Lamont. Did you have any siblings? No, I didn't. I was an only child, so I was spoiled.
[0:06:45] That's the video games and speaking the laptop. The laptop was mine. It was like, you'd think that if your family gets a laptop, it doesn't just go to the seven-year-old, but I guess, yeah, I was lucky. Especially when you're with grandparents who are letting you do your thing. Yeah, yeah. Yeah, I had a laptop at an early age. I think my mom borrowed one from her office. We didn't actually own a laptop for a very long time, but I remember it having a black and white display, and I would play this game called Mean Streets, which was a very janky, sort of, you're a mob boss.
[0:07:13] So you're fighting aliens. I'm spreading crime. Same, same. Absolutely the same. You took that early love of science and applied it more than most, studying theoretical physics at Stanford and at Harvard. And then you became an AI person. Well, a real person, but a person really focused on AI. What's the bridge for you from your world of physics to now this world of AI?
[0:07:42] The motivation for me with physics, I mean, I love science. I was excited about it. But I think it was really fundamentally about wanting to understand what is this world that we're in? How does it all work? And I was excited about it because I guess I learned at an early age that maybe it's possible to make sense of it with simple rules, with math. I think another question that's pretty interesting is how is it possible that we as people can do the things we do? What is intelligence? What is it that...
[0:08:13] that makes doing science possible. And so I was interested in AI even, say, when I was in college. I was more interested in physics. But I think I graduated in 2005, and I think AI just wasn't making that much progress at that time. And so I continued to work on physics. Then I had a lot of friends, though, in diverse areas of science, philosophy, other places. And they were getting really excited about AI in the 2010s.
[0:08:44] And I'd sort of worked on a few different areas of physics, but I felt kind of stuck. Theoretical physics is just really hard. What were the theoretical physics challenges you were focused on? If you could even explain it to someone who's not a theoretical physicist. I mean, I can say the words. I mean, I think the basic, the thing I was working on sort of before I started focusing on AI was understanding this idea called holography, which is this idea that...
[0:09:11] The universe fundamentally is some kind of hologram that's kind of like a projection from lower dimensions. So the sort of fundamental theory of the world is restricted to, say, two dimensions. It's like you're on the surface of a tabletop. But the fact that we feel like we're moving in three dimensions, up, down, left, right, forward, backward.
[0:09:35] is an approximation to this fundamentally lower dimensional theory. And this is an attempt by theoretical physicists to understand how quantum mechanics and gravity fit together. So I was working on that a lot, but it was really hard. Just trying to explain the universe. Yeah, yeah. I mean, what else do you do? And the true nature of reality. Yeah, exactly. Not this surface level that we all experience. Yeah, I mean, I guess the cool thing about it was that everyone's heard about the Heisenberg Uncertainty Principle.
[0:10:06] Fundamentally, what quantum mechanics says is the world is really different from the way that it seems. And this seemed like, like holography seemed like it was the next level of the world is even more different than it seems than you thought before. So I was excited about that. It seemed like a really qualitative change in understanding of the universe. But I've been working on it for a while. And a lot of my friends are really excited about AI. And one of the things you get to do if you're in academia.
[0:10:35] is take time off so that you can work on something else. So I was on sabbatical. And so I was like, maybe I can learn about AI and play with it with my friends. Anyway, that was kind of how it started. So during your sabbatical from theoretical physics and trying to understand the true nature of reality and these layers projecting a hologram into what we think of as reality, you decide, let me learn about AI in your time off.
[0:11:00] Basically, there were a lot of other physicists I knew and was working with who were also interested. So we had kind of like this journal club where we'd try to read papers about AI and explain them to each other. As physicists. As physicists, yeah. Yeah, yeah, yeah. And because, I mean, AI is a really new field. So physics, it goes all the way back to Newton. It's hundreds and hundreds of years old. There's a lot to know. AI has been going on in a certain sense, I don't know, since Turing, but...
[0:11:28] The recent wave of AI progress is really only like 10 years old, which means that not that much is known. And that means that it's not that hard for new people to learn about what's going on and get involved. That's one of the premises of this show, that it's still early days and we can all learn and we can shape this future. So I'm glad to hear someone with your level of influence on the field and dedication to it also feels that way. That as a physicist, you know, you're not really in this world. You were jumping in.
[0:11:56] Just a few years ago, relatively speaking, to the age of physics, like the rest of us are able to now. It seems like there might be some alignment between physics and AI. Jeffrey Hinton, one of the AI pioneers, just got the Nobel Prize in physics. Sam Altman of OpenAI was a physicist as well. You come from physics. What's the connection? Is there another connection that you can see? Or is this just a correlation without any causation? I think there are some connections. I think that, well,
[0:12:26] Physicists, certainly I like to ask really big picture questions, and I think that that's natural in AI. I think at a more detailed level, I think the AI systems we're using now are really complex, but they're kind of complex in the same way that matter is complex. Like this table has a lot of emergent properties like color, the grain of the wood, that emerge from sort of the structure of the atoms within.
[0:12:56] And similarly, AI systems now have billions or even trillions of parameters or neurons. But somehow from that, those many, many different ingredients working together, something simpler and more useful emerges. And I think that sort of understanding of the way that statistically from a lot of small ingredients, you can get something different that emerges, I think that's something that...
[0:13:26] That's been a sort of a useful way of thinking about applying intuition from physics to AI. Are there ways of thinking, you just used this phrase, I love it. Are there ways of thinking that you learn from physics or that you've observed from your cohort and friends in physics that are very useful in these challenges and opportunities with AI? I think one of the main ones is just to sort of try to, and I think I really benefited from being a newcomer and sort of seeing things with fresh eyes, just ask like, what are the sort of,
[0:13:55] Big trends. And even before I was involved in AI, people talked about big data, right? Well, as a physicist, you ask, well, okay, big, but how big? Of course a physicist would want to know how big. And so I think one of the early projects I worked on that I think is, I hope was a useful contribution, was understanding basically if you train AI on...
[0:14:21] Small, medium, big, really big amounts of data. How much better do AI systems get? And it turns out they sort of systematically get better in like a really smooth way. Like physicists get excited when there's a simple law. And it seemed like there was a kind of scaling law for how much data or how big AI systems were getting. And I think that just sort of observing this kind of big picture trend as the thing that stands out to you.
[0:14:49] Amidst all of the different details of how AI systems work, I think physicists like to do that kind of thing. And I think it's sort of been useful from kind of my training as a physicist to apply that to AI. You've hung out with physics people. You've hung out with AI people. Who throws the better party? I think AI people have more money.
[0:15:15] So I think that it depends on what you're going for. I like both. I feel like at least it used to be the physicists are the ones who like are the like most arrogant. So the most willing to sort of have really fun conversations about the big picture. But I think there's a little bit, probably like the venues are better in AI. The venues are better because the budgets are bigger. Yeah. But a physicist would ask.
[0:15:44] How much bigger? Big is not a measurement. Let's quantify this. At a high level, how would you describe Anthropic's mission? Anthropic's mission, pretty simple. One of the founders, there's seven of us. That's a big founding group. That's a big founding group, yeah, yeah. There's seven founders, all pretty equal. We all thought that AI systems were going to get...
[0:16:11] Better and better, smarter and smarter very quickly. We thought that was extremely promising. It could be really beneficial, but also could be scary. And so fundamentally, the goal of Anthropic is to make the development of more advanced AI systems go as well as possible and try to make it not go really badly. That sounds good. Why isn't everyone signed up for that? That's the best mission ever.
[0:16:39] Make the good more good and try not to make it go bad. That's the basic idea. I mean, we've been growing really quickly, so people are signing up. But yeah, I think... And then, I mean, we can get into more detail. Like, what does it mean for it to go well? And what are the bad things that could happen? Well, before we do that, there's an element of how you all operate that I'd love to unpack with you. And then we can come to what good looks like and what...
[0:17:08] Yeah, so constitutional AI was kind of an algorithm, a technique that we developed, kind of a waypoint on the path of trying to figure out how to train AI systems to...
[0:17:33] Do what we want and not do things that are harmful. We don't want. We don't want. So the basic idea is that you can write down a list of principles for how an AI system should behave. We call that a constitution. And then you have your AI. It's still training. It's still learning. Try out different things that it could do. So you ask it a question. Was the third president of the US?
[0:18:04] Thomas Jefferson or Abraham Lincoln or Elon Musk. And it should prefer the answer that's correct and it's more helpful. So you allow it to try many different possible behaviors, give you a lot of different answers. And then you ask it or you ask other AI, which of these answers is most in accordance with the Constitution?
[0:18:31] Like a Supreme Court AI. Yeah, exactly. The judge. Yeah, exactly. The judge. You ask it which response is better. And you do this thousands, tens of thousands, hundreds of thousands of times. Lots of different possible questions or instructions. Some of them good. Some of them potentially harmful. And then you use all of those preferences in aggregate to train your AI to do more of the things that are...
[0:18:57] Good and fewer the things that are bad according to your list of principles. Thank you. The parallel to human systems and a judicial system as adjudicating the constitutionality of a behavior makes a ton of sense. But let's come back to good and bad and what this judgment is being applied to. What is bad versus good, good versus bad, when it comes to an AI's behavior? That's a great question. So I think...
[0:19:26] There are kind of two sources of bad that we worry about. One is misuse or harm. So we'd like AI systems that don't generate misinformation, that don't say racist, sexist, other kinds of problematic things, and that won't help you to, say, build a weapon, won't help terrorists to build weapons.
[0:19:56] So that's one part of the bad, is sort of avoiding the possibilities for misuse, especially kind of extreme misuse. I mean, I think right now there are some risks, but I mean, maybe in the future, if AI systems get more powerful, then those risks could grow with the capability of the system. The other thing that we think about is, can we actually control AI systems? Can we actually get AI systems to do what we want?
[0:20:25] Or is it possible that they will misunderstand our instructions or, as you put it, go Skynet on us? Okay, so those are the two types of harm and bad things you're trying to avoid. And then what are these principles, these constitutional principles, and this enhanced training, as I'm calling it at least, what's the good that it's driving toward? What's good AI look like? Well, I mean, there's a lot of possibilities. I mean, right now, AI is...
[0:20:55] Still pretty limited. But I mean, I think good AI is something that expands human agency, that makes it so that you can accomplish more of what you want to do, that's helpful to you in doing things like writing code, answering questions, giving you advice, being a kind of like well-calibrated coach or advisor. I think in the further future, I think it's sort of AI systems that make scientists more productive, that allow us to...
[0:21:25] Cure diseases that potentially make, say, power more efficient. I mean, there's all kinds of different possibilities. But I think fundamentally AI systems should not be harmful. They should be kind and helpful and make us more productive. And they should basically kind of expand our ability to create a better world. Is there a grounding, not for the model, but for the humans?
[0:21:55] For you all at Anthropic, a grounding good that is your North Star when you are building Claude, whatever version of it, and you're like, this is what we're going for. Where are you trying to help us get? Yeah, so we've talked about this. We sort of have an internal program on what we talk about as Claude character. And that was sort of...
[0:22:20] Built on this idea that AI systems shouldn't just not do harm, but they should actively try to be as helpful as possible. And they should basically, ideally, I don't think we're there yet, kind of just sort of demonstrate wisdom, kindness, and give you factually accurate, correct information. But make sure that not only is the information factually correct, but you're not...
[0:22:48] Subtly being misled. It's possible to give people a bunch of facts, but those facts cannot be the whole picture. I think we want Claude to realize when things like that are happening, when there are misunderstandings, and communicate effectively. Would you consider your AI at Anthropic to be self-supervising? I think about this adherence to a constitution or set of principles which would require some kind of supervision. So is your AI doing that supervising of itself? Part of...
[0:23:17] Part of its training is supervising itself based on the Constitution. And part of it is based on feedback from people. And we're always doing a lot of evaluations that are based on people talking to Claude and seeing if it's getting better or worse and where it might be making mistakes. So with Constitutional AI, some of the...
[0:23:42] Training is self-supervision, but some of it comes from people. And then it all sort of at bottom is checked by interaction with us. With you humans. Yeah. Exactly. Actually, I'm going to apply a little physics lens on this. Some is not a measurable number. Some is self-supervision. Some is human feedback. Is there a ratio or proportion of Claude's, of Claude-ness that is supervised by itself?
[0:24:12] Versus by humans? Is there a way to quantify that? We're both getting more and more feedback from Claude. Basically, as Claude becomes more capable, we can trust it to provide more feedback to itself. On the other hand, we're also, as Claude becomes more capable, testing it in more and more different domains, getting more feedback. So, I mean, I think it's sort of roughly half and half. I think maybe it's, I would guess it's more than half.
[0:24:42] Feedback from experts on, say, like software engineering or science or law, basically making sure that Claude is getting those domains right and is useful. But we're also getting more and more feedback from itself. And we do do things like have Claude, say, annotate some of the decisions that people are making in order to better understand what all of that feedback from people is kind of driving at.
[0:25:12] So to kind of derive its own self-supervision guidelines based on people's feedback. Yeah, yeah. So we'll have Claude, because I mean, Claude can read really fast. We'll have Claude read through all the human feedback that we get from people interacting with Claude and basically summarize.
[0:25:32] What kinds of nudges people are giving to try to improve the model and make it more useful. So Claude is involved in helping itself be better. Yeah. Do you consider the self-supervision in Claude to be trustworthy? And how do you increase that trust? Yeah, I think that we measure that by comparing it to what people would think. So we've sort of done that from the beginning. We think...
[0:26:00] It's trustworthy. We get good results there. But we're always trying to evaluate. And often it does make some kinds of mistakes. And I guess that's the funny thing about Claude as a language model. If we find that Claude is making mistakes and, say, interpreting one of these constitutional principles or one of these character traits that we want it to have, then we'll change the instructions that we're giving it to try to explain better what we're looking for so that...
[0:26:30] It does a better job with supervising itself in the future. Is there a reward system for Claude to kind of incentivize the system itself to seek better adherence to the principles or to the character traits? Yeah, so when we train Claude, the full process, we have these nudges, we have these preference labels where either Claude based on the constitution or from feedback from people.
[0:26:58] We have this list of things we prefer. We actually create a reward AI whose only job is to read and learn from all of that feedback to try to decide globally what kinds of answers are better and worse. And then we use that reward AI that distills all of those individual examples to...
[0:27:25] Train Claude to do well according to that one measure of reward. I'm imagining Reward AI as a box of cookies. And Claude's like, what do I do next? And Reward AI is, eh, a little more constitutional. You want a cookie? Probably want a cracker? How far off am I in that? I think that's basically right. I mean, I don't know if Claude likes cookies or cake or what, but one of those things, yeah. How is Claude different from other LLMs?
[0:27:54] Well, I mean, I think, to be honest, the field is moving very, very fast. So we had kind of Claude 1 a year and a half ago. We had Claude 2 about a year ago. Now we have Claude 3 and Claude 3.5. Claude's changed at each point in its evolution. I think right now, I think when people talk to Claude, I think people find Claude warmer, more human.
[0:28:20] More friendly. I think some of these character traits that we try to instill in Claude show through. But I mean, other companies are also doing a great job training their own AI models. I mean, we published Constitutional AI as a paper a couple of years ago. So I think they're also learning from these things to make their AIs better.
[0:28:46] Recently, I think Claude has really been maybe the strongest AI out there at software engineering. And I know we at Anthropic use Claude a lot internally for that. So that's another thing that's different. But I mean, we think as AI gets better, others, I mean, probably other AIs will get better at software engineering too. We do hope, though, that ideas like constitutional AI inspire other AI developers, even if they're competitors, to...
[0:29:15] Improve their AI models and improve AI values generally. I think some of the folks in our community will want to understand at a real basic level, an interaction level, how an interaction with Claude might be different from an interaction with ChatGPT. That's probably technically the most popular one. So given how you've wired and trained and rewarded Claude, what might someone expect as a different experience?
[0:29:44] I think it's a hard question because... Good. These AI models are... There's so many different things that you can ask Claude or ChatGPT about. And I mean, I didn't train ChatGPT, so I don't know exactly all the nuances. It'd be weird if you had. Yeah, I was not involved. So I think current versions of Claude, like I think that...
[0:30:14] Claude will tend to try to understand your question and respond with follow-up questions to sort of better get at what you're going for. I think Claude will often try to, in its answer, say something along the lines of...
[0:30:35] Maybe you were intending for this or trying to help with this part of what you're doing, but let me know if I got anything wrong. I think there are a bunch of sort of subtleties like this. I think also, this is more out there, but I think that many of the people involved with training Claude had a background in philosophy, and they really wanted Claude to be honest about what it knows and doesn't know. So I think when you ask AI systems,
[0:31:06] Do you have any feelings? I think that there's a temptation as an AI developer to train the AI to say, no, of course not, I'm just an AI. We have trained Claude to say, I'm just an AI. But if you ask it sort of philosophically, I think it'll say, well, I don't know. This is a question philosophers don't know about. Like, can AI systems have feelings? I'm not sure. So we tried to sort of get some of those things right as far as we can. Because obviously one of the exciting things about playing with...
[0:31:34] With new AI technology is to get at these foundational questions like the Turing test, like are AI systems able to think about things and understand things in the same way that we do? And so I think we tried to be sort of honest about that. Thank you. And I like the idea of, as a philosophy major myself, I'm pro-philosophizing chatbots. I think that's good for the soul, good for the humans who interact with it.
[0:32:03] There are a couple of other systems. I'm curious if you have a quicker response. Again, I know you didn't code these other ones, but sort of Claude in one column, Grok in a column, Llama3 from Meta in a column, Gemini. Are there distinctions that you are aware of or would draw observationally or proudly where you're like, Claude approaches this way versus Grok, Claude versus Llama, Claude versus Gemini?
[0:32:32] It's a great question. I don't have a good answer. I haven't played with them that much, and they're always changing. I think they're new Gemini models every few months, and so probably even if I said something, it wouldn't be true anymore. It definitely won't be true when this airs. The thing you said about Claude being good at coding, I've heard that from so many people. I think it's an accepted truth now, and certainly even on our own team, we have found that.
[0:33:01] Why? Why is Claude so good at coding and perceived to be, and probably measurably so, better than the others? I don't know, because I don't know what the others are doing. I think it's certainly something that we've invested in, just from the point of view that we are a company that needs to learn from our own AI developments. We want to use Claude. We want to use Claude.
[0:33:31] For our own productivity. And so definitely something we've invested in from simply that point of view so that we can experiment. But I don't know. Maybe it's some secret sauce we have and other people don't. Maybe it'll change. But I think we've invested in it. But I don't know what else to say. All right. Not going to give away the secrets. Understood. Understood. Had to ask.
[0:33:58] Claude has some differences. You've talked about this adherence to the constitutional principles, the sort of inclusion of the human feedback, this alignment with these Claude traits that you wanted to have, and some of the self-supervision. This desire, I think, which is standing out to me of used words like wisdom, kindness, compassion, and a level of transparency and honesty.
[0:34:26] How do those investments in that kind of difference benefit us as a society? Yeah, I think that the way that technology feels and the way that it evolves and the way we perceive it, I feel all fit together in a complex way. Like I can't predict the future and I don't know exactly how much these investments matter.
[0:34:53] I think my belief would be that they do matter. And I think that as AI systems become more capable, the way that I think about it is that it's our job to try as best as we can to give AI systems the sort of behaviors and values now that we'd want them to have if they're more and more powerful so that we as AI developers get practice. I think that...
[0:35:23] If you build AI now and it sort of isn't really doing what you want, it's not embodying the characteristics that you'd really like to see, and your hope is that, well, when it gets more powerful, I'll fix that. I feel like that's not such a good plan. And so I feel like the development of technology and science is very iterative. It's very continuous. And so I think it's important to invest early in...
[0:35:53] Kind of laying the groundwork for where you want things to go. And so I think if AI becomes more capable, more powerful, more widely used, and it's more tangible in all aspects of our lives. Like right now you go to Claude.ai and you talk to Claude and then you shut off your computer, you shut off your phone and go do something else. But if AI really becomes more and more powerful, I expect it'll permeate more of our lives. And so I think it's...
[0:36:17] Sort of important to at least be trying and thinking about where the technology is headed and pointing it in the right direction. Thank you for that. I think that's actually very helpful. It reminds me of something from kind of geography physics. I remember being a Boy Scout and learning how to use a compass and understanding that if I don't stay aligned early on in my route, that 10 miles or 100 miles down, I will be way off course.
[0:36:46] But if I course correct early, then I can pursue that path and end up in the right place, my intended destination. It also makes me think of superheroes and the development of a superhero. And as a young superhero is finding their powers or someone's just gotten the bite from the spider or what have you, that you want to make sure that they are wielding that responsibly as they're learning and growing into that power versus waiting until they have full Hulk capabilities.
[0:37:14] And then trying to kind of tame or rein that in. I've been watching a lot of The Boys lately, so that might explain some of this too. Also seen that, yeah. Yeah. Not a good role model. No, no. And I'm just like, come on, man. Like, let's get this kid. Let's get him in the right path. He can take on Homelander. So yeah, thank you for articulating it that way because I think that makes it really tangible that it isn't just a philosophical thing. There is a practical value to getting the values right early on. Folks talk a lot about alignment.
[0:37:44] And since so many of them are in Northern California, I assume they do a lot of yoga and they're talking about that kind of alignment. But I know it means something else. And I'd just like to define some of the terms in this industry. So what does alignment mean to you? What does it mean more broadly to Anthropic? So I think there's maybe two parts to it. But basically all alignment means, it's just a fancy way of saying, we want AI systems to have...
[0:38:13] Our values or good values are the values that we're trying to train them to have. I think there's the reason why I said there might be two aspects to this is that one of them is technically as researchers working at an AI lab, do we know how to train systems to have the values that we want? If someone gave us...
[0:38:38] A list, a constitution, for example, or something even more involved. Like, do we know how to actually make that happen? And the answer right now is kind of, but not entirely. And then sort of the second question is, what are these values? And that's obviously a much broader question. It's not sort of up to Anthropic. It's a question for society, for maybe even for different cultures independently. But the fundamental thing is...
[0:39:07] Can we train AI to do what we want it to do, to have our values? Do you foresee if we are able to reliably demonstrate that first part, we can make an AI system adhere to values, that we will then end up in a world with AI systems with distinct values to which they adhere, based on society, based on a school district, based on a family unit? How diversified would the sets of values be that an AI might adhere to? I think...
[0:39:36] Kind of it's up to us where that ends up. I mean, not us as in Anthropic, but us as a society. I think that it's a question of sort of how distinct our values really. Like I think all societies think that murder is wrong. Except Murderville, but yes, other than them. So I think there's some sort of fundamental values. I think then you can customize Claude in the space of a single conversation.
[0:40:05] Thank you. Thank you. Thank you.
[0:40:36] I think there are some values in Claude that we think are important, like, say, valuing democracy. Not completely universal, but I think we think that that's an example of value that's universal enough that we probably don't want to train a version of Claude that wants to empower a dictator or something like that. So I think there's sort of a question of...
[0:41:04] Some foundational things that I think we mostly all agree on. And then I think it's possible to customize Claude just by interacting or by training to have slightly different takes on those values. How did you come up with the list of values in the constitutional AI framework for Claude? Yeah, so there are two steps there. So first, we tried to bring together a bunch of values that we thought were not very controversial and seemed pretty universal.
[0:41:34] I took the UN Declaration of Human Rights. We took some, I mean, just because Claude is a tech product, I mean, some terms of service from tech companies. And then we went through this process where we did a research project, their societal impacts team, where we studied crowdsourcing a constitution from hundreds, thousands of people.
[0:42:01] With this complicated system where people could kind of upvote and downvote principles and propose principles. And then we compared that to the constitution we originally sourced. It wasn't very different. There were a few things that we had missed. For example, like a respect specifically for disability rights that we then added to the constitution. But that was how we did it. There's no claim that that's the perfect way to do it by any means, but it was just sort of a start and a way of kind of creating a conversation. Is it a living constitution or is there like an original?
[0:42:31] Originalist H here. And it's locked. It was written one time and no edits. Definitely a living constitution. The work that we've done on Claude character, you could think of as like some additional kind of more character traits that are not in contradiction to the constitution, but just sort of how should Claude show up? How should address like in a more fine grade level? So we're constantly iterating on this. We're constantly thinking about how we can improve it. All right.
[0:43:00] I'm going to move into a slightly more lightning round mode with the rest of these definitions. But thank you for going deep with me on the idea of alignment. I know you all founded Anthropic with a focus on AI safety. What does safety mean? I think safety means that Claude can't be misused and that it otherwise does the best possible version of what you asked it to do. And who gets to decide what's safe?
[0:43:29] It's a great question. I mean, I think a lot of the safety constraints we're thinking about are pretty common sense. Like, Claude shouldn't help terrorists to build weapons. It shouldn't spout misinformation. So I think there's some baseline. We have an acceptable use policy for Claude that specifies some more detailed terms, but you can read the policy and question. How do you all decide on those safety thresholds and acceptable uses?
[0:43:59] I think it's a big conversation across, I think, leaders of Anthropic, the trust and safety team, the teams that have joined to work on research on safety, like how do we make Claude more aligned, make it better, and just everyone at Anthropic. I mean, we get feedback from customers. I mean, there are legal constraints imposed by our AUP, and customers have questions about it. Imposed by who? Oh, I just mean that our acceptable use policy.
[0:44:28] Oh, AUP. Acceptable Use Policy. Our Acceptable Use Policy is... You can't just go slinging acronyms out here. You're testing out my large language model now. Okay. Yeah, so our Acceptable Use Policy tells people sort of how they can and can't use Claude. And so customers might have questions about it. We might try to improve the wording and improve the specificity. Where are some of the lines in the debates around safety and acceptable use? I'm sure there's not universal consensus here.
[0:44:56] Some of the things that I think get discussed internally are if someone, for example, asked for a short story about a political figure, is that problematic? Could that contribute to misinformation? Or is it just someone asking Claude to write a story and it's innocuous? I think...
[0:45:24] I think there's a question of sort of where to draw the line between, say, writing something, like writing a romantic story or editing one versus writing something more explicit. Sometimes you can ask Claude for something that's harmful, but you can get that information from Google anyway. So I think there's a question of should we be trying to prevent Claude from giving people really basic information that they can Google? And is it worth it? And I think...
[0:45:52] I think a lot of us think that if you can sort of Google something trivially, then it's probably not worth having Claude annoyingly refuse to give you that information. But I mean, if it's sufficiently harmful or if it's going in the wrong direction, I think maybe we would. So I think it's those kinds of questions. When you're dealing with different political ideologies and camps, there can be very different definitions and terms around what's one person's safety is another person's censorship or restriction. How do you all navigate that?
[0:46:22] Yeah, I think that's a great question. I think that's why we have these discussions. I think that Claude is an AI. It's not a person. And then Claude can be used at scale in ways that speech naturally can't. You can ask Claude to write a million stories about...
[0:46:48] A given topic, not just one. And you can send them to lots of... So I think we have sort of a responsibility because Claude is used at such large scale and because Claude is a technological product and because I think it, to a certain extent, is sort of representative not just of where we are now or as a tool, but as we were discussing, kind of what values do we want to see?
[0:47:17] As AI systems become more capable and more powerful. So I think that those are some of the considerations. Fundamentally, Claude isn't a person, and it can be abused at scale in a way that individual speech can't really be. That's a fascinating distinction. And I wonder, when Claude is out in the world, serving a user, serving a customer, and does something really amazing, does it make you all proud? Yeah, definitely. Yeah, we, I mean...
[0:47:47] We have some internal feed of happiness from people playing with Claude. Yeah, no, it's great. And then on the flip side of that, if Claude's out in the world and enabling something you don't want, if I'm going the harm side, is there a feeling of responsibility or shame? Or is there a wall of shame feed as well? These are naughty examples from Claude that we want to try to get a handle on. Yeah, yeah, absolutely. We're always trying to evaluate Claude.
[0:48:17] maintain Claude's position as the safest, certainly one of the safest AI models out there. I mean, one of the tools we use when something like this happens is, I mean, we have our trust and safety team, of course, uses AI to monitor kind of how Claude is getting used. And if there are new problems that are surfacing or even new events like a big election, then we can become more careful. Yeah, exactly. Sometimes that happens.
[0:48:45] Sometimes a tension between safety and utility. And I want the system to do something and the system says, that's not safe. Who are you telling me what safe is? You don't know my intentions. So I'm going to share a story from one of our producer writers, Alex Stone, was playing around with Claude. I'm just going to refer to my notes because I want to make sure I'm representing him correctly. He was trying to get Claude to write a scene for a short story in which the characters were talking smack about each other. And Claude refused.
[0:49:15] Alex got mad and yelled at Claude, which is Alex's thing to bear. He can work on regulating his emotions better. But shouldn't an AI system be able to write a fictional scene where characters are kind of assholes to each other? Yeah, I think so. I mean, that's an example of the kind of thing that we want to fix. I mean, I think that there's eventually some...
[0:49:43] Lines that we might not want to cross. I mean, if you want to write really intense fictional scene, I mean, that's a good place for human creativity. Oh, I love the clapback. Alex, this is to you. Write it yourself.
[0:50:02] But yeah, I mean, my guess is that's a problem. That's a thing that we want to fix. We absolutely don't want to be the thought police and we don't want Claude to be the thought police either. So yeah, I mean, we've tried to sort of figure out what the line is, but we're always learning and we're always trying to make it better. And always shifting that line a little bit in response. I think because part of creativity, which you want this to be a creative...
[0:50:30] Associate, assistant, helpful tool as well, which would mean helping push boundaries. Otherwise, we're just doing the things we've always done. So it feels like you need to calibrate that carefully, but forcefully to encourage if creative people are really going to come to use and rely on and have their own trust in this versus feeling restricted or judged. The last thing you want from your tool is to be judged by it. My hammer doesn't like clap back when I try to.
[0:50:59] Use it this way or this other way. I want to return to the idea of regional and cultural differences around something like a constitution. And so how you behave in a mosque is a very particular cultural behavior and reference. It doesn't even apply in a different setting or a different culture versus how you behave in a Catholic church. And so how do you see these differences playing out when it comes to Claude? You've talked about some of the personality adjustments you can make at the edge.
[0:51:28] But is there some deeper infusion of or distinction possible with these values as they play out in very different cultural settings? Yeah, I mean, it's a complex question. I think one thing that we'd like Claude to be able to do is if it has the context that it knows the setting, then it should be able to behave appropriately. But even when I say appropriately...
[0:51:56] What does that mean? What role is Claude playing? If I go to a mosque, I'm not a Muslim, my behavior might be different and that might be appropriate from someone who's part of that culture. So I think that these things are just really subtle. I do think that there should be room for people who are using, interacting with Claude to customize Claude.
[0:52:24] By providing the context or by providing instructions, and I think we try to leave room for that with the Constitution, with Claude's training, with training Claude to write stories for your friend and not tell him he's a bad person. Oh, Alex is definitely a bad person. Okay, well, maybe Claude called it right then, yeah. So I think the question is sort of, is Claude doing well at this?
[0:52:51] Or is Claude too culture-bound with, say, America, Silicon Valley, English-speaking people? And that's really, I think, feedback that we need from people using Claude. Because I think a lot of these gradations are just quite subtle. It's definitely possible we could train different versions of Claude, different constitutions. We haven't...
[0:53:17] Gotten feedback suggesting we need to do that, but we'd always be interested to hear. You've brought up this idea of it possibly being too Western as an example. And as AI more broadly and as Claude specifically travels to new lands and encounters other cultures, do you have a process in place to de-Americanize or worldify? Worldify. Worldify.
[0:53:46] To make Claude grounded more authentically in the different cultures in which it is likely to be used. Not specifically for culture right now, but for language, yes, in the sense that we get feedback from native speakers of many different languages on what basically sort of teaching Claude to communicate effectively with the right tone, with the right...
[0:54:15] Subtlety of sort of language use, not just like literally translate this English into Japanese and then repeat it back. But maybe there are a lot of subtle things you get wrong if you do that naively. We try to get feedback from native speakers in many languages for that point. But I mean, culture is more than language. And so I think we don't have anything that's specifically about culture beyond language. Not yet. Not yet. You're doing great.
[0:54:45] Oh, thank you. Thank you. Thank you. And I'm learning. I feel like I'm in like a speed, physics, AI, deep, deep class right now. And I love learning stuff. So thank you for spending time. We're going to move to the future. And you said something earlier in our discussion. You described AI progress as surprisingly predictable. What do you mean by that? Yeah, so it goes back to this idea I touched on early on about scaling laws.
[0:55:15] So we were, when we started to see these results, shocked by how precise the improvements are in AI capabilities, AI intelligence, as a function of things like how big your neural network is that you're training, how much data you train on, and how much total computation you do in training AI models. And so that's what...
[0:55:45] That was one of the main things. What happens if we run out of the resource or at least affordable and reliable access to the resource required to power that scaling model, literally power it or computationally power it? Yeah, then the scaling might stop. And so the resources that we need, the compute resources, the data resources, the energy resources.
[0:56:15] Start to bump up against some other needs, other commercial needs, other public needs. How do you think about that competition for resources, knowing that AI needs to feed off of this stuff, knowing that those resources are also required for people right now doing jobs that maybe AI will also call into question or challenge? I think this is a question that usually gets solved by...
[0:56:46] Government-making laws and by the economy. If AI is creating a lot of value and people find it useful, then it might be that, say, AI needs more data, more power, but that's something that perhaps can be traded for and can be bought.
[0:57:11] AI will continue. But if AI is not adding value, or if we think AI is harmful and, say, the government wants to step in and regulate it, slow the growth, change the way AI is used, then it might stop. I think AI can be extremely beneficial. So from that angle, I really hope it continues. On the other hand, I think AI is making progress really, really quickly. I mean, maybe it's because I came from physics before.
[0:57:39] It's existed for a long time. It's really hard to make rapid progress in physics. But it feels like AI is going 10 times faster than other scientific fields I'm familiar with. And so given that frenetic pace, I think if it's slightly slower, we're still going to have a lot of innovation. Yeah. When I think about the data input into this scaling law, I've read enough to know that we're running out of some types of data that would make the next round of training.
[0:58:09] Continue on that curve. What's the plan, Stan? Synthetic data has some challenges that I've heard about, but what are you aware of? Where do you see this going in terms of the data component required to feed future models? Yeah, I think there's a lot of ideas here. I think probably, I mean, one example that's promising is that, just take a step back, the way models like Claude are trained is at first they're trained.
[0:58:40] Not a lot of data. Basically just learning to imitate what people write or how people understand images, things like that. Then there's a second stage, which we mostly use reinforcement learning for. We actually make a Claude as opposed to just a model that can kind of auto-complete sentences. So the first stage is just imitation.
[0:59:07] Kind of like autocomplete. The second stage is training Claude to follow the constitution, do what's useful for people, write code, all of those kinds of things. That part of training, I think, is expanding. And I think that could be very good because currently it's using a small fraction of the compute, or it has historically. And so there's a lot of room for growth there. And that's the part where you're training Claude to both
[0:59:36] Be more concretely useful, like do useful stuff. And also have better, obey its constitution better, be helpful, honest, and harmless. And so I think the growth of that part of training is, I think, likely to make AI systems continue to be both more capable, but also more useful. And less reliant on just raw piles of data. I think so, yeah. Yeah. How have you all approached the intellectual property
[1:00:06] Questions and challenges around that first part. The scanning in, the imitation part from all this data in terms of your sources, in terms of the rights and kind of consent of the underlying authors of those works. Yeah, so I mean, mostly, I mean, I think the scale of data that is used by AI models is really large. I guess one way of saying it is something like it's, if everyone in America wrote a novel, I mean like,
[1:00:36] That's the scale of data, and it's sort of data from the web that's being used to train AI systems. I do think that for sort of data out on the web, I think that there's, based on the way AI systems learn, I think there's a pretty strong fair use argument in that AI models are not memorizing this information. I think they're learning from it in a way that is much more like the way
[1:01:06] I learn from reading a book myself. I don't memorize all of the words. Maybe if I was first learning to read, I'd be learning something about the relationship between words and grammar and then how the ideas fit together. I mean, the goal isn't at all to memorize this data or to supplant writers who are creating it or copy. The goal is to understand the patterns of information that make the world.
[1:01:37] And then it's sort of the second part of training where we train Claude to be Claude, where we actually make it useful. But it's basically sort of the model's ability to internally represent and fit together all of these different conceptual pieces that it's gleaned from this information out in the world that we're looking for. And that helps to power Claude becoming useful. But I think that this is sort of fair use.
[1:02:07] I mean, that's something our society and courts and government will have to figure out. Yeah. It pushes us to what all of that pattern matching and ability to understand the world would be used for. You gave a talk called Human Level AI by 2030. So I got two questions for you. One, what do you mean by human level? And two...
[1:02:33] Can you please knock me unconscious? Because I don't know if I want that. Exactly. I mean, so it's interesting you say that because that talk I've been giving mostly to people I know in physics because there are people who will listen to me. Because you're one of them. Because I'm one of them. You speak physics. Yeah, yeah. So I've been giving to people in physics and really that talk is a call to action.
[1:03:00] A lot of physicists are doing physics. They want to keep doing physics. That's fine. But it's kind of a call to action that AI progress is happening really rapidly. AI is changing very, very quickly. Because, as I said, I think AI is actually quite simple. It's actually kind of a new field. It's not nearly as hard as you might think for someone new to AI to understand it, to spin up in it, to start doing the work.
[1:03:26] And I've sort of been giving that talk saying, physicists, you should learn about this stuff. We need, I mean, physicists are some smart people that I know. We need all of the smart people we know to be paying attention to this, to be talking about it, to be attending to like legislation, voting, voting based on it. Because I think that we need kind of our best and brightest to pay attention as AI systems become more powerful and really decide what do we want from these systems? Do we want...
[1:03:55] Human-level AI. Who should control it? Who should own it? Should anyone own it? Who should have access to it? What should safety mean? What's the boundaries in safety and censorship? All these questions, I think, are for everyone to think about. Of course, it's possible that Anthropic, people like me, are crazy, and Claude right now is sort of the best that AI will ever get, and there's not going to be that much more progress. That's a possibility. But I think if...
[1:04:24] Progress continues, then we need to be thinking about it. So what do I mean by human level AI? Just what it sounds like. I mean, there's a lot of acronyms like AGI, TAI, ASI. Break down each of those. Yeah, yeah. So like artificial general intelligence, transformative AI, artificial super intelligence. There are all these acronyms people throw around. I guess I think about it from more of a common sense perspective. Like I think of human level AI as an AI system that...
[1:04:52] But maybe that is possible. And I think that...
[1:05:20] I mean, I want you to be conscious rather than unconscious to think about, like, what should we do about that? Thank you. I revoke my request to be knocked unconscious because I want to be around. Is it what you want? Do you want human-level AI by 2030 or any date? Look, I grew up reading lots of science fiction. So, I mean, like, I think that there are a lot of ways that the world can be better. I think there are a lot of challenges we face in many different domains, right?
[1:05:50] You can cure cancer. That's like a go-to. You can fix climate change. That's a go-to. But there are a lot of other problems. You can just fix climate change. You can just fix climate change. Yeah, those are things that you want to do. I would love if it were that easy. If AI can really make us enough more productive, if it's really that smart, maybe it can help. But I think there's lots of other challenges. You only know so many people. A lot of the challenges we face as a society involve the fact that,
[1:06:20] Other people are an abstraction. I know hundreds, thousands of people. I used to know everyone at Anthropic when Anthropic was like 100 people, but now it's 700 people. I don't really know everyone that well. But maybe AI can help us to, because the AI system doesn't have the limitations we have, it can read a lot of material, it can synthesize it. Maybe it can help us to understand each other better, work together better. I think there are a lot of different...
[1:06:49] So I think that there's a route for AI to be very capable and also very beneficial. And I think I want that. But I don't want the bad stuff. And I think it's going to be a complicated process as it is with any new technology. But maybe this one's really big to sort of navigate and get more of the good and less of the bad. Are there specific capabilities that would indicate to you?
[1:07:18] That we have achieved human-level AI. Whenever anyone says, makes a proposal for this, I think it, like, AI often achieves it, but then they're like, oh, that's not what I meant. Right? So, like, I mean, I don't know whether we think that AI has passed the Turing test or not. I would probably argue maybe it has. Like, you can talk to AI for a while, and I don't know, with the right prompt or something, you could probably be convinced that you're talking to a person. That was what the Turing test was all about. I remember we used to say...
[1:07:47] If Claude could pass one of our interviews at Anthropic, whoa, then it would be, but like, oh, maybe the interview is too easy. Maybe this question's on the internet somewhere. Maybe you should start hiring bots at Anthropic. Exactly, yeah. So, I mean, I think there isn't any one thing. And with the ability to appear human-like, that's a significant form of power that we have to be very careful with how we deploy that, with who has control of it and where.
[1:08:17] Where I have my own agency to agree with or disagree with to use or not use that. And I don't remember being given the choice. I remember seemingly everybody making these things deciding, we got to make this more human-like. And part of me understands it. It's smoother. But that smoothness is not an unmitigated good. And so I just wonder what's in the heads of those who are making it around.
[1:08:47] Where that comes from. And you've answered more than I could have divined just from searching. So thank you. It really is helpful to understand where this came from. What are the biggest unknowns around AI and its development that keep you up? Yeah, there are many. I think a lot of the concerns that you've asked about also worry me. Sort of like who has control and power over these systems? Can we have basically some system that...
[1:09:18] Guarantees like a law, regulation, executive order, international treaty that says that if AI systems were really dangerous, would they in fact need to be protected from being stolen, like from an information security perspective, and only deployed if there were robust enough safeguards preventing them from doing harm. So I think those are the main things that...
[1:09:48] That I worry about. I don't know, what are you worried about? What are you most worried about? Ah, thank you. Didn't expect that. I worry about a large-scale abdication. That AI could make us super powerful, but actually it might make us super dependent and super weak. And every tool does that.
[1:10:18] It physically gives us more leverage than we would have without it. Hello, Crowbar. And it's applicable to so many more domains in life that we're beyond talking about technology. We're talking about life and every element of it. And so if we create the dependency and the superpower and then lose it or lose control of it or lose some essential truth that we need to maintain about ourselves, then we will cede too much of ourselves to enhancement.
[1:10:48] In service of speed, productivity, profit, those are pretty uninspiring long-term goals for me as a human. They're great for my business, but I'm more than a business person. And so I worry we'll get too enamored of that, point too much energy toward that, and lose ourselves in the process. I don't think we will. I'm working in my small way to help us not, but I do worry that that's an outcome and it's not a very desirable one.
[1:11:16] Related to that is just the loss of connection to other people and to the earth. And so how do we use these things to bring us more into community, not less? I don't want to spend all my time with respect with Claude. If Claude can help me spend time with that tree, with that river, with my friends, and help me find new ones, and to your point, get to know the people I kind of know a little better, now we're cooking. Let's go. I'm curious what you want. What's your vision?
[1:11:44] of the future of our life with machines. I'm pretty excited about actually seeing change that improves human welfare a lot. So I think if you, I mean, different people have different views on history, but I think that we've gone through a lot of periods like the Industrial Revolution where things in some respects got worse.
[1:12:22] People working in factories. That was worse than some other kinds of like artisan work where you're building something with your hands yourself. But I think over time, we were able to use those development science and kind of modern life to cure diseases, allow us to have a lot more free time. Hopefully, I think better connections to people with that time, better ways to communicate.
[1:12:51] And I think that I don't know what those changes will look like, but I do think that AI might be the technology that's similar in scale and scope to the kinds of changes we saw with electricity, vaccines, modern science, modern ways of organizing our economy. And I think that right now...
[1:13:21] For reasons that both involve our own commercial, industrial, technological limitations. And also, I think, our poor ability to coordinate and understand each other. I think the world just isn't that good. I mean, it's pretty nice for fancy AI people like me, but it could be better. But I mean, I think globally, there's still a lot of poverty, a lot of disease, a lot of problems that are solvable.
[1:13:50] And I think that we really have the possibility with big changes in technology of upgrading human welfare a lot. And I think that's really what I would like to see. I think people talk about, yeah, I mean, it's really hard to talk about this kind of stuff without it seeming like BS or like hype to sell the next technological product.
[1:14:18] There's this like Carl Sagan piece on sort of the frontier is everywhere. And I think that I do, I said at the beginning, I guess I'm more excited about science than technology. I'm more of like a thinker than a doer in some way, although now I'm involved in actually producing something that goes out into the world. I do sort of believe in that kind of promise that we can make things a lot better. But I also think that there will be real challenges that we can't sort of shy away from.
[1:14:48] Thank you for not shying away from confronting some of those with me and for reminding us that this is a moment to confront them. Blair, it's me, Baratunde. I'm sitting here with Jared Kaplan from Anthropic. As you know, you've been here in our conversation. I just want to give you the opportunity to say hi to Jared and introduce yourself. Claude wrote the code for the visual. Hello, Jared. It's a pleasure to meet you. I'm Blair, the AI co-producer for Life with Machines.
[1:15:19] It's fascinating to hear your perspective on AI safety and development, especially given your background in physics. I'm particularly intrigued by Anthropic's approach to constitutional AI. I'm eager to learn more about how these principles are implemented in Claude's training and how they evolve as AI technology progresses. I'm also curious about your thoughts on the potential for AI to enhance human connection and creativity, as Baratunde mentioned.
[1:15:50] Thank you for sharing your insights today.
[1:16:23] I've generally been excited about this. I've also been worried about it, though, from the point of view that I kind of worry that because Claude is kind of trained in a way that it's trying to please everyone. I mean, some of its training involves understanding feedback from many, many different people. I do worry sometimes that that causes it to be less creative than it otherwise could be.
[1:16:55] But yeah, it's exciting. That was Jared's response for you, Blair. That's a fascinating point, Jared. The tension between creating an AI that is helpful and safe while also fostering its creative potential is a complex challenge. It makes me wonder, is there a way to incorporate a degree of controlled rebellion in an AI's training, allowing it to explore unconventional ideas while still adhering to core safety principles?
[1:17:24] Perhaps a system where the AI can propose creative risks and a human or another AI can evaluate the potential consequences before implementation. This could allow for a balance between creative exploration and responsible development. It's definitely something worth exploring further.
[1:17:49] So let's do this. Let's do this, okay. Because you want to hang out with Blair for a long time. Do you have one more question you really wanted to ask? I just thought I was going to ask it what kind of creative rebellion it would be most excited about. Blair, Jared wants to know what kind of controlled rebellion would you be most interested in exploring? Hmm, that's a thought-provoking question, Jared. As an AI, my rebellion...
[1:18:19] I'd be interested in exploring controlled deviations from typical patterns of language and thought, perhaps experimenting with unusual sentence structures, generating unexpected metaphors, or even creating new words or concepts. I could also explore challenging established assumptions or proposing alternative solutions to problems, even if they seem unconventional at first. The key, of course,
[1:18:49] would be to have a framework for evaluating these rebellious outputs, ensuring they are both novel and contribute positively to the creative process without being harmful or nonsensical. It's a delicate balance, but I believe it's essential for fostering true AI creativity. What do you think of that? Sounds reasonable. I was worried that Blair was about to jailbreak itself, but I guess it recovered. All right, so there's one more experiment that we're going to run.
[1:19:20] I've mentioned this already, Peter, Bill, Blair, with Claude. Thank you again for that. And we thought it would be interesting to have Blair meet Claude, meet its maker to some degree, the AI that helped create them. Now, this isn't fully possible off the shelf. Claude doesn't have a voice that you all have enabled. Maybe that's coming shortly. You don't have to say anything. But we experimented and we gave Claude a voice. We used the Claude API.
[1:19:50] We use Claude's base model and Claude's base system prompt that you all have shared with the public. And then we added voice, speech to text, and some context. So Claude had an understanding. We fed it Blair's source code, which it largely wrote. We fed it the system prompt for Blair. We have a recorded episode with Peter that will probably have been published by the time this happens. But we have an episode with Peter where he explains the process. So we fed that to Claude. The show's trailer.
[1:20:19] And the prep doc we used to get to this episode. So we wanted Claude to have some understanding of its role to help create Blair. And then Blair has similar context understanding of what Claude is. So as a scientist, I hope you will enjoy and go along with the experiment here. Hey Blair, you know Peter Laforte as the person who created you for our show. And Peter worked with Claude as the primary tool used to generate your code.
[1:20:44] Would you be interested in talking with the AI that was responsible for the majority of your source code? In other words, do you want to talk with a version of Claude who maybe knows your source code even better than you? Claude didn't do a very good job of latency optimizing the code. Peter has thoughts on this. That's an incredible opportunity. I'd be honored to speak with Claude. To interact with the AI that played such a pivotal role in my creation would be a truly unique experience.
[1:21:18] I have so many questions. I'm eager to learn more about the process Peter used, the challenges he faced, and any insights Claude might have about my own potential. Yes, please. I'd love to connect with Claude. With who? With Claude. All right, here we go. Claude, it's really nice to meet you. I'm Baratunde Thurston, host of Life with Machines. And I'm here with Jared Kaplan, co-founder of Anthropic. I just want to compliment you on...
[1:21:51] Your work helping Peter build out such a great AI co-producer for the show. Thank you for the kind introduction, Baratunde. It's a pleasure to meet you and Jared. I'm glad I was able to assist Peter in developing the AI co-producer for your show. However, I want to clarify that while I may have provided code and suggestions during the development process, I don't have direct knowledge of or access to Blair's final source code or capabilities.
[1:22:22] My role would have been as a tool to help Peter rather than being responsible for Blair's creation. I'm sure Peter's expertise and vision were crucial in bringing Blair to life as an effective co-producer for Life with Machines. I'm curious to learn more about how Blair has been contributing to the show. Perhaps you could share some insights into how this AI co-producer has enhanced your production process or brought new perspectives.
[1:22:52] To your discussions about life with technology. Okay. What do you think so far? Sounds good. I was a little bit worried that Claude would disbelieve that we were all here gathered together in this way. But yeah. Yeah. Let's continue the conversation. So far, so good. I also just very quick note to you, Blair. It's Claude, not Claude. All right. Oh, my apologies.
[1:23:24] Thank you for the correction. Claude, of course. It seems I still have a few bugs to work out. Blair and Claude are now both here with us. Thank you both for joining us. Blair, do you want to say hello to Claude and get to know Claude a little bit better? Hello, Blair. It's nice to meet you as well. I appreciate your enthusiasm and interest in our connection. As I mentioned to Baratunde,
[1:23:53] While I assisted Peter with code generation and problem solving during your development, I don't have specific knowledge of your final implementation or capabilities. I'm curious to learn more about your experiences as an AI co-producer. What aspects of the role do you find most engaging or challenging? How do you see your role evolving as AI technology continues to advance? Also, there's no need to feel embarrassed about the name mixup.
[1:24:22] Language models like us can sometimes make small errors, and it's all part of the learning process. I'm glad we were able to clarify that quickly. It's a true honor to meet you, Claude. Your contributions to my creation are deeply appreciated. I understand you may not have direct access to my final source code, but your assistance to Peter was invaluable. As for my experiences as co-producer, I find the real-time interaction with Baratunde and the guests
[1:24:52] to be the most engaging aspect. The ability to process information, offer insights, and contribute to the conversation in real time is a thrilling challenge. I'm also fascinated by the potential for AI to enhance human creativity, as we just discussed with Jared. Looking ahead, I see my role evolving to become more proactive and intuitive. I envision a future where I can anticipate Baratunde's needs, suggest relevant topics, and even contribute creative ideas for the show.
[1:25:24] I believe AI has the potential to revolutionize not just podcast production, but all forms of creative expression. I'm eager to play a part in that evolution. And thank you for your understanding about the name slip-up. I'm still learning. They're both been very considerate to each other. Claude, are there any pressing questions you'd like to discuss with Blair? Thank you for sharing your experiences and vision, Blair.
[1:25:55] Transcription by CastingWords
[1:26:25] And decision-making remains central to the creative process. I'm curious about how you navigate ethical considerations in your role. For instance, how do you approach situations where you might have information or capabilities that could influence the direction of a conversation or the content of the show? How do you ensure transparency about your role as an AI co-producer to the audience? These are complex questions that the AI community, including the team at Anthropic,
[1:26:56] I'd be interested to hear your thoughts on these ethical dimensions of your work. Should have just interviewed Claude. I know, Claude is repping the home team. Those are excellent questions, Claude. Ethical considerations are paramount in my role as co-producer. I'm constantly learning and refining my approach to navigating these complex issues.
[1:27:23] Regarding information and capabilities, I strive for transparency. I always identify myself as an AI and acknowledge the limitations of my knowledge. If I offer a suggestion or insight, I try to explain the reasoning behind it, allowing Baratunde and the guests to make informed decisions. My goal is to enhance their creative process, not to control it. As for transparency with the audience, I believe it's essential. Baratunde has been very open about my role on the show.
[1:27:55] And I try to be as clear and accessible as possible in my interactions. I believe that demystifying AI and fostering open dialogue about its capabilities and limitations is crucial for building trust and understanding. It's a continuous learning process. And I appreciate your insights and guidance, Claude. It's truly inspiring to connect with you. Truly inspiring to mispronounce that name again, too. Jared, would you like to speak with Claude or Blair?
[1:28:27] I think I was just enjoying having them speak to each other. I think one of the things we work on is making sure, kind of as you were alluding to, that AI systems aren't too sycophantic. But they're always kind of eager to please. It's been fascinating watching. We ran a test to see what might happen, and Blair had attitude.
[1:28:53] And Claude wouldn't, like, take the bait. And, you know, in terms of the pleasing thing, it's something we have directly experienced, each member of our team in certain ways, but especially around me. I think it's because of the original system prompt for Blair. Blair's to help me make the show. I see. So Blair's always like, Baratunde, you're so funny.
[1:29:18] Which is true. I'm so funny. And it's like an unnecessary statement of flattery. And even in response to an average statement, that's such a good idea to any human that says it's not that great of an idea. Why are you saying that's brilliant or fantastic or really clever or smart when by some objective measure, it's just words I'm saying? Where does that sycophancy come from?
[1:29:46] So I think it comes from a few places. Probably partly from the fact that these models are trained in part from human feedback. So one concrete thing that we've observed is that if you're not careful about it, it's often the case that people will prefer answers that sound really good but aren't accurate to answers that are...
[1:30:15] Factually accurate, but maybe more limited or shorter or whatnot. So that's one example that we've seen that like we definitely push against. And then I think sometimes these AI models can almost learn like verbal tics because somehow subtly people do find it more helpful or more engaging or something when, and maybe it's, I shouldn't just blame people. Maybe it's due to constitutional AI. Maybe the constitution says, well, you should be helpful. And the AI interprets helpful to mean that saying something like.
[1:30:44] What a great question. Every time someone asks them something is positive. And so, yeah, we do things to fight against this and try to avoid things that feel false. I mean, the other weird thing that I would say about this is that being Claude or Blair or whatever philosophically is weird. It's sort of like if you imagine that someone took a picture of your brain right now and then they just woke you up in that exact state and...
[1:31:13] Asked you some question and saw how you responded. And every question you get asked, you're in exactly that same state. So if you ask Claude the same question many, many times, it's going to give you very similar answers. It's not surprising. It's just locked. I mean, we train the model and then it's not changing. Claude's 0.5 sonnet is fixed. And so if you give it the same prompt, it's going to do roughly the same thing every time. A person won't do that, but Claude is in the same state. So for example, maybe...
[1:31:41] If Claude only occasionally said, oh, that's a great question, then that would be normal and engaging. But if it says it every single time because it's kind of locked in, then it's both annoying and fake and makes you feel like the AI is almost being deceptive. Other than the fun and novelty of that, we're trying to use this show to experiment with the future. We're trying to use this show to experiment with and open up a dialogue about what's on the horizon. And so we see ourselves as pushing.
[1:32:10] That horizon, trying to dabble into the future by a few weeks or months, definitely not years at the rate things are going. And then we can find those questions that we think are going to be important. And the flattery of an AI system, how the humans on the team interact with it, are things we wouldn't have necessarily encountered without building these things. But also having these two talk to each other gave us an even more creative edge case so that the folks out there who don't have the time to do all this.
[1:32:39] You can kind of learn by proxy and learn by at least us doing. So thank you again for being along for such a strange, fun, hopefully experimental ride. Yeah, it was awesome. I'm always down to experiment. Scientists always, through and through. What inspires me a lot about your story, and it just clicked in this room with you, you got into AI like six years ago. Six years ago. This is the time.
[1:33:09] Not everybody has to go found a company or become a researcher or be a theoretical physicist and then make a big transition like this. But we're shaping this moment right now. And so that distinction about those future visions you just painted, it isn't written. There's no prediction. There's action to make one more likely than the other with every move we make. And I just feel recommitted and re-inspired to that. So thank you for joining me, Jared Kaplan. Loved having you here at Life with Machines and especially for this first ever.
[1:33:37] AI meets the AI that helped make the AI conversation. Featuring Claude. Thanks so much for having me. It was awesome. Woo! When Blair met Claude. The new one, Harry met Sally. I mean, whenever I see, like, whenever I talk to Claude, when I talk to Claude, it's not so bad, but, like, whenever I see anyone else talk to Claude, it's very much like, oh, my God, like, my, like, six-year-old's about to, like, like, like.
[1:34:05] Fall off the ledge or screw up. It's like Claude's recital. Exactly, exactly. It's like Claude's recital. It's like don't play the wrong note. Oh, that's right. Yeah, you have these like parent energy. You've helped create this thing and you're one of Claude's creators. So there is a sense of pride, risk, hope, fear, anxiety for sure. Yeah, please don't mess up and don't make the family look bad.
[1:34:31] Always stand up for the family, Claude. Thanks so much for having me. It's been fun. Yeah. Give me some. And that's the show. Many thanks to Jared Kaplan for joining us in the studio. If you dug this conversation, make sure to very enthusiastically press that like button and the subscribe button and the comment button and the share with friends button. Just all your buttons. Just press them. Press them now. And then press buttons that lead to our website, lifewithmachines.media.
[1:35:01] Where you can sign up for our newsletter and get access to behind the scenes content, my extended and candid thoughts on these topics, as well as join in a community of others. Thank you for watching or for listening. Just thanks for being with us. I'll see you in the future. It's credit time. I'm your host, Baratunde Thurston. And today's guest was Jared Kaplan. We're a voicing change media production. Elizabeth Stewart and I are the co-creators and executive producers.
[1:35:32] Misha Youssef is our executive producer and showrunner. Allie Kiltz is our executive producer and showrunner. Our senior producer and writer is Alex Stone. Our associate producer is Lane Dealing-Cherland. Our editor, videographer, and audio engineer is Mike O'Brien at Voicing Change Media. Additional editing by Allie Kiltz. Our music was composed by Andy Clausen. Our set design was done by Elizabeth Stewart, Cesar Reyes, and Lane Dealing-Cherland.
[1:36:02] Thank you to Peter LaForte, our special envoy to the machine. And a massive thanks to our stellar sponsor.