[0:00:06] I'm so glad to have you here. I'm really excited for this. Thanks so much for having me. I'm really excited to be here. One thing I want to ask just to start is, what does it feel like being back in the Stripe office? I have to tell you, this is the most surreal experience I've ever had. Just to bore you all for 20 seconds with the last time I was at a Stripe office. I'm two offices back, and I left Stripe in 2018 after about five plus years.
[0:00:33] I just remember thinking like, my God, this company is so amazing and so impressive, but it's so big. It was like a thousand people or something at that time. So it's just incredible to walk in and see this beautiful office, all these incredibly talented people. I mean, it's just like totally mind boggling. So it's nice to meet you all. So congratulations are in order. You guys had your launch the last couple of days, Claude too. How's that going?
[0:00:56] Well, first of all, thanks. We're feeling good about it, feeling excited. The launch obviously happened on Tuesday, and Claude 2 is now available for businesses. So if you're a business user, you can access us through our developer console, and then individuals can also play around with Claude at claude.ai. Are you seeing any interesting uses in the launch, or anything interesting? Yeah, I think something that we have been noticing is that
[0:01:26] People are really, really excited about the file upload system. It's, you know, I mean, you probably all know this better than me, but something about, you know, launching a new product or a new feature. It's not always what you expect is the thing that kind of people are excited about, but have been hearing, you know, some positive feedback just about the ability to have this longer context window and then to really be able to sort of upload these longer files and have Claude.
[0:01:53] Read through them and pull out information, summarize them. So it's definitely been an awesome learning experience over the past few days. So tell us a little bit more about like what led you to decide to co-found Anthropic? What was that story that led you there? Yeah. So after Stripe, I went to OpenAI and I was there for about two plus years. And really, you know, my co-founders and I had this vision.
[0:02:19] I think we had this vision of...
[0:02:48] Really training models with sort of safety and humans at the center of them. And we felt really excited to just go and do that together as a group. So how is Claude in particular different from some of the other models out there? What sets us apart? So I think, you know, we were talking about this a little bit before, but one of the things that I think is so sort of interesting about this moment in time.
[0:03:11] Is that these different models are almost taking on like slightly different like personalities, right? So it's a little anthropomorphizing, but I think when you sort of play around with different models, you'll see these kind of different strengths and weaknesses emerge. And again, I think, you know, what we've heard from our users is that Claude tends to come across as friendly and warm, conversational.
[0:03:37] Claude is very apologetic, so if it won't answer a question, it'll be like, I'm so sorry, I can't answer your question. Very polite. I tend to think of Claude as sort of a very, like, eager junior assistant, right? It's a tool that really wants to help. That's kind of on the personality side. I think in terms of kind of skills and capabilities, really, we've aimed to make Claude, you know, as safe as possible. So I think we saw some improvements from Claude 1.
[0:04:06] On things like harmlessness and honesty, of course, no large language model on the market today is perfect on any of these dimensions. But I really think one of the sort of things we feel sort of most proud of is that Claude is as capable, but also has made progress on honesty, harmlessness, and helpfulness. I love that point about personalities. Matt Webb describes this as an AI sommelier for the different personalities and knowing exactly how to...
[0:04:33] So I think it's interesting to see how these things are so unlike software. As engineers trying to make sense of these things, like, wow, this is actually pretty bad at following the instructions I gave it to. It's like, yeah, it's not software. As Ethan Mollick has described it, it's pretty good people. And I love the notion of that assistant. So trying to be focused on safety, I imagine has a lot of trade-offs and challenges. Can you talk us through how you navigate those and where some of the hard trade-offs are? Yeah, that's a great question.
[0:05:03] easiest one or the one that most brings to mind is in this kind of approach to training with this HHH framework, you know, helpful, honest, and harmless, there's this kind of inherent research trade-off between helpfulness and harmlessness. You can imagine that it's actually very easy to have a perfectly harmless model. It will just not be very helpful, right? You can say, hey, can you, you know,
[0:05:30] I don't know, like write a line of code for me or could I borrow something? And Claude will just say, I'm so sorry, I can't help you with that response. It's a perfectly harmless model. And so really we spent a fair amount of time kind of, especially in the late stages, you know, sort of fine tuning between a little bit more harmful, a little bit more helpful, a little bit, a little bit less harmful, a little bit less helpful. And of course, you know, we try to have.
[0:05:53] The scores on both of those be as high as possible, but there is a little bit of an inherent trade-off just between those two. What are some of the techniques that you use to make these more safe? Yeah. So a few different things. Also, if this is like a topic that you're passionate about, we publish our safety research and we have about 17 papers available on our website that you can read. A couple of things that just sort of jump out, you know, especially for, you know, Claude 2.
[0:06:19] So we use something called reinforcement learning from human feedback. This is a technique we've written about. And this basically is just human labelers basically giving feedback to the models of, hey, this is a little more harmful, this is a little more helpful. But really, I think one of the kind of big breakthroughs for us over the past several months was constitutional AI. And we've also written about this, so you can read about it on our blog. But constitutional AI was this idea that
[0:06:47] Rather than kind of scoring or giving feedback on particular responses at kind of a very high rate, you can actually instruct the model with a constitution to say things like, hey, it's not nice to be angry or mean to people. You should be thoughtful about all different types of people from all around the world. And we've sort of shared some of the documents that we use to train that.
[0:07:15] The UN Declaration for Human Rights and trust and safety, you know, best practices from the industry. It's a really long list. But I think that was a really, you know, interesting kind of research breakthrough on our part. So one of the things that makes Claude different is a huge context window. Yeah. 100,000 tokens, I think, which I think is the biggest of any available model. What were some of the challenges in extending the context window that long?
[0:07:42] Yeah, I mean, I will definitely say my research colleagues would probably tell you more tales of woe and toil than I will be able to do justice. But, you know, it's again this sort of issue of trade-offs, right? Extending the context window is going to trade off against other things that basically just take capacity and usage, right? And so sort of we chose to prioritize that because we sort of saw this.
[0:08:10] This potential use case of people kind of giving Claude just a lot more information, right? And I think some of the value that we're sort of starting to see from LLMs in general is this idea that they can really search over a very wide corpus of information. And I'm probably not the right person to opine on like how hard it will be to be, you know, 200,000 tokens or 500,000. But my sense is that that's probably where the industry is going.
[0:08:39] Yeah, I found that with like, we've talked a lot tonight about semantic retrieval and a bunch of techniques for doing embedding-based indexing. And when you give these things the facts that you want to summarize, they do a significantly better job than kind of free recalling facts just out of their training. And so it's interesting to see with the larger context windows what kind of stuff you can cram into the context. And I think in Claude 1, you talked about Great Gatsby, the whole novel in the context window, which is pretty amazing.
[0:09:04] Are there any interesting use cases that you found in the wild of people using that extremely large context window to do anything cool? Yeah, this is a great question. I think it's honestly often for what I would describe as kind of important, but like a little bit banal tasks. And this is sort of where this concept of a junior assistant, I think, comes in, right? If you're trying to read a lot of information, like long memos or legal briefings or...
[0:09:32] You know, many, many hours of recorded Zoom meetings or SEC filings or just earnings reports. That is a great use case that might take a human or many humans, you know, hours or days to do. And Claude can do that almost instantly. And then you can ask it information about that, right? What are the takeaways from the earnings reports or what were kind of the interesting things that you discovered in these articles over day X or Y?
[0:09:57] So one of the ways that you all differ as well is a very explicitly enterprise focus. So how does that manifest? What does that mean for what different things you prioritize? So I think from kind of an organizational perspective, you know, what that has meant is sort of investment in this team that we call product research. And that sounds like a hodgepodge of, you know, two parts of the organization that you mush together.
[0:10:22] But I actually think it's a very kind of interesting and unusual challenge, which is how do you kind of build this bridge between researchers that are kind of experts in training and improving LLMs? And how do you actually turn that into something for a larger business that's kind of tailored to their use cases? And I think part of what's been so interesting for us about that is we've really learned.
[0:10:50] A lot about how to generalize some of those trends, right? Bigger businesses often say, hey, you know, I want to, you know, fine tune on some data and, you know, use it to apply to, you know, concept X or Y. But if you just change a couple of those details, it often applies to other parts of the industry as well. And one of the things that's interesting about your company and others like it are a huge focus on research within, you know, normally startups are like, you know, mostly about productization. This has a significant portion of the resources.
[0:11:20] Are there any other challenges that that leads to and like navigating and figuring out what to build, how to ship it? That's also a good question. I think, you know, there's a lot of overlap in terms of values and goals and mission for everyone that works at Anthropic. And I think sometimes there are, you know, like stylistic differences, right, between a more kind of academic, often research-based approach.
[0:11:49] You know, it's harder to sort of set immediate concrete goals, right? Sometimes you have to let researchers play around and explore, right? They want to sort of go down a rabbit hole. And I think, you know, interpretability, which is another kind of research area that we're known for, is like sort of the pinnacle example of this, where it's really hard to predict exactly, you know, where this research is going to go. So doing things like writing OKRs is like not necessarily those teams favorite activity.
[0:12:17] And then on the product side, you know, of course, there's things like, well, how many users do we want to ship to and what features do we need? And so I don't necessarily think that their intention, but I do think it's important that there is sort of, you know, energy put into harmonizing the strengths of both of those kind of perspectives and worldviews. So you mentioned the users getting let in. And you all have, I think, more than 350,000 people on the wait list. We do, yeah. Can you tell me more about what's going on?
[0:12:46] Someone who badly wants an off-the-weight list. You and Twitter are both asking this question this week. So, you know, here what I would say is, you know, number one, if that's you, thank you very much for your patience. We're trying our best. If you're an individual, Claude.ai is the best place to go now. You'll be able to, you know, get on there very quickly. For businesses, we have been, you know, as quickly as we can kind of moving people off the wait list. We're just, we're actually a sort of shockingly tiny effort on the product and business side. We're about...
[0:13:15] 40 people total. And so just from an operational perspective, we want to make sure that we give every user that is on Anthropic an excellent experience and that we're really able to help them answer their questions, support them. And so we appreciate your patience and we're working as hard as we can to get that waitlist down to zero. So the New York Times did a piece on you all in the last couple of days, which is really interesting. And one of the themes that comes out of that is a lot of people talking about their existential fear for what might happen if the technology goes in the wrong direction.
[0:13:43] So what keeps you up at night? Yeah, I think what I would say is it's really kind of a broader reaction to AI and just how quickly things are moving as a whole in the industry. And maybe the thing I most worry about is that we have this incredible opportunity for AI to create positive change in the world.
[0:14:12] And of course, there are real risks that come along with that level of kind of transformative potential and engagement. And I think maybe, you know, the thing that keeps me up at night is just, are we as, you know, an industry, but more broad than that, right? Policymakers and civil society and academics and consumers, will we all really be able to invest the time needed to get together and kind of thoughtfully walk into this next
[0:14:42] I think that the potential power of this technology is a little bit mind-boggling, right? Anyone who's played around with an LLM would generally say, of course, they're not perfect. There's all kinds of things they can't do, right? I think that's talked about a little bit less.
[0:15:11] If you sort of extrapolate out from, you know, looking backwards in time, like four years ago or five years ago, GPT-2 and, you know, the BARDs and GPT-4s and Claudes of today, that's an incredibly small amount of time, right? And if you sort of look forward another five years and imagine, well, what won't these models be able to do? I think that's sometimes scary, but also incredibly inspiring, right?
[0:15:40] Imagine the kind of breakthroughs that might happen in healthcare or science or renewable energy as a result of having these powerful tools that can work with humans to create all kinds of things. It's funny. It's such a fast-moving space. It's so hard to figure out what kind of business models are going to be really powerful. People keep on asking me where the moats are going to be. I'm just trying to keep track of what's going on. I'm just curious, what are your thoughts? What do you think companies found in this AI world will look like? How will they differ?
[0:16:11] Will they be training? Will they be using fine-tuned models, just off-the-shelf models, their own models they're building? Are they going to specialize it themselves? What's it going to look like? I mean, first of all, hard to know the future, of course. But I think I would expect that there's going to be a lot more innovation coming, right? I think we sort of, it can feel easy to kind of imagine that, oh, you know, there's only four or five players that are sort of building these LLMs.
[0:16:41] People are just only going to build on top of them. And how much diversification can there really be? But I think that the sort of commoditization of these tools is something that is just in its infancy. We're seeing all kinds of ecosystems, much like Stripe, of platform resellers like Bedrock and individuals that are building things and businesses that are building things. And the layer of depth and the level of different applications that are totally unexpected that we're seeing people use these for.
[0:17:11] I think, again, we're sort of in chapter one of a long book. And so I expect that we're going to see more surprises in the next year, two years, five years. Somebody recently, I'm a big AI booster, and someone recently asked me a really interesting and thought-provoking question. And she said, would you rather that society had Isaac Newton or large language models? I'm curious what your answer to that question is.
[0:17:37] It's a big, it's a meaty question, right? It's a very meaty question. I mean, I feel like a cop-out is, could you fine-tune a large language model on Isaac Newton and then sort of retain that? Maybe not a fair answer. I mean, one sort of presupposes the other, right? Could we have done large language models without Isaac Newton? Would there have been another Isaac Newton? Would they have come along later? What would that have looked like?
[0:18:06] I don't know. I mean, maybe there's an equivalent of like an Isaac Newton today, right? Would we rather have a brilliant mind of today versus large language models? I don't necessarily know how to make that trade-off. I feel like this is not the answer you were looking for. I would go with the fine-tuning answer. I'll just quote while I'm ahead. Fine-tune Isaac Newton, and then you get them both. For the record, that was also my answer. Yeah, okay, good, good, good, great. And then changing tack a little bit.
[0:18:35] How has it been as a co-founder? This is the first time you've been a co-founder, I believe. How is that role different from what you've done before? Yes. Oh, man, it's different. I think, you know, Stripe and OpenAI were incredible training grounds for me in terms of unbelievable opportunity and learning that I got at those two companies. And two companies that I can't think of two other companies that I respect more in the world, genuinely.
[0:19:03] And I think being a co-founder is, it's a different experience in the sense that like you just, you go through different phases, right? I mean, I joined Stripe when it was around 40 people. I joined OpenAI around 60, but I never did that sort of initial gross period. And I'd always been curious about it. And the truth is it's by far the least fun and glamorous. It's the most exciting, but you know.
[0:19:29] Our chief science officer, one of my co-founders, was literally running payroll. He was like, how do I click this button? Did we pay everyone this month? And we're sort of like, we're sorry if we didn't pay you. Could you email Jared? So it's the prototypical kind of classic startup story. But it's a humbling experience. You realize how many things you're bad at when you're a co-founder. You also really learn to lean on the other people that you're...
[0:19:57] Thanks so much for the talk. My question was, when you left OpenAI, you were there for two years.
[0:20:26] Did investors ask you, how are you going to differentiate from OpenAI when you were raising money? Yeah. So the question is, when we were fundraising our first round, people said, how are you different from OpenAI? How are you going to differentiate? I think, you know, really a lot of that question at that time, and of course it's evolved. We really sort of, again, talked about this kind of research and safety focus as this kind of primary emphasis, right?
[0:20:54] And I think we have this view that safety is something that is incredibly, you know, valuable and economically viable. And I think that was a story that felt kind of just given the nature of sort of who the co-founders were, you know, pretty intuitive for us to tell. Yeah. Is there somebody over there or not? Maybe we'll do one over here then. Less running around. Yeah.
[0:21:27] Hi, thanks for the engaging talk. Quick question about enterprise use cases. Are there any surprises either to the upside or the downside that you found around specific use cases where either enterprises are really excited to adopt or less excited than you think? So I think, first of all, very early for us, so I think probably too soon to tell, but I think we have...
[0:21:51] I've been surprised the degree to which more traditional businesses are kind of excited about the power of LLMs. And I think I sort of went into it expecting, you know, larger tech companies and maybe financial services, obviously I have a background bias here, but would be the types of businesses that sort of might, you know, see the potential power of these because they're kind of, you know, in that world.
[0:22:17] But it's really been kind of across the board, right? I think many industries are feeling the potential for what this can do to kind of revitalize areas of the economy that maybe missed the last wave or the wave before. And I think that's been an incredibly interesting and genuinely inspiring opportunity to think about how to take this technology and help other.
[0:22:44] Sectors that might not have come along for fintech and help them really bring forward a vision of how to continue their story. Thanks. Dr. Adam, we'll go over here. Thanks for the incredible discussion. So at the FOSTI open source conference, a coalition of top orgs was putting together the next 10 principles of what it means to have openly accessible AI.
[0:23:19] And the golden rule just guiding this discussion was basically, if I like an AI, I will share it with other people who like it. And then one person proposed adding, unless it will kill them, which they accepted and added. So if you could weigh in on principles guiding the open source AI community, what would you add and how might we bring open source AI leaders into the conversation on AI risk? Great.
[0:23:45] Basically, thoughts on kind of open source and potential risks and how to kind of include that community in sort of this broader discussion. I think this is a great question. And I'll start by saying that the AI ecosystem is very broad. And I think the diversity of perspectives that are emerging are really valuable. We've just never grappled with a question quite like this one before. And I think it would be...
[0:24:12] wrongheaded to say, you know, only industry gets to weigh in or only academia or only, you know, policymakers. And so I genuinely think that there's room for discussion for many different views here, including open source community. I think sort of beyond that, I would say it's just kind of interesting to see sort of how that ecosystem is developing. And I think much like
[0:24:38] All of these different sort of sectors of the economy or society or whatever, it's still really early, right? And I think there could be, you know, future iterations of kind of open source with different, you know, beliefs and values than the ones today. Hi, what's your take on mechanistic interpretability for like the future of alignment of AI models? Yeah, I love mechanistic interpretability.
[0:25:14] And I've known for many years, so potentially just biased. But I think mechanistic interpretability is one of the most interesting areas of research happening today, really in many fields. And I'm not a researcher, so I probably won't do it justice here. But I just think this idea that we could look inside this black box of generative models and better explain what's happening in them is...
[0:25:44] Just intellectually fascinating, right? And I think from kind of an alignment perspective, there's probably no silver bullet. But in general, the more you understand what something is doing, probably the more likely you'll be to be able to notice behavior that might be concerning, right? That's just kind of a heuristic. I'm sure there's more details you could go into there. But I really think interpretability is just an incredibly...
[0:26:13] Like powerful tool that we're just, we're really just scratching the surface of, right? It's sort of, if you were to use a neuroscience analogy, we're probably 15, 10 years behind where neuroscience today is. And we're trying to kind of close that gap. Awesome. Thank you. Hi, thank you so much. And I'm quite impressed by Claude compared to Bard and ChatGPC 3.5. So great job. What do you think the role of data?
[0:26:45] And the quality of data, especially with the recent publication of textbooks is all you need. From Microsoft indicating that that's crucial. Along the lines of your main vision, right? Safety for AI and, you know, alignment, I guess, is a tough problem. Let's not get there, but just safety in general. And how are you able to address this in the training process? And are you in the same?
[0:27:11] Second part, are you in the same issues as other companies in terms of usage of data? Crawling Twitter and whatnot and other sources of data that are on the internet. How do you plan to handle that if you're challenged? Yeah. You know, the topic of data and kind of how it overlaps with privacy and security and copyright, all of these issues, I think we're sort of seeing almost like a...
[0:27:39] parallel track of acceleration of discussion of these issues along with the kind of acceleration of AI, right? And that's because these issues didn't really apply in the same way as much before. And I think here, you know, I would say we are in the midst of all of these decisions and norms and laws like being written, right? Like case law is literally being written about these things today because of decisions that
[0:28:09] I think that we have sort of invested almost a little bit extra in building relationships with policymakers and folks that are interested in these questions because it feels very important, right? It feels kind of fundamental to how these large language models are developed or really generative AI in general, right? It's not just language, it's other modalities. And my sense is that
[0:28:38] There will need to be a very thoughtful, engaged, nuanced approach to how we kind of solve those fundamental questions for the field more broadly. Could you repeat the first part? I'm so sorry. Oh, the quality. Yes. The quality. So, yes. I mean, the short answer is data quality is incredibly important. I mean, how do you measure it? Yeah, I think, again, probably a research colleague would give you a better answer. But my real kind of...
[0:29:08] Answer here is that it's not just the quality of the data itself, but it's how you treat the data too, right? So how do you clean it? How do you curate it? How do you analyze it? How do you train it? And I think that's actually a really interesting sort of emerging like jobs space, right? Like I think the concept of like data engineering, they're hard to hire, right? Like excellent data engineers. And they're actually some of the most valuable people at AI companies because so much about
[0:29:37] The overall outputs and the end sort of results of the models happen at that kind of initial phase. So sorry for forgetting the first part of your question. Yeah. Yeah, I probably have time for one or two more questions. Thank you for the chat. It was great. My question is, can you mention one thing that you took from Stripe or OpenAI culturally to Anthropic and wanted to embed it in the culture at Anthropic? And then one thing.
[0:30:06] That you decided to do differently or like wanted to fix, let's say, about your prior experiences culturally at Anthropic. I love that question. So I, again, like Stripe and OpenAI, different companies, both incredible, both very impressive. I think probably from Stripe, the thing, I mean, I should also say we've hired a lot of Xstripes. So there's some of this is just natural kind of osmosis.
[0:30:34] I think the kind of intellectual rigor of Stripe is something that I always found so impressive. And I think that there's a nice kind of synchronicity because our founding team are like mostly PhDs. Like we have like seven physics PhDs and like our first 10 employees or something like that. There's this kind of desire to really like understand and know the truth, right? To be kind of intellectual, empirical.
[0:31:02] And I think that that is definitely something that I feel is sort of synchronous between Stripe and Anthropic. In terms of things we didn't want to pull over, I don't know that we've succeeded at this, but I think a flip side to that kind of intellectualism is sometimes like almost overthinking decisions, right? So sometimes there are decisions that you're like,
[0:31:29] We should just have some snacks in the office. Like maybe peanuts are okay. We could also have cashews. I do think that the sort of over-intellectualization is kind of the Achilles heel of that approach. One last question. I was wondering how you're thinking about AI literacy as a fundamental skill together with other forms of literacy, because that's the other part of safety in terms of policymakers, your users, journalists.
[0:32:02] You can ensure that these models are as safe as possible and there are lots of guardrails in place. But ultimately, the level of knowledge that people have about the systems and how to use them will determine the outcome. Yes, great question. So I think in terms of AI literacy, a few thoughts. One is that it is a more kind of narrow group.
[0:32:29] Part of why we publish so much of our research is for this exact reason, right? To kind of the earlier question around mechanistic interpretability or, you know, constitutional AI. Part of why we try to be as open as possible about our safety work and our research is so that that will become propagated in the field more broadly and so that it's something that's talked about because we think it's so critically important. Beyond that, we have an entire team in our policy organization called Societal Impacts.
[0:32:58] And what they do is they're sort of the bridge between the research teams and the policy team. And their job is to take the very technical topics that are probably challenging for policymakers or others to kind of understand intuitively and turn them into something that's easy, right? Something that you can arm our policy team with or other policy teams with when talking to.
[0:33:25] Government officials or regulators and say, here's why it's important that we put guardrails here. And I think that's just one example, but I think there are ways to communicate this information in a way that is not just academic or challenging to sort of understand. Cool. And we'll end it there. Thank you so much for your time. This was amazing.