[0:00:00] The holidays are supposed to be the most wonderful time of the year, but let's be real. Between family gatherings, endless plans, and trying to make everyone happy, they can also be overwhelming. Sometimes the biggest win is setting a boundary. Maybe it's saying no to an event you don't have energy for, or choosing to stay calm during a stressful family conversation. Those little moments can make a big difference. That's what growing feels like, and that's where Grow Therapy comes in.
[0:00:28] Therapy isn't just about navigating life's biggest challenges. It's about those small, everyday moments that help you feel more grounded and in control. Grow Therapy makes finding support easier by connecting you with licensed therapists who fit your needs and accept your insurance. Book your first session in minutes and start in as little as two days. Sessions average $21 with insurance. Some pay as little as $0 depending on your insurance plan.
[0:00:54] Prepare for holiday glimmers instead of holiday stress. Visit growtherapy.com slash ACAST to get started today. That's growtherapy.com slash ACAST. Availability and coverage vary by state and insurance plan. This episode of the Times Tech Podcast is sponsored by Adobe. AI is changing how brands reach customers, but potential only becomes business value with the right system behind it.
[0:01:21] Adobe helps brand leaders get discovered by new audiences and deliver better customer experiences with AI that puts your team's insights into action at speed and at scale. It starts with Adobe. This episode of the Times Tech Podcast is sponsored by ServiceNow. Dani, one thing we keep hearing from business leaders right now is AI sounds great, but how do you actually make it work inside a company?
[0:01:47] Exactly, because most organizations aren't neat, shiny systems. They're layers of software, legacy tech, and teams all doing things slightly differently. ServiceNow sits across all that, acting as a control tower for making work move seamlessly through the organization. Connecting people, systems, data, and increasingly AI agents so that things don't happen in silos. Learn how ServiceNow puts AI to work for people at servicenow.com. What are you wearing? What?
[0:02:20] What do you mean? What's on your face? Oh, these? Yeah, those. The blue flashy things. It's almost Christmas, as you know. Okay. So I figured I'd add some flair to today's recording. In London, we do Christmas jumpers. Yeah, well, I'm doing Christmas blinking, flashing plastic glasses on my face. You look great. I really see you. Yeah, thank you. Did you get those from the Humanoid Robot Summit? They look like tech conference stash. They actually do.
[0:02:49] It's Tech Conference Stash. That's a whole other episode. Oh, for sure. I figure for today's recording, I'm going to wear this kind of pulsating neon glasses the entire show. It's upping the mood. Glowstick, glowstick, glowstick, you know? Yeah, I think they're great. You don't have to take them off until New Year. I think I will. On conferences, how was the robot, humanoid, humanoid robots? Humanoid robot. People as robots conference. Strange.
[0:03:17] Because you're in this, it was at the Computer History Museum. You go there and there was like a whole like hall full of all of these different robots made by these different companies. They're all various forms of humanoid. And the thing that struck me was they're all kind of like action figures. So I moderated the panel. On stage was me and five other dudes.
[0:03:46] And I was like, like, look at our panel. Classic tech. Classic. Welcome back to the Times Tech Podcast brought to you from two sides of the Atlantic. Yes, with Katie Prescott in London. And Danny Fortson in Silicon Valley. Katie, powerful AI or artificial general intelligence to you and me could be here by 2026. What does that mean in real life? Well.
[0:04:16] Today's guest has a very clear idea of what that could mean. I've been speaking to Dario Amadei, who is the co-founder and CEO of Anthropic. And Anthropic obviously is one of the big, most important developers of AI models in the world. And according to their website, they are, quote, dedicated to building systems that people can rely on and generating research about the opportunities and risks of AI.
[0:04:46] And they are basically, there's seven co-founders. They were refugees from OpenAI, so they were there for years. They left a few years ago and said, we're going to develop AI in a bit of a different way. They've raised a ton of money. They've been valued at $40 billion, so $0 to $40 billion in three years. And their model is called Claude, which is kind of the alternative to ChatGPT.
[0:05:14] I always love hearing about Claude because they talk about it as the sort of soft and gentle face of AI. And they spent a lot of time working on its personality, haven't they? Yeah. And obviously, we have an interview today with Dario, who's the CEO, and he's totally fascinating. But I spoke to four of their other top executives and co-founders. And we talk about the amount of work they put into, for lack of a better term, Claude's.
[0:05:43] Personality. The kind of the outcome, you know, speaking to companies as well who are kind of working with ChatGPT and Claude, they're like, it's just a bit warmer. It's like a vibe almost that feels better and a bit more human than ChatGPT. And reading about it, it feels almost like they're sort of bringing up a child.
[0:06:06] For want of a better word, you know, guiding it on how to behave in certain situations and what's appropriate and what's not appropriate. It's incredibly fascinating. So what was he like? You know, he's really, he's super enthusiastic as you'll hear. He's obviously very, very smart. He was at OpenAI for years. He actually helped kind of lead the development of GPT-2 and GPT-3.
[0:06:32] GPT-3 was the model that powered the initial version of ChatGPT a couple of years ago. And he was in academia before he got to OpenAI. And, you know, if you recall, OpenAI had this kind of founding mission to do this for the benefit of humanity, all that stuff. Since then, they have turned or they're in the process of turning themselves into a for-profit company. Their latest valuation is $157 billion.
[0:07:01] They're being used by 300 million people a week. And it is now seen by a lot of people as like something very different, including Elon Musk, who has sued them over this, who helped found it and fund it initially, as becoming something else, kind of just a big profit-seeking corporation. And at some point, Dario, his sister Daniela, and five others were like, you know what? We don't like how this is developing. We're going to start our own thing and really focus.
[0:07:30] And what's interesting has happened since, of course, is that they've gone very, very fast. Claude is kind of right on the level of the latest chat GPT model. Some people think it's better, it's more capable. If we're drawing the battle lines here on this very small group of companies that are developing large language models, you've got OpenAI backed by Microsoft over here. As you said, it's worth, what, $150,000?
[0:08:01] Amazon is the big backer of Anthropic, right? It's put in about $8 billion so far. Something along those lines, yeah. So it is, you know, everybody's kind of partnering up, finding their dance partner for this new kind of era. And this is very much in...
[0:08:18] Amazon backed. They also have backing from Google. But yeah, their big, most important partner is Amazon. So you have kind of Amazon, Anthropic on one side, Microsoft, OpenAI on the other. And then the third, of course, is Google with Gemini. But it feels like when we look back at this moment 10 years from now, these are the three companies that are going to be the most important. And I would say probably a fourth would be Meta with their open source model.
[0:08:45] These are the companies building this new base layer upon which everything is being built and why the world is changing. And the reason I wanted to have him on is that in October, he wrote this 10,000 word essay called Machines of Loving Grace. The idea or the kind of conceit of it was, what if this all goes right? Because Anthropic, because of their kind of founding mission, has seen...
[0:09:11] Has been seen as a lot more like a doomery, like it's all going to go, you know, this could all go terribly wrong. This could be an existential threat to humanity, all that stuff. And he's like, let's like take a kind of a first principles approach and talk about what could happen if this keeps developing the way on the trajectory it has so far. And by the way, they don't see any reason why it won't like technically. So his.
[0:09:40] Kind of thesis is that we could get, you know, super powerful AI, another word for AGI, by 2026. And that is going to basically unleash this new era for humans that's going to compress 100 years of progress in the next five or 10. So we've got a chunk of that essay now, and it's read by a real human, not a chatbot. Kiara Brown, who's a commissioning editor at The Times, has kindly read it for us.
[0:10:10] Powerful AI is smarter than a Nobel Prize winner across relevant fields, biology, programming, math, engineering, writing, etc. This means it can prove unsolved mathematical theorems, write extremely good novels, write difficult code bases from scratch, etc. In addition to just being a smart thing you talk to, it has all the interfaces available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access.
[0:10:37] It can engage any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of most capable humans in the world. It does not just passively answer questions. Instead, it can be given tasks that take hours, days, or weeks to complete, and then goes off and does those tasks autonomously.
[0:11:07] In the way a smart employee would, asking for clarification as necessary. It does not have a physical embodiment, other than living on a computer screen. But it can control existing physical tools, robots, or laboratory equipment through a computer. In theory, it could even design robots or equipment for itself to use. The resources used to train the model can be repurposed to run millions of instances of it. And the model can absorb information and generate actions at roughly 10 times to 100 times human speed.
[0:11:35] It may, however, be limited by the response time of the physical world or of software it interacts with. Each of these million copies can act independently on unrelated tasks or, if needed, can work all together in the same way humans would collaborate, perhaps with different subpopulations fine-tuned to be especially good at particular tasks. We could summarize this as a country of geniuses in a data center. Okay, so, I mean, maybe I'm limited by the response time of the physical world, but I find this depiction of utopia...
[0:12:05] A little bit far-fetched. Well, so this is what's really interesting is that I really like the analogy of this is a country of geniuses in a data center. And presuming that we reach this, and again, this is, I've talked to their CTO, their head of product, Dario himself, his sister, they're all very convinced.
[0:12:30] That there is no reason we're not going to get there. It might be a little bit slower. Maybe it's not 2026. Maybe it's 2027, 2028. Who knows? But they're like, you know, according to the scaling laws and everything they've seen so far, there is no reason we're not going to get to a point where these machines are better than any human at any task effectively. Like you have just a country of Nobel Prize winners and you're like, okay, Nobel Prize winners, go sort out climate change. And you just send them off.
[0:13:00] And they're not alone, right? Because also Sam Altman published his own essay around the same time called The Intelligence Age. It is not 10,000 words. It's much shorter, but it's the same idea. And he's like, we are about to enter this age of abundance that people don't really comprehend. But is it any surprise that these very powerful billionaires...
[0:13:23] Yeah. Yeah.
[0:13:51] Out of this world. The reason why, I've read this essay now like three times. I know, you really love this essay. It's a commitment. Well, because I think it's really interesting because it's very specific. And he's like, let's take four or five different areas of life and think about what this could mean. And he's talking about like just biology, for example. Elimination of most cancer. Prevention of Alzheimer's. Doubling of human lifespan. Most mental illness.
[0:14:19] Being cured or treated. Any one of those things, you're like, no, that's crazy. And he's like, no, this could do it all. And of course, to your point, we should be skeptical. It's a little bit like the internet. Like, oh, the internet's going to change everything. And we got overexcited. And people spent a lot of money, invest a lot of money. And a lot of it didn't work at the time. But it's all working now. And their whole point is like, this is going faster and is far more powerful.
[0:14:47] It's a good point. I was reading an interesting Guardian article from 2000 the other day where it was talking about the tech crash. And it was saying, you know, ha, the idea that we're all going to be walking around glued to our phones is kind of a bit far-fetched. Totally. Yeah. Yeah. Well, so should we get to, should we hear a little bit of what our friend Dario has to say? Yes. Did you ask him why he left OpenAI? You bet I did, Katie. But.
[0:15:15] I did naturally start by asking why he decided to write this essay and why now? I'd kind of been following like the AI discourse for a long time, obviously. And I was kind of unhappy with both sides of it. The two sides being? Being kind of like doomers, people who are afraid of, you know, bad things happening with AI.
[0:15:39] And on the other side, AI enthusiasts. I felt like the doomers, you know, with which Anthropic was sometimes, like, wrongly associated. It was very focused on, like, you know, here are all the things that can go wrong. Like, you know, kind of bad things are going to happen. And often not a mentality of, like, there are these concrete challenges. And how do we overcome the concrete challenges and get to the other side? But just a focus on fear.
[0:16:03] And conversely, the enthusiasts of AI, it seemed like they were doing a surprisingly, weirdly bad job of just selling the benefits of AI, right? If you looked at the e-accelerationists who were one of the folks who were kind of promoting it. The EX. Yes, EX. It was always just this kind of...
[0:16:26] This like weird vibes based, we're going to build great things and it's going to be great and let's accelerate forward. And it was just kind of very off-putting and not describing like a humanistic world or even being very specific at all about the positive things that would happen. And I think most people found it quite off-putting.
[0:16:46] And conversely, the people who talk about an AI singularity was like, okay, well, we're all going to upload ourselves to the cloud or build Dyson spheres around the sun or whatever. And I think the most common reaction of people is just like, you're not serious, right?
[0:17:06] But do you have a sense of why those were the two narratives that were so popular and so kind of transcended above everything else? I don't know. I think it has something to do with social media where it just kind of like selects for kind of extreme points of view or like things that get lots of clicks rather than like...
[0:17:27] Things that make sense or actually appeal to like a broader set of people. I think people just get rewarded for saying kind of extreme or outrageous things or like cheering or booing more loudly for something. And so, yeah, you get these kind of weird ideologies and extreme views. And so my thinking with Machines of Loving Grace was...
[0:17:54] You know, let's actually take this seriously, right? I'm someone who actually takes, you know, the potential for AI and where it could go very seriously. It's possible that I'm wrong about, you know, where AI is going to go and what's going to happen.
[0:18:10] I think, I'm not sure that the impacts are actually going to be very serious. And so let's think about how this can go well. What could AI actually do for humanity that looking back a decade or two from now, people could say, wow, that really actually made our lives better. I think people actually thought that particularly around the kind of what's possible with biology and health, that it was saying something that hadn't been said before. Well, I want to talk about that because I imagine,
[0:18:39] Most people listening to the podcast may have heard about the essay. I imagine most people haven't read it because, as you say, it's long. It's rather long. It's rather long. But I thought the way you put it is like, you know, powerful intelligence, not AGI. It was a different terminology you're using. Could come as soon as 2026.
[0:18:56] And the way to think about it is that these data centers would be like a country of geniuses that you could then deploy to do any number of things. And then in terms of biology, there's a whole bunch of bullet points in here. I'll just go over a few of them, and then we can talk about how you got there. Yeah. That this could lead to the reliable prevention or treatment of nearly all infectious disease, elimination of most cancer, effective prevention and cures for genetic disease.
[0:19:25] Prevention of Alzheimer's. Biological freedom. So you basically will be able to control our weight, physical appearance, reproduction, and other biological processes. And that we might double our lifespan to say 150 years old. This sounds crazy. This is based on what? Why are you comfortable making some of these kind of predictions? I think what has made me comfortable is the time that I've spent in biology.
[0:19:55] It's taught me a couple things. One is certainly it's taught me that the complexity of these problems has made solving them difficult. At the turn of the century in 2000 or so when we sequenced the genome, there was this hope that now we understand the recipe so we can cure all these diseases.
[0:20:14] The challenge, of course, was, well, it turns out diseases aren't caused by one gene. They're caused by a whole bunch of genes and the way that interacts with the environment. And so then we started to look into RNA and protein expression. You know, we started to look at multifactorial causes and all of that. And, you know, around that time, around 2010, you know, I was doing my work in computational proteomics and computational neuroscience. And I think the thing that occurred to me is like,
[0:20:43] There's nothing about these problems that's beyond humans' ability to understand. There's just more facts, right? You know, one thing I say in the essay is like, you know, there's this complicated, you know, like, you know, diagram of metabolism or something. Like, you know, it's not like, you know, string theory or general relativity or theory of everything where, you know, conceptually it's very difficult to understand. It's just like, you know, it's just like a circuit board with a bunch of components.
[0:21:12] And I think when I left for AI, while I didn't know everything that I know now about AI and didn't have as much of a picture, I did have this conviction that machines might be able to do a better job than humans of sorting through this complexity. It's hard for humans to store it in their brain. And the skills need to go across thousands of different humans. And then you run into human institutions and the difficulty of humans working together.
[0:21:39] And all of it felt, when I was a very small part of it, felt very inefficient. And so my reaction was, well, the humans need help. And I think that led me to work for many years in AI. It's been 10 years now. And maybe 2023, 2024 was the first year where I really felt that we'd gotten to the point.
[0:22:02] Where the AI systems felt like they were knocking on the door of really being able to help with these problems, right? You know, we look at PhD level biology problems and the models are finally in the last six months.
[0:22:15] Getting to the point where they're able to solve a lot of these problems. And that doesn't mean they can anywhere near doing what we described there. But if the exponential continues as it has, and I don't know, it's hard to predict these things exactly. But when I said 2026, I think what I meant is it's soon. Like it could happen very soon. If you literally extrapolate it, that's what you get. People should be ready for it and they should be willing to be optimistic.
[0:22:43] As this develops, at a certain point, kind of the whole economy, the way we think about it, the way we think about value changes. And specifically for a company like you, if you are building something that is this base layer for the rest, for like a whole new economy effectively that propagates into every corner of what we do, the potential for wealth concentration.
[0:23:11] I mean, this is, you know, this could be a trillion dollar company in terms of like revenue, not value. Do you see that? Do you aspire to that? How do you think about that? Yeah, I mean, you know, I think these are going to be some really important questions, right? Where, you know, I think traditionally when new technologies are developed,
[0:23:34] They lead to the overall pie getting bigger, but they lead to all kinds of disruptions. And they can lead to concentration of wealth in one place and disruption of what people were doing in another place. We saw this in the 19th century during the Industrial Revolution. You had your Rockefellers. You also had skilled craftsmen who used to do jobs that...
[0:23:56] You know, became disrupted in various ways by the industrial revolution. So I think that's going to happen again. And, you know, we need to have some humane and pro-humanity way of dealing with it. So, you know, I certainly think the whole universal basic income in some form seems pretty important.
[0:24:17] But I think it's only the beginning of the solution. It's not the end of the solution, right? Because a job isn't only money. It's like a way of structuring society, right? People place value in it. I think actually in the end, as I say in the essay, we're kind of all going to be in the same boat, right? The AI is going to be able to do my job as well. It's going to be better at us.
[0:24:39] All of us at everything. It's going to be better than all of us at everything, right? There's going to be, for a long time, there's going to be complementarity and increase in, well, long time could be a few years. But for a long time, there's going to be complementarity. And I say, even if AIs do 95% of what we do, comparative advantage is very powerful. But we're going to reach a point, I think, where that is no longer the case.
[0:25:02] And my hope is actually, it's the fact that everyone is in the same boat that's going to save us. Because if all of us are in the same boat, then we'll have empathy and we'll all have to figure it out together. I'm actually more afraid of a world in which it's like, you know, half the jobs are automated, right? I worry that that could lead to a world where half the people just say, screw you to the other. They're just like, oh, well, I guess my jobs must have been intrinsically more valuable. I must be more valuable as a human.
[0:25:31] And of course, that's an illusion, that's motivated thinking, but we won't be able to engage in it if it happens to everyone. I don't know for sure what the solution is, but it just must have something to do with the idea that, you know...
[0:25:46] Our meaning can't derive exclusively from the economic value that we generate, right? Like somehow we have to have some other way of structuring our lives. I mean, maybe, I don't know, maybe we can generate economic value by augmenting ourselves with the machines or something. I don't.
[0:26:05] I don't have the answers here. In fact, I suspect this is one of these questions that you kind of can't work out ahead of time, right? If you went to like hunter-gatherers, you know, 5,000 years ago, and we said, you know, someday there will be more food than any of you. And so, you know, we won't need to organize our society around, you know, building spheres and, you know, who can do a better job of, you know, taking down a mammoth and distributing the meat. And you were like, guys, ideas, you know, how do we organize? I don't think you would have gotten anything very good.
[0:26:34] But we worked, as we built industrial civilization, we worked out a new way to do things. The challenge now is that I think it's all happening very fast. That's the source of all the problems here. It's all happening very fast. I mean, I think it might be better if it all happened a bit slower. But, you know, it's...
[0:26:59] When these things are compelling economically, it's very difficult to hold them back for more than a very small amount of time. And, you know, I think all I can do to help with that is to kind of start the conversation and offer my ideas, which probably aren't very good. But, you know, you got to start somewhere. This episode of the Times Tech Podcast is sponsored by Adobe.
[0:27:23] AI is rapidly changing how brands reach customers, but simply adding AI to your tech doesn't guarantee results. That's where Adobe comes in, personalizing customer experiences and turning AI into real business value. With data at the center of decisions and governance built into every step, you can use AI safely.
[0:27:44] And with Adobe, your teams are free to focus on what matters most, setting strategy, defining goals, and bringing the creative insight and nuance that only people can. Turn AI's promise into marketing reality. It starts with Adobe. Today's episode of the Times Tech Podcast is sponsored by ServiceNow. ServiceNow describes itself as an AI control tower for business reinvention, connecting people, systems, data, and AI.
[0:28:11] So workflows more smoothly across an organization. The platform integrates with different clouds, models, and data sources, bringing them together in one place to help your team. To learn more about how ServiceNow puts AI to work for people, visit servicenow.com. The holidays are supposed to be the most wonderful time of the year. But let's be real. Between family gatherings, endless plans, and trying to make everyone happy, they can also be overwhelming.
[0:28:40] Sometimes the biggest win is setting a boundary. Maybe it's saying no to an event you don't have energy for or choosing to stay calm during a stressful family conversation. Those little moments can make a big difference. That's what growing feels like. And that's where Grow Therapy comes in.
[0:28:57] Therapy isn't just about navigating life's biggest challenges. It's about those small everyday moments that help you feel more grounded and in control. Grow Therapy makes finding support easier by connecting you with licensed therapists who fit your needs and accept your insurance. Book your first session in minutes and start in as little as two days. Sessions average $21 with insurance. Some pay as little as $0 depending on your insurance plan.
[0:29:23] Prepare for holiday glimmers instead of holiday stress. Visit growtherapy.com slash ACAST to get started today. That's growtherapy.com slash ACAST. Availability and coverage vary by state and insurance plan. Why did you leave OpenAI? Why did I leave OpenAI? Something I think I've said on podcasts previously is...
[0:29:50] In an organization, you know, when you have like, you know, an organization or a company, you know, whatever its governance structure is, kind of ultimately, you know, the founders, the people in charge are kind of the one who set the direction. And so if you feel like someone isn't going about something the right way, or if the direction isn't quite the right...
[0:30:11] You know, I think I tried for a very long time to point out concerns, to say, like, you know, this isn't the way we should do things. This is how I think we should do things.
[0:30:26] I think at the end of the day, it's just not that effective. It's just not that effective because if someone has a different vision, then either they'll frustrate your vision or they won't fully understand what you're getting at. And I don't think it works. I think the better way to do it is through the market in a broad sense, right? And both in the sense of make something that is a...
[0:30:54] Show that you can do it in a different way from a commercial, from a market perspective. But I mean market also in the broader sense of market of ideas, right? Where if you think you see another way of producing or rolling out this technology or navigating these very difficult things that I've talked about, it's probably better, instead of fighting about it within one organization, to have two organizations, each of which tries to do it their own way.
[0:31:24] How does it work out? How viable commercially are the two efforts? How much are they able to stick to their values? What happens inside the companies? It's like an experiment. You run it for a few years. I feel pretty good about how our side of the experiment has gone. I won't speak for the other side or for anyone else. And certainly it hasn't been perfect.
[0:31:48] And, you know, all the difficulties I've described of the experiment, of the challenge, certainly make me think that there's always room for improvement, right? You know, someone else should come along and try to do it even better. That's what, you know, I've used this phrase, race to the top, right? Where, you know, there's some, I think there's some importance in, again, especially the marketplace of ideas to come up with ways to do this that are better, more innovative, but that also address the safety and ethics component.
[0:32:15] Just wondering, have you had any moments over the past, well, not just three years here, but also at OpenAI where you had a kind of a pit of your stomach moment of like, oh, this thing is doing this thing, which is very worrying. Yeah, a few things. One, I think in 2018 or so when I first saw like GPT-2. Did you develop or lead that?
[0:32:38] Yeah, yeah. There were a few people involved, but I was kind of, you know, kind of led the team that did the key scaling, picked up on GPT-1, which was done by others, but then kind of led the GPT-2 effort. And the, you know, the ability of the model to kind of generate, you know,
[0:32:59] Content that maybe just kind of wasn't, like its ability to just generate slop and make things up and kind of do a good job of it concerned me. Not that I thought this was a concern at the time, but, you know, looking at the exponential and where it was going. And, you know, I think the way that was communicated, again, it was like filtered through an organization, you know, who's...
[0:33:21] Way of communicating I didn't necessarily fully endorse, and whose history and baggage I didn't necessarily fully endorse. But I think the point of the message around GPT-2 is, hey, this is going in a direction that could be concerning, and we should take slow steps around this and take a responsible attitude. I think some of the stuff around coding that we've seen, I think there are companies that are working now to use...
[0:33:50] Our coding models, like 3.5 Sonic, to do things like automated vulnerability discovery and penetration testing. So like automated hacking. Yeah, well, not to do it offensively, to help us defend, but of course it's dual use. It can be used for both sides.
[0:34:08] Being able to do that responsibly, I think, is a very important thing, right? That we need to make sure if the capabilities exist, we need to use them to identify vulnerabilities and patch them so that we can defend our computer infrastructure better than we have in the past. And so responsible disclosure and the order in which these things are done is, I think, really quite important. Right. Any other kind of, for lack of a better term, oh shit moments?
[0:34:36] Yeah, I feel like every new model, in a certain way, every new model is an oh shit moment because it kind of, you know, it validates the curve is continuing. And every new model is unsurprising because it's just the curve continuing. So that's kind of the duality. Everything is extremely surprising in its details and what exactly it says to you, but unsurprising in that we're now expecting the trend. And is that the relentlessness of the scaling?
[0:35:05] Surprising. Yeah. And how much is that? Because it feels like that's kind of what drove you in part to write Machines of Love and Grace, right? Is that you're just like, this doesn't appear to be slowing down. And if we continue on this trajectory, then dot, dot, dot. So you basically never know. I mean, my highest level expectation is it's going to continue because I've seen it do so many times. But every part of that curve, I've been watching that curve for 10 years.
[0:35:33] And every part of that curve, you know, every year there's like a point where it looks like it's going to slow down. You train the model and it looked like it leveled off because you weren't using the right kind of data or, you know, you kind of had numerical problems at this new scale or the cluster wasn't quite working right. And so there's like...
[0:35:53] Problems at every stage. And at every stage, it looks like it might slow down. And every time so far, it's kind of cleared. We always kind of find a way around it. The force and the number of things you can try is always such that the curve...
[0:36:09] Roughly with little wiggles manages to continue. So again, there's this duality. It's super surprising because somehow it always gets through and the results are so eye-popping. But at the same time, it's not surprising because it's the curve we predicted many years ago. It's surprising and it's not. How do you think about the kind of competition with China Peace? And like this kind of setting out as this is kind of the...
[0:36:36] This is the most important race to win of this century. Yeah. So, you know, I think this is a very subtle question, right? So, you know, I think, first of all, I think that's just very important, right? So China is an authoritarian country. I am afraid of what they will do with...
[0:36:55] Powerful AI. Typically, to enforce a dictatorship, you need people to cooperate with you. And if people become unhappy enough, if you do things that are bad enough, then the people can rebel. But if you have an army of AI systems that are programmed not to rebel, then that kind of can raise things to a much greater danger than we've had in the past.
[0:37:19] Similarly, what autocracies can do in the international stage. It's my guess that this country of geniuses in the data center is something that could provide a really serious strategic advantage, right? That could make up for, you know, in every new...
[0:37:35] Conflict, old technologies become much less relevant, just like battleships replaced by aircraft carriers in World War II. And in Ukraine, we see that these tank-seeking missiles are making tanks much less valuable. And so is just about everything about to become much less valuable, and AI will become the thing that's most valuable. So the reason I ask is I was talking to somebody who is the founder of a tech-based defense contractor. And he was saying that...
[0:38:04] In his world, like in the Pentagon, AI is seen as what they call, I can't remember if it's the third or the fourth offset. And the previous offsets being, once someone has technology X, it renders the other side already defeated. Like it was a nuclear bomb, and then it was intercontinental ballistic missiles, be able to deliver those on your shores, et cetera, et cetera, et cetera. They're like, this is the next one. And I'm just wondering, it sounds like you...
[0:38:33] That would be my guess. Again, we don't know. The scaling loss could slow down. So that's my guess. And I think the...
[0:38:42] You know, the democracies of the world, I think, need to get together and, you know, and I think recognize the situation they're in and address the threat. Now, at the same time, you know, I want to say an international arms race is just the sort of thing, it's worth acknowledging, that, you know, can accelerate things in a way that makes us skimp on a lot of the safety concerns that we have.
[0:39:06] So, you know, we're in a difficult situation. Right. So, you know, I think I mean, and other people, you know, can get a lot of criticism on both sides. Right. Where, you know, you know, are being supportive enough of, you know, winning the race against China versus is this reckless because we're not slowing down on say. And my point of view on this is just look like the situation we're in doesn't it doesn't have to have some easy, correct or moral answer. Right.
[0:39:34] You know, I think the more we can have a lead against...
[0:39:45] China, the more we can use that lead, the more we can kind of spend it to make sure that we're doing things safely. So I think the combination of a posture where we try to get ahead of and slow down authoritarian countries and then use the surplus that we have, which, you know, if we're lucky, could be like two or three years to, you know, to be careful, to put in place judicious regulation that.
[0:40:10] You know, make sure that, yes, we're building these very powerful things, but, you know, make sure that there are rules of the road and that we build them carefully. You know, some things that I think help with that. I mean, you know, I think some of the semiconductor policies, the export controls on chips and semiconductors have helped to hold back authoritarian states, right? When we do interviews with...
[0:40:34] You know, there have been interviews with, you know, folks in China and they say we can't get enough chips. If we got enough chips, we could catch up. And so that's something that gives us more room on both sides. I think we also shouldn't give up just talking to China. Like maybe they can recognize the dangers themselves and, you know, we should.
[0:40:52] Thank you so much for having me.
[0:41:21] Naive about when the world works, about how the world works. What a fascinating interview. What a fascinating interview. And what a fascinating time to speak to someone who is one of the leaders in this technology as it unfolds as well at this kind of really interesting inflection point. How do you feel? Are you feeling hopeful? Are you feeling freaked out? Are you feeling skeptical? I feel a little more convinced than I did before I heard the interview. I have to say, I came out, I don't know what the word is, Claude-pilled.
[0:41:51] You drank the Claudade. Claudade. Yeah, well, because I was able to, as I said, because I'm doing this larger piece for the paper, the magazine, I got to speak to the whole kind of top team, basically. And I had an interesting conversation with the CTO, who's leading the actual building of these models. And he was like, I remember a couple of years ago, we were so excited.
[0:42:18] The day that Claude figured out how to rhyme. Now it's doing computer use where it's like, hey, Claude, go fix this problem in another part of my computer. Get me a plane ticket. And it just takes control of your mouse, takes control of your computer, goes on the internet, does stuff. It's like writing really high level computer code. Taking the bar exam. Exactly. And then two years ago, they're like, yay, it can rhyme. Fat cat, Matt. And you're kind of like, whoa.
[0:42:48] You know, just the kind of, as he was talking about there, this kind of the scaling laws being so relentlessly going in one direction. But I don't think we should lose sight of the funding point, which is, you know, as you said, the dance partner analogy is brilliant. They are, this is big tech becoming even bigger. You know, I asked because there's a tension here, right? You know, one of the things they talk about is they've given Claude a constitution. They call it constitutional AI.
[0:43:14] It's a mix of like terms of service, like from companies like Apple, but also like the UN Convention on Human Rights and things like this. It's like this governing document that Claude must abide by. But at the same time, they've got one of the largest companies in the world as their biggest backer. They're in this huge race with OpenAI. And I was like, isn't there a tension?
[0:43:41] The number of companies out here who talk about their principles and what's important to them. And then the financial imperative just completely washes it out at some point. But they were like, look, all seven co-founders are still here. And sometimes it's a little bit messy, but we're all on the same page around keeping that safety mission core and front and center. And then you look at OpenAI and it's just Sam Altman left.
[0:44:08] Well, this is the point, isn't it? All the other co-founders are gone. There's that famous photograph with the crosses through them. I mean, it's a race, as you say, with OpenAI. It's a deeply personal race. These are people who all were working in the same very small company together who've split off. And Dario is being quite cautious. And I think it was pretty candid in the interview with you, but he didn't insult.
[0:44:33] OpenAI too much, but you get the impression. No one did, and I tried to get them to. I bet you did. Yeah, I'm like, why? Because it's very clearly like I see them and I think everybody sees them as the counterweight to OpenAI. And you feel if you scratch the surface that there's probably a bit of animosity? You know, I think they've been asked this question a lot, right? Because they all were like, we're out of here. We're going to do this, you know, without saying the exact words. We're going to do this the right way.
[0:45:01] Therefore, OpenAI is doing it the wrong way, the irresponsible way. So maybe animosity is the wrong word, but it's certainly a personal thing because they've had relationships and they've worked together. And I haven't spent the time I've just spent with them, I haven't done this at OpenAI, so I don't know that company. But that would be great and we'd love to do that if they would have you there. Exactly. But they do feel quite grown up and quite serious and earnest about like they've...
[0:45:28] They are building this thing really fast and they do, they're very aware like this could, if this continues, kind of change everything. But they're also really, they're worried about it and they're trying to kind of be responsible here. Like all of these companies are like, we're asking for regulation. Well, I would say Anthropic, don't worry.
[0:45:51] Regulation is coming. I mean, the UK's got the AI Safety Institute, which I think is looking at Anthropic's models, it's looking at OpenAI's models. I don't think they've got the teeth to do anything yet, but you are starting to see these institutions. You've got one in America as well, the Safety Institute there. And there's certainly one in Japan looking at what can be done or what should be done in terms of regulation. But it's just that.
[0:46:20] Kind of sort of thing that we always have in tech, isn't it? Where how do you, is it possible with things that don't have neat country lines and fit into neat jurisdictions to do that sort of global regulation? Yeah. Should we end on one of the final paragraphs of Dario's essay again that you've read three times? Because it's really, really interesting. And yeah, Kiara Brown, our colleague, has very kindly again read a chunk for us.
[0:46:47] If all of this really does happen over five to ten years, the defeat of most diseases, the growth in biological and cognitive freedom, the lifting of billions of people out of poverty to share in the new technologies, a renaissance of liberal democracy and human rights, I suspect everyone watching it will be surprised by the effect it has on them. I don't mean the experience of personally benefiting from all the new technologies, although that will certainly be amazing.
[0:47:12] I mean the experience of watching a long-held set of ideals materialize in front of us all at once. I think many will be literally moved to tears by it. Happy Christmas, Dani, and I look forward to reading the magazine piece with all of the other people from Anthropic. It's a kind of a mind-bending thing to kind of sit down and try to get your head around, but it was like you kind of, when you sit down and have hours talking to the people who are building these things, you kind of come out and you're like, whoa, is the world about to change?
[0:47:42] And obviously I feel like it is, but I feel like it's going to be messy. It's probably going to be slower, but the world is going to change in some interesting ways, I think. Yeah, it's an amazing time to hear from that company and a very rare interview to get as well. So it is Christmas, but I think we've got an episode next week coming up anyway. The podcast never sleeps. It's always summer, what is the phrase? It's always sunny.
[0:48:11] Happy holidays, everybody. We'll talk to you next week. This episode of the Times Tech Podcast is sponsored by Adobe. Brand leaders know the future of reaching customers lies in AI. But getting there means creating more content and managing more data. Adobe is the foundation that turns the opportunity of AI into business value.
[0:48:38] Get your brand discovered by new audiences, personalize every customer experience, and give teams the insight to set strategy with confidence. It starts with Adobe. You've got big plans for your money. You invest. You track. You plan ahead. But does your financial app actually help you decide what to do next? Quicken helps you go beyond balances and basic reports. Project your cash flow.
[0:49:03] Dig into your investments and use Quicken AI to understand what matters across your finances. With Quicken, you can make sense of your options and decide what to do next. Quicken, know what to do next. See how at Quicken.com. Today's markets move fast. Get the insights you need in 10 minutes with the Barclays Brief, a new podcast from Barclays Investment Bank. Through sharp dialogue and scenario-based analysis, our leading experts analyze key market themes each week.
[0:49:31] So whether you're managing a portfolio or leading a business, the Barclays Brief podcast can help you make smarter decisions today. Stay sharp. Stay brief. Find Barclays Brief wherever you get your podcasts.