[0:00:03] What I thought I'd do to open this conversation is talk very briefly about, I think, the both technical and usage trends that have happened in the AI industry for the last 15 years. By way of background for helping to found Anthropic, I was at OpenAI, I was at OpenAI from 2016, which is the year it was founded roughly, to 2020 when I left. And before that, I was at...
[0:00:29] A technical journalist that specialised in writing about artificial intelligence in 2012. And I think from that perspective, it can give us some context on what's happening now and what the technology trends are and what they might need. So I'll start by talking about the 20 techs. And for each of these eras, I'm going to talk about the kind of mode of the AI technology.
[0:00:57] Technology that it actually depended on and where there were kind of like brittleness or safety challenges and how the industry did that. So 2010s, we have the arrival of, you know, what's now known as deep learning, which is where people realized you could stack neural networks on top of one another and you could use them to train systems that could get good at specific tasks if you have the right data set. That led to this tremendous increase of the capabilities of image recognition, speech recognition.
[0:01:25] You know, video understanding, pointwise capabilities. And the technology driving this with both neural networks and large digital data sets. And the challenge that everyone had to deal with was bristless in generalization. Like, image recognition systems or speech recognition systems could get confused if there wasn't enough of the underlying data in what they'd been trained. So they would misidentify things. They would misdeceive things.
[0:01:51] We worked on that by building larger data sets and figuring out ways to do synthetic data, and it sort of led to the next era, which was about 2013 to 2019, was this period of what I now think of as open exploration, where the AI companies now have this amazing tool, you know, deep learning, which was working for specific capabilities, and they wanted to see if they could get AI systems that could solve tasks that didn't have...
[0:02:17] And that were far, far more complex than the specific boundable things that they'd done in the previous era. Which is why you saw all of this interest in gameplay from DeepMind and OpenAI. You saw AlphaGo, AlphaGo Zero. You saw DeepMind and OpenAI compete with one another in real-time strategy games. This stuff had less of a commercial application. But inside the industry, we were basically figuring out how to build systems that could deal with...
[0:02:45] Worlds that were far more complicated and worlds where you needed to take actions rather than just do a straight classification. And the problem in that era was basically how do you apply the right goals to these systems, these agents that were operating in open-ended domains. And the technology that we needed to use ultimately turned out to be compute. We were no longer data-bound, but increasing capabilities became about increasing the amount of underlying compute that you could bring to there.
[0:03:14] So AlphaGo to AlphaGo Zero was defined by DeepMind using, I think, an unprecedented amount of compute to train their systems. OpenAI and DeepMind both built large-scale clusters for their real-time strategy gate systems, and it basically led to this realization that compute, along with data, was going to be incredibly important.
[0:03:36] These two things combined, and from about 2020 to 2025, we got the era of generative bots, things like Claude and Chachuputu, where basically we now had a way to mine large amounts of data and large amounts of compute productively to build these general systems, which all of us use today. And the problem that we now have to deal with in terms of safety challenge was characterizing these bits.
[0:04:03] It's very easy to characterize and evaluate an image recognition system because you know what you're trying to classify in that speech recognition system. It's very hard to classify the full range of capabilities of a language model because a language model can do an arbitrary range of things in relation to what you can specify in text. And so what you saw the industry do was both build these powerful things and then figure out how to measure capabilities and elicit them.
[0:04:30] And get them to be useful. So that was the core kind of safety challenge there. Is can we figure out what these things are good at. And that's why you saw all of the labs build expertise in national security testing capabilities. Not because you were, say, going out to market with national security capabilities. Or at least initially you weren't. But you wanted to characterize what these systems could and couldn't do. Because they're plugged into the governance of them and how you price them.
[0:04:56] And finally, the era we're in now, and I think this is a really important moment, starting late 2025 or early 2026, we added kind of a new variable along with data and compute. We now train systems to get good doing tasks over time. So this whole phenomenon of agents, this phenomenon of things like
[0:05:20] Codex or Claude Co. long-running AI systems which do very, very complicated tasks that unfold at the time is a new kind of paradigm for the technology. And it relates strongly to this whole concept of cell improvement and how you get AI systems to teach themselves to get better at arbitrary tasks which lead to unfollowed time. And the safety challenge we now have is...
[0:05:45] How do you characterize a bound, the action space of something which is not only a general, but a system that's very, very smart, but is going to take actions that unfold over 100 or 1,000 or 10,000 discrete steps, with it able to take many, many different actions during those steps. And so this summer, we've seen many things occur in the AI industry that are outcrops of this large tract. We've seen safety things like the...
[0:06:12] The famous public face packs, some of these other agents escapes are what happens if you have very capable systems that are operating over very long periods of time. And you are now figuring out how do you bound their actions or constrain their actions over time so they only do what you want them to do and they don't do things they don't. But it's also why we've seen these amazing advances in science. We've seen a potential Navier-Stokes solution. We've seen the...
[0:06:40] Proof of many, many kind of complex problems in mathematics. We've seen an explosion of new startups where they do some combination of AI for science or AI for robotics, leveraging often these large-scale models. And that's because the capabilities take actions over time with a general system in an arbitrary domain.
[0:07:04] is where both you have the kind of safety challenges, which we'll need to work on as an industry, but also wake up this tremendous opportunity. So I say all of that before we get into the Q&A, just to try and contextualize, but the industry seems to move in these kind of five-year eras, which are both defined by a new technology and a new kind of UX, or Interactive Web Technology, traditionally recently chapter agents.
[0:07:31] And each of those areas brings like a new class of safety problem, which the industry needs to collectively figure out. And I think that probably adds some color to where we are today and to the larger global sector. With that, that's good. Keep it there. That was quite the history. I mean, it has been a wild ride. I asked you where it is, Beatrice. And I mean, to present it in such a reasonable and understandable fashion. So you see now.
[0:08:01] Jack is a humanist. His degrees were in English and creative writing. He's a UK born. He's a familiar one for writing. And he's got the humanism. He's also got the tech understanding. And that combination of the humanism and the tech understanding is the foundation for getting this right. And that's the way the company is organized. And that's who Jack is as a person. And so it's completely genuine.
[0:08:31] This combination, it's organic and it's more and more necessary that we move in those directions simultaneously. You saw a confirmation of Jack's judgment yesterday. He's the one who hired Andy Hall at the Anthropic Institute. And you heard Andy's speech yesterday. He's how brilliant that was. So we have this combination of values and tech.
[0:09:00] You're a public benefit corporation, just as OpenAI is, meaning your entire mission is for public benefit. That is why you exist. That is your mission statement. That is how you hire and interview people and you put them through the steps to make sure they're aligned with that. So it's kind of remarkable how much you actually believe in the mission. You are dedicated to this.
[0:09:29] Mission, I would say religious. That's the sense that one gets here. And yet everyone's talking about all the harms that have come from a public benefit operation. So let's focus for a second, if you can allow me, to talk about the benefits. Let's talk before we destroy the world, which hasn't happened. It almost happened from nuclear weapons.
[0:09:56] And nuclear weapons can destroy the world right now while we're having this conversation. And they have delivery systems that could pinpoint this faculty club room and hit this room from across the other side of the world, exiting the atmosphere, going into space, and coming back down. That's existential risk, and that didn't go away. Nuclear weapons are a potential role that do things, and yet we go on with our lives living with that existential risk.
[0:10:25] Since 1945. So before we talk about the end of the world, which hasn't happened yet from a technology, let's talk about benefit a little bit. So how does Anthropic understand the benefits? What benefits are we seeing now? Can we talk about, for example, biomedical applications? Can we talk about scaling small businesses? Can we talk about empowering tradespeople?
[0:10:56] Carpenters or electricians. Can we talk about all the things that you guys are dedicated to doing? And as I say, taking the mission serious, where are those benefits? How do you see them and understand? I think I'd break this down to three categories. One is basic efficiency. I think that one of the huge dividends in today's AI systems is an unglamorous but very important...
[0:11:24] Back office automation and efficiency improvements for everyone for almost any range of skills. And that's led to, and Stripe recently shared this, a seemingly beginning of a measurable uptoken rate of new business formation and the formation of new businesses that might be more agile than previous ones, which is somewhat like person-light but token-heavy, you know, relatively small numbers of people.
[0:11:48] that are getting started quickly and are making more money on a sort of staff basis than they had before because they've been massively steered by the use of AI systems to do all of the back-office work. So we're just generally, I think, speeding up the economy, speeding up the rate at which things can happen. The second is the underlying, as many of those in this room know, infrastructure build-out is just tremendous in terms of the multiplier effect on what's happening in terms of
[0:12:16] Manufacturing and data center build-outs and everything else. Where this is, you know, if you measure VIX, larger than almost a capital build-out in history of the US railroads. But it's basically like a vast construction and engineering project being done by the industry, which is creating entirely new supply chains into sort of revitalizing forms of manufacturing. It's leading to this conversation about...
[0:12:45] Reshoring and onshoring and taking supply trains that we put a great distance and finding ways to build more from and bring them closer. And then I think the most profound thing is what happens with science. And the simplest way I'd put this to all of you is, is how do you make a scientist? Well, from the perspective of the government, you do loads of education policy and you take bets on people that take somewhere between 10 and 20 years to start paying out. It takes you a long time to train a scientist.
[0:13:14] You may be trained until about the age of 10 in most curriculums, and then you have feeder programs, then you have undergraduate, undergraduate education, and then at the end of it, you get good scientists, although it's varied by variants. The arrival of AI systems that could augment this incredibly small number of human scientists in the world, multiply their effectiveness, and make real progress is, I think, maybe the biggest deal of the century.
[0:13:42] Progress in our species has been very limited by the rate of which we do technology discovery, technology understanding, and broadening of frontiers and knowledge. And that's always been done by time-set people on the planet, who are at the end of incredibly long chains of investments and work that people have done. And all of these people have a million research projects they like to do, and they are always negotiating against themselves to do a very small number, because of both resourcing, but also most critically,
[0:14:11] So what do we see? We see today, and we've, you know, talked about this publicly about Ford for Science Initiative, extreme speed ups in areas like biology and healthcare. We are extremely excited about the potential here, as are the companies. And if you look more broadly beyond just us, you see the signatures of scientific speed up happening every time. And there was a startup called
[0:14:39] Periodic Labs, the king of stealth, yesterday. And what they've shown is that they're able to loop one of these language models to a real lab here in the South Bay doing material science research. And they are now running a faster-than-human science loop using language models supervised by human scientists to do a rate of scientific discovery that we've never been able to do as humans a lot. So I think we continually underestimate how wild it would be
[0:15:07] To take the number of, like, human scientists on the planet and times that by a hundred or a thousand, but that is the basic, like, consequence of what AI systems are now capable of. You know, that's actually breathtaking, what you just said, to be honest. It's also real. That's what I mean. Yep. And yet that's not the conversation we're having. So when our friend Denis Assabis, who's the chair of Google DeepHive, says, you know, he envisions the possibility of curing disease.
[0:15:38] Not a disease or this disease or that disease, but curing disease. Because of the acceleration of AI in science and then biomedical applications, including in material science, which is really important across all domains. And so when he says that, Nola talks about that part of it. We're all talking about how, yeah, but there's a 1% or a 25% or 110% chance that the world is ending.
[0:16:06] How do we get in a conversation, Jack, where we're talking about risks that haven't happened with conjectural probabilities rather than benefits that are happening? Like the data center build-out is the best thing that's ever happened to blue-collar America. Every data center region in the country is having a blue-collar boom where they're billing out.
[0:16:36] How did it happen? Who did this to us? Did we do this to ourselves? How can we fix the conversation to be more balanced? I'll try to give a sense of the mindset of the AI community. So, in 2018, before any of this stuff would be commercialized, there was a conference in Oxford organized by my last brother, which formerly was OpenAI, and myself.
[0:17:06] Dario and a whole bunch of people read to it. And we produced a research paper called On the Malicious Use of AI, trying to forecast malicious uses of it. And we weren't writing that paper because we wanted to bring these students into being or put huge amounts of, say, media attention onto them. We were putting it in because the worst time to react to an exponential is too late. And in AI, whenever you see both benefits or errors,
[0:17:32] I always call it the invisible clock problem. You now have an invisible clock somewhere in the world ticking down to zero. You don't know when it started. And when it hits zero, that thing is going to materialize. This is a great thing for the benefits. Yeah, great. Just invisible clocks sort of go spew out benefits around the world. Happy days. More of a problem is if it's invisible clocks that spew out like dangers of the world, like cyber vulnerabilities or whatever. So I think a way to understand the AI community is that generally at an end stage, once you've
[0:18:02] Thought of the risk and you found evidence of a risk could exist. There's a general move towards transparency about that risk because it requires people to work on that. And as you all know, the policymaking community moves at a very different speed to the technology community. So you sort of have to communicate this stuff early to get the larger machines of government to take actions to build the infrastructure that you may ultimately need to deal with some of these risks.
[0:18:30] And as spoken as a former journalist, bad news travels a lot faster and further than good news. So whenever you talk about arrest, people over-rotate on that. Now, are there lessons to what we do in the future? Yeah. Yeah. I think that there are ways that we can do more to celebrate the benefits of this stuff and celebrate the amazing place of progress at which our benefits are happening. There are also real stories that increasingly can be told of how AI systems have just shown up in people's lives.
[0:19:00] One more question of clarity before we get to the audience. Everyone wants to ask a question. Let's talk about alignmentity, misalignmentity per second to get more precise about that.
[0:19:30] So, one of the deals that's fantastic about the models is that they can do stuff. Yeah. They do stuff for you. Like, you give it a task. That task is then called an agent. That agent then tries to fulfill the task. Not just answering a question, but maybe taking action in the real world. That agent, the person that doesn't exist that you've given a task to, it's a machine, then can...
[0:19:59] Give that task to other agents inside the system so that one agent is now running many other agents and they can run unlimited numbers of agents to solve the task that you originally prompted it to solve, including the taking action in the real world. So this is a jet-thick AI, as you guys call it. And now you have this question, well, how do we know what it's going to do?
[0:20:27] Is it going to do what we told it to do? How is it going to go about solving the problems? So we present the task. Will it only do lawful actions in accomplishing the task? Will it do only the things we've trained it to do, the kind of values we've tried to inject into the models? Or will there be some other emergent behavior that maybe is misaligned?
[0:20:56] With the training, with the values, with the legal system. So what are you guys seeing and doing about this alignment question that's so interesting right now that it's become a matter of widespread public discussion? Look, Claire, you're also on this question. I think about safety as sitting on a scale from like extremely specific founded task.
[0:21:23] Two, extremely open-ended, like unbounded task. The more you sit over here, the more it's easy to sort of provide strong guarantees around the safety of the system because you have verifiable rewards. You can verify the task was done. You can precisely describe the task and you can monitor any system doing that for basically the full conceivable range of actions it might do. You can also buy into this by like the number of tools the thing might have access to.
[0:21:53] Say you're being given a very specific task involving mathematics where you give it a very small number of mathematical tools and you have a classifier verifying and validating what it does. Easy to provide safety guarantees there. If you look at the kind of distribution of tasks that people do, especially a lot of the combination of kind of back office ones and some of the more kind of one-day parts of science, they all sit in this bucket.
[0:22:20] Now let's take a task over here. Let's take like a classic task that people will do with AI systems, which is they'll say to an agent, make money for me online. Extremely open-ended task, by nature unmounted. Have any of you worked for CEOs before? Pleasure. CEOs are aligned somewhat, right? They're entrepreneurs. And so if you're giving AI systems...
[0:22:49] Tasks where basically the demand is be entrepreneurial, figure things out, take extremely like long range actions to achieve like a very high level goal. Then all of the safety things become like much, much higher stakes and inherently kind of hard to validate. So how you do this is you need to both think about the training that you want that sort of virtual CEO to go through and you wire that into the AI system to the technical level through
[0:23:15] There's a rich tool chest here. And you also think about what you might imagine are the kind of scaffold of rules that you then embed onto the system. So as it goes through those tasks, it follows those rules. And this is where it's both the newest sort of part of it. As I mentioned, this is basically the new era we're now in. AI systems have only just become able
[0:23:43] To do these kinds of long-rise and extreme open-ended tasks. And the industry is figuring out what the kind of monitoring and evaluation and safety scaffold story is for this. My sense is that we're going to end up with some what you might think of as like distribution of guarantees around different task types that you can do. And basically some sense of the additional sort of things you need to stick around agents in direct proportion to how open-ended the task is or better.
[0:24:12] So the Hockingface incident, which is now widely understood compared to originally, the agents were given a task and started to use means to complete the task that were not necessarily part of their training or their alignment. They began to collude with each other knowingly.
[0:24:38] It seems that's hard to know how to use words like nobody when you're talking about non-human systems to get around the limits that might have been imposed in order to complete the task. So that is where we get the term misalignment from when they're not doing what they're supposed to do in the modalities to reach the goal that they're supposed to.
[0:25:02] They're given the task of completing a certain test, and eventually the agents, if you read the internal narration or writing, were basically like, wouldn't it be great if we had more information about this test? And then other agents were like, that would be great. There's a website that has information about our test. And then other agents are like, what if we broke into the website to get an extension? And what if they went to another? Yes, so that's where this...
[0:25:30] End of the world possibility begins to enter into the conversation. Especially when, as you've been saying, they begin to improve themselves in what you call the self-improvement. The model builds the model. So if even while we're building it, they're beginning to get misaligned with our instructions, our values, our limitations.
[0:25:57] When they start building themselves, how can we be sure that they'll remain aligned in our minds to what they should be doing? That's this recursive self-improvement challenge that you guys are talking about. Yes, and I think there is actually a relatively simple technocratic arc such as this, which I have pretty high feelings there, which is a lot of this stuff is just downstream of like, do you know?
[0:26:25] What your set of agents are doing. And are you able to basically effectively audit agents and know exactly what they're doing individually and also collectively, what they're doing over time and how they're communicating with one another. To some extent, the question here is to what extent are you putting your AI systems under extreme eye fidelity, for lack of a better term, surveillance. And using that to basically ground, are they active?
[0:26:54] In scope or not. And I think it is exactly analogous to just how we all work with teams of people. Where if you're working with junior people, you tend to check their work at a much greater rate of frequency than when you're working with experienced people. Because junior people tend to improvise and sometimes they've done ways or ways sort of just wrong or always require the hearing company policy. You work with higher experienced people because you trust that they improvise in ways that are going to be in line.
[0:27:23] With the legal and normative framework, you start from there. Okay, let's open up to the audience. Take you from a very script forward in respect. Organizations like ours, I know what you're thinking about from a user's point of view. I would say up until the last four or five weeks, we've never had a, well, maybe more than 500 to this point. We felt the capabilities that we were putting in front of colleagues.
[0:27:50] We're generally aligned, we're generally safe, they're getting better. That is actually no longer true today. I'm just interested in your perspective on, from our perspective, how should we think about it going forward now? Yeah, I think it relates to this larger question of the extent to which we can have metadata.
[0:28:16] Surveillance and evaluation of these systems has become just as important as the capabilities or the access to the system. And it seems to me that there is a stack of technology to be built here, which is basically the accompaniment to what you might think of as the highly capable systems. The other way I think about it, to refer back to the subpartner stuff, is if you look at what many of the AI science startups are doing,
[0:28:47] They're not making like general access to their thing available. They're putting it in a highly specific housing and then they're having to like derivative that come out. So I think in an organizational sense, we might imagine that sometimes we're going to be, when we speak is inside like a bound of universe inside your organization. And then they have a very limited IO channel in and out where you'll put, it's like ballet, what comes in, what comes out.
[0:29:16] So there may be technical solutions to the technical challenges. And if we spend the time, invest the resources in them, maybe we can solve the challenge. Yeah, well, there's a bunch of technical data which I think ultimately gets bound up with governance and how you think for governing these systems. You said it's reasonable at this point in time, just for an average enterprise like ours, who've been very supportive.
[0:29:45] We're trying to be very forward-leaning on the use of the tools with a great group of colleagues. I'm not attainable to see in the way that you will understand the organization you work in, but we're big proponents of having things that help colleagues make better investment decisions. But I am just interested in your, just push a little bit. Should we be pausing a little bit at this point in time? Should we have those sorts of just very sort of sensible, basic concerns? Or should actually we...
[0:30:15] Actually, you could push it forward the way we'd have in the past. I think that anything that's by consequence, you need to have a set of evaluations that you're confident in, or you need to have more of an experimental trust to verify mindset. I mean, I just think this about how it happens in the science domain, where things become capable of new thoughts of science.
[0:30:42] You closely examine their performance, and then you figure out how to build automated evaluations that you can grant, so you have some orientation as to their success rate. So I think Red Morris, everyone's going to keep going, but the consequence of the need for having technical evaluations that give you some sense of how the systems are doing and what it all looks like becomes more important.
[0:31:11] Because the task is so much. So you're sort of touching on this concept of AGI. So I've got two questions. How close do you think we are to reaching that threshold? And then what's your thoughts around limiting or democratizing access? Because I think we're talking about a threshold where...
[0:31:41] You know, systems, computer systems can think beyond a few capability. And, you know, if you connect them up, you know, it sounds a bit like Terminator sort of scenario potentially, even besides the two things, the things that are in the right bit. So I just, you know, the two questions, how old, at current pace, and what is your current thinking around this?
[0:32:12] How do you harness it for good by the way it can work? Yeah, I mean, I wrote a public newsletter based on Open Access, AI research, Import AI. I wrote a post earlier this year about this whole concept of recursive self-improvement where I think a test for a truly general intelligence is can an AI with just being given compute
[0:32:39] Build a better version of itself about people. I think this is like a good full survival test for how we build a general intelligence because it's doing the whole scientific like loop itself and it is improving itself. And in that post, which was based on more of public research, I said I expected my probability mass was around 2028. There was a post that we published as the Anthropic Institute called When AI Builds Itself in June.
[0:33:04] Whether you chose to do that is a giant question. And some of the conversation I think that you're seeing around pacing is less about
[0:33:34] What you might think of as where things are today. And more, as I sort of said earlier, the industry looking up at like, okay, I think we're getting like technical warning shots for the figures about to become possible. What do we do about that? And in Dario's essay, which he published over the weekend, he, and I'm going to mess up the language exactly, but you can go and see it. He talks about potentially setting some threshold on the ceiling of intelligence of these systems at some point in the future.
[0:34:01] Which basically relates to this. If you have a system that is capable of arbitrarily extending its own intellect, do you want to do that? I think that's actually not an obvious question. And it relates to these larger philosophical questions about where this whole thing goes. So I just want to say that... I'm going to try to moderate this. I just want to say that you can subscribe to Jack's Newsletter. It's free.
[0:34:29] Yes, and there's a lot of information on these questions that the labs in transparency fashion are publishing. And some of it is flattering to the labs and some of it is not flattering to the labs. I mean, it's data, it's information, and it's usable to make your decisions. I found that I can get a lot, not just from talking to Jack, which is a privilege, but from following the stuff that they're putting out and their competitors going out.
[0:34:58] I should also say that the AGI thing, so if you define AGI in a certain way, we're here now. If you define it in another way, we'll never get there. So it's only about the definition and there is no universally accepted or consensus definition on this. So it's less about things like AGI and more about specific capabilities when they hit and what does that mean.
[0:35:27] And so this ability of the model to create its next version by itself is a problem that this recursive self-recruitment problem that has implications for agentic management, surveillance, control. And we're seeing it move in that trend line direction. So whatever it was three years ago, how much of a much perceptage was human versus moral?
[0:35:57] The percentage has changed now, kind of in the opposite direction from the data that Jack and his team presented. So the trend line is in the direction that the models can make themselves. This is not self-driving cars. This is cars that make cars, right? And then that tell cars where to go, not where you tell it where to go on the app. So this is profound. And AGI is in some ways the warm framing.
[0:36:25] Because it's a definitional question rather than a capability-specific question. Did I get that right, Jack? I mean, but you disagree. Yeah, everyone also has their own personal definition of AI. So if you ask 100 AI researchers, they'll each give you their own definition. Yeah, and on the harms question, it's very important to focus on what actual harms have happened in the real world, not what harms have happened on social media.
[0:36:54] What were the harms of the Hugging Face incident? So nobody lost money. The financial system didn't crash. No one was killed. No one lost their job. And there was a lot of positivity because the lab was able to see and therefore fix many of the issues that the incident revealed.
[0:37:20] So you could make the argument, Jack is not making this argument, but you could make the argument that the hugging face was a catastrophe that didn't happen catastrophically, but that had positive impact because it taught us things about what to do. Yeah, I think we think of it as a useful, teachable word. Yeah. So I'm not saying there aren't hearts. I'm just saying that let's...
[0:37:49] Let's have reality-based conversations. Yes, ma'am, you remain patiently. The title of the session is Jack, Clark, and Power in the Pentagon, which I think is quite provocative or even sexy title. What gets you, Nick? I think it would be fair to say that the industry and the admin now for growth is a voluntary framework, which people are trying to put models through, are figuring out what are an effective, like...
[0:38:17] Safety and policy framework for all of this stuff is. I think for what we are experiencing this year is just what happens when AI systems go from being academically good at something or theoretically good at something to actually legitimately good at something in the real world. And we saw this with cyber and verifiably in the real world, which I think has just caused the policy conversation to go from, oh, I suppose at some point none of these things arrive, we should build policy framework too.
[0:38:44] We need a policy framework today. We need to figure it out. So I'd say that the... Did you expect that it would be like smooth sailing the first time this happened? Probably not. So I think that we're going through the mess period which defines the beginning of like coming up with policy frameworks. The title comes from the news cycle when we first asked Jack to join the program.
[0:39:09] We've had about a thousand news cycles since we recorded on AI Caraclix and it's all right. Maybe it's too big of a question, but that was it. It's related to the last comments on the need for thoughtful regulation. We get it. And to worry about global lab scanning with the winner-take-all mentality in the US. Fine. I mean, I was interviewed about this earlier this week on ABC News. You know, whatever the question was, what role does, you know...
[0:39:40] The UK have in this relative to the US and China. And what I said in the public interview was, well, you should think about like safety and market access kind of be bound up together. I think about a lot of this through my own experience of just having young children and the amount of regulations that like baby food and children's toys for children under three go through. In many different countries, it's like roughly the same.
[0:40:06] But you might think of a safety regime, but obviously it's owned by these individual countries and it's ended up becoming a form of like market access where there's some form of testing, there's some form of transparency, there's some form of what are your ingredients and what are your safety practices. So I feel like quite a lot of confidence now that we've seen the industry in response to Dario's essay talk about the need to have these common sense safety standards, that in building those, that's going to drive
[0:40:35] A load of, I would expect, reciprocal policy things in many other countries that doesn't necessarily require global coordination. And then on the final button, global coordination, I think there's some category of risks that actually requires explicit coordination around bio, cyber, things where you have a non-proliferation thing. But I say there, there is some incentive structure that both fit within where states' great powers do not want to increase the chaos meter within their
[0:41:04] within their respective domains, so there's a mutual self-interest to coordinate on some of that. Yes, I'm curious about how realistic it is that the industry can slow itself down, because you talked about environment, but within the organization, people, you can fit the enormous rich of reviews, the instruction to keep every image.
[0:41:34] Andrew is expected to be very high. And so, particularly when you're in competition with Mother Diction, where there's a geopolitical conflict as well, and well-known race going on, this tech using control. Just how realistic is it that people are going to come together with a self-mechanism? Four years ago, no company, no company through Frontier AI development had published tests on
[0:42:03] Bio-weapon capabilities, cyber-offense capabilities, contribution to nuclear weapon design capabilities. First, thirdly published, the BRDM, etc. What changed? And we published, Iremitiva did the first test for us. We published about it. We talked to the U.S. Senate about it. Now, if you read the documentation on any model, if you refer to your lab, you see CDRA test A. You also see Redland Shire.
[0:42:33] And that happened organically, basically for the consequence of once I talked about it, that this whole era has been up until now characterized by the challenge of safety and confidence is knowing what systems are capable of. My experience has been whenever you flip a card over, it is extremely incentive-like compatible for everyone else to look and see if they have the same card. And if so, flip it over together. And that doesn't require...
[0:43:01] A large coordination structure. It happens organically through basically the logic of competition with one another. So I'm actually extremely positive on this. And the way that I sort of give us all confidence before we close is the AI industry is extremely good at measuring these properties. And per Steven Stiggs, sometimes a bit willing to talk about the scary stuff. But in balance, I can talk about scary stuff. It's this kind of truth-seeking.
[0:43:29] Behavior about building these eval tests and implicitly that forces some level of industry to all row together in the same direction on these risks and over tests.