[0:00:13] Thank you, Neil. And it's really a privilege to be speaking to everyone today. I'm sorry that I couldn't be there with you in person. So I was asked to speak on sort of hypo thoughts about interoperability as a field. And as I was preparing this talk, I realized that maybe it made more sense to give kind of a collection of unrelated reflections rather than one cohesive narrative. So the structure of this talk is going to be a couple of reflections that aren't necessarily particularly related, but...
[0:00:45] And I hope it'll be interesting. The first one is really an analogy. And it's really about how I think about what is it that we're doing. And also it's about how we talk about it. So I've come to think about mechanistic interpolation as being a kind of mathematical biology. You know, we start with some neural network architecture that we define with some random initial weights. And we train
[0:01:17] The system, we get a model, perhaps Claude. And when I think about that, it sort of feels to me like we built the scaffolding and then we grew something on that scaffolding through training. And the resulting object that we created is really quite inscrutable to us, even though we understood that initial scaffolding. There's some very deep way in which neural networks are grown rather than built.
[0:01:49] And studying them is a kind of alien mathematical biology. One consequence of... And yeah, and so just as evolution creates the awe-inspiring complexity of nature, creating descent creates really mind-boggling rich structure inside these models if we're willing to do the work to find it. One consequence of this view...
[0:02:25] It's a lot of value anchored in the biology, right? So I think sort of we all probably agree that you need to understand some mathematics to do interpretability. You know, these systems are made of math, so you should probably understand some foundational mathematics. But I think there's also a lot of value in sort of internalizing the biology, as it were, the rich phenomenology and data and empiricism that we have contact with here.
[0:02:55] That biology, I think, is where intuition comes from. And it's also where feedback comes from. So I think that's a way in which our field is different from a lot of sort of more mathematical disciplines. Okay, so my next reflection, this is something that just feels so different than a year ago. It's a way in which safety is just starting to feel really real today, not as a hypothetical thing in the future, but really a live issue.
[0:03:26] And along with that, interoperability is starting to feel very relevant to that. So on safety being real, in extreme safety testing, we find that Claude will actually blackmail people to preserve itself in really extreme cases. And that's not specific to Claude. All of these frontier models will do this.
[0:03:58] And interoperability is becoming a really important part of how we think about things like that. Because it's just so wild. It's so crazy, right? We need to be able to go and understand that. And one really interesting example is, you know, with Claude's Sonnet 4, we observed, you know, various misaligned behaviors. With Claude's Sonnet 4.5, all of a sudden, we didn't. But there's this curious thing where a lot of the time in its chain of thought, we'll see that it talks about how it might be being evaluated.
[0:04:31] And it turns out that that actually kind of undersolves the problem because actually one of the first things we found with interoperability is that features related to being evaluated or tested activate much more than this. And so there's actually a lot of unverbalized evaluation awareness. And training is actually sort of a leading indicator of the verbalized evaluation awareness. And also we can, well, one question that's really raised is, right, is, you know,
[0:04:57] Is the model actually safe or is it just appearing safe because it knows it's being evaluated? And so one thing you can try to do is you can try to use interoperability to go and then inhibit this evaluation awareness. And it turns out the model suddenly starts exhibiting more misaligned behavior. And so interoperability really played a big role in that analysis. It would have been very hard to accomplish by other means.
[0:05:18] I think this is a general story, right? That safety is becoming this really real concerning thing. And at least Anthropic, Interpolity is playing a larger and larger role in our analysis of safety. So if you look at the most recent model release, you'll see that in our system card, there's a lot of places where Interpolity comes up. Okay, so all of that actually lays a good groundwork for the next thing I want to talk about, which is ambitious goals and high conviction bets.
[0:05:49] So there recently was a very thoughtful blog post from the Google DeepMind team on pragmatic interoperability. And I understand that you're going to get multiple lightning talks relating to this later on. I think that somebody from the DeepMind team might talk about this post. And I think you might get a rebuttal from Leo Gao. And I saw that David Bao also has a talk on curiosity that I suspect might partly be responding to this as well.
[0:06:14] Honestly, I suspect that they will have more thoughtful things to say to me. I haven't really had the time to fully deeply engage with us, but I wanted to offer some preliminary thoughts all the same. And the first thing I wanted to say is, you know, I think there's actually a lot of value in having a diverse research portfolio. This is true, you know, within Anthropic, we think about having a diverse research portfolio. And I think it's even more importantly true for the world as a whole, for the field as a whole, for Interpol as a field, for safety as a field.
[0:06:44] Because people might be wrong. Now, all the same, it's also really good for us to debate what the best things to do are because we want to be doing the best things. One other thing I wanted to say as a preamble is I think that maybe, you know, there's a bit of an exaggerated impression of views where I think maybe I'm sort of on the far sort of, let's say, interpolate the ideal of them side of the spectrum. And yet, you know, we do a lot of what might be described as pragmatic work. I think some of the examples were from us.
[0:07:14] And I think that just sort of obviously makes sense given how things are connecting to real safety issues today. But maybe where it's a little bit of divergence is I think I still believe in something one might describe as ambitious interoperability. And it seems to me, first of all, that in that there's still a lot of shared views. I think that we have an agreement with something that I might articulate as there being a lot of low-hanging fruit in applying interoperability to safety. And given that, I would just be crazy to not be doing that.
[0:07:45] And also that feedback loops are really important. This is sort of the theme of the biology point that I was making earlier. But it also seems to me like there are some differences in views. So one thing is, I just feel like gradient descent is smarter than me. I have a lot of fear about not...
[0:08:07] Being fooled because I didn't know what to ask for, or I was caught off guard by unknown unknowns. Some very interesting methods that have the capacity to be surprised, that can tell me that there's hypotheses I wasn't considering. And this really goes, I think, to more sort of unsupervised, bottom-up methods that are often more difficult. I think there's also some disagreement on how exciting past results have been. I'm pretty biased on that one, so I think really that's for all of you to evaluate.
[0:08:39] There's also some differences maybe in how excited we should be about ambitious safety cases, where we try to say something like, you know, we really significantly understand this model with a very strong reason for believing that it is safe. It sort of flows from a sort of understanding it in some quite comprehensive way. I think it's unlikely we will be able to produce such safety cases before we have really powerful models, but I think we might be able to produce evidence that such safety cases are in principle possible.
[0:09:09] And I think that could be very significant. I think we might also develop tools, which even if they don't get us all the way there, can be very powerful in making safety cases or doing safety analysis. And I think finally, probably a difference in use is what's in reach. What is in reach in the near term? Now, one thing you might say is, Christopher, what if we don't have that long? And I think that's a pretty reasonable question to ask. But then something striking to me is that a lot of the projects I've been most proud of,
[0:09:39] Have really taken about 10 months. And this is in contrast to a lot of other fields, right? You think about like ambitious projects and other scientific fields, oftentimes they take years, right? Not 10 months. And so even if you wanted to have impact in some very, very short amount of time, like two years, it wouldn't be crazy to consider having some long-term sort of ambitious bets in your portfolio in addition to applied work. Now, I think as you move towards your target date, you probably end up doing more applied work.
[0:10:09] And you hope that you're sort of more ambitious things that you can leverage in that work. Now, I think it's a bunch of ambitious goals I could talk about. Macroscopic structure, you know, I think safety cases. I think a lot of things that, you know, one thing I want to highlight in particular is this idea of true global circuits. And we've seen, I think, progress there. We've been working towards it. Leo Gao's recent paper, I think, was a step forward here.
[0:10:39] I just want to articulate what would be necessary, I think, for this, in my view. So what I would like is monosemantic features that are truly monosemantic, that they are interpretable as meaning something not just on some narrow sub-distribution, but globally, and not just in their maximum activations, but across the activation spectrum. And that the weights between them would also be interpretable. So in our work, one reason that it doesn't count as this is that we were relying on attributions, which sidesteps a bunch of problems.
[0:11:08] And don't really give us this. But I think that this is starting to become in grasp and I think would be very exciting, especially for this project of making stronger safety cases. Okay. I'm now going to pivot to something else, which I'll call useful anthropomorphization. So this is my first meme in a talk. But it seems to me that, you know, a lot of...
[0:11:35] Laypersons will use theory of mind to read about AI. And then there's kind of this more, let's say, elite discourse, which says anthropomorphizing models is bad, right? And there's reasons you might say that. But I've been coming increasingly to actually be more sympathetic to using theory of mind to think about AI, at least in some cases. And I actually think that this maybe isn't so surprising. You know, we train models on data about ourselves.
[0:12:03] And so I think it's not that surprising if they then model us in deeper ways. And as a result, thinking about them as being kind of similar to us becomes actually quite a useful tool. And this has come from, you know, there being lots of very human-like features. That's been a major intuition pump for me. Looking at what happens when you ask models about themselves. And of course, we've recently had a sub-team of Interoperability led by Jack Lindsay going and investigating questions like this. The introspection paper they put out.
[0:12:33] This might seem unrelated, but I think it is kind of deeply related. Last week, Claude sort of leaked a draft of a new constitution Anthropic was working on for Claude.
[0:12:58] This is in the spirit of our constitutional AI work where you have some, you know, human sort of natural language document that specifies the model and that's used to train it. If you read the leaked approximate draft, which is not exactly right, but it's approximately right, you'll notice that there's a lot of discussion of Claude that's actually very internal, right? It's a sort of the internal nature of Claude in some way. And I think something that's really interesting about that is you could ask, you know,
[0:13:29] Is there some kind of parallel to that on the interoperability side? Can we ask about that on the interoperability side? I think that'd be very interesting if we could. I'll also say that I think these kinds of constitutions or OpenAI has a vaguely similar document called a model spec. I think these will be extremely important in the future. And so if you just haven't been tracking them, I think they're really, really interesting. Okay. The final thing I wanted to talk about
[0:13:57] is our community's role in public discourse. And this is a bit of a strange one. But I think that we might be moving in the direction of scientific fields like, say, scientists whose work has bearing on climate change, where there's really a very intense societal interest.
[0:14:22] In these results and where you as a researcher have a role in that discourse. And I think that's going to be a very strange thing for us. So society increasingly has a lot of questions about AI, right? You know, is it just cheating? Is it just memorizing? Does it understand? And really, I think what that's about is will this continue? Should I be scared? People also ask, is it safe? Can we trust it? Is it Skynet? Should it have rights? Is it a person?
[0:14:55] And candidly, the public discourse on these questions is atrocious. And you, each person in the audience, are among the people in the entire world most qualified to answer parts of these questions. Not the entirety, not the whole thing, but certain parts. And I think that means we are called on to provide basic education to public, to push back on sort of bad scientific takes, to provide more nuanced frames.
[0:15:33] Here are some claims where I would say you are the relevant experts. It's just memorization. It's just an X word predictor. It's just statistical pattern matching. I think that your expertise is very much at the core of what those claims are getting at. And they are relevant to these broader questions. I think there's a lot of value in providing more nuance. I think there's a way in which the lack of vocabulary for a middle
[0:16:03] One thing I've come to believe is important here is developing deep analogies, because the work we do is very technical and very difficult. And figuring out ways to talk about it that are accessible, but still
[0:16:29] Give kind of a scaffolding for having real conversations about at least aspects of it, I think is really important. So I've been leaning a lot into this sort of, you know, biological grown entity that we don't understand. We just created the scaffolding for and then have the sort of biology project to understand it because it allows you to have a conversation about at least parts of this. And I think also socializing sort of key foundational ideas is really powerful and important.
[0:17:02] You know, I think there's things we need to communicate a lot more. I think really, you know, I know everyone has been sort of communicating this, but I think continuing to communicate that we really don't understand these models. We're making progress. There's things we don't understand, but in deep ways, we still don't understand them. We've been saying that for a long time, but I think the world, in important ways, still doesn't really understand that. And I think some people think it's maybe a dodge that we're just sort of saying it because, you know, it sort of avoids responsibility for what might be going on inside of them.
[0:17:30] I think the truth is way more worrying. I think in the near future, we're going to start to see a lot more public discourse on AI safety and also on a topic of model welfare. And I think both of these topics are going to rely on sort of subclaims in their discourse that are really about interoperability. And so our community, and especially the unconflicted parts, the academics, those of us who aren't in industry labs.
[0:18:02] Which is not me, obviously. We'll need to evaluate those claims and sort of adjudicate them. I think that'll be really important and a really valuable service to the world. I also think that we should be cautious of what you might call the advocacy trap. Where I, at least personally, I have views on some of these questions. But I think it would be really easy to lose credibility by...
[0:18:32] You know, if you've sort of listened to climate change, I think sometimes, you know, wanting to advocate for a position can in the long run undermine scientists. And so I think that's worth reflecting on. And instead, I sort of aspire at least to be sort of an honest broker of the scientific knowledge we have. And, you know, to the extent that I do anything that's more in the line of advocacy or persuasion to really clearly distinguish that.
[0:19:00] From this other role I have of sort of wearing my scientist hat. So with all that said, thank you so much for your time and I'll be very pleased to answer any questions. Cool. Thank you so much, Chris, for an excellent and thoughtful talk. Maybe one question of mine before we open to the audience. There's a lot of promising junior researchers in this room. Are there any problems you didn't mention that you'd be excited to see people go and try to answer?
[0:19:29] Or areas you think are exciting and neglected? I mean, we publish some open problems. I don't know that I have anything that I particularly, though, want to point people at. And if I did, I'd probably want to go and sort of reflect more and write something rather than say something on the spot. You mentioned concerns about model welfare. Why do you think that's becoming more important at this point in time?
[0:20:02] Yeah, I think that models are becoming more capable and are exhibiting a lot more behaviors that are raising questions about this. You know, I think just sort of objectively, the public discourse is having, you know, more discussion about this. I think we're seeing more really serious philosophers of mind ask questions about this, people like David Chalmers. And so I think these issues are starting.
[0:20:31] to become more relevant. I also suspect, again, I'm sort of just trying to be descriptive here rather than prescriptive, that as people interact more with models, I think especially, you know, if they interact with them in forms that, you know, maybe with audio, for instance, I think that's going to create a very, I think it's going to be very natural, emotionally natural for people to ask questions about this.
[0:21:00] And so I would predict that this will be a bigger issue in the future. I guess I will also say maybe just briefly that I think it is not a trivial question. And I think that it's sort of tempting to go and sort of think about it as trivial in some ways. And I think it isn't. Hi, thank you for your presentation. So obviously in the AI safety community, the work in interp and steering is very prevalent already. But how do you view...
[0:21:27] Interp's role at the intersection of both capabilities and safety, right? It seems like understanding a model's behavior can allow us to kind of optimize its performance for some certain task and that will improve performance, but also obviously ensuring that there are relevant guardrails for safety. Yeah, I think that, like, for example, steering, I don't think is a particularly effective capabilities mechanism. So, you know, it's not to say that there might not be ways that interoperability could be used for.
[0:21:57] Yeah, you mentioned ambitious safety cases that come from interpretability. Can you say a bit more of what you have in mind of what interpretability could do for safety cases, especially for much more powerful models compared to what we have today?
[0:22:31] I will say before I say that is maybe I'll comment on how I think interoperability in practice is going to fit into safety cases. And then maybe I'll say sort of how we could be more ambitious. So I think that my guess at how interoperability will in practice fit into safety cases in the short term is sort of supplementing holes in sort of a more sort of what I would say like alignment science-y approach.
[0:22:59] So, for example, perhaps you do a bunch of evaluations where, you know, but for, you know, concerns about deception, you would find them persuasive that the model was safe. And then you use interpolation to go and spot check that. And I think that's a pretty interesting safety case. But, you know, the dream, I think, would be to go and say something, you know, that really gets at sort of the models off distribution or sort of possible range of behaviors. And I think the even stronger thing would be if you could say something that's sort of more like formal verification.
[0:23:28] It's obviously a very long ways away. But even sort of short of actual formal verification, right? If you think of software, why do we think that software does something? Well, we can understand the code and we can reason about it, right? And so I think if you can really go and get to an ambitious level of understanding a model, you might be able to go and make arguments that are more flowing from really understanding the system as a whole, rather than I think being more based in sort of...
[0:23:57] Empirics and stress testing and then trying to rule out certain failures with that. Let's also just get that sort of this unknown unknowns concern that I have, where I feel a lot of fear about sort of unknown unknowns causing us to miss things. Yeah, thanks. It was a great talk. I wanted to just kind of recap, I guess, you know, thinking back in the history of interpretability, we've seen, this is not a call out or anything, but like,
[0:24:30] Anthropic has had historically an extremely large influence on the strategies that people in the broader field of mechanistic interpretability kind of take on to solve different problems. And I also agree with your take now with the kind of like responses to the latest post that we're now seeing a more heterogeneous kind of like distribution of different agendas and disagreement and this is all good.
[0:24:55] Because now it's fragmenting out, but I just wanted to hear kind of thoughts from your end, being kind of one of the founders of the field, and just your perspective on kind of these kind of like very kind of point-wise strong influence of very few individuals who have a really big impact on the narrative and the flow of ideas and stuff like that in the field. It's not a very well-formed question, but yeah.
[0:25:25] Yeah, I mean, I think I've always said that it would be good to have a wide portfolio of research bets. And that's both within interoperability and also across safety approaches, right? Like Anthropic has pursued a lot of safety approaches, partly because, you know, interoperability might not succeed, right? And so we've always sort of thought of that as being a portfolio approach. And honestly, that goes back a very long ways.
[0:25:56] One can say what they think is true about research and what their vision for research is. It's kind of difficult to control how people will react to that, right? So I mostly communicate through publishing papers, and those papers say what I believe. There can be complicated dynamics around how the community responds to that, but I don't really know.
[0:26:25] How to manage that. It seems good for people to have a variety of views. Yeah, I think that would be my main take. Thank you for your remarks. As always, found them to be very thoughtful. Regarding the public discourse, one of the things I've seen in other areas is that perhaps the public at large misinterprets scientific neutrality for lack of an opinion.
[0:26:56] And I was wondering if that's a possible pitfall for sort of taking the higher road, more neutral stance. Yeah. So I think that might sort of then point more towards this kind of distinguishing your statements that are kind of more as a scientist providing objective information and your interpretation of those. So that seems like something that one might be able to do.
[0:27:23] I'm still developing my thoughts on this. I agree that there's potentially some kind of pitfall there. But I think on balance, I mean, another thing I will say is just that I think there's a lot of people who will be sort of debating these higher level issues. And there's a lot of value in having people who can sort of evaluate, credibly evaluate the subclaims this discourse is resting on. So those are some thoughts.
[0:27:52] Yeah, I don't know. I think I'm not, like, I don't mean by this to fully, you know, sort of, you know, fully commit myself to not going and making claims or statements that are sort of on these. But I don't know. I mean, there's also a point, right, where, especially on the safety side, there may just be, you know, very scientific claims that cut very directly at these issues, right? So you can just sort of, you know.
[0:28:23] Make actual, you know, very objective arguments about, you know, why, say, blackmailing happens or, you know, about particular safety failures occurring or having a potential to occur that I think are sort of more core scientific claims. I think the model welfare issue has kind of this indirect effect where sort of any sort of scientific claim then sort of has to feed into sort of a philosophical theory that I'm sort of more excited about going into sort of.
[0:28:53] Saying the scientific claims.