[0:00:00] Well, we're now ready to actually understand two-layer attention-only transformers. And I think there's actually some really cool stuff inside them. So I'm excited to chat about this. In particular, there's going to be some really interesting stuff that they're doing that kind of is meta-learning relevant and seems to generalize to much larger models. And so hopefully we'll learn some interesting and generalizable lessons. Now, if you watched our previous video on eigenvalue analysis,
[0:00:30] You'll remember that eigenvalues are kind of a nice way to summarize the behavior of attention heads without having to look at them very closely. And in particular, positive eigenvalues for an OV circuit, for the output value circuit, mean that an attention head wants to increase the probability that the output is the same. It'll increase the probability or increase the logits for that the output is the same as the token that it's attending to.
[0:00:58] So it really just means the attention head is implementing copying. And if you, you know, last time we saw that really the behavior of our two-layer transformers is mostly described by understanding the second-layer attention heads. And there's 12 of those. And of those 12, seven are doing copying. So it's a little bit less than when we were looking at the one-layer model, but still the majority of our attention heads are doing copying.
[0:01:27] And we're going to see that they're doing a much more interesting type of copying and a much more powerful type of copying than the one layer model was. Yeah, so we could try to go and study this by analyzing the QK circuits. The math would get a little bit complicated. So an alternative is that we can try to just empirically understand the attention patterns. And then we can go from there if we want and try to understand more about the math of the QK circuits. And something that we see a lot of
[0:02:01] is what we call the induction pattern. So, for instance, if we look here, and it might be a little bit hard to read, so let's focus on this D token. So that's the present token. And if we look back, it attends really strongly to ERS, which is the next token after D. And if then we go and when we look at when the present token is ERS,
[0:02:34] It tends to Lee. And this is a very general thing. So, you know, similarly, when the present token is pot. So this is all from the first paragraph of Harry Potter, by the way. I think probably maybe we've mentioned that in a previous video or maybe we haven't, but I've been having some fun using that as a standard piece of text to go and play around with. But you can also observe this on lots of other texts and it works very reliably. But yeah, so if the present token is pot.
[0:03:06] Well, we have the potters, so we attend to the next token, the token that follows pot previously, ters. Okay, and so we're starting to get a pretty good hypothesis maybe of what's going on here, which is that somehow this attention head always attends to the token that follows previous copies of the present token. So if we're on pot, the previous copy was pot, and then we get ters. Okay, so how could it be? Well, that's kind of interesting. Now, a natural hypothesis is...
[0:03:36] The reason it does that is that it would allow it to go and predict then what the next token is because the goal of the present token here is to predict the next token. And we can actually just measure that. We can just take the output of the attention head and it just has a linear path down to the logits. And so we can just run that into the logit. And even without analyzing the terms, we can just see the empirical effect on the logits. And it's indeed...
[0:04:04] Going and predicting errs, going and predicting li, going and predicting a little bit the pot on potters. So we have the pot, the pot, but then the terrs and potters really strongly. And so it's increasing the probability of all of those. And in each of those cases, it was attending to that token. So it's copying the errs to the errs. And then we have this hypothesis that maybe somehow the way it's doing that is because the previous copy of D.
[0:04:33] Or the present token is D and that was the token that proceeded where we attended. So we'll call that the induction pattern. And we can actually verify our hypothesis that it's actually looking for the fact that the fact that the previous token is the same is what it's looking for. And if we just look at how the key for this, so let's pick a particular case. So where we're on
[0:05:03] Well, it turns out that if you look at how the key is computed, the key isn't computed from the present token, but it's actually computed from the previous token. So we shift the, we grab the content of the preceding token and inject that into some subspace and then use it to construct our key. Whereas we construct our query mostly from the present token. And so by going and looking for then looking, looking to go and match the present token with a subspace that we stored this in.
[0:05:33] We're able to go and look for tokens that follow a token that matches the present token. So it's a little bit more of a complex algorithm than anything we've seen. It's almost a little bit like nearest neighbors in that you're almost treating your text up to this point as being a little bit like a training set and you're looking for places where there's a match and then you're going and copying that or something. Now, it does other interesting things too.
[0:06:03] Sometimes it's just the present token that matters, but there's also cases where, like here we have this D, and if you look at this, I don't have it here, but if you interactively play around with this, you'll see that it really is that the D of this query matches the D of this key, and the Mr of this query matches the Mr, and to a lesser extent, the Mrs portion of the key over there. And so they match, and then that causes us to attend to errs.
[0:06:30] Which then increases the probability of error. So it really is this copying thing. And we're calling that general mechanism of looking for the token that in the past has preceded your present token. Looking for situations that were analogous to your present situation. And then predicting that whatever happened next in the past is what's going to happen next now. That's what we call induction or an induction head. And that seems to really be kind of the workhorse.
[0:07:00] of meta-learning in a lot of transformers. We see it in large models and it seems to be playing a big role. And it's not just one of them, one head like this, we actually see a lot of heads that are doing slightly different flavors of this. Some are sort of very strict versions that are really looking for exact matches and copying, whereas others are a little bit more flexible, like here we have
[0:07:30] Dursley, probably Mr. Dursley was, he, and then we copy was. Or you'd, and then didn't. But probably if you're using apostrophes, that's in contractions, you're likely to do that again in the future. So yeah, we see a lot of this kind of induction. Now, yeah, is there some way that we, we have this really handy trick for detecting heads that do copying by just looking at the...
[0:08:04] The OV icon values. And we mentioned previously when we talked about their video on icon value analysis that having positive QK icon values would mean that we were attending to, that we were trying to attend tokens that are similar. And so we can apply that same trick. And here our QK circuit is a little bit more complex. It's rather than looking for a positive icon value, meaning that we're attending literally to the token that's the same as our present token. It means that we're attending to a token.
[0:08:34] Where wherever the key is being constructed from is the same as our present token, or really the same as wherever our query is being constructed from. But that's still if the keys, if the QKE eigenvalues are positive, that's a very strong sign that we're doing something like induction. And the OVE eigenvalues mean that we're doing copying. And so if we see if both are positive, that's a sort of very strong signal that we're seeing induction. And we can see that there's
[0:09:03] Six heads that are quite clearly doing this, and then another one that is maybe doing this a little bit, but it's a little bit messier. Now, that leaves a number of other heads that aren't going and doing copying and aren't induction heads. One of the reasons I'm focusing on induction heads so much is because they seem to be genuinely really generalizable to larger models of the transformers generally and seem to be
[0:09:33] Sort of a very important story about how transformers work. I don't know that I have any really interesting generalizable takeaway about these other heads. Not based on the study that I've done on this model so far. A lot of them seem to be doing sort of like slightly engram-ish stuff. And yeah, so I'm not going to focus as much on them. Oh, one final thing that's worth mentioning is just like we were able to go and do a histogram in the video that we did on eigenvalues to go and sort of quickly...
[0:10:05] Quickly summarize the heads in a really dense way. We can put the sum of the QK icon values divided by the sum of their absolute values on one axis. And the same thing for the OV values on another axis. And then attention heads that have both very, very positive on both of those. So tend to really copy and tend to prefer to look at places where a token is the same. They'll be up in that corner and other tokens won't be. And so that's a...
[0:10:34] An even denser way to go and really systematically in a really automated way pin down the induction heads. Okay, so a couple of takeaways. These induction heads seem to be really a major mechanism for meta-learning. And they seem to be a lot more effective than simple copying heads. And we can automatically detect them, which is pretty cool. I mentioned this earlier, but the other attention heads
[0:11:09] They tend to be doing kind of localish stuff, and it's not clear that we have a really generalizable story, so we're not going to pay too much attention to them. Well, in any case, I think it's pretty exciting that we can really mechanistically understand some of how these two-layer models work, and with enough effort, if we wanted to, we could probably really understand them. And a lot of the ideas that we've built up here, the general sort of form of this equation of thinking about
[0:11:39] Thinking about attention-only models and this idea of virtual attention heads are actually, I think, kind of generalizable and will be useful as we start thinking about larger and more complex models.