[0:34:33] Great, welcome everyone to the NeurIPS panel on the consequences of massive scaling. I'm Jacob, and I'll be the moderator. So to set the context for this, machine learning as a field has always prized algorithmic contributions, but many recent breakthroughs have been driven by just scaling up the same basic algorithms and architectures. And in some cases, this scaling can even lead to qualitatively new emergent behavior.
[0:35:01] As was the case with GPT-3 and few-shot learning capabilities. So our panel today is going to discuss how this new trend should affect the practice and evaluation of research, whether it's a paradigm shift or a dead end, and the safety and social concerns raised by scaling. I'll be the moderator, and we'll be joined by five excellent panelists spanning industry and academia and bringing perspectives from machine learning, cognitive science, complex systems, and physics.
[0:35:27] Our panelists are Joelle Pinot from McGill University and Facebook, Jared Kaplan from Anthropic and John Hopkins, Noah Goodman from Stanford University, Melanie Mitchell from the Santa Fe Institute, and Oriol Vinyals from DeepMind. To keep things moving, I've asked panelists to keep their responses to two minutes or less. And we'll start with introductions. So I'll ask each of you to say a sentence or two about what you hope to get out of the panel today. Let's start with Jared.
[0:35:56] Okay. Well, this is my first panel, so I'm not entirely sure, but I'm really interested in hearing everyone's perspective on this topic. It's something I've thought about a bit, and I think I have kind of a unique perspective on it coming from theoretical physics where we have experiments like the Large Hadron Collider that are very large and very collaborative. Great. Thanks, Jared. Let's do Melanie next. Hi.
[0:36:22] I'm really interested in this issue because I'm very interested in the question of how do we assess AI systems for actually understanding the data that they're dealing with. That's kind of an ill-formed notion of we don't really have a great definition of understanding, but I think it's really important because it comes up over and over again when discussing sort of the robustness and predictability and trustworthiness of our models.
[0:36:51] The scaling issue is interesting from many perspectives because it really, in some sense, shows kind of where machine learning engineering departs from machine learning science, where the science of machine learning is concerned, hopefully, with...
[0:37:11] Understanding how the mechanisms by which these systems do what they do, whereas machine and learning engineering might be more concerned with getting these systems to actually perform tasks in more accurate and efficient ways. So I think hopefully we'll talk about that divergence in using these very large models for achieving tasks. Cool. Thank you, Melanie. Let's do Noah next.
[0:37:43] Yeah, I guess I'm excited to kind of think with everybody about the sociological issues in science that surround big models and kind of how we make space for new and unpopular ideas while also kind of exploring the phenomenon that we're discovering with big models and, you know, of course, letting industry get the most out of the new capabilities. Cool. Thank you, Noah. Let's do Joelle next.
[0:38:14] Hi, it's great to be here today. I'm really interested in this topic in large part because I have a deep interest in reproducibility and in open science. And I find with these very large models, it becomes very difficult to ensure the same norms of reproducibility as we have for other of our results in the field.
[0:38:35] It also makes it very difficult in some cases for a large number of people to access the results or even access the models. And so I think there's a lot of work that we can do as a community to take a strong point of view on how we incorporate this type of research and these types of research artifacts into our scientific landscape. Definitely agree with that. OK, and then finally, Oriol.
[0:39:02] Yeah, thanks. Thanks, Jacob. It's great to be here. I guess what I'm looking forward to the most maybe is to meet and learn maybe in the new spirit, right? This is a virtual conference, but we get to connect with folks that are in our field, but they're not necessarily in our institution. So this way to...
[0:39:25] Kind of connect with a group of people that I usually don't talk to and be challenged and hopefully challenge them. That's really what I'm looking forward to generally when I join panels. So looking forward to that. Yeah, that's great. I think we'll get lots of that. And it sounds like we have a lot of common interests around scientific norms, understanding what these models are doing and learning from each other. So I want to move on to research practices, which actually
[0:39:54] Many of you have already mentioned. So NeurIPS has traditionally valued methodological contributions, but it's now evident that scaling a known method can also produce insights. So with that in mind, I guess two related questions. One is, when does scaling actually constitute a valid research contribution? And then relatedly, when thinking in terms of scaling, are there new problems that become interesting and any that become uninteresting?
[0:40:21] I'll address this to all five panelists, but maybe we can start with Noah. Start with me. Gosh, okay. You can pass if you want. No, I mean, I think it's a pretty important question. I guess with respect to, you know, is scaling alone, you know, a story that we should be interested in in our research contributions? You know, I guess I feel like...
[0:40:52] We should really think about what counts as a scientific contribution, where certainly one kind of thing is a new method that allows us to do something new. And that's kind of traditionally what we've looked at in AI and machine learning. But in the rest of the sciences, kind of identifying a phenomenon and exploring the nature and boundary of that phenomenon is considered a main kind of contribution.
[0:41:22] With respect to scaling, there's a kind of contribution where there's a new phenomenon that happens as a result of scaling, and that should be absolutely valid. I really like the deep double dip or double descent kind of line of research for kind of like identifying interesting phenomenon as a result of scaling and then really kind of across now many authors and papers diving into that. I think, you know,
[0:41:51] What that does well, we can put in contrast to a different kind of scaling result. So what that does is it really analyzes what are the phenomenon and when do they occur. In contrast, if all you do is you say, you know, I ran it on 23 instead of 12 computers and I achieved a better score on something, that really isn't a kind of scientific analysis. And so I think we should, as a community, get a little bit better at distinguishing kind of, you know, proper scientific.
[0:42:20] Results with proper controls from just the result of some more runs. And that's really what it's going to take to make sure that we're doing science and not just reporting the results of our bigger hammer that we hit something with. I guess the other half of your question was, do I expect that there will be some things which become interesting and other things that become uninteresting?
[0:42:47] I guess the answer to that has to be an unqualified yes, independent of whether we are talking about massive scaling or anything else in science, because that's just what happens, right? So I think there are things that we used to think were really hard, like part of speech tagging or something, that no longer seem hard and therefore don't seem as interesting to work on, although maybe that's a bad example.
[0:43:10] And, you know, other things that we didn't think we could do, like a bunch of stuff in providing feedback in education that I've been really excited about in the last year, that like all of a sudden is a thing we can work on. And so that becomes interesting. Cool. Thanks, Noah. Yeah, I definitely agree with the distinction that you're drawing. I guess I'll just let in if anyone wants to jump in with reactions and also feel free to raise your hand if you want to get queued up as well. Maybe we could.
[0:43:47] I could go on to Jared if you have thoughts, especially since you were one of the people who worked on GPT-3. So you have a lot of firsthand experience with some of these scaling results. Sure. Yeah. So I think I agree with a lot of what Noah said. I think that if you're able to train larger models, I mean, I do think it's an engineering contribution potentially to be able to do that.
[0:44:16] Beyond that question, I think often there are new phenomena that are revealed. And I think that deeply investigating those phenomena and understanding them seems important. And I think, as others have hinted at, I think understanding which problems that we thought required a new idea actually are seemingly resolved by larger scale seems very interesting to understand. So I think that
[0:44:45] Trying to differentiate between those two things is useful. I also think that a lot of methods, at least I think it's common to see methods that seem to improve performance or unlock some capability at small scale. But those methods might not be as important as models scale up. Maybe the differentiation you get from a new method diminishes with scale. I think it's interesting to understand which methods
[0:45:11] Get better and better as you scale up versus which methods kind of become less relevant. Because regardless of what you think about scale, presumably in 10 years, in 20 years, larger scale models will be the norm. And so I think that's an interesting question. Great. Thanks, Jared. I see a few hands. So I think Oriol was first. Sure. I can go next. I guess the first question, right?
[0:45:39] What constitutes a valid research contribution? I mean, that is a very complicated question because our field is evolving. And I remember, for instance, my very first NeurIPS paper had a theorem because papers had to have a theorem and a proof at the time. So at the time, this answer would have been quite different. To me, more in general, rather
[0:46:07] I want to learn something, gain some understanding, and generally be surprised about research that I read on a paper. There are lots of papers, so that's generally what I tend to value. And so any result, you know, regarding scale that provides any of the three, or ideally the three I find, you know, they're worthwhile research contributions. And as a reviewer, area chair, et cetera, you know, obviously I'll try to highlight those, right? Then in terms of...
[0:46:36] Problems that become interesting and uninteresting. I mean, interesting, there are lots of them. I'm curious to hear, Melanie, on the point or the distinction between engineering and research, because I think a lot of interesting questions, maybe scientifically about hardware.
[0:46:54] might become relevant thanks to scale. And in fact, our tools in machine learning have evolved quite a bit as well. Thankfully, actually, reproducibility being much easier thanks to the software and the hardware that we have much wider at our disposal these days. And then to name maybe a couple of areas, I think I like sparsity or better optimization since we optimized these methods for a long time are clearly cool research directions that are
[0:47:22] Ever more important at scale than maybe without scale. And then maybe just to answer fully, an interesting one. So maybe it might seem like generalization, which is a very important topic in machine learning, especially on the scientific side of things, to understand how generalization works.
[0:47:42] It might become less relevant if you are in the infinite data regime as we might be perhaps if we consider all the data ever recorded, given that we cannot train possibly on all of it. That being said, we also see that these large models are then further used downstream in applications that actually might lack training data. So maybe there are new interesting generalization questions to be asked about what happens when we fine tune these models or we do few-shot fine tuning, et cetera.
[0:48:10] So these are a few of my reflections. But great question to start the panel. Thanks, Ariel. Yeah, let's actually move on to Melody next, because I saw that you had something you wanted to say too. Yes. Yeah, I think that this question of science versus engineering is really interesting, especially in machine learning, because it's one of the fields where it's hardest to kind of
[0:48:40] Separate those two. And often people in machine learning conferences and so on will give results like scaling up to this new numbers of parameters in our system yielded this new performance on this benchmark. But it's really hard to understand exactly why. And it doesn't really seem to be...
[0:49:09] It's essential to look into mechanisms of that rather than just reporting it. I hope that people will spend more time looking into these mechanisms of why scaling has these effects. Scaling phenomena have been fascinating. I've been particularly interested in these power law.
[0:49:31] Scaling phenomenon in machine learning. And that's something that intersects with a lot of work in complex systems on power loss scaling in a lot of complex networks, for example. And I'd love to see if there's relationships. There's also these very mysterious scaling phenomena where, for instance, GPT-3 seems at some level of...
[0:49:54] And so the source code isn't available. And, you know, most...
[0:50:21] People in academia don't have the compute resources to do the kind of science that's needed. So I think that's something that we're going to talk about later. And I just wanted to make one more point, which is that it's funny because the usual process in science, especially physics, is to not scale up.
[0:50:40] A problem when you're investigating it, but scale it down to make it as simple as possible and to idealize it so that it can be studied. So a big question here is how do we do that to make a more scientific study of machine learning? Thank you, Melanie. Yeah, I really like the examples you gave of kind of new phenomena. And I'll ask for some more examples of that.
[0:51:07] Later, since that's something that a lot of people have brought up. Before doing that, though, I want to finally move on to Joelle. Thanks, Jacob. I wanted to add a little bit of a historical perspective on this, actually, because I think it's sometimes interesting to look a little bit back about where we came from. Some of us have been in the field for a few decades, and maybe some of us listening today have at least studied the fast, fast evolution.
[0:51:34] But we've had several of these steps in scaling since the last three or four decades of machine learning. In a sense, we went from having a few hundred training examples to having several thousand training examples to having millions of training examples. And every time we sort of stepped up significantly, in this case, the size of the data, but of course, the compute has to go with it.
[0:51:59] This led to a flourishing of new ideas, whether it's algorithms, whether it's theoretical understanding, whether it's really opening it up in terms of application domains also. So I think this notion of scaling has been a meaningful driver for progress and for innovation in our field. The one thing to keep in mind, though, is that as you progress along this, the size of the step necessarily has to get bigger.
[0:52:27] And so it's not going to be a linear scale. And so I think we're now at a size where when you want to get to really the next size of scaling where you're going to be really different in terms of the underlying mathematics of the models that we're building in terms of the applications that we're trying to unblock and so on, it's going to be a really big step. So I think that's something to keep in mind as we dig into this question.
[0:52:56] The other thing I'll add is as much as I do believe that scaling up to the point of some of my colleagues earlier can really lead to new knowledge, which is, of course, a valid research contributions and deeply exciting, it doesn't mean that all scaling leads to that. And so we have to be a little bit discerning of what's the question we're asking, what's the knowledge we're gaining. And just by scaling is not necessarily what we want in terms of research contributions.
[0:53:24] Yeah, that's great perspective. And I think lots of you have kind of drawn the distinction between, okay, just scaling up doesn't necessarily always provide insight, but there's some work, maybe work that identifies new phenomena or that does something else that is insightful. So for the benefit of our audience, especially, I think it's helpful to have some examples of that. Melanie gave some great ones earlier.
[0:53:52] Noah, I know you mentioned you want papers that identify new phenomena, so I'm wondering if you have examples in mind that you particularly like. Yeah, I mean, I already mentioned the double descent, triple descent line of things, which I haven't been involved with, but I like as an example of doing science on scaling. The other thing that kind of comes to mind that's sort of blown me away in the last few years is the flourishing of research on unsupervised learning or self-supervised learning.
[0:54:24] You know, I think the observation that in some sense, all of a sudden we could do useful and supervised learning by doing things like, you know, InfoNCE and these other contrastive learning or self-supervised learning methods, and that, you know, somehow the basic ideas there didn't feel new, but, well, obviously there were many specific techniques that were added on that were new, but the basic ideas seemed like,
[0:54:53] Things that we had tried before, but it started working in a way once we kind of did it with a certain scale. And then I think there's just been an amazing amount of research extending and exploring, and there's still a lot to do. I think we actually don't understand yet why self-supervised learning works and what's the deal with negative examples or not having negative examples and things like that.
[0:55:18] So that's another domain where I think there are phenomenon and people are exploring them. And I should say, critically using mathematical tools like mutual information and these kinds of things to analyze what's going on and what's required for representations to emerge. Thanks, Noah. Yeah, those are great examples. So I want to move next on to a question that I'll first address to Melanie and Jared, but other people should.
[0:55:47] Feel free to jump in as well. And I guess this also kind of relates to phenomena. So GPT-3 demonstrated emergent behavior in the form of few-shot capabilities which arose from scaling up. Since you both kind of have physics and other science backgrounds that kind of deal with emergence, I wanted to get your perspective on what other emergent properties we should expect at scale.
[0:56:14] And also what safety or other implications emergence might lead to. So let's start with Melanie. Yeah, that's a hard question to answer, what we should expect. Because part of what I think of as emergent behavior are ones that are fairly unexpected. So I did some investigation of how well GPT-3 could make analogies.
[0:56:46] And it was really surprising that it could do really well, except in some cases it would fail. And it was really hard to predict exactly when and why. So I'd like to know that. I'd like to understand it a little bit better, but it's hard to probe it. So one of the things that...
[0:57:12] I've been a little bit mystified as this notion of prompt engineering, which I think of is also related to scale. So the idea there is, you know, you have to carefully figure out what prompt to give the system and maybe to get the behavior you want. And maybe scaling, you know, I'd like to see an investigation of that with scaling, sort of how does...
[0:57:37] The requirement for super careful prompt engineering change as the system scales. So that might be some emergent phenomenon. It's also the kinds of ways that we measure performance in these systems I think needs to be expanded.
[0:58:00] As someone mentioned earlier, we use these systems as pre-trained models that can then be fine-tuned for specific tasks on specific benchmarks. And it's often been shown these systems will way outperform humans on certain benchmarks, but will fail miserably.
[0:58:22] When the benchmarks are changed a little bit and become more out of distribution, they don't generalize well. So I think the ways that we probe these systems and the ways that we measure their performance has to be... That's a big issue in this entire field that we have to spend more time on. Thanks. Yeah, I definitely agree that emergence definitely makes science harder.
[0:58:50] That's one of the conundrums. So Jared, do you have ideas of a way out? I don't know if I can solve any problems. I guess one comment that's sort of both about emergence and sort of new capabilities is that I may be kind of a naive observer, but it seems to me like there's been a kind of unification where historically people might be working on image processing or language and they would view themselves as doing something very, very different.
[0:59:20] And I think at least part of the reason why there's a convergence there where maybe one kind of model can address all of these different data distributions is scale. And I guess I also think of this as something where it's maybe simplified the picture of what's going on from a situation where there are a lot of different details you have to know to work in a very specific area to a few different techniques that seem to apply very widely. In terms of emergence, I think
[0:59:49] I guess my very zeroth order picture is that reasoning is the kind of a capability that seems like it's most missing from current large scale models, language models. And so I think a big question is mathematics seems to emerge suddenly in this kind of a grok like way during training or as a function of model size. So I think there's a very kind of interesting and also perhaps scary from the point of view of safety.
[1:00:18] Question of whether or not reasoning capabilities more generally might suddenly improve with scale or whether they need some kind of specialized technique. I think self-supervised learning also seems like a potential example there where you might imagine that beneath a certain level of capability, sort of the model is just sort of.
[1:00:37] Not teaching itself anything. It's kind of garbage in, garbage out. But you might imagine that there's some kind of, I don't know, like negative feedback loop where above some scale models can get a lot better. So I think those two things seem like simple examples. In terms of safety, I think it, I mean, I'm someone who is very concerned about kind of alignment and robustness for these models. So I think it means that we kind of need to go slowly and try to
[1:01:04] Build the best scientific understanding of what's actually going on as possible ahead of time. Cool. So a follow-up question for everyone. We've, I guess, discussed two examples so far of emergent behavior, one with few-shot learning and GPT-3. And I think Melanie and Jared also mentioned this kind of emergent mathematical performance.
[1:01:27] So are these the main examples that we've seen so far? Or are there other examples of emergence that we should keep in mind when building intuition? I just want to put in a very short plug that we should remember cognitive science when answering that question. Deep networks are not the only systems that exhibit emergent intelligence in interesting ways. And there's kind of a long, especially analytic and philosophical tradition of figuring out, how do we understand the brain in terms of the mind? And I think
[1:02:04] You know, pleasantly, we get to think about that now for AI, too. Melanie, it looked like you had something to say, too. Yeah, I mean, I think just to follow on to Noah's comment, you know, one of the things that people, I think, used to think that scaling up the brain, like between, say, certain organisms like insects to humans, that it was like the number of neurons that really mattered. But then I think that's been shown that
[1:02:35] It's not really the focus. That's not what's relevant in scaling. In scaling, it's something else that we don't really understand. And so it would be interesting to pair the study of scaling in machine learning with the study of scaling in natural intelligence. And I don't know if those two fields, I know people do research in both of those fields, but I don't know if they've actually come together for any discussion.
[1:03:06] Cool. Thanks, everyone. So we'll move on to the next question. This will initially be addressed to Oriol and Joelle. So even at smaller scales, researchers are often pressured to demonstrate their ideas on larger scale experiments. And both of you have experience both doing smaller scale and very large scale machine learning. So I'd be interested in your perspectives on what are the cases
[1:03:32] Where this practice is necessary versus superfluous. Let's start with Oriol. Yeah, great question. Sorry, I'm still thinking about the previous question. I might have something to add, but maybe let's do at the end a bit of miscellaneous discussion. So I think that this comes up quite often indeed, that you submit a paper and one of the feedback is, well, what about how does it work at scale? But maybe...
[1:04:02] The one that maybe is most useful is as a field, we have done a reasonably good job at standardizing data sets and benchmarks so we can talk to one another and understand sort of what it means to achieve certain level of dexterity at classifying images, et cetera. So I think that to me feels like that comes tight with scale usually because the data sets that maybe the community now is accepting as the de facto are.
[1:04:31] And as a result, to achieve maybe results that are competitive enough so that you would get an interesting insight might require some skill, right? So maybe the question is, are there interesting research data sets that don't require skill? And I think absolutely, I love these ideas of
[1:04:53] Even if it requires some amount of pre-training or downloading some set of weights, but then fine-tuning maybe at the small scale, I think that can be a very fruitful line of research. But I think, to me, the important bit is, you know, MNIST, I think, played an important role and maybe it still has some role to play.
[1:05:15] Definitely, if you're working on computer vision, ideally, you would want to demonstrate things on the very least ImageNet. And in fact, deep learning was only taken seriously when it demonstrated, actually, it had the same issue, right? When it demonstrated results beyond CIFAR and other smaller data sets. So it's an interesting, I guess, turn of events. But I would say there is an advantage to some extent, or at least being resource constrained.
[1:05:44] Creates novelty as well. And I mean, there are probably many examples, but one in which I was directly involved and keeps coming back as a very important one, actually, given how the field has evolved, is at the time we were working on machine translation. We developed sequence to sequence model, which we were able to train on eight GPUs, which at the time, I mean, it was kind of a lot, I guess. And then the group at Montreal had a much smaller
[1:06:11] Capacity or GPU budget. But as a result, they invented a new mechanism beyond LSTMs called Attention, which obviously has been a very profound transformation thanks to transformers, no pun intended, and beyond.
[1:06:27] You know, the fact that they couldn't train possibly for as long as we could. I mean, we trained for maybe 30 days on ATPUs, something like that, right? That created, you know, very cool insights for not scaling up, so to speak. So I think, you know, that should keep happening. And I keep seeing this over and over. So that I'm looking forward to keep seeing, regardless of whether you're interested in scale or actually you cannot or don't want to scale. I think there is a lot to learn from everyone here. Cool.
[1:06:55] Thank you. And Joelle, what are your thoughts on this? Yeah, I want to pick up on Ariel's last point. I think it's a really interesting one, not because it's a Montreal story, but specifically because I think a little bit of friction can be great for the creative juices in research. And in an era where you have no constraint on data, no constraint on compute, sometimes you don't really use all of those creative ideas to really
[1:07:26] Think of other paths to solving a problem. So I think we do need to keep that top of mind when we do research, that friction of different kinds, whether it's constraints brought in by the real world or by resources, can be actually really fruitful. It's not always obvious at the beginning. The other part I would love to take on in this particular question is where I've seen some problems and pressure to scale.
[1:07:54] is actually when I've tried to do projects taking on machine learning applications, for example, in healthcare, where typically the data sets are quite small, very heterogeneous. And in that case, I've often been in situations where collaborators come in, you know, having done their review of the literature, they're very excited about all these large scale models. And you know, they want to do reconstruction and data imputation using the latest and the greatest in self supervised learning, and they want to do prediction using some large scale models.
[1:08:23] And there's really a gap between their expectation of what the models can do at scale versus the scale of the data that they have. And there's a lot of work that we have to do in terms of just educating our partners and really deriving the right models for the application cases. And the number of times I've had to sort of bring them down from their grand vision to a simple linear model, usually at least as a baseline to get us warmed up is quite often.
[1:08:53] I think we just have a responsibility to take that on. As experts in this field, we have a responsibility to characterize carefully the cases under which the scaling pays off. And I find too often we don't do that. We're not really incentivized to do that. I love the positivism and the energy in the field, but I think sometimes we have to do a lot of cleanup afterwards in terms of what real problems we want to solve.
[1:09:22] Cool. Thanks to both of you. So you mentioned friction is good for the creative juices. So I'm going to use my next question to hopefully create some conflict on the panel. So the next question is, is scaling a paradigm shift or a dead end? So what are the limits of scaling? How far can it take us? And where can it not take us? And I'll let anyone who wants to jump in to that.
[1:09:51] To that question. Melanie? Yeah, I think that's always been the question in AI, you know, from the very beginning. It's been an important question. And, you know, we've seen so many surprising effects of scaling, even pre-deep learning, like in speech recognition. You never thought that you could just do so well without, with just kind of brute force, huge probabilistic models.
[1:10:30] But I guess the question is, what do you want to accomplish with scaling? And I think that one thing that we've certainly seen, and I think someone brought this up with respect to reasoning, that scaling, at least so far in language models, hasn't produced great abilities to reason or great abilities for this kind of umbrella concept of common sense.
[1:10:59] This is a big question to me. Can you accomplish these things just by, say, training on language, by having your system ingest the whole web or Wikipedia and so on? My intuition is no, but it's hard to prove that. The question is, can you produce a human-like model of the world with...
[1:11:26] Just language. And I think that is the question that people should be addressing. And I think it's a very profound and deep question. I don't know the answer. But I don't think it's... I would guess no. Okay. It looks like a lot of people want to jump into this one. So I think Noah had his hand up next. And after that, we'll go to Joelle and then Oriol. So I guess, I mean, it's a good inflammatory question.
[1:11:59] The way you phrased it in terms of paradigm shifts, I think, is really a good way to think about it. If I try to analyze what's going on in Kuhnian terms, I think this is absolutely not a paradigm shift. What we're seeing is science as usual that's having some unexpected results. We're not actually seeing a profound transition in the scientific terminology and the culture that we use to investigate questions.
[1:12:29] From that point of view, this is great, but it's not a paradigm shift. And optimistically, it will be a dead end. I say optimistically because what we want is that these are good ideas that will become solid enough that we don't have to work on them anymore and we can move on to more radical paradigm shifting kind of change of scientific questions. I'd love to see a future where there just exist foundation models that we pull off the shelf and use when we need really rich features.
[1:12:56] But then we instead investigate questions like Melanie was raising about reasoning and such. I think you gave us the quote of the day there, Noah. OK, let's go on to Joelle. I'll try to keep it brief. I tend to agree with Noah. I don't think we're seeing a paradigm shift in terms of scaling. But I do think we still have a lot of mileage in scaling, quite honestly.
[1:13:26] And in particular, when it comes to scaling across modalities. And so we've seen a lot of work on quite narrow modalities. We've talked about language models. We could have talked about computer vision models. But really, taking data across all of these modalities, I think there's a lot more that can be done. Taking into account interaction, a lot of the work we're seeing on scaling is quite passive. It's not interactive. It's not dynamic systems. And so in that sense, I feel
[1:13:53] I'm not going to take a strong stance on whether it's a dead end or not, but I don't think we've really explored nearly as much as we can on that thread. Cool. Let's go to Oriol next, and then Melanie. Yeah, I agree in general that this is not a paradigm shift. As an active observer, I must say, and you can go and read papers that led to the current ones, there is not, I would say, that...
[1:14:21] Many surprises, at least in hindsight, when you look at how language models have been used.
[1:14:29] Chad bought research going for him for a while. I think the difference is that it's just so much better now that, of course, then new questions arise. But even Shannon had that Ngram language model that he was sampling from. So I think it's just the performance indeed is maybe scaling as well. And that creates new questions. But I don't think it's necessarily a paradigm shift.
[1:14:56] In terms of, is it at that end? I don't believe so. Look, I think, I look at kind of existential proof. We have intelligence in this planet, we think, and clearly we had, like evolution has done a lot of iterations, a lot of SGD steps or steps, if you think about years spent energy, probably parallelism through different planets, et cetera.
[1:15:23] I think scaling up the learning towards more intelligent systems with the data that we collect, I think is quite necessary. And in fact, as in humanity, we see that all the time because we write down things and that way we also scale ourselves. But I think scaling will be part of the solution. But yeah, will it be the only element? I don't believe that. Cool. Let's go back to Melanie.
[1:15:54] Yeah, just one more comment. We recently had a visit here at Santa Fe Institute from Mustafa Selim from DeepMind. And we were talking about whether systems like GPT-3 could actually understand in some sense. And he said something that really struck me, which was that what scaling is showing us is that understanding will become irrelevant.
[1:16:24] That it's not going to be a relevant concept because you can get intelligence without understanding. And so that's, to me, quite a paradigm shift, which I don't agree with necessarily. But I think it's an interesting point that I think is something that people didn't used to believe. Well, it sounds like so far everyone agrees with each other.
[1:16:55] I have a quick question for Melanie about that. That was fascinating, and it makes me wonder how much of human intelligence, the things that we think we've been shooting for, will turn out to be a spandrel, right? Is it just this understanding? Is it all of qualia? What is it? I'll bet everybody has ideas about this. Do you want to follow up on that briefly, Melanie? No, I think it's a great question.
[1:17:23] It's possible that that understanding will be irrelevant and consciousness will be irrelevant, but we have to make sure that we understand what we mean when we're talking about those things. Okay, and it sounded like you wanted to weigh in, Jared. Sure, yeah. I mean, I think that to me, this is most interesting in terms of the next question, which maybe I'm anticipating of sort of who has access to large scale.
[1:17:57] Because, I mean, it's been very strange to me working in AI where a model that's, say, a thousand times less expensive than the Hubble Space Telescope or the Large Hadron Collider is viewed as unbelievably expensive and inaccessible to researchers. And I think that the reason why academics in physics have access to such large-scale resources is because they sort of agree.
[1:18:23] They talk about it a lot. They have conferences and they agree that to sort of make further progress, they need access to those resources and they're able to get them. And it seems like given how impactful AI is and its apparent promise after like a decade of tremendous progress, I would expect that academic researchers and AI could have access to those same levels of resources that would compete with or exceed those of industry if they were in agreement on this question.
[1:18:52] If most academics disagree and think that scale is not that important, it's something to be avoided, we hope it's at that end, then it seems like that's unlikely to happen. And I do sort of perceive that the culture and the way that people talk about these ideas is very influenced by the fact that industry has access to all of these resources. It seems like large-scale models are viewed as this kind of tool of revenue for big companies.
[1:19:19] Whereas something like the Hubble Space Telescope is viewed as this scientific instrument that tells us something profound about the universe. And I do think that intelligence, whether or not it's conscious or not, whether or not it understands or not, is a pretty profound and interesting thing about the universe. And so I think it's kind of a shame if that isn't one of the major themes of thinking about AI, whether it's large scale or not.
[1:19:45] So what steps do you think academia should be taking if they agree with you that scaling is fundamental? I mean, I think that I guess there's two comments. One is that it does. I think it is important that there's kind of agreement. I think at least a lot of people have to believe that scale is important and then prioritize it over things like.
[1:20:09] Perhaps day-to-day concerns like, can I get this paper finished in time for a conference, et cetera. It has to be something that some people prioritize. I think there has to be agreement. And then I also think that this possibly delivers something that solves some of the problems associated with the dual-use nature of these large models. Sometimes when I say, oh, it seems like the LHC
[1:20:36] is a good institution. And something like that should exist for AI. People say, oh, well, that's scary if you're just giving away increasingly powerful models that are unsafe. But even as a theoretical physicist working on LHC physics, I don't have access to the LHC machine. I don't even have access to the data. So I think that there's some middle ground between everyone in the world having access to the most powerful model in the world as soon as it's trained.
[1:21:00] And only one person or a few people at a large corporation. I think there can be thousands of scientists part of a collaboration examining all aspects of these models and then publishing results on them. So I think that's kind of a paradigm that I think would be interesting if people want to investigate these large-scale models in a kind of academic, rigorous way. Joel, it looked like you had your hand up. Yeah, I think that's a really interesting point.
[1:21:29] That's Brad here. I think there's a couple of things that might make it a bit harder for AI right now to get that level of cohesion. I think one of them is you do need to have a high level of alignment across the academic community on what are the questions that are worth answering. If you've got thousands of researchers all wanting to ask thousands of different questions, it's going to be hard to get that level of alignment to invest.
[1:21:53] Even in terms of infrastructure, that could potentially answer several of these questions. So I think that makes it a little bit hard. And I think also the pace of progress in our field, it's still really exploding. And so in a sense, yes, there is a gap of access to computation, sometimes data between different institutions.
[1:22:14] But there's still a lot of ground for research. And I don't think we're seeing a slowdown yet that would allow the time for that convergence to happen. But that may come, right? That may come and that may be the model that emerges in coming decades. I saw a hand from you as well, Melanie. Yes. So I just wanted to say that there is a lot of talk, at least here in the US, about...
[1:22:43] Developing infrastructure, cloud infrastructure for researchers through, say, National Science Foundation, which is a great thing. But I also wanted to push back a little bit on the physics analogy, like with the LHC, that it seems like that in physics, there's been centuries of building up a theory that informs a lot of the experiments that people are doing on these big experimental facilities.
[1:23:13] And that's really different from what's gone on in machine learning and AI, that we don't have this foundation of theory that guides the questions that we're asking. We're just kind of just trying stuff out and seeing, oh, there's these strange phenomenon and how do we explain it? So I think that there is a role absolutely for these huge models, but there's also a role for developing theory that I think should get more attention.
[1:23:42] Cool. So in the last, we're in about the last five minutes of the panel. I just wanted to open things up to any general discussion or any points that people wanted to go back to that they have more thoughts on. So I'll kind of open up the floor. Feel free to unmute if you have something to say. Oh, I see Joelle has her hand up. So I guess we can go to you first. Yeah, Jacob, I want to bring back one of the key questions of the panel that I feel we didn't really
[1:24:11] Dig into as much perhaps as we had intended, which is really what are the risks of these models, right? You know, there's been a lot of safety concerns have been raised, whether it's about GPT-3 or some other models, risks because of misuse, because of dual use and so on. And I've heard a lot of different points of views across the field of whether these risks are justified or whether people are sort of playing up the fears and creating more concern than are there. So I don't know.
[1:24:41] If you'll allow me to pose a question to the group. But I would love to hear what this group has to say specifically on that. Should we be concerned about the risks associated with these models in a different way than we have with previous models? I think that's a great question. So I'm very happy to pose that to the panel. Aurel? Yeah, I can go first. I mean, I think we should absolutely be concerned.
[1:25:12] I tend to be like immediately what's the solution, like problem solving mode. And I guess the challenge here is how to make, and I think it was Melanie, right? Like, I mean, you play with these models. I mean, how you prompt them. I mean, it's kind of a bit like not very quote unquote scientific, right? So, you know, the question and maybe a challenge for the community. And again, I agree our field is very, very rapid.
[1:25:42] We probably are behind, but we better get to work. So how do we benchmark? I think, as I was saying, I think benchmarking is a great tool for progress and for like maybe a bit of slowdown, like normalizing kind of claims, et cetera. So how do we benchmark what, you know, let's say language models, of course, being at the center of attention probably these days are, you know, what kind of biases they exhibit. And there are, you know, I see some
[1:26:11] At least attempts, right? I mean, they're never perfect. Even if you think of established areas in NLP like machine translation, I mean, people still think that the metric is completely bogus, right? So we're not going to get it right. But I think I see some interesting data sets and ways in which we can prove our models that maybe we are at the MNIST stage of this. But as a community, we totally should.
[1:26:39] Be thinking about investing, maybe try to accelerate that gap between MNIST and ImageNet, which had the vision to spend a lot of money on annotation and thinking about advancing computer vision, not through models, but more through data sets. So I think we should just maybe encourage that from our positions and from, you know.
[1:27:00] The instigating thoughts through this panel. And I absolutely would like to see that, although there are some early signs that I start liking from knowing a little bit at least on the NLP front of things. Great. I think Jared had his hand up next. Yeah, so I agree. I think that questions of both bias, fairness, use in current models, and alignment
[1:27:29] And robustness of future models are extremely important. I think if you extrapolate progress in the field forward in time, it seems like we're really going to be confronting potentially scary issues. And I think we have a responsibility to address that. The other point is that this seems very important to me is that Melanie raised this question of theory and understanding how these models work. And I guess perhaps because of my background, I tend to take a natural science approach where
[1:27:58] These models themselves are kind of a subject of study, just like a star or an organism might be. And I think that having theories about sort of how these models are working that are ideally actually
[1:28:11] predictive of what empirical results we'll have in the future seems very important for kind of forecasting where we're heading and trying to understand whether or not some of these challenges associated with safety and dual use can be met or whether they need dramatically more resources before we kind of move forward. Cool. I think Noah was next, and then Joel. Yeah, I guess I, well, at this point, had two separate things to say. So one had to do with the kind of the risks, and I guess they are
[1:28:42] On the one hand, I agree any new technology has profound risks that we have to understand. But the way to do that historically is to get it in front of a lot more eyes and a lot more hands and let people bang on it and explore it, not to keep it locked away. Also, just to put a little bit of cold water on the notion of risk, right now we have
[1:29:05] Billions of human-level intelligences out there in the wild, each of which has extremely profound biases. And yet human society works, even though people are out there being smart and being biased. And so I think we should worry and think about what AIs will do when added to that community, but with a little bit of temperance. Separately, I wanted to make one small point about
[1:29:34] Kind of related to what Aurel was saying, that hasn't come up yet, which is I think the unsung hero of all of this scaling stuff has been hardware. We can only do all of this because somebody decided to make video games run fast, and now all of a sudden that made intelligence. Who would have thought that, right? And I think it's very likely that if we invest seriously in hardware, or just if...
[1:30:03] It happens because it's going to happen anyway. It's going to change the story about what counts as scale and who can do what anyway. And so I think we should keep that in mind when we're thinking about projecting into the future. If tomorrow or a year from now, all of a sudden, I can fit 87 GPUs on my phone, it changes the story just a lot. Let's go to Joelle.
[1:30:30] I feel as a panel, we have a little bit of a duty to be coherent in our points of view. So having said earlier that we don't see a paradigm shift happening, I feel we maybe have to take the position that there's not a paradigm shift in terms of the safety concerns either.
[1:30:48] And that whatever safety concerns and framing we have for these questions and our expectations of these models have to span the whole regime from smaller models to these larger models that we're seeing emerge in the community right now. I'll leave it to see if others disagree on that point, but that's certainly how I'm thinking about it. I do think we need to holistically consider different aspects of responsibility for these models as we would for other models.
[1:31:17] Point on which I will perhaps insist is I think we need to keep a very high bar in terms of transparency and the ability to audit these models. And that's one place where I feel we've been sort of getting a little bit loose on this, on the reason that these models are so large and not that many people can run them and so on.
[1:31:37] And I feel as a community, and since we're having this panel for NeurIPS, which is playing a leadership role on those questions, I do feel that really high standards for transparency for open science should continue to prevail to help us sort of monitor, audit as needed the behaviors of these large models. So I guess from Noah and Joelle, we heard that maybe these aren't
[1:32:06] I mean, I can jump in. I guess the obvious one is that we can always replicate this. And in fact, in social media, that does happen a lot.
[1:32:40] They might not be fueled by the state-of-the-art language models, but they are delivered at the large scale population-wise. I mean, one million was mentioned, but how many models can you run online? I mean, there's probably quite a lot more. I think that is a differential factor that should be considered, definitely, that maybe wasn't mentioned, and I think it needs to be clarified. I'm sure Noah agrees, but yeah. Melanie?
[1:33:12] And then Jared. So I would add that one of the dangers of these models is that people give them too much credit. They sound really human. They can do all these things. And so people, not just the general public, but also AI researchers themselves, sort of anthropomorphize them too much and give them, I think, too much credit for understanding.
[1:33:40] And therefore, perhaps are allowing people to use them in ways that they shouldn't necessarily be used. So we should emphasize not only their capabilities, but also their limits. Cool. We're running up on the hour. So Jared, you're going to get the last word. Oh, no. I don't deserve the last word because I will a little bit disagree with what other people have said. I mean, I do think that...
[1:34:08] I think of scaling up as a kind of baseline. I do think that even larger models will perform better. But I agree with other panelists that new techniques will also be discovered and introduced alongside these larger models that will make them better still. And I do think that if we're thinking things like maybe someone is going to have a much larger deep D3 model trained with reinforcement learning that can do some forms of reasoning, I do think that the capabilities of those models in five or 10 years
[1:34:37] Seem pretty scary to me. And I do think that at least as a kind of tail risk consideration, we should really be taking seriously models that may be sort of more broadly capable like humans and sort of how we make sure that those are used safely. But I apologize for having the last word. Sorry, everyone. It's OK. Technically, I have the last word because I'm going to summarize.
[1:35:03] So yeah, thanks everyone for the great discussion. This was really informative, certainly to me, so I hope the audience enjoys it as much as I did. We've covered lots of topics, kind of the distinction between just making something bigger versus gaining insights through studying new phenomena, perhaps where academia should be headed in terms of building large-scale collaborations or maybe even national labs.
[1:35:32] The safety concerns and how they are and are not different from smaller models. And so I hope this can stimulate continuing discussion within the NURBS community. And I hope all of our audience enjoys the rest of the conference. Thanks.