[0:00:00] Hello, everyone, and welcome to the podcast. Life is a story we tell ourselves. My guest today is Dr. Jared Kaplan. Dr. Kaplan is a theoretical physicist with broad interest in effective field theory, particle physics, cosmology, scattering amplitudes, and the conformal field theory bootstrap. His present research uses conformal field theory to learn about quantum gravity via the anti-de Sitter space conformal field theory correspondence.
[0:00:27] In the last few years, he's also been collaborating with both physicists and computer scientists on machine learning research, including on scaling laws for neural models and the GTP3 language model. Today, we will discuss the nature of gravity, the curvature of space, quantum gravity, and how those things contribute to our understanding of reality. So welcome to the program, Dr. Kaplan. Great. Thank you, and thanks for having me.
[0:00:54] Well, it's really a pleasure. I'm excited to have you here today. You know, we're going to talk a little bit about physics from the point of view that, you know, physics started out with people wondering about life and wondering about reality, wondering about spirit, purpose, and meaning, you know, of course, which gave birth to natural philosophers. And then from natural philosophy ensued, you know, the scientific method and physics.
[0:01:22] It leads me to be curious about you and why you became a theoretical physicist in the first place. Yeah, that's a great question. So I'm one of those people who's been really excited about and fascinated by physics from a pretty early age. I was always interested in math and I always liked learning math with my dad. I remember the first time I learned that you could write.
[0:01:49] Really, really, really, really big numbers using exponential notation. And my dad explained to me while we were waiting for dinner at a restaurant that you could multiply two times two and then two times two times two times two, et cetera. And you could represent numbers that were as big as the number of stars in the sky, that kind of thing. So I was always excited about math and its relationship with the physical world. And that was very inspiring.
[0:02:16] I think then also I was very interested in more philosophical questions about what the world really is. I guess if you had amnesia and you just woke up here in the world, you might ask, well, what is this place? Where am I? What's going on here? And I think it's natural to continue to ask those questions about this world that we all find ourselves in. Another particular memory I have is when I was 13 or 14, I remember
[0:02:42] I realized that I didn't know whether free will was possible or not. And I didn't necessarily have the most sophisticated conception of free will, but I wanted to know, was it the case that the laws of physics were fully deterministic, meaning that me and my brain were all just moving forward through time like clockwork in the way that it's often discussed in classical physics?
[0:03:09] Was there some room for choice? Was there some room for randomness? And I knew there was some association of quantum mechanics with randomness, but I just sort of remember having this revelation. I just don't know. And I went to the library at my school and I started looking through science books, trying to figure out what I needed to read to figure this out. So I think a lot of questions like that really motivated me to keep on going in physics. And I really just wanted to understand what the laws of physics were and kind of what it meant about...
[0:03:38] About life. Yeah. And then this leads you to becoming ultimately a theoretical physicist. This will be a little bit of a digression, but I can't help it because you mentioned mathematics and that being something that inspired you in the beginning. And I remember having a love for mathematics as well. But then when I got to imaginary numbers, I had this complete, almost meltdown. What is this? Until I read,
[0:04:07] Sir Roger Penrose's book, The Emperor's New Mind. And I don't know if you're familiar with that, but in that he did the most brilliant and elegant representation of how an understanding of probability and imaginary numbers led to the discovery of the strong and weak forces and how they interacted. And that like blew my mind. It was like magic.
[0:04:33] So I can like totally relate to what you're saying with mathematics. And I was finally cured of my frustration with imaginary numbers. Yeah, I mean, I think I had a more pedestrian moment like that as well, where when I was in high school and I was learning about circuits, someone told me, well, you know, like it's easy to understand circuits.
[0:05:02] Including oscillating circuits with capacitors and resistors and all these sorts of components. If you introduce imaginary numbers, and I remember thinking I'd heard of imaginary numbers, I knew the basic idea, but I was kind of surprised that that was a place where they find application. And of course, as a theoretical physicist now, it's not surprising at all. But I mean, we all have these moments where...
[0:05:27] Something that seems extremely abstract and almost nonsensical actually has a lot of bearing on the physical world. It made me happy, actually. I don't know. People would think that's crazy, but anyway. One of the things in physics that people really relate to and I get the most questions about in my seminars is gravity.
[0:05:53] There's Newton's conception of gravity. Of course, there's general relativity and Einstein's conception of gravity. And then now there's quantum gravity. But if we started with Newton, what's that conception of gravity all about? So I'm not the best historian, but I can tell you how we think about it in modern language. I guess the startling realization that seems obvious to us now is that the same force is responsible for the
[0:06:21] Right.
[0:06:50] So most people in the beginning, I know me as a young student in graduate school, when I learned about that, I mean, that kind of was intuitive and it made sense to me. And then I go to advanced physics and along comes this notion of general relativity and the curvature of space. And then somehow Maxwell and his equations of electromagnetic fields had something to do with
[0:07:18] You know, Einstein coming along and saying, you know, this, you know, conception of gravity is a force between two masses and the force being, you know, inversely proportional to the square of the distance between them, which seemed intuitive. Now we've got this other conception. And where did that come from? And why is that important in physics today? Well, I think it has a long history.
[0:07:46] So you mentioned Maxwell's equations. I think one thing that inspired Einstein early on, and this was before general relativity in his sort of famous 1905 paper about special relativity. So there are these two relativities associated with Einstein. Special relativity is about the symmetries of space-time, but...
[0:08:15] Of course, in 1905, Einstein hadn't come up with general relativity. He didn't know anything about it. So it's really a statement about symmetries. And it's a statement about what the relationship is between symmetries that I think are very intuitive, like that the laws of physics are the same no matter what orientation you're in. So in other words, if I'm facing east or I'm facing west or I'm lying on my back looking up at the sky, the laws of physics are the same no matter how I orient myself. So that's the symmetry of rotations.
[0:08:45] Special relativity was about uniting the symmetry of rotations with another, maybe less intuitive symmetry, which is that if I'm running and you're stationary, I see the same laws of physics as you. And what Einstein noticed in particular was that in electromagnetism,
[0:09:08] Electromagnetism had a different set of symmetries from what everyone assumed. It has the symmetries of Lorenz transformations. And so that was this first inkling of uniting space and time because a consequence of these ideas were these very, very surprising observations like that if you're in a spaceship going near the speed of light.
[0:09:31] Going so fast that you're approaching the speed of light, then there will be time dilation. Time won't flow the same way for you as it would for an external observer. And there's also these length contractions. Objects might appear shorter or smaller than they would seem. And so that led Einstein to sort of this idea that space and time should be united in this beautiful way by this enhanced set of symmetries. And then I think it's...
[0:10:01] What motivated then moving on to general relativity was this idea that acceleration and falling were somehow related, that any freely falling observer, no matter where they are in a gravitational field, basically experiences the same laws of physics locally.
[0:10:27] This led Einstein to general relativity, but it was really quite miraculous because there's a sense in which an adaptation and enhancement of Newton's law of gravity could have been sort of good enough. Newton's law of gravity is very, very simple. It, in some sense, has a force field, which is just like a number everywhere. It just says, what is the gravitational force or the gravitational potential at any given point in space?
[0:10:56] Einstein's theory of general relativity, this theory of curved space time, goes way beyond that. It's a more complicated seeming theory, but it's very, very beautiful features that it explains why all different kinds of objects, no matter what their mass is, experience the same kinds of effective gravitational forces.
[0:11:22] Right, and that's a lot to take in. But that leads people to ask questions about, you know, this notion of, you know, the curvature of space-time. And, of course, everyone's familiar with the famous experiment that was done during a total eclipse that saw the bending of light around a nearby star that validated the notion that space itself is curved.
[0:11:51] So could you say a few words about what it's, you know, curved by? I mean, most people know there's mass in space and there's energy in space and there's some sort of, you know, pressure acting upon, you know, space that produces this curvature. But I think, you know, people's minds get blown when they think about looking out into space and thinking about something being curved. And so they ask, you know, well, what is it that's curving?
[0:12:20] I don't see nothing out there that's current. Yeah, yeah. This is a classic challenge of general relativity. So yeah, the idea, I mean, the answer to that question is that if you imagine that you don't live on the globe of the earth, but you imagine, say, an ant living on a globe that you have sitting in front of you, just a ball or a sphere, then if...
[0:12:46] You're only sort of able to move around on the surface of that object. And you don't have any way of getting out. You don't have any way of getting off of the surface and sort of out into the world. Then you think that you basically just live in a two-dimensional world because there's only two different ways that you can go. And all you see is the intrinsic curvature of that surface that you live on.
[0:13:12] And that is the kind of curvature that we talk about in general relativity. And now you can abstract away. It doesn't matter whether that globe is sitting in your living room or whether it's the things that are sort of outside and inside the globe exist at all. If you imagine something that could only move around in that surface, in that thin layer of the surface, then that's what we're talking about. And so the idea is that...
[0:13:37] Our universe isn't two-dimensional. It has three spatial dimensions, and then it has time all wrapped up together. But really, our universe isn't sort of curved inside something else. It's the sort of intrinsic curvature of space and time itself that we see. And you also asked sort of what's bringing the curvature. Well, there is mass, as you mentioned, and in general, there's energy.
[0:14:05] And then you can imagine, well, what happens if energy moves around? If energy moves around, then in different reference frames, it has momentum. And if you sort of fill out all the possibilities, basically the possibilities are spanned by what is the motion of an observer? And then how is the object being observed moving? So that basically requires you to specify both.
[0:14:29] The motion of an observer and the motion of the energy source that's moving, that basically means that you need 16 numbers to specify that. And physicists call those 16 numbers the energy momentum tensor. And that sort of beautifully symmetrically wraps together the energy, the mass, the momentum, and the pressure that you mentioned. And it's ultimately that.
[0:14:54] All of that stuff that acts on space time and causes it to curve in general relativity. Right. And it's something that's hard for people to conceptualize. And I was going to talk a little bit about desitter space and anti-desitter space, but I think that'll be a little much for some people to get to and conceptualize. And there are so many components of that, but it's a...
[0:15:21] It's a beautiful way of conceptualizing space and understanding how things work in space, especially using those particular models. When we talk about gravity, people also think of it as something mysterious. Out of that mysterious notion, even if you talk about it in terms of general relativity,
[0:15:47] And even if you, like I have, tried to understand Einstein's field equations and get into them in depth, it's still mysterious and beautiful. And so it makes you question, you know, what's reality and how do things work? So do you think that ultimately getting a grasp of gravity will...
[0:16:15] Give us a deeper understanding of reality. And then maybe you can work into your answer a little bit about why now we're trying to understand gravity on an even deeper level in terms of quantum mechanics and quantum gravity, something that you're working on. Sure. So I think a lot of the progress in physics has really been uniting.
[0:16:41] Very, very early on, there was Newton, as we discussed earlier, uniting the motion of the planets with the motion here on Earth. I think through electromagnetism, sort of electric forces and magnetic forces were combined together in one description. Then we had this super important series of developments with thermodynamics and statistical mechanics, where it was realized that
[0:17:05] Heat was really just a form of energy. And the kind of energy that Newton was talking about in terms of motion and potential energy and gravity was just one form of energy that could be turned into many, many others, including heat. And that was responsible for the phenomenon of temperature. So that was a further uniting. And then with quantum mechanics and the rise of the standard model of particle physics, we were really able to kind of describe almost everything in reality.
[0:17:35] Using this one simple set of rules. So I think that's the way in which physics is really dramatically demystifying in a certain way in the sense that we can get to these simple, these very, very simple, beautiful underlying theories that can explain things like why does the periodic table of the elements in chemistry have the particular structure it has? Why do the elements behave the same?
[0:17:59] Behave the way that they do. How do you build a GPS satellite so that you can get signals anywhere on Earth and sort of figure out where you are? I mean, all these different things, like GPS requires general relativity. So I think that's one of the things that's very demystifying. I think the thing that's mysterious is that as we learn more and more about these fundamental laws of nature, they really apply in regimes that are very, very far from our day-to-day experience.
[0:18:28] Humans evolved to deal with a particular set of environments. I mean, as humans, we can't see things that are much smaller than a millimeter across. It's hard to see things that are a lot smaller than that. We can't really travel further than, I mean, if we're traveling on foot, we can only travel maybe tens or hundreds of miles. Maybe if you're an adventurer, you travel thousands of miles, but that's it. So there's a certain range of environments that we really evolved to live in.
[0:18:53] In our day-to-day lives, even in the modern era, there are only certain kinds of things that we encounter. And as the laws of physics begin to apply to distance scales that are vastly smaller than anything the human eye can see or vastly bigger than any place we can travel or experience, and on timescales like the age of the universe that are much longer than even human civilization lives, I think the laws of physics become less intuitive for the reason that they don't talk about
[0:19:21] They don't conceptualize the world in familiar terms. And so I think that's a lot of what makes it mysterious. The other mysterious aspect is really that there are these simple laws of physics at all. Right. I mean, from a basic point of view, I mean, like the laws of physics could have been a lot more complicated. They could have been arbitrary. There could be random events that happen that are just completely unpredictable. But none of that seems to occur in our universe as far as we're aware.
[0:19:49] And so this extreme predictability with such simple rules, I mean, I think it's very mysterious, but I don't think we really have any idea why it's the case. No, I know. And we could even talk about why the constants in the universe are what they are and the mass of an electron, why it is what it is. And if it was just a little bit different, we wouldn't exist. And so, yeah, those are the kinds of things that...
[0:20:18] That do lead to making it a little more mysterious. But it seems to me that you're also saying that physics has been able to put things together to the extent that it's demystified a lot of things. There are some fundamental mysteries that really do still exist. And our understanding of reality is still a little bit challenged because there are those kind of fundamental mysteries like the universal constants.
[0:20:48] That make life possible in the universe. Those types of things still remain kind of outstanding. And so when you get down on the small level, and so people out there, you started talking about math and using numbers. And so the Planck scale is like it 10 to the minus.
[0:21:15] 33 centimeters, if I'm not mistaken, something very, very small. And that's when you're getting to the quantum level. And so that's where you start talking about quantum gravity. And does gravity behave different at that level? Is it not the same? Is it totally in conflict with what our understanding of gravity is? So that's a great question. So, I mean, we, of course, are very, very far away from doing any kind of experiments at such short distance scales.
[0:21:45] But a phenomenon that's actually recurred in a lot of areas of physics is that a lot of theories of physics predict their own demise. So there are many situations where... So an example of this sort of classically was people were trying to think of what is an electron 100, 150 years ago. And so there were these classical models of the electron as some sort of...
[0:22:13] Shell of charge. And the shell has some size. It's just like some classical object like a ball, but it's just very small. And you cram all this charge in there. And people wondered, well, if there's all this charge at a tiny little place, it must have a lot of energy because of the repulsion of the charge against itself. And therefore, it seems like something's wrong with our description of the electron. Because if we keep making it small and small, it'll have bigger and bigger energy. And that was a problem.
[0:22:41] And of course, our theory of physics was wrong in a lot of different ways at that point. But it sort of presaged both the development of quantum mechanics and the existence of the positron. So this fact that you would think that the electromagnetic energy of the electron should give it a total energy, which eventually becomes much bigger than its mass.
[0:23:03] was a real problem. And it was resolved basically because quantum fluctuations of electrons and positrons together cancel out this effect and leave the electron's mass fixed in quantum mechanics so it doesn't become arbitrarily heavy if it becomes an arbitrarily small point particle. So that's an example of one theory of physics predicting its own decline.
[0:23:29] That helped lead theorists to develop further theories. And the history of physics is littered with tons of examples like this. So in general relativity, the problem we face is that general relativity also predicts its own demise in the same way. And that's where the Planck scale comes in. So if you try to do experiments with general relativity at energies at or above the Planck scale, then and really what I mean
[0:23:55] Distance is smaller than this distance scale you pointed out, 10 to the minus 33 centimeters. Then general relativity appears to predict its own demise. It stops making sense. One runs into singularities. One can't really do any calculations anymore. So that is one of the classic problems with general relativity. And it really, if you take it very seriously, it suggests a breakdown in reductionism.
[0:24:23] So in physics in general, we've made a ton of progress by asking sort of what is nature made of? What are protons made of? Well, they're made of quarks. What were atoms made of? Well, they're made of nuclei and electrons. And with gravity, because gravity breaks down in the way that it does, you actually, beyond the Planck scale, can't really apply reductionism to gravity.
[0:24:47] The reason is that if you try to make something smaller than the Planck scale, what general relativity predicts is that you'll actually just make a black hole. And if you try to sort of build a microscope to see even smaller, you'll actually just end up with a bigger and bigger black hole. Basically, to see really tiny things, you need a lot of energy. And all that energy just collapses into a black hole. And so that's kind of how general relativity breaks down when we try to probe it at the Planck scale.
[0:25:17] And so how are you trying to resolve that? Great. So I think fortunately, these black holes that you form when you try to study gravity at very, very, very small scales have a lot of interesting properties that we can reason about in a...
[0:25:40] So those kinds of black holes are so big that they're actually under control. We believe general relativity makes sense as a description of them to a very good approximation. And Hawking and Bekenstein and Wheeler and others in the 70s were thinking very hard about what black holes really are. How should we think about them? And they were bothered by facts like
[0:26:09] If I take a system with a lot of information in it, like an encyclopedia, and I throw it into a black hole, is that information somehow lost forever or isn't it? And this gets back actually to the point I made at the beginning about are the laws of physics deterministic or not? In a sense, quantum mechanics really is deterministic. The evolution of the wave function in quantum mechanics is deterministic.
[0:26:34] What that means is that information actually can't ever be destroyed. You can always run the laws of physics forward in time or backward in time, and you get back to where you started. You don't sort of forget what happened and then lose track of where you are. And so it was strongly believed that, say, an encyclopedia thrown into a black hole, somehow that information has to be there somewhere. And by thinking very carefully about this,
[0:27:02] These researchers came up with this idea of black hole thermodynamics, that black holes actually have a characteristic entropy, which really means a characteristic number of possible configurations they can be in. And of course, they also have famously a temperature. They radiate via Hawking radiation, et cetera. And so those...
[0:27:23] Those observations made by Hawking and Bekenstein and others in the 70s have really been, I think, the inspiration to a lot of progress that we've made in quantum gravity. And I think one of the most promising ideas is really sort of to take this very seriously and postulate what at first seems very implausible, that there's some kind of theory of physics that is...
[0:27:50] Describes our gravitational universe, but lives in fewer space-time dimensions. So this is called the holographic principle. And the very, very fancy acronym you mentioned when you were introducing me, the anti-de Sitter conformal field theory correspondence radius CFT is a very concrete direct instantiation of this idea.
[0:28:19] Bekenstein, when he was analyzing black holes, found that it seemed by far the most natural conclusion to reach was that the amount of information in a black hole or the different number of configurations a black hole can be in is proportional to its surface area rather than its volume. And this should be surprising because if you have a hard drive, if you say...
[0:28:45] Fill a box with a bunch of hard drives. The bigger the box that you have, the more hard drives you can fit in the box and therefore the more information you can store. And it's the volume of that moving box that you have that governs how much information in terms of hard drives you toss into it you can store. It's volume, not like...
[0:29:08] Thank you.
[0:29:37] The universe really wants to be described using something that's in lower dimensions, that's an area rather than a volume. So areas are two-dimensional, volumes are three-dimensional. And so we should describe our universe with a lower-dimensional theory. And that's something that I've tried to work on. Are you kind of saying, we did a seminar on this not long ago here in my community. And so on the universe is...
[0:30:07] Thank you very much.
[0:30:37] Yeah, absolutely. So yeah, what you said is right. So yeah.
[0:31:01] The word hologram here is an analogy rather than exactly what's going on. So when you have a hologram, you see something looks three-dimensional, but fundamentally it's just a flat piece of glass. And here the idea is that our universe as we see it with general relativity as our gravitational force and the rules of how space and time relate, all of that is in some sense an approximation.
[0:31:31] In a way, you could call it an illusion. And the fundamental laws of physics really live in two space and one time dimension, something like that. So one fewer dimension would be the idea there. And so somehow, in order for this to not be obviously wrong and obviously crazy, you have to find a system with two spatial dimensions that...
[0:32:00] You can reinterpret all of the interactions and dynamics of that two-dimensional system as a three-dimensional system with the force of gravity included. And that is what the great accomplishment of this ADS-CFT correspondence, Anti-Decider Conformal Field Theory, that's what it achieves is it turns out there is...
[0:32:27] This thing that sounds incredibly technical called a conformal field theory. And there's this kind of space time, which is described by general relativity, but it has a particular kind of curvature called anti-de Sitter space. It has a kind of negative curvature. It's sort of like a saddle shaped universe. And the gravitational dynamics of particles and fields and strings and black holes in anti-de Sitter space.
[0:32:54] Is exactly the same thing. It's the same system as this conformal field theory. And that's very surprising and involves a lot of fascinating mathematics that it's possible for dynamics in two dimensions to be the same as dynamics in three dimensions. But that's what's exciting and is being accomplished by this description. So has someone actually taken...
[0:33:21] A particular field that exists, let's say just the electromagnetic negative field, which everybody understands pretty much because they have cell phones and all that stuff. So they kind of have a day-to-day living understanding of electromagnetism. So has someone done the math or a calculation or a simulation that takes a two-dimensional space and then create it?
[0:33:49] Electromagnet, an electromagnetic field that exists in our three-dimensional space. I mean, it would seem like that would be something you would want to do, you know, to show that this might make sense because we have all these fields permeating the space we live in. So they have to come from somewhere. And if it's holographic, you should be able to mathematically show that they can emanate from that two-dimensional space. Yeah, so.
[0:34:16] So at the level of thought experiments and computational experiments, the answer is definitely yes. People, I mean, ADS-CFT was discovered in 1997 and 1998 by many people. But I mean, the very first paper was by Juan Maldacena, one of the greatest theoretical physicists alive. And so people have been thinking about it now for 20 some years.
[0:34:46] And there are a host of different examples you can work out in detail. If you have a particle or a few particles, they could be charged particles under the force of electromagnetism. They can feel the force of gravity and be orbiting each other in some way. If you have some system like that, we can actually give you a, we can mathematically write down a two-dimensional description of exactly that same dynamics in a conformal field theory and show that
[0:35:13] They're the same thing. They behave in the same way. So at the level of sort of computational or thought experiments, it's very well established. But we certainly don't have any kind of physical experiments in our universe where an experimentalist goes and measures something that confirms or denies this. And I mean, it should be noted that this version of holography, I mean, it's...
[0:35:37] Generally believe that holography is a very general principle, this general idea of lower and higher dimensional systems being related. But this specific realization of it doesn't apply to our universe. We live in an accelerating, expanding cosmology. And this really applies to a universe that's more, that you'd think of as being more static and actually has kind of the opposite kind of curvature as our universe. So we haven't yet really been able to use this to...
[0:36:06] Satisfyingly address any fundamental questions about our particular universe, like questions like what happened at the Big Bang? What was the initial singularity of our universe? What is the far future of our universe? What's that going to be like? This is something that people have thought about a lot, but we don't really know how to apply these ideas to answer those deep questions yet. And that's why people are still working.
[0:36:30] Well, that's interesting. So you started off giving us an understanding about this, talking about how physics, some physics and theories have within itself its own destruction. And you can think about the original model of the atom, which was this, you know, electrons orbiting a nucleus like planets orbiting the sun. And soon it was found out that that's just like, no.
[0:36:56] The electrons would crash into the nucleus and our model would be destroyed. I'm bringing it up because, of course, that led to a more sophisticated understanding of the atom and a quantum mechanical understanding, as you mentioned earlier, of the electron and how the electron works. And that was a huge breakthrough.
[0:37:24] But what I'm not seeing or hearing in this explanation of quantum mechanics breaking down or having within it its own destruction, even though you've gotten to this notion about a holographic universe, it doesn't seem to be on the same level as understanding the atom in a quantum, since there seems to be just this huge gap between...
[0:37:53] You know, the theory and anything that might be real or measurable or understanding from a laboratory sense. Yeah, no, I think that's just absolutely true. So I think physics and theoretical physics is both a beneficiary and a victim of its own success. So it's I mean, it's basically very, very difficult. I mean, you might say impossible to find any experiment that can't be explained using.
[0:38:22] Using the standard model of particle physics and I would include in that general relativity, there aren't any experiments that really show major violations of this. There are some loose ends and a lot of them are interesting, important loose ends, but a lot of those theories work very, very, very well. So that's a blessing and a curse. I think it's a curse because it means that
[0:38:51] These questions about quantum gravity, while very deep, are not very relevant for upcoming experiments. There are some kind of ideas out of left field where maybe they would be in a couple of different ways that might be surprising. But basically, I think it's very, very far afield. Now, I think there's this underlying question, which is, to what extent is progress in science driven by experiment versus by theory?
[0:39:21] And I think that generally speaking, I think experiment, I think basically nature is smarter than we are. And so experiment is crucial. I think the thing that gives theoretical physicists a lot of excuses is that it really depends on the sort of age and level of development of the scientific field. So, I mean.
[0:39:45] People have been doing physics for 400 years or more, I mean, depending on what you count. And that means that we have an incredible wealth of information about all sorts of different constraints. I mean, we have predictions from the standard model that go out to 10 decimal places that are correct. And so there's a huge, huge number of different constraints. And the theories we currently have, like the standard model and general relativity,
[0:40:14] Have been tested extremely well. And they're themselves very complicated, sophisticated theories with a lot of different pieces that have to fit together perfectly and do. And I mean, that's why it takes a long time to study physics in college and grad school and become a working physicist. Because you have to absorb all of this knowledge and all of the implications and constraints that it implies. Because all of those constraints say that a lot of ideas that might sound good
[0:40:43] at first are really just sort of wrong in an obvious or not obvious way because of existing knowledge. So I think it's the incredible level of constraints from all the things that we've done so far and the precision and detail with which we understand physical theories like quantum field theory built on quantum mechanics and then the standard model being an example of a quantum field theory. There are all these different components. And so I think the reason why it's possible for theorists
[0:41:13] To do speculative theory work and not be completely wasting their time is because if you come up with a theory, it's so constrained by what we already know that there isn't a lot of wiggle room. And so you can really test your ideas very, very, very quickly. And so when like autodidact.
[0:41:34] I think that's why it's not crazy to do theory work, but who knows?
[0:42:04] Well, that's a good segue for me to ask you this, and we're just about out of time, and I appreciate you spending so much time to explain some of these things. You also work on GTP3, which is AI and artificial intelligence and a way of, well, for our listeners who don't understand GTP3 or what we're talking about here, when you put in a,
[0:42:34] When you're typing on your computer, for example, in Gmail, and Gmail completes a sentence for you, for example, and anticipates what you're going to say, that's a lower level, maybe first generation form of what we're talking about with GTP and a type of artificial intelligence. So now, in this artificial intelligence that we're talking about, we're talking about
[0:43:03] It's incomprehensible to me, you know, data sets that are in the billions or trillions or quadrillions or something that goes into this AI and allows them to complete different things and to give us, our listeners, an understanding. For example, you can say in common language, and this is something that you'll find on the internet, that my landlord won't come and fix the radiator. And can I...
[0:43:32] Legally withhold the rent, and I want to write a legal document that says that. You can say that in plain English, and this AI will actually put that in legal language for you. It can also create music, and it's also now writing, as I understand it, computer code as well.
[0:43:55] Something you've been working on. And that a lot has to do with neural nets. So can you just say a few words about that and how physicists come at that and help create that kind of artificial intelligence? Yeah, so you described it beautifully. It read something like 200 billion words of text. There's way more than that out on the internet. So it read something like 200 billion words of text.
[0:44:25] Through that, through all of the patterns in that text, because it doesn't know anything about the world except what it learned from that process, it's able to then write, to generate seemingly impressive text through the correlations that it understood in its training process. And so I came to work on this mostly just because I
[0:44:52] I mean, I guess at a very fundamental level, maybe two of the most interesting questions you can ask are, what is the universe and what is intelligence? What is the mind? That sort of thing. I think another perspective on this also is that what we can actually do with intelligence is almost in itself kind of a feature of our universe. It's not a law of physics, but the fact that certain kinds of algorithms are...
[0:45:22] possible to execute in a reasonable amount of time and others aren't, I think is actually kind of an interesting feature. There's a way that any of us could beat Shakespeare. We could write every possible book that could ever exist. And then we could just read all of them and go through and pick out the ones that were as good or better than Shakespeare. And if we had good judgment, all we would need is good judgment and we could do that. But that's obviously a ridiculous idea because all of the books that could ever be written are
[0:45:51] Completely computationally infeasible to construct or write. And so that's sort of a silly example, but sort of points this idea that what kinds of things we can and can't do in terms of processing information really kind of fundamentally constrains what's possible. So those are some super abstract philosophical reasons.
[0:46:13] Mostly I care because I think there's been really dramatic progress in AI using neural networks in the last 10 years. And I expect that that's going to continue. And I think a perspective that I as a physicist have on this is that they're actually scaling laws in AI. And this is something that I worked on directly and it helped inspire GPT-3. So if you look at sort of how smart
[0:46:42] A program that's writing is, and you look at how smart it is as a function of how much data it reads. We talked about 200 billion words, but it could have been 20 billion words or 2 billion words, et cetera. If you look at how smart it gets as a function of how much data it has and how big the neural network is, how big its brain is, then it turns out there are actually very precise scaling laws. They're actually very reminiscent of things like thermodynamics.
[0:47:11] In the sense that these networks are so big and the data sets are so big that there's a kind of law of large numbers that when you try to optimize their performance, you get some consistent, predictable result. And so I think that's a perspective that is very much the way that a physicist would think about a system, just sort of look at what happens when you change certain parameters and then try to formulate a precise.
[0:47:36] Kind of law of nature, like a mathematical formula that tells you what changes as you change these inputs, like the amount of data or how big the neural network is. When you say neural network, I want our listeners to understand we're talking about analogous to the neural connections between synapses in our brain, and there are trillions of them.
[0:48:01] In an analogous way, we're talking about these neural networks in AI being comparable to the brain and the connections that are made. And so the more data, as we were saying, whether it's 200 billion words of text or whatever it is, represent all these different connections. And when you say there are scaling laws, you're talking about after something gets to a certain point of connections or you get a certain amount of data, then you can predict that there's going to be
[0:48:31] A certain level, and to use this word, I think it would be correct to use the word intelligence, emerges. Exactly, exactly. Laws that govern that. Yeah, that's exactly right. So yeah, these are artificial neural networks that are being run on big clusters of computers. And it's exactly what you said. As you either make these neural networks bigger,
[0:49:00] Or you increase the amount of data and there are various different ways you can combine these things. There's a predictable sense in which the intelligence of this system improves. And I mean, you can quantify that very straightforwardly. So for example, in our GBD3 paper, you can look at...
[0:49:18] The SATs, these SATs are these tests that American high school students take to get into college. And they're analogy, there are various word problems in the SATs, they're analogies. And you can ask how well does this neural net do at answering SAT questions? Could it get into college in the US or not? And the answer is the most recent model on some of these questions, even though it didn't specifically, it wasn't taught to do the SATs.
[0:49:47] It just read. It's an autodidact. It read everything on the internet in some sense. The largest model was actually better than the average American high school student at these multiple choice questions. And you can also see that there's this steady improvement. So if you take models that have less data or that have smaller networks, they don't do as well. But there's kind of a steady, predictable scaling of
[0:50:14] How intelligent these networks are as you increase these inputs. And I think this is one of the many reasons why I don't think that this whole neural network AI deep learning field is sort of overhyped. I think there are these predictable trends which indicate that we're really going to be able to make even more intelligent systems in the relatively near future. And I mentioned to you in our early correspondence and
[0:50:43] And I wasn't sure if you were responding to this part of what I was saying or not. But can an AI with this technology, with this GTP technology, does this really lead to some sort of sophisticated AI mind? Because this technology and the algorithms that you employ to run the system, it's self-learning.
[0:51:09] So this is machine learning that's actually self-learning. After a point, you're not putting any more input in. It's actually using the input that it has and going through it constantly and learning from what it has already generated. Is that correct? Yeah, that's all correct. It's what people often call this unsupervised learning because we just give the model all this data, all these words, all these...
[0:51:36] Web pages and essays and books downloaded from the internet and it just reads them and learns and we just sit back and watch. So that's absolutely true. Whether or not what kind of learning this is, whether or not it's achieving some notion of true intelligence or true understanding is fairly controversial. There are a lot of people with different opinions. I tend to be fairly kind of optimistic on the side that it sort of really is learning.
[0:52:05] A lot of subtle, important features of the world and how to think about it. And I think sort of combining this with other learning techniques that maybe are more specific, more aimed at actually doing a specific task, I think we will have models that really do kind of understand and really can kind of think, at least in all external respects.
[0:52:36] You see a duck and you call it a duck because it looks like a duck. And similarly here, if you have something that's capable of doing all of these different things at once that we associate with human intelligence, then I think I would sort of give it the benefit of the doubt and imagine that it actually does have some model of the world that means that it understands the world and is sort of making predictions and doing tasks.
[0:53:06] Based on that model. So coming full circle on that with physics, do you think that you can take this AI and apply it to some of the deep questions of physics, feed it a bunch of data and the questions we're talking about? Because it wouldn't come to it with some of the prejudices that physicists come at it with. It would be totally, I don't care what the answer is as long as it's right.
[0:53:34] So do you think it's possible? Yeah, I actually do think it's possible. I don't think that it's going to happen super very, very soon. But yeah, I think that, I mean, look, I mean, humans seem like we're very dominant as a species on the planet because we're intelligent and because we are able to work together and preserve knowledge within our civilizations and pass it on to our children.
[0:54:02] But there's a whole spectrum of intelligence. There's, I mean, there's paramecium, there are mice, there are dogs, there's dolphins and us. And I don't think that we're, there's no sense in which that continuum of sort of intelligence, of course, I'm oversimplifying a lot by calling it a continuum. I mean, all these different species have different skills. But anyway, there's no reason to imagine that.
[0:54:28] You can't have a creature or an AI system that is much more intelligent than we are and therefore would be much better at understanding the laws of physics than we are. And I do think that that is achievable. One of the benefits of being a physicist is that the field of artificial intelligence research is very traumatized. There was this idea in the sort of 60s and 70s that they were going to be able to build systems that were smarter than people.
[0:54:57] In just six months or a year. And I'm not even really sure that very many people believe that or claimed it that widely. But anyway, it has this reputation that the field of AI over claimed. They said that they were going to make tremendous progress and then they failed and there was this AI winter. And so people in the AI field, I think, are very reticent to make any kind of ambitious claim because they're afraid of being accused of just kind of crying wolf.
[0:55:23] As a physicist, I don't really have any of that kind of collective trauma. I'm just sort of coming at it with fresh eyes. And it really just seems like there's going to be a trend of significant improved performance in intelligence in the sort of foreseeable future. Well, Dr. Kaplan, thank you. I mean, this has just been absolutely fascinating. I could talk to you into the...
[0:55:48] Midnight hours about all of this stuff. But I know you have better things to do than to talk to me for eight hours about this. But I certainly appreciate the hour you've spent with us sharing your knowledge on these subjects. And would you come back again and talk about feels? Sure, sure. I would be happy to come back. Yeah, it was a lot of fun. And thanks for all the questions and for having me. Yeah.
[0:56:15] It's a great program. Well, you have a wonderful Thanksgiving with your family, and we'll talk to you again soon. Wow, what a great discussion. I sure hope you all enjoyed it. If you want to learn more about the concepts we discussed on the program today, like anti-de Sitter space or conformal field theory, or the latest advances in artificial intelligence, you can go to our website at lifeisastorypodcast.com.
[0:56:45] Our next podcast will take a retrospective look at the discovery of the Higgs boson, or more accurately, the Higgs field, and ask whether its discovery has contributed to our understanding of reality. So remember to stay safe, share happiness, and never stop questioning. Curiosity has its own reason for existing.