[0:00:00] Some great guests here. I think you guys know me and obviously the prime minister, but if you guys could introduce yourselves and say a bit about you and, you know, you've done both done amazing things. So please don't be shy. Sure thing. So I'm Greg Brockman. I'm the president and co-founder of OpenAI. We started OpenAI out of my apartment in 2015, 2016. So yeah, it's been a really exciting time to just, you know, be able to help not just move this field forward, but try to help steer it in a positive direction.
[0:00:29] I think that's really foundational to how I think about AI. What got me excited about this field is the fact it's going to be so transformative. And I think it can benefit everyone in huge ways, but we also really need to mitigate the downsides. And so I think that's something that really resonated with me about the earlier conversation. I hope we'll talk about a lot in upcoming minutes. And since Greg was too modest to say so, I'm going to add something to your introduction of yourself there, which is just a few years ago when...
[0:00:54] Greg and his colleagues were saying, oh, in a few years, we believe we can actually build AI that can master human language and common knowledge. A lot of my colleagues in AI were like, that's crazy science fiction. It's going to take decades. But you did it. We literally had an intern in 2016, so our very first summer, who we had this conversation about.
[0:01:19] How to solve the Turing test and we had all these ideas and whatever and we were like three years we'll solve the Turing test and that intern told me I'm out like I'm not joining full-time like you guys are crazy and three years later we had GPT-2 so we didn't quite solve the Turing test so he was right but I think that we had far more progress than I think most people thought possible. So I'm Max Tegmark, I'm a professor doing AI research at MIT and I think it was spot on what we heard earlier here from you that artificial intelligence
[0:01:48] We'll be both a blessing and a curse. I'm very, for a long time, very excited about the blessing part, how we can use it to solve all the problems that we haven't been smart enough to solve before and go far beyond and help life flourish for billions of years on earth and beyond. And I also, though I'm very concerned about the curse part. So nine years ago, I founded this nonprofit called the Future of Life Institute. Together with Elon, we launched the first ever attempt to the mainstream.
[0:02:16] AI safety by launching a worldwide grants program and educate and do conferences and so on that you, Greg, have participated in also to make sure we can steer this ever more powerful technology away from the curse towards the blessing direction. Well, can I say that? I want to plug this book. You wrote this book and I read it. It's a good book. It's a very good book. Was it ChatGPT or? I have an alibi because not even Greg was fast enough to have it written by the time. ChatGPT.
[0:02:46] Ain't good enough yet to do this. But from what I hear, you guys saying it's good. Yeah, give it a minute. 30 seconds. Well, you see, let's talk about that because, you know, I think that the kind of things that you bring here are, I think, are monumental. You know, when I see the talent on this stage, excluding myself, I'm reminded of what JFK, John Kennedy said, you know, when he brought his gifted team into the White House. He said, this is the most gifted team ever since, that sat here ever since.
[0:03:15] Thomas Jefferson died alone. So I think you're about to surpass that and you are changing history. And I'm sure with such power comes enormous responsibility. And I think that's the crux of what we're talking about here is how do we inject a measure of responsibility and ethics into this exponentially changing development? How do you do that? That's really what I'd like to know from you. I think that this should be a very interesting discussion. And I think I know.
[0:03:44] Almost everyone in AI, and I think I really can't think of anyone who's got bad motivations in AI. I think everyone I know in AI has good motivations. They want to create technology that helps the world. But one could also say, if you think of someone like Einstein, who was obviously a very peace-loving person, and...
[0:04:03] He didn't think that there would be a nuclear bomb created as a result of his discoveries in physics. He wasn't thinking, let's make super weapons. He was thinking, let's try to understand the truths of the universe. So it's just important to, I always want to just preface any criticism of AI development with it not being a criticism of the people doing it, but just bear in mind that just as Einstein didn't expect his work in physics to lead to nuclear weapons, we need to be cautious that even...
[0:04:33] With good intentions, the best of intentions, that we could create something bad. That is one of the possible outcomes. And we want to try to take whatever actions we can to ensure that the future of humanity is good and that AI is much more of a blessing than it could be. That's definitely why I work on it. Not for evil? It turns out. But I think that balance is really key. And I think that one of the things that is important is to really calibrate the choices with respect to the technology.
[0:05:03] Because I think this technology, it has this way of being very surprising. The way that people pictured, like you read Asimov, you read kind of all the science fiction, you read Eliezer Yudkowsky and kind of all this thinking people are doing about very smart machines that are smarter than humans. And somehow none of it really predicted GPT-3. Somehow the technology that we're building, it's just surprising. It has very different properties. And I think that even from, you think about like chess, people used to think chess would be, we solve that, we solve human intelligence. And it turns out that's the first thing to go.
[0:05:31] And so I think that what we're really seeing is that we need to be calibrated and see not just what the current systems are doing. And this is one of the reasons we deployed ChatGPT is it's just really important to not get in our heads and think, oh, this is how it's all going to play out. This is how people are going to misuse it. But you need to actually see that. You need to actually have people spending a lot of time thinking about where this slots in, where it works, where it doesn't, but also to really try to peer ahead. And so we've been putting a lot of effort into capability prediction. One of the things that...
[0:05:56] I think what's kind of undersung in our GPT-4 release is that we actually had very good predictions of exactly how good GPT-4 would be before we even trained it. And that's something we can never do with previous models. And I think that this and doing that at societal scale, like I think that's one answer for how do we actually get this right? Because if you can see around the corner, then you have an answer. And in AI, it's always been the opposite. We've always been taken by surprise. Well, I want to ask you, because there are sort of layers of questions here, Max's book.
[0:06:24] It takes you to the existential question of whether you project basically machine intelligence or human intelligence into the cosmos. Human intelligence turned into machine intelligence into the cosmos and so on. That's a big philosophical question. I'd like to think we have about six years for that, okay? But the other questions that I'm dealing with as a leader of a country that is blessed with a lot of AI talent, rated about number two in talent relative to population size in AI.
[0:06:53] But I want it to be among the top three because it's important to advance the country. But I think also it raises questions that we raised before. But here's another one. OK, as ChatGP4 or ChatGPT8 comes into being and you have Hollywood strikes already about screenwriters and directors and actors. OK, what do you think will happen to the job market? I mean, we had this conversation. What do you think will happen? I mean, you know, you can tell me.
[0:07:20] You can comfort me with all previous revolutions that there were more jobs, many more jobs created than lost. But do you really believe that now? Yeah, I think it's far from obvious. I think it is a correct question. And I also think that it's important to kind of look at the whole picture of for sure there are jobs that are going to go away. What's going to be the balance of like, you know, marginal job creation? And you can tell the story of, oh, it's going to be very comforting. You look at every previous technology. I do think AI is not like every previous technology, right? That this is usually we kind of are.
[0:07:49] Seeing automation of mechanical parts of creation. And here it's almost this creative aspect of creation. But I think that we want to even go one step further from where we've been, right? And not just say, okay, like, you know, I could have been an artist and now an AI is a competing artist. But you want AI that can help us come up with ideas and solve problems that we just couldn't before. And so what does that world look like? And, you know, your post-scarcity world where people don't have to work to survive, you know, what...
[0:08:13] Set of people, you know, because people identify themselves, you know, your whole identity is around your work. You identify yourself by your job. Yes, exactly. And so what will happen then and what does that look like? And so I don't come with an answer to this question of exactly what the balance will be like. I think there will be fundamental changes in how we even relate to work and to meaning that will happen and may result in a landscape that just looks very different that the answer to that question is, you know, if you're a hunter-gatherer and someone's like...
[0:08:42] What's going to happen to our hunter-gatherer job post-agricultural revolution? It's just a weird question, right? It's like the framework changes on you. And so that's not a cop-out, right? I think that it might be the case that actually many more jobs go away than get created and there's a lot of chaos and turmoil. But I think we've got to be ahead of it. And the more that we can predict it, the more that we can have concrete answers. And we have teams that try, right? But I think that we're just getting started and we need help. Well, it'll change our economic models, okay? I'm a clear free market guy.
[0:09:09] I help liberate the Israeli economy to make it a sort of a technological powerhouse, but that's because of free markets. And free markets means open competition, and it's fine, but you stop at the monopolies and monopolies edge, okay? Well, you do that. I had a conversation with one of your colleagues, Peter Thiel, and he said to me, oh, it's all scale advantages. It's all monopolies. I said, well, yeah, I believe that, but I think we have to stop it at a certain point because we don't want to depress competition, okay? Well, AI is producing.
[0:09:38] This wall, and you have these trillion-dollar companies that are produced overnight, and they concentrate enormous wealth and power with smaller and smaller number of people. And the question is, what do you do about that monopoly power? What do you do about competition? And if you are going to cannibalize a lot more jobs than you create, then we have to change the structure of our economic policies and our political policies to take care of...
[0:10:08] of the people who are not going to find jobs, who are not going to contribute added value to the economy. We have to make sure that they have a living, a decent one, and they have all the services. We'll probably have the money to do that, but it requires a challenging model, certainly for a free market guy, free market disciple like me. And that's coming very fast. It's not going to come slowly and we have time to adjust. I like how you challenged the question, what's going to happen? I'd like to continue.
[0:10:34] And that theme, because I think if we ask this very passive kind of question, what's going to happen to us? Let's just sit here and eat popcorn and wait for the future to happen. Then we're heading straight for the curse, I think. This is a sort of very passive approach, which I think is just going to...
[0:10:51] We've done this experiment before when a new species showed up on the planet that was smarter than all the other ones. And what happened was the Neanderthals went extinct. If we just very passively go into this, I think it's very likely humans will go extinct also. And I think that's the wrong attitude. Even in economics, in a country, you don't just sit back and wait to see what happens and get child labor and the super monopoly of doom. No, you ask yourself, what do you want to happen?
[0:11:18] And then you put in place institutions, you ban child labor, you put an antitrust. And in this case, I think the analogous question is, how do we make sure that with this ever more powerful tech, it's we who control the AI rather than the other way around, right? So I was honored that you had this book. If this book still hasn't put you to sleep, because this is a technical nerd paper I wrote with my- This one's written with- Technological approach to how we can control machines that are even smarter than us by actually having other AIs-
[0:11:47] Formally prove that the AI is safe. And the reason why I'm so excited about the proving part is, you know, the only way you can trust something much smarter than you to still do what you want is if you can prove it. Because no matter how clever it is, it can never do what's provably impossible. And it turns out it's much harder to come up with proofs about math or about what a program is going to do than it is to check that the proof is correct. You can write a 300-line computer program.
[0:12:14] Which will check any proof claiming anything about anything else. We can understand that. If we force AI to prove things to ourselves, to us humans, that's going to meet our specifications, then we actually have a reason, a way we can control things that are smarter than us. How do you know, how do you know that the proof is not disputable? That's the beauty of mathematics. You can actually check, even if it's a brilliant proof, right, that I couldn't come up with myself. I can just say, I just plug this into this and this.
[0:12:44] Yeah, you know, it checks out. And there's a big science in doing this with proving things about programs for cybersecurity because companies are so fed up with losing millions of dollars by getting hacked all the time because yet another bug was found and secure shell or whatever that there's been years of investment. And Greg and I had a fun conversation actually just before we went on stage here about how this field of trying to prove that things are safe is still in the Stone Age.
[0:13:12] In the pre-large language model phase where it's mostly humans trying to make the proofs, I think it's ripe for a revolution where the kind of technology that OpenAI is building can totally turbocharge our ability to prove things about code. And I think there's a bigger picture too. Because in my mind, if you think about cybersecurity, it's always a cat and mouse game. You have attackers, you have defenders, each one's leveling up.
[0:13:35] Defeating the other. Is there ever going to be a time when the defenders totally win? Is there going to be a time when the attackers totally win? And I think with AI, that balance can change. And I think that formal verification is an example where maybe you can have defenders just win once and for all. And I think there's a lot of asterisks there. I think it's definitely a maybe. But I think that at least at a technical level, formal verification has been this idea of if you could mathematically prove these systems are safe and good and that you don't have bugs in your Linux kernel, then...
[0:14:01] All of these bugs we've seen, we were talking about Heartbleed and Shellshock and all these bugs from the past 10 years. Those would be solved. Those would go away. You would know there'd be no future one of those. Now you need to be very careful because you need to make sure that your underlying machine model is like actually correct and a bunch of details like that. But it's just interesting to think that these fundamental things we believe about how security works might just be up. And we've never had an opportunity for that. That'll give us a lot of free time. Exactly. And it's worth adding for the nerds listening to this who like the code, you know, normally debugging.
[0:14:30] All that ever does is, all it can ever accomplish is prove that there is something wrong. If you don't find any bugs, is there one more or not? When you prove it, when it's what's called formal verification, as Greg said, that means you're guaranteed. You seal the deal. Seal the deal. All right. That assumes, by the way, that mathematics is, you know, that's it. It's the law of the universe. It's not going to change. You're not going to get alien mathematics.
[0:14:59] You don't need to assume that, actually. You get into Gödel's incompleteness theorem and things like that. That comes in a big way when you're dealing with infinities of various kinds. All the computers, even Greg's very biggest servers, it's all finite. I would say you still got to watch out for the alien mathematics. But you can totally formalize what it means that it's impossible to hack into his laptop. And if you have a rigorous proof, I think that seals the deal. So I think that beyond these enormous questions that you raise,
[0:15:28] I think for me, the immediate problem is to fashion a national policy, which I intend to announce in a few months. And you guys are stimulating me, I have to say, this visit. But you sort of think, okay, what is it that you're trying to achieve? You're trying to better all the systems, transportation, health, education, agriculture, production, everything, for government, for the private sector.
[0:15:53] For philanthropy, for academia and so on. And what is the role of the government in this? And, you know, various countries are seeking solutions to this. What do you do? What kind of instruments do you build? I think the larger question is, you still have the monopolization of AI power in small concentrations because of what you said before, Elon. You said basically it's too big. You know, you need the big data. You need the big, you know, the big computational power. You need the talent.
[0:16:21] for creating the algorithms. And that's concentrated in a few countries. We have another test, not only to make sure that we take care of ourselves, but how we have these benefits percolate to, you know, the rest, the scientific and technological have-nots. And I think that's another real issue. But so far, you know, the advances that we've made, at least in Israel, I don't know, anything from Disconkey, remember that? Used to have something like that. And I don't know, a drug, the camera.
[0:16:49] Well, first of all, I want to say just on the talent.
[0:17:16] Density front, we have a number of Israeli engineers, scientists, managers, and they are top-notch, no question. We have a manager who runs one of our core teams who is ex-special forces, and there is nothing that fazes him. So I think there's something about the Israeli way that I think is actually very conducive to this field. And I think that it is really important to counteract this rich-get-richer phenomenon if it comes at the expense of everyone else. I think that the world we should shoot for is one where
[0:17:45] All the boats are rising. And I think that there will be some sort of inequities in the landscape. And I think that's something to address. And that's something that I think that leaders such as yourself, I think, need to be very cognizant about. You know, I think that we've talked about UBI, like OpenAI actually has run, I think, the world's most comprehensive UBI study. It's something people talk about, you know, universal basic income, the idea of just the government just distributes money, right? That's just what it does. If most people are not working, producing, you know, dollars themselves, and there's a world of plenty.
[0:18:11] Is that a mechanism that can work? Maybe it can, maybe it can't. There's a lot of assumptions that would go into a world where it works. It's clearly, I think, also has a lot of downsides to it. But I think that we need to have creative solutions because, again, I think that just like defense versus offense might fundamentally change, economic production and what it takes to actually build even a stable society, I think, is going to fundamentally change. Well, Elon, you said that when I asked you this question in one of our nighttime conversations, I asked you, well,
[0:18:38] What are you going to do when people don't have jobs and we shell out money for them? And you said, I think you said, oh, what's bad in living in paradise, right? I think you said that. Yeah. I mean, this is a question of like, you know, the concept of heaven. Now in heaven, I don't think you need to work in heaven. I haven't read that anywhere. They're not laughing. Feel free to, you know, if you want. So, you know, the very positive scenario of AI is actually in a lot of ways.
[0:19:07] The description of heaven in that really nobody would need to work. I wouldn't even call it universal basic income. I would say it's probably universal high income. I'm describing the best case scenario here. So I'm not saying this is definitely what will occur. There's a range of scenarios from very negative to very positive. Very positive scenario is basically sounds like heaven.
[0:19:26] You can have whatever you want. You don't need to work. You have no obligations. Any illness you have can be cured. When do we die? Well, that's a good question. I think it'll be a choice. I think it probably ends up being somewhat of a choice. You want this world? Well, I'm not saying... I don't know. There's a sort of philosophical debate. Is the concept of heaven as it is normally described actually what you want? I mean, do you want to be in a future world where, you know, in the AI sense, that AI can do everything better than you by far?
[0:19:55] That's because, you know, you have like man's search for meaning type of thing. Good work. And what meaning, if AI can do everything better than you, do we live a life of hedonism? I don't know. It's hard to say. But that is, where things are headed in a positive scenario is that there would be no scarcity of goods and services. The robots and computers can make as much of whatever you want for as many people as you want with really no limitation.
[0:20:22] Any scarcity we have would be artificial, meaning if you define that a particular artwork done by a human is unique and special, then it's artificial scarcity, but there will be no actual scarcity unless we arbitrarily define something to be scarce. So I think for a lot of people, that is a good future. I might be personally a bit frustrated because I'm like, well, what am I supposed to do now? You'll think of something. I think that...
[0:20:49] This is a very interesting set of challenges. What would we do if people have no jobs? But I think it's important to remember that we're not automatically going to end up in this situation in the way you're thinking about it now. We could also, I think, very likely end up in a situation where people have no jobs because there are no people anymore.
[0:21:07] Where we literally go extinct. You know, we had a statement that came out in May, right? That I think Sam Altman and you both signed, right? Saying, and Ursula von der Leyen came out with it a couple of days ago saying we have to take seriously human extinction. And if this sounds like very crazy science fiction, you know, we were joking, but just a few years ago, all your success sounded like science fiction too. And you said that we don't have a lot of time. These are things that could happen relatively soon. And there's...
[0:21:34] It's very important to me because I want the good future to think about how we steer away from those sort of scenarios. One way in which we could end up in this kind of bad future is pretty obvious. The idea that we can lose control to something more intelligent is not new at all. Alan Turing himself said this in the 50s. That's kind of a natural thing to expect. That's what always happened on our planet before when a new smarter species came in. It took control over the older ones and then, you know,
[0:22:03] Bye-bye Neanderthals, bye-bye mammoths, whatever. More concretely, I think, even though Hollywood is full of scenarios where someone somehow loses control of AI and it takes over, there are more sneaky ways in which we could end up in this sort of bad state, even more willingly. So for example, right now, I'm a big fan of the free market too. It's a fact that if people are competing against other people, they will often cede control voluntarily to machines.
[0:22:33] To get an edge on the other guys. So companies who replace workers with machines out-compete companies who don't. Armies who cede control to machines out-compete armies who don't. Once we have a really powerful AI that's better at running companies and countries than humans, then countries that replace their prime ministers by AI might out-compete others. And then we end up in this future where all these...
[0:22:56] Things are happening, but it's not our future anymore. And we're like, wait a minute, where did we go wrong here? It was supposed to be for us. But that's the point, you know. Forgive me for borrowing a page from your book, literally a page from your book. I'm confessing, I'm a speciesist. Hard not to. Okay, so I'm for the human species. Okay, we can talk about other evolutions and, you know, machine-launched intelligence and so on. But I'm, you know, I'm for...
[0:23:22] You know, in the Jewish tradition, there's a saying, how do we translate this? The impoverished of your own city, of your own town, precede the impoverished of other towns. So the human species, as far as I'm concerned, is something that I'm vitally interested in maintaining. I'll tell you that the problem we have is not merely extinction, which I hope you can, maybe you can extemporize, just elaborate on. But here's something else. Even if we don't go extinct.
[0:23:49] How do we define ourselves? You talked about that, Greg, okay? So we defined ourselves. We had this myth in the Bible. We had heaven, you know, paradise. You can just pick off the trees, you know, pick off the fruit from the trees. You'd have to do nothing. And then the serpent comes and messes things up. And then we're condemned to toil for our labors. Life is a struggle. It's defined as a struggle. It's defined all the time as a struggle where you're competing.
[0:24:14] With the forces of nature or with other human beings or with animals and you constantly better your position. This is how the human race has defined itself. And our self-definition is based on that, both as individuals, as nations, as humanity as a whole. Now that could be challenged and it is challenged. So it's both our self-definition throughout human history and evolution that is being challenged and also the question of our continued existence, which, you know, you can please explain why you think.
[0:24:43] We're in danger of extinction. Well, I was going to say, I mean, I think this whole arc in my mind is all about paradigm shift, right? And I think that even the question of what would that heaven post-AGI positive future look like, I think even that is hard for us to imagine what the true upside could be because it's not just material abundance. It's not just we all get...
[0:25:03] Nice cars or something. Hopefully all Teslas. But also that everyone gets a great education, right? You know, everyone has that story of that one teacher that they had who paid attention to them and inspired them on a specific subject. Imagine if you had that teacher for anything you wanted constantly and just how much better of people would we be? How much better would we relate to others? Like that is just one example of the kinds of upsides that are possible. And I think the question of, well, what does it mean to be a human? You know, if you back to how do you predict what's going to come next? Actually.
[0:25:31] The thinker who I think had the best foresight about how the AI revolution was going to play out is actually Ray Kurzweil. I agree. Yeah. And his book Singularity is Near gets like a lot of shit. I think that people kind of assume it's going to be this almost like just sort of religious text, but instead it's a very dry analytical text. And he just looks at the compute curves and he says, this is the fundamental unlocker of intelligence. Everyone thought that was crazy.
[0:25:55] It's basically true. It's basically common wisdom. And part of what he says is, look, what's going to happen is in 2030s. First of all, he says AGI 2029. Yeah, I keep telling people it seems to be almost exactly right. It's spooky. It's spooky. 2030s is when the merge happens. So we've got neural link coming and, you know, maybe other systems like that. And what does it mean once you actually are kind of merging with an intelligence? Well.
[0:26:17] Neuralink is necessarily moved slower than AI because whenever you put a device in a human, you have to be incredibly careful. So I think it's not clear to me that the Neuralink will be ready before AGI. I think AGI is probably going to happen first. But, you know, we'll have the sort of AGI singularity. Sometimes digital superintelligence is called like a singularity, like a black hole, because just like with a black hole, it's difficult to predict what happens after you pass the event horizon of a black hole. And we are currently...
[0:26:46] So, you know, circling the event horizon of the black hole that is digital superintelligence, the event horizon. So the reason for Neuralink, although initially we're very helpful to people who have brain or spine injuries, just doing basic stuff like enabling people who are tetraplegics or like a Stephen Hawking to communicate actually even faster than I'm communicating with a fully functional body. So, and ultimately to improve the bandwidth between.
[0:27:14] The Cortex and your AI version of yourself. In fact, I think a lot of people perhaps don't quite realize that they're already a cyborg. So you've got your Olympic system, your sort of basic drives, your Cortex, which is the thinking and planning. And then you have a tertiary layer, which is your computers, your devices, your phones, laptops, all the servers that exist, the applications. And in fact, I think probably a lot of people have found that if you leave your cell phone behind. Take it away. If you forget your cell phone.
[0:27:44] It's like missing limb syndrome. You know, you're like, where'd that thing go? Losing your cell phone is like missing limb syndrome. So because it is, your cell phone is an extension of yourself. The limitation is bandwidth. So the rate at which you can input, or I should say output information into your phone or computer is very slow. So with a phone, it's really just the speed of your thumb movements. And with, you know, best case scenario, you're a speed typist.
[0:28:09] But even that data rate is very slow. We're talking about tens, maybe hundreds of bits per second, whereas a computer can communicate in trillions of bits per second. And this is admittedly somewhat of a Hail Mary shot or whatever it is, a long shot, is that if you can improve the bandwidth between your cortex and your digital tertiary self, then you can achieve better cohesion between what humans want and what AI does.
[0:28:38] At least that's one theory. I'm not saying this is a sure thing. It's just one potential iron in the fire. If ultimately, you know, hundreds of millions or billions of people get a high bandwidth interface to their digital tertiary self, their AI self effectively, then that seems like that probably leads to a better future for humanity. So I'm getting somewhat esoteric, but hopefully some of this is making sense. And like I said, it's a long shot. But what I find interesting is that I've...
[0:29:06] I've not found any human who wishes to delete either their cortex or their limbic system. I suspect nobody in this room wants to do that. Or their cell phone. Well, no, I've met people who don't use a cell phone. It's rare. Well, I confess, for years I avoided this, and I avoided it. I didn't even have, not only did I not have a cell phone within my immediate radius, I didn't have a television set. My phones were 35-year-old.
[0:29:35] Phones, because I was Prime Minister of Israel and I knew what this means. Okay. I mean, maybe your audience doesn't know, but we knew. Now, having spent some time in the opposition, I lapsed into the use of the cell phone and I find it very hard to dissociate from it for the reasons that you said. It's become an extension of, you know, a repository of memory, the ability to get information very fast, which you say correctly is very slow, but compared to what it was before.
[0:30:03] Remember you had research assistants that have to go to libraries, things like that. Where's that? That's gone. But this will be seen as vastly primitive compared to what you're talking about. Yeah, I'll just add something. I'll go to Max. I mean, it's sort of a funny thing. If you assume a best-case AI scenario, imagine if you're the AI and you just want the human to tell you what it wants. Just please spit it out. But it's speaking so slowly like a tree.
[0:30:31] Like trees communicate. If you watch a tree, like a sped up version of a tree growing, it's actually communicating. It's communicating with the soil. It's trying to find the sunlight. It's reacting to other trees and that kind of thing very slowly. But from a tree standpoint, it's not that slow. So what I'm saying is we don't want to be a tree. That's the idea behind a high bandwidth neural interface is just even when the AI desperately wants to...
[0:31:01] Good things for us that we can actually communicate several orders of magnitude faster than we currently can. I love that you mentioned the trees there, because if anyone is still struggling to understand why it's so likely that we could get wiped out by AI, you know, to a super intelligent AI, we are like trees. They are as much faster than us in their thinking than we are to trees. So if some trees in the rainforest are a little bit worried that some humans are going to come chop them down, you know.
[0:31:28] And they're like, oh, don't worry. We're so smart. We'll stop those humans. Good luck with that, right? I love how you went philosophical here and talked about what it means to be human. And I wanted to follow up on that. But before that, maybe just continue that. Okay, so how you're assuming that the danger of human extinction is that you have machines, super machines, super intelligent machines that outpace humans by orders and orders of magnitude who will not necessarily value.
[0:31:56] The continued human existence as we know it. And basically program is out. That's basically what you're saying. So if we were to be so dumb as to create the entities that are much more powerful than us that don't share our goals, yeah, that's a really bad idea. I mean, in Israel, I think you understand really well that if you have a bunch of other beings who are intelligent and don't share your values, you know, it can go really badly, right? And let's not make that mistake.
[0:32:23] By giving too much power to machines don't share our values. But of course, we can tackle that in various ways. So coming back to what can we concretely do? I just want to say one thing about the philosophical point about being human. Just for myself, one of the things that I really cherish the most about being human is the agency that I can actually...
[0:32:43] Make choices and make a difference. And one of the most beautiful things about what you are all doing with technology is giving us more agency. Back in the Middle Ages, we had a life expectancy of frigging 30 years. And knowing that we might starve to death next year because of the weather, you know, what kind of... It's so much more inspiring now that we're becoming more of the captain of our own ship with technology, able to actually have more agency as a species. So I say let's use that agency to create a wonderful future.
[0:33:11] Not throw it away. And three things I wanted to just suggest for concrete action items we can do here. I'll say a little more about each of them. One has to do with pausing risky stuff. One has to do with controlling the machines. And one has to do with inspiration for the future. So for the controlling, we already talked quite a lot about the importance of staying in control of these machines. So I don't need to dwell on it anymore. But we are, I think it's saying things like, oh.
[0:33:39] What is going to happen to us? Bring out the popcorn. That's too passive for me. Let's celebrate our agency and ask ourselves not what we think will happen, but what we want to happen. And clearly, we want to control these machines. We're building them. Second thing, so I agree with Greg on many things, and I greatly respect your work. One thing where we have a friendly disagreement is I signed this pause letter saying I think we should pause the riskiest things, and you don't.
[0:34:06] I think that it would be a good strategy for us to emulate what biotech does and just have clear safety standards. You know, if you want to release a new drug, you can't just sell it in the supermarket. You have to do a clinical trial, persuade some experts somehow that this is safe. And I would like to see really high stakes, very powerful systems, some sort of system like this with standards. And then the companies have an, it's on them to prove that they're safe before they get released. Who does the standards? Who makes the standards?
[0:34:34] Well, and we've solved this. It's been solved in Israel. It's been solved in the U.S. for biotech. Not for biotech, but here, you know, do you want it regulators or do you want it legislatures? I mean, it's not the only choice, but, you know, it's a...
[0:34:48] Pretty clear dichotomy. I don't think we need to reinvent this wheel. The people who decide whether a drug gets approved in Israel is not people in parliament or in the Knesset, right? It's Israeli scientists. But the government is involved in approving them. Exactly. We can emulate that. And Elon can tell us all about how we deal with cars. I don't want to dwell on it. I just think this is not rocket science. We can put these standards in place and then we can continue.
[0:35:14] Doing all sorts of wonderful stuff with AI tech. Some stuff will wait a little longer for it, but it'll be worth the wait because it'll be safe. The last thing I just wanted to say is I think for us to really seize this agency and work together, not just between companies, but also across borders, we really need a shared positive vision. And this is something I really appreciate so much about you, Elon. I have the guts more than most to talk about like really audacious, positive visions for the future.
[0:35:42] What is it that makes people collaborate and put little squabbles aside? It's always a shared positive vision where people realize that if we collaborate, we'll all be better off. Technology is not a zero-sum game. You can take countries like Sweden and Denmark that had been killing each other for centuries and flat GDP, and then technology came along and they both got dramatically better off. It's so obvious that the technology that you're building
[0:36:12] Your building can make all humans just spectacularly better off with AI. And not just on this planet. I like the list before we started you had there about how it can revolutionize healthcare and so many other things. And I would add to that, why limit ourselves to this planet when there's this beautiful universe where life can flourish for billions of years? The more we can just appreciate how huge the upside is, if we do this right, the more incentive.
[0:36:40] We will have to actually be a little patient where we need to be a little bit patient, make everything safe, and therefore get this wonderful future. Well, maybe we're sort of lagging behind in our social and political development. This is like having nuclear technology in the Stone Age. What do you do with it? Which I think is more or less the thing that I talked about in terms of the pace of development, the pacing, basically, what solutions we need to put in place to...
[0:37:07] Maximize the benefits and limits the risks. I think we're there. I think we should have a meeting of minds. You were in the Senate the other day. I presume you were talking about this. I mean, you don't have to divulge what you can divulge, but divulge what you can. Well, there were a lot of people in the meeting, so I don't think it was secret. It was just a discussion on AI regulation.
[0:37:27] And actually, Sam Altman from OpenAI was there and a number of leaders were there. And I think generally it would be a good idea to have some kind of AI regulatory agency. And you start off with a team that gathers insight to get maximum understanding. Then you have some proposed rulemaking. And then eventually you have regulations that are put in place. And this is something we have.
[0:37:51] For everything that is a potential danger to the public. So if it is, you know, food and drugs, we've got food and drug administration, we've got aircraft with the FAA and rockets. You know, anything that is a danger to the public over time, we have learned, often the hard way, after many people have died, to have a regulatory agency to protect public safety. I'm not someone who thinks that regulation is some panacea where it's only good. Of course, there are some downsides to regulation in that things move a bit slower.
[0:38:19] And sometimes you get regulatory capture and that kind of thing. But on balance, I think the public would not want to get rid of most regulatory agencies. And you can think of it also as the regulatory agency being like a referee. What sports game doesn't have a referee? You need someone to make sure that people are playing fairly, not breaking the rules. And that's why basically every sport has a referee of one kind or another.
[0:38:47] That's the rationale for AI safety. And I've been pushing this all around the world. And when I was in China a few months ago, meeting with some of the senior leadership, my primary topic was AI safety and regulation. And they, after we had a long discussion, agreed that there's merit to AI regulation and immediately took action in this regard.
[0:39:09] So sometimes we'll get this comment of like, well, if the West does AI regulation, surely then what about what if China doesn't and then leaps ahead? And I think they're also taking it very seriously because, you know, the opposite of whatever moral constraints you program into the system. I mean, Greg, I don't know what you think. What are your thoughts on this? Yeah, I mean, I think, again, it's just like this technology is almost backwards from how you'd expect it to work. Like I think that a lot of people talk about the paper clips.
[0:39:35] If you think of that as if we literally program into a machine, make as many paperclips as possible, and it gets really smart, and it's like, well, there's all these houses and factories and other things that are not producing paperclips. They're all going to be just paper production now. All these humans that are in the way, paperclip would be a better arrangement of their atoms. And that's just not the technology path we actually seem to be on.
[0:39:55] You look at GPT-3 and it is a system that gets the nuance. It gets the common sense. Now it has other problems, right? It makes mistakes that no human ever would. It like kind of loses its chain of thought and says all these weird things and sometimes make things up. Like we have real problems we have to encounter. But I think that for me that the hard thing that I think gets lost in some of the conversations is the nuance, right? Of really being coupled to what's the technology we actually see and particularly how does it depart from our intuitions about where we thought it would go.
[0:40:21] Actually, it's even interesting, you know, Max mentioned the pause letter. The pause letter, one of the key parts of it was to not train GPT-5 for the next six months. And we looked at that and we're like, well, we're not training GPT-5 for the next six months anyway. And so, you know, it's just like, again, if you kind of miss the sort of nuances of what's going on at a mechanical level, and some of this does require getting like really in the weeds and coming and talking to us and also opening up and showing kind of our in-progress work. And I think that's something.
[0:40:48] That will be very important as you think about moving to these very powerful systems. And I also think there's another blessing and curse of how the technology is playing out, where we talk about digital super intelligence, and we also talk about chat GPT. These are very different things, but somehow there's going to be a smooth continuum between them. And so I think that the blessing is that we see a little bit into the future. We have a little taste of a system that, you know, I talked about a
[0:41:11] You know, personalized tutor for everyone. You kind of have it. Now, it's not a perfect tutor. It's got a lot of problems, but like, you know, you can go use ChatGPT for free to ask about anything. It'll give you something sometimes actually pretty insightful. Well, fair enough. I'll withdraw the seven, seven and a half, but you'll get there. You'll get to a 10. We will. We will. We will definitely. That's right. And I think it's great for people to see it while we can still laugh about it.
[0:41:34] Right. I think that when it starts coming up with new ideas, you type in some problem you're having at home, trying to figure out how to do something. And it just comes back with, oh, you should say this to this person and you should do that. You should, you know, here's how you should rearrange your schedule. And it's just like great ideas. Like that's going to feel so different. And then one day when we're actually having a system that can actually just autonomously go off and do a bunch of research and say, here's this crazy chemical formula, like please mix them together in these ways.
[0:41:58] You want to make sure that thing is trustworthy, right, before you start mixing those chemicals. And so I think that this question of how do we separate out the sort of diffusion of today's AI technology and application and really, you know, all the debates that we're having, let those happen. Open source technologies like everyone's building, we should not squelch those. We should actually promote those. But I think that also looking ahead, being ahead of the game here and saying there is going to be a point and it's we're hitting that wall where we're going to be building the most massive machines.
[0:42:26] Humans have ever built. Like Elon talks about these massive rows of data centers with all these computers. Like it's one of those things where you kind of have to see it to really feel it. Like I've walked through some of these data center halls and you just get a sense of the scale of what's being built. And it's very different when it's abstractly just like you're looking at, oh, is this many GPUs? And, you know, looking at numbers, it's like you just realize the mechanical force that is behind what you're building for cognitive intelligence. And I think that kind of thing we need international answers for.
[0:42:55] I fully agree with you. I think, first of all, you know, with all due respect to the pause button, we ain't going to get it. It's not going to happen. I think you can't put the genie back into the bottle. And I'm not sure you can slow it down. The only thing you can do is just accelerate our own efforts. And I think you're right. It has to be international. I think you start with a like-minded and like-valued states. Start with that. And frankly, I think...
[0:43:18] Obviously, you know what AI stands for. It stands for America and Israel, obviously. I'm just plugging my own country, but it's not an empty plug because I think that there's a concentration of talent in Israel that makes it more than a mouse compared to the American elephant. It's more than that. It's a very nimble mammal, very fast, compact, and it's also got this feed of military AI which moves very fast and takes a very large part of the population. The talent's there.
[0:43:47] It screens it and trains it very quickly. So I think that you start with the like-minded states. You definitely should have a very robust conversation. First of all, decide among us what is something. There are things that we wouldn't want to share, and it's obvious, related to our security. There are standards that we have, ethical standards and other regulations that we have to achieve quickly between industry and government, quickly. And then I think we have to conduct a robust discussion.
[0:44:17] So here at Tesla,
[0:44:46] They have a different name for pause button. They call it the break. And when we launched this pause letter, I personally didn't think that it was immediately going to cause a pause. For me, the real purpose of it was to mainstream the conversation about these issues, which really did happen. And I really appreciate you and Sam and your colleagues for saying very openly that there may come a time when you will want to pause certain things if you meet certain criteria.
[0:45:13] If Elon says that there may come a time when he's going to slow down his Tesla, I would feel, I would trust that more if I knew that Tesla actually had brakes in it so that they had like a mechanism by which he could actually slow it down. So this is something I'm very curious actually to hear from you about. What criterion would you say that you all have? Can you say it?
[0:45:35] In the future, say it already now, what are some things which would make you want to pause? And then won't it be difficult with the actual mechanism to resist commercial pressures to continue, even if you and Sam want to pause? Well, I actually kind of agree with both of you, right? That I think that my picture of how the future is going to play out in terms of AI development is not as simple as, oh, you're scaling models, and then you see something a little bit weird, and then you like...
[0:45:59] Stop all scaling and you all go home or something, right? It's definitely not going to be that. I think that we need to be making maximum velocity efforts towards a good future, right? And I think that is something that you don't want to pause. You actually want to put the foot on the pedal. Now, of course, the question is, what actions actually get you towards there? Which ones are against it? I think that nuance is something that I think is what we should be debating. But that's why I actually bounce a little bit off of...
[0:46:24] The framing of a pause, because I think even to the point of GPD4 doing all this prediction work ahead of time, like if that prediction work was a little bit wobbly, we weren't quite there. And we actually do this all the time that there was like some experiment that like came back a little bit strange. We're not really sure we understand it. We actually spend a lot of time to fix that. And it's not even, you know, even if you forget about the safety reasons, and actually one of the reasons we do it is safety motivated. But if you even forget about that, you're just not going to build a good system. And so I think that there's a lot of places in AI.
[0:46:50] Where there's actually great alignment between capability and safety, between building something useful and something that is safe. It's not perfect, but I think it is actually surprising where there are divergences. Actually, the single biggest divergence we've seen has not been, oh, too much pressure to deploy quickly. Because I actually think that getting things out, like you can see the market, it hates hallucinations. It hates it when the model makes something up. All the jokes we made about, oh, it doesn't give you good insights. The market hates that stuff, right? And actually, I think it's very aligned with safety to fix them.
[0:47:17] What the market actually pushes us against is actually when it comes into conflict with other values. For example, privacy, right? So that the way we launched the initial GPT APIs was that we said, hey, we're going to monitor everything. We're going to record everything. Like if something sketchy comes up, we'll go and look and we'll be able to look back in the log. We'll be able to see what you did. People hate that, right? Like, you know, you kind of want that for everyone else, but for yourself.
[0:47:38] You want privacy. And so how do you balance those two factors? And we actually see this very much in terms of to actually win deals, to decide to be able to give companies the assurance that, hey, we're not going to retain any of your data. We'll delete it immediately. That removes our ability to do any investigation post hoc. And so this is actually a place where I think government can be very helpful, where norms can be very helpful to say, hey, even though this is what would be commercially most viable, that this is what the rule is going to be. And everyone has to play with it. You don't ever race to the bottom.
[0:48:08] So I think that, again, it's all going to be nuanced. It's all going to be, I think, like I've just accepted that this field, it's totally counterintuitive. Like everything we learn just is like, it just would make you just like, if you told yourself 10 years ago, you would have said, there's no way that sounds totally crazy. And somehow we have to navigate through that. Well, the only thing that I can add to this conversation is to invite you all to Israel. I'd like to, in fact, continue this conversation. In fact, bring a few more of your colleagues and a few leaders to discuss this.
[0:48:37] International strategy for like-minded states. To begin with that, you know, first like-minded states, then the world, okay? How about that as a plan of action? But I think that the work that you are doing is amazing, nothing short of it. You know, Elon was saying, what is a human being? It's sort of this two-legged thing, ape-like creature with a cortex and a computer and its wants, desires.
[0:49:04] And a brain, but the brain is developing in ways that we couldn't possibly imagine. And it just doesn't stop. Well, from my earlier comments, the thing that perhaps should provide some hope for digital, not being controllable in a very specific way, but acting in a way that is beneficial to humans, is that our cortex and our limbic system, the cortex is actually in service to the limbic system. So the cortex, the thinking part, is constantly trying to make the instinctual part happy.
[0:49:32] Even though it is much smarter than the instinctual part. And so I think there's hope there that digital superintelligence, even though it is much smarter than our cortex, will nonetheless want to make the cortex happy and ultimately the limbic system happy. I mean, not to get too off color here, but the sheer amount of thinking that people have done to have sex is a lot. But not sex for procreation, just to make the limbic system happy. So, you know, given how much...
[0:50:00] Thought has gone into getting laid essentially for the sole purpose of getting laid, not procreation. And all that does is make the limbic system give a spark of joy. I think that actually suggests that there is a good future where even if we don't understand the digital superintelligence, just as the limbic system does not understand the cortex, it might still actually at the end of the day be trying to make our limbic system and cortex happy. Is that a happy note to end? I think it's a great note to end on. Thank you. Thank you, everyone. Thank you.
[0:50:29] Epic, so it's really fun.