[0:00:13] Boom, what's up everyone? Welcome to Simulation. I'm your host, Alan Sakian. Very excited to be talking about safe, artificial, general intelligence that benefits everyone. We have Greg Brockman joining us on the show. Hello. Hey, glad to be here. Thanks for coming on. Really appreciate it. Love your work. You're the chief technical officer and co-founder of OpenAI, which is...
[0:00:40] Doing incredible work in artificial intelligence, safety, security, building artificial general intelligence. And previously at Stripe for five years, prior to that doing a lot of work at Harvard and MIT, building up the math and computer science skills that you now have today. So this is a very pressing time, probably like the most pressing time that we have is building out.
[0:01:07] Passing the torch from the biological to non-biological intelligence. What is your current take on the state of humanity? I guess the way that I think about it is I don't think about it as there's a torch to be passed here. The purpose of technology is to enhance human lives. You think about the march of progress even starting with the steam engine and all of the wonderful technologies we've created over the past hundred years. Think about the computer. Think about the internet. Why do we like these technologies?
[0:01:36] And I think that when technology serves people and enhances the human experience, helps us create a world of more plenty, a more equitable world, it's good. But it doesn't always work out that way, right? Sometimes you look at more recent technology, and I think that we've introduced new societal questions that now just someone with an idea can generate such wealth, such value, that you end up with a very inequitable system. And so looking forward, if you can actually build smart systems, if you can really build...
[0:02:05] Machines that can automate human intellectual labor, which is the kind of thing we're talking about when we say artificial general intelligence. That all of those questions that we've seen are just going to be amplified. That there's this potential for this huge benefit. You think about what are the problems that humanity faces that we're just not really seeing answers on the horizon. Think about health care. Cheap health care for everyone really works. Think about human doctors today. We're addressing symptoms.
[0:02:35] We can't go and do root cause analysis, right? There's no human doctor who can go and look at your genome and say that these are the specific risks that you have and so starting today at this age that you should start eating these foods or exercising in this way or you should do these preventative things. There's no human doctor who can see every single health record of every patient across the entire world and be able to correlate all of these treatments with outcomes. Those kinds of questions, how are we supposed to solve that?
[0:02:59] We're just so many steps away. Right now, we're just stuck in this world of trying to figure out how we can get our medical records transferred between systems. And so I think that we have this huge opportunity. We're on the cusp of having technology that can really enhance the human experience beyond anything we've seen to date. And we're excited about that. Yeah, interesting.
[0:03:19] When we think about it in this big history perspective, we've always been developing technology that has enhanced human life. And this is another one of those technological advancements that we're aiming to make as enhancing of human life as possible, as benevolent and benefiting as possible to everyone, instead of thinking about it as a passing of a torch. That's right. Absolutely. Interesting. Okay. Okay. And then...
[0:03:46] So then how did you get to this point of being here five years now, developing, you've went and given, you've delivered statements to congressional committees on the importance of artificial intelligence. So tell us about how you became Greg Brockman that we see today. Well, I actually started out as a very different Greg Brockman. For a long time growing up, I thought I was going to be a mathematician. And I read these stories of Galois and Gauss.
[0:04:16] And they were operating on this 100, 200-year time horizon. Galois was this young mathematician. He actually ended up dying in a duel when he was 21. I didn't want to emulate that part. But the part that I did want to emulate is that almost out of nowhere, he developed Galois theory. And it really took another 100 years for mathematicians to build up the machinery where they really should have been able to develop Galois theory. And so that was how I thought. I was like, I want to do abstract mathematics where it's never used in my lifetime. If anyone uses it, it means that I wasn't thinking,
[0:04:46] Deeply enough. I wasn't abstract enough. Because I just loved the idea of that deep impact. The impact that's going to last hundreds of years. And really that's going to help set the future for humanity. And that was kind of the best way I could imagine to contribute. And after high school, I took a year off. I was writing a chemistry textbook because I had happened to come up with some very mathematical way of thinking about chemistry that I wanted to immortalize. One of my friends said, you're never going to get this thing published. So I learned to program to build a website.
[0:05:15] And the thing about programming is that you do a very similar process to mathematics where you think hard about a problem, you write it down in obscure form rather than a proof it's a program, but then suddenly anyone benefits from it immediately. That feedback loop, the idea that you could build this castle in your mind, and then suddenly anyone can live in that castle. That for me was just like, why didn't anyone tell me about this before? Instead of this multi-hundred year timeline horizon, it's an instant deployment to billions of people. Absolutely.
[0:05:45] It's such a powerful thing. It's such a powerful thing. And I think we still don't really appreciate how beautiful that is. And so at that point, I said, I'm just going to focus on building. I want to build software. I want to become as good at programming as I can. And so I showed up at Harvard for undergrad, for freshman year. And I discovered the Harvard Computer Society, which was a group of people who just want to build services for the Harvard community and got really involved in that sophomore year.
[0:06:12] Suddenly I was running the club. The two seniors who knew what they were doing, who had spent all freshman year teaching us things, they graduated. And I was the one who was supposed to teach people things. And I was like, I'm not ready. I'm just a sophomore. I kept looking down the street at MIT where there were all these people who I wanted to learn from. And so I ended up transferring. I was there for another year. And throughout all this, I was working on the thing that I felt was going to have the highest impact, which were startups. And so I tried a bunch of different things and never went anywhere. But from each one, I learned another thing not to do.
[0:06:42] And I ended up meeting some people working on this payments company. It was like two or three people at the time out in Palo Alto. And when I met them, I realized these are the people that I want to work with. These are the people that I've been looking for the whole time. And so I was in the middle of a semester. I had this whole path charted out for myself. And I said, never mind. I'm going to drop out and go work on this. I didn't think it would succeed. I expected that we'd work on it for a year and we'd fail and probably be back in school and have to retake my classes.
[0:07:12] But I just knew that these were the right people. What was it about them that made them the right people? For me, it was just we really clicked in a way that I think is very rare. And I think that we have very complementary skill sets, but also overlap in terms of the way that we wanted to affect the world. I think one thing for me was I was 21, 22, and I had this belief that as a 21, 22-year-old, you can't really do real things yet. You have to be in school. You have to go to grad school, come with something cool there.
[0:07:42] Turn into a startup if that's the route you want to go. But 25, 26, you can do stuff. Not yet when you were my age. And these people were my age. And they were already out here. They had already started to build a company and had raised some seed around. And I was like, one of the two of us is wrong. One of the two of us has completely wrong. And I just want to know who it is. And so in some ways, that questing to kind of probe.
[0:08:07] The different systems in the world and to see how can you have impact and how can you have outsized leverage, that was something that really drove me. Yeah, that's so, so important to get to greater local maximums of intelligence and drive to make new systems that obsolete old ones that maximize.
[0:08:32] are society's potential and a great place to do that is Silicon Valley and a great place to do that is with people that are building something like Stripe, like something regarding payment deployments. That's a big idea. So then you guys wrapped on that, you wrapped on that big idea. Stripe is still roaring. Stripe was purchased, right? No, no, no. Stripe is still independent. Okay, Stripe is still independent. So then what was the transition from Stripe to OpenAI?
[0:09:01] So for me, I think in that transition period where I was going from thinking on this 100, 200 year time frame of lay some mathematical foundations and have impact that way to you can actually build stuff. I discovered Turing's 1950 paper on the Turing test. And it's this wonderful paper. Have you read it? I haven't read the exact paper. I highly recommend it. And there's really two big ideas in it. One of which everyone knows and the other...
[0:09:28] I think does not get played. The one that everyone knows is the Turing test. The idea of, can a machine think? Turing's like, I don't know what that means. So instead, I'm going to devise a concrete test that seems just about as good, which is, can a computer fool a human into thinking that it's a human as well? That it thinks. That's right. And so it's like, thinking is not really operational. Everyone has a different definition of it. But this test is very concrete.
[0:09:52] And if you can actually operate, you can answer all the questions. And over some text channel, I try to teach this other being calculus, and it learns calculus. Sounds like thinking, right? So that's the idea that everyone's familiar with. The idea that people aren't that really captivated me was Turing said, how are we going to pass this? How are you actually going to build a system that can pass my test? He said, it's going to be too hard to program an answer. So instead, we're going to have to learn an answer.
[0:10:21] And he says that imagine if you could build a machine that learns just like a human child, that you program in whatever rules you need to have that learning process. And then you've got to grow up. You've got to experience the world. You've got to have 20 years of experience in an interesting environment. This is a way that you could actually get all of the crazy things that are required to pass that test into a system. And for me, what I realized was that here was the key. You think about programming.
[0:10:49] It's limited by what I, the human programmer, can understand. I have to understand the process for how a domain works. You think about building a payment processor. You have to understand all these details about where you're going to move the money in, all the different error codes that could come back. You write those all out. And then you have an automated system that executes it. But that core understanding of the process has to come from a human. But with learning, I, the human programmer, can specify the goal.
[0:11:19] And the details as to how it actually works, that's something the system can figure out for itself. And to some extent, this sounds abstract, but we experience it all the time. You think about children, and how do children learn and grow? And as a teacher, you don't get to program in all the rules. You don't get to say, here's how you have to think. But if you're a good teacher, if you're doing a good job, you can help that child.
[0:11:44] Learn the skills that they need and the right ways of interacting with society and interacting with others and doing good things for the world. And so I think that that idea is a really core powerful one. And is that both of the pieces? Those are the two pieces. So here's the scoring test, but the thing that I think is undersung is the way we're going to pass it is through learning. Oh, okay, okay. So then the thinking machine would need to learn in order to pass.
[0:12:14] Along the way. That's right. And so the idea of human programmers, we can only push our system so far. There's a limit to the problems we're going to be able to solve. And we do great things. Don't get me wrong. You look at all the software systems around us today, it's pretty amazing. Imagine describing that to someone in 1900. We're going to have all that. We're going to have the internet. No one will believe you. So this is a buildup of thinking capacity.
[0:12:38] That is really important for machines and artificial intelligences. That's right. In a more general, potentially, the better at reasoning in a human society that is also very general and complex. Yep, exactly. And the way that I would think about it, just to add to that, is that you can think about the systems we can build today are very brittle, very narrow. The systems we're all used to, you have a computer, and if anything goes wrong,
[0:13:06] It crashes, and you get a blue screen of death. We've kind of hopefully moved beyond blue screen of death in particular, but that same idea of that there's just some programmer who forgot to add some if statement, and now it's all broken. It's not really able to handle new circumstances. And so the way you can think about it is that the computer systems we can build, which really do benefit us, that there are huge benefits from these technologies we've already built, are just such a narrow slice of all the computer programs that could be out there. And so the question is,
[0:13:35] Are there new technologies that we can add that increase the space of these beneficial programs we can build? And that's where I think this really powerful idea of having the machine learn the process, have the human specify the goals. We should be in the driver's seat, but why should it be that I have to spend all my time really looking through the details of how all these little nitty-gritty pieces of the system should work? Specifying the goals and then having the thinking slowly become...
[0:14:05] Better and better over time. So it's constantly running iterations of thinking until it gets closer and closer to the goal, and that's how it's building itself. Well, and it might be instructive to look at today's AI technology and how it works. So if you look at the history of the field, starting in the 1950s is kind of when the term AI was coined, and when people really started to set out on this journey of trying to build machines that can
[0:14:35] Be smart. They can pass the Turing test. They can automate intellectual labor in some way. And there were two big camps. There was the camp of, this is a little bit of a caricature, but let's say the symbolic systems camp, which was, let's try to write down the rules. And they ended up building the expert systems. There were various other approaches of just having machines that can search through lots of things, try to encode some human knowledge in there. If you want to learn about language, well, let's encode how parse trees work.
[0:15:04] And then there was the learning camp. And the learning camp, in some ways, ended up really focusing on neural networks. And neural networks are this simple, simple idea that, as time has gone on and as we've gotten more computational power, have proven to scale much better than any other technique. And so the first neural net probably dates back to 1940s even, before we even had computers. It was a mathematical model of a neuron. And 1959, this person named Rosenblatt released the perceptron.
[0:15:35] It was the first implemented neural network. It was a hardware device in addition to a learning rule. And it was able to learn some very basic stuff. And in the New York Times, it was announced that the Navy funded this project. And the Navy says that one day, Perceptrons will recognize people, will call out their names, will instantly translate speech between languages. And basically, the other camp of AI people looked at this and they said,
[0:16:02] Your machine can't do any of those things. It's overhyped. This is false. You're misleading everyone. And the funny thing is, you fast forward to today, and neural networks are doing all of those things. Neural networks are the cutting edge of artificial intelligence technology. That's right.
[0:16:19] So in the recent history is that in 2012, so neural nets were kind of this idea that existed in the 60s that actually Marvin Minsky and other researchers who were very big names on the other side of the fence spent a long time really trying to discredit that approach. They didn't believe in it. They thought, look, you said you could do all those things. You can't do any of it. You're kind of sucking all the oxygen out of the room. You're getting all the funding. We need to put an end to it. 1969 published a book that actually did curtail research for about the next 10 or 20 years.
[0:16:48] So that was the beginning of the first AI winter. Fast forward to the 80s, the democratization of compute hardware caused this resurgence of neural nets. And suddenly, all these people were trying these different things. They developed a better learning rule called back propagation. They developed a lot of really awesome things. But once again, somehow there ended up being a winter where people got really excited, weren't able to make the progress that they were hoping. And so now you fast forward another 20 years.
[0:17:18] 30 years to 2012. And in 2012, something really magical happened. So two grad students at University of Toronto, so one of them is my co-founder, Ilya Setskever, the other is Alex Pachevsky, and their professor, Jeff Hinton, were able to train a neural network on two GPUs, which are just these gamer cards, literally developed so that gamers can have really fast graphics. They were able to train deep neural network to recognize images.
[0:17:48] So is there a cat or is there a dog in this image? Better than anything else, by a huge margin. So you imagine what's been going on is that for the past 40 years, all these computer vision researchers have been trying to figure out, how can I recognize if there's a cat in this picture? And so they come up with all these rules of, oh, maybe if there's an edge here, and if there's a whisker and a whisker, but maybe one of the whiskers occluded. And so you could try to make all those rules work. And they built these very brittle systems that just didn't really work. And then two grad students with just $2,000 worth of hardware.
[0:18:17] Are able to blow all that out of the water with a system that it just started out knowing nothing. And you just showed it examples. And you just keep showing examples. And something gets really good at this task. And so that was 2012. Everyone was like, whoa, this is a wake up moment.
[0:18:33] This is supervised. This is supervised learning with neural networks. Because there is a decision of, yes, you identified correctly or not that there's a cat. That's right, yes. And so the data set that you have has a bunch of labels. And in fact, the big data set that everyone uses, as used as the benchmark, is called ImageNet. And when ImageNet came out in 2010, it was just viewed as intractable.
[0:18:56] It was like millions of images that were pulled from my Google image search and a bunch of other things. And it's like these pretty high resolution photos. And they have 1,000 different possible things that are in there. It's all sorts of varieties of dogs, and cats, and motorcycle, and 1,000 different categories of things. And so all these computer vision researchers using the traditional non-neural net based stuff were like, this thing is totally impossible. We're going to be working on this for the next 10, 100 years.
[0:19:24] And two years later, you know, so I think 2010 is I think about when ImageNet, I think they first started running the competition. That might be slightly off in terms of the years. Two years later, this neural network comes out from these two grad students and they're able to just push so far. They move the state of the art to like, you know, I think it was maybe like 30% of the time there would be a mistake with the old stuff and then 15% of the time there would be a mistake with the new stuff. And that gap is actually very, very significant. Huge, yeah.
[0:19:53] Because the ones that you don't make mistakes on are the easy ones, right? So now you're going to the pretty difficult ones. And everyone's like, all right, we've got another 20 years worth of research to go from 15% to really solving this data set. But actually what happened was that every year from there, there was this exponential decrease. And now we surpassed human level on this data set probably 2015, 2016. And now it's basically this particular task is viewed as obsolete. It's solved.
[0:20:22] And the funny thing is we still have more to do, but this data set is totally saturated. Interesting. So there's a certain amount of human level tasks that are already become obsolete by artificial intelligence. And so there's a little asterisk too.
[0:20:42] And so here's where things start to become very subtle, and you really have to look at the details. And I think one of the hard things about figuring out what's really going on in AI today is that simultaneously, the details tell a very interesting story. But when you really zoom out, the narrative looks very different. So when you really look at the details, we have not solved computer vision. Yeah, yeah. Otherwise, we would have fantastic vehicles that could drive themselves. That's right. All this other stuff. That's right. That's exactly right.
[0:21:10] And these neural nets make mistakes that a human child never would. That it maybe misidentifies an object as a human or something. Exactly, right? And just like, you know, you think about like even a small child is able to never make a mistake of thinking that a human is a cheetah or something, right? It just never happens. And so we have to solve that problem. But at the same time, if you look at the contrast as to where we were, it's night and day. Before, we couldn't even, we didn't even have a place to start.
[0:21:40] And so I think that if you zoom out, that the story, again, is a really telling one. So first we had computer vision. And basically everyone else in AI looked at that and said, cute trick, worked for your field. My field, way too complex. These neural nets that start out with no knowledge and just learn the answer, never going to work, especially for machine translation. Can you imagine, for machine translation, you think about what you need to understand. You need to understand languages and syntax and all these things.
[0:22:10] It actually turned out, 2015, that neural nets had surpassed every other technique in machine translation. And it was like there was this march of computer vision, speech recognition, and a whole host of others. And so this is general. This is generality. This is, if you have a labeled data set, so you have your inputs. Maybe they're images. Maybe they're sentences in one language. And you have your outputs. Maybe they're, is there a cat in this image?
[0:22:39] The same sentence in a different language. We now have the technology that can learn to map one to the other. Yeah, that's massive. Okay, so any data set that has to be structured, it can be unstructured. Yes, so that advance that I refer to, that's supervised learning. There it has to be structured. And the thing that people have been saying is that, well, if you look at what data is out there,
[0:23:10] There's huge amounts of unstructured data on the internet. Huge, huge amounts. But we can't use it for anything. The only data that's useful for these systems is labeled data. Is labeled data. That's right. Okay. Okay. And that would be very interesting to figure out how to train artificial intelligence to maybe label data. Yeah. That's interesting. Yep. So, okay, so now you, so basically with any, right now we're trying to make it so that with any inputted,
[0:23:37] Structured data and a goal that in mind we can get outputs through a neural network. Yep. And so how did you and Ilya go like, let's start OpenAI, let's have this charter as well. Let's be very ethical, heart-centric. Let's really be thinking about this on a benefiting everyone around the world level. Yep, exactly.
[0:24:07] So in 2015 is kind of when I felt like I'd gotten Stripe into a very good place, and I could really start to think about how can I have the kind of broad impact that I want. I think within Stripe, pretty amazing, today it's a $20 billion company, and I could contribute to that. But I think what I really like to do is try to find problems that I think will not work, will not play out in the right way without my involvement, where I can really try to steer them from one place to another.
[0:24:35] And AI had always been the thing that I felt was the most important problem. It was just a question of when. And after three years, 2012 was when Ilya had that breakthrough. And three years later is when we were starting to think about this. We actually had this dinner with Sam Altman and Elon Musk and others. And I think that the question there was really, is it too late to start?
[0:25:03] A company trying to make AI play out rather than in a negative way, in a positive way. Is this something that you can actually do from scratch? And the conclusion from the dinner was, it's not obviously impossible. But you're competing against the top talent at Google and in China. This is very, very tough stuff. Apple, Facebook, Microsoft, Amazon. And they can pay 250, 500 millions of dollars a year for these positions. So how does one...
[0:25:31] Come up into the rankings. These were the questions that we had to ask, right? And really, I think that AI development is not like a normal science in some ways. You almost think of it as very underfunded and very kind of backwater, right? And it's like, neural nets, in fact, were this for a very long time. The only reason that we have what we have today is because this Canadian moonshot organization was willing to fund a couple of labs to go and work on this stuff. No one else was willing to do it.
[0:25:58] And today, it's very different, as you said, that lots of corporate labs are involved and that there's a large number of people who are working in this field. And so to have a differential impact, it's pretty non-obvious. But we felt that this technology is just so important. What I described is supervised learning, but if you can really make unsupervised learning work, you can really learn from all the data around us.
[0:26:23] Like that suddenly unlocks a whole new host of problems. And if you can really build an AGI, if you can really build machines that can automate, you know, meaningful amounts of all of human intellectual labor, it's going to affect the world more than any technology we've seen. And so if you believe that that might be possible, it's really hard to sit by idly. Yeah, you want to dedicate your life to it. That's right. And so, you know, kind of it became a question of tactics, right? And it's like we have a mission, right, that we knew was important. We want to make sure that AGI benefits everyone.
[0:26:54] That the world that we deliver, we in the sense of humanity, not in the sense of any of us as individuals, to the next generation, to the post-AGI generation, is one that is equitable, that is a better world than the one that we're in today. And I think it's very achievable. I think that AGI can be the most beneficial thing ever. I think that, again, you think about the amazing applications.
[0:27:21] And how many people were saying, oh, we'll be weary about planes or be weary about antibiotics or be weary about all of these advancements. And so it's equally as important to push through the adversities because it does end up catalyzing a lot of reduction of suffering and increase of health. That's right. And I think it's really important to be realistic. If you're developing a transformative technology, you've got to think about the ways it can go wrong.
[0:27:51] And you've got to think about the ways it can go right. And then your job becomes trying to sift those out and trying to figure out what path do we take that minimizes these downsides, maximizes the upsides. Yeah, yeah. Okay, so then how do you figure out how to minimize downsides, maximize upsides? You know, you made the charter. And the charter is a, would you say, kind of like an ethos of OpenAI. That's right, yeah. And so it took us some time.
[0:28:20] You know, we started out with a team that, you know, I'd done all the recruiting for that, and we brought together a team of about 10 people to start, which is a really weird thing to do as a new company, right? Normally it's just you and a co-founder, and you've got some time to build a product and get some users. But here we needed a critical mass of these people to push forward AI technology. And so we spent about two years really trying to figure out how do we want to operate? What is our strategy? What is our path?
[0:28:47] To actually making the future be better than it would be without us. And the OpenAI charter is, in a lot of ways, an embodiment of that thinking. And so there's four sections. It's very short. It's on our website. Anyone can read it. And I think that there's a few points that are pretty non-obvious. The first one is that the core mission of OpenAI is to ensure that AGI benefits everyone.
[0:29:12] And in terms of how do you actually implement that, we can talk about that in a bit. But I think it's just like AGI is just going to be so transformative and it's going to be so impactful that it shouldn't be something that's 100x more concentrating of wealth than anything we've seen before, 1,000x, 10,000x, whatever the number's going to be. That creates a lot of instability in civilizations if that is the case. It does. That's not a good world. I don't want to live in that world. It doesn't matter which position I sit in. I don't think that's a world that's good.
[0:29:41] There's a lot of beautiful creativity in people's minds that can be unleashed, that we would love to see unleashed. That's right. That's right. Yep. And so there's a second point, which is as we talk about safety and security and what's it going to take to get to that good world. Well, one thing we're very concerned about is, first of all, people not investing in safety research. And the more that there is an arms race to get to the end, you have a bunch of different companies or countries.
[0:30:09] Other actors that are trying to build a transformative technology. What's the first thing that's going to go out the window? The safety. Exactly. And so we actually have built into our charter a pretty weird provision, which is that if someone else is pretty close to succeeding at building AGI and they're value aligned, so they kind of want the same thing that we do, we'll stop competing with them. And we'll actually help them deliver on that. Because the core, like one thing that I think is really important about our mission is that
[0:30:38] We don't actually have to be the primary actors. We don't have to be the ones to make AGI benefit everyone for AGI to benefit everyone. And that's something I think really sets us apart from other structures, from other companies. Yeah, well, every company is trying to figure out how to maximize their vision, but they don't think most often about it. Like if another company's...
[0:31:05] They're doing that same vision, and they're doing it potentially better. We can maybe merge with them and help them. Yeah, that's, yeah. Yeah, and so one thing that, yeah, it's a pretty non-typical thing. Yeah, yeah. Yes, okay, so then there's that safety component, of course, and then there's fourth as well. That's right. Well, so we have, you know, I think it might not be too important to break out all of the provisions here. But the technical was that.
[0:31:34] The fact that you have technical professionals doing it as well. Yes, and this is the core strategy. The core strategy for OpenAI is to push forward AI development on that exponential until we successfully build safe AGI, which is, there's really three components that go into that. There's a technical piece of, how do you actually make this technology work? And that's hard. And how do you actually...
[0:31:59] Like figure out how to build a general intelligence. Yeah, this is the hard problem, right? And so for us, a lot of what we do is we have a number of different teams that work across different domains. And to some extent, the actual problems we work on are secondary because what we try to do for any problem is we're trying to develop general purpose technologies that can solve a problem that no one's been able to solve before but are applicable across different domains. I have a question. Yes.
[0:32:25] If your intelligence that you're building can maybe switch between intelligent tasks that are considered narrow intelligence, would you consider that a step towards a more general intelligence if it's able to do two or more narrow intelligence? I think this is a step, and I think that there's this core technical problem called transfer learning that people haven't quite figured out yet, which is how can you use knowledge from one task or domain in a new task?
[0:32:54] Humans are great at this. I read a book on Paris. I go to Paris. Suddenly, I'm better at Paris-related tasks. Our AIs don't do anything like that right now. But we're actually starting to see, with some of the models that we've been working on recently, we're starting to see inklings of this. So one example is this model called GPT-2, which is a model that was trained on internet data to just predict the next word in text. So you show it at the beginning of some snippet of text. You say, what's the next word? It just plays that game. It gets really, really good at that task.
[0:33:24] And then- It's like an n-gram now? Somewhat? Well, so it's an n-gram for a very large value of n. Yeah. Right? So the context here are like thousands of words. And so it turns out this model is able to, first of all, you can ask it to generate text for you, right? Because you just show it some snippet, and then you say, what word should come next here? What word should come next here? Right? You keep feeding its output back into itself. And so it's able to write text. And the text that it writes ends up being coherent paragraphs, coherent pages on any topic you want.
[0:33:54] We have this example that I think was very visceral where we prompt it to basically say, recycling is good for the world. No, you cannot be more wrong. And then let the AI complete it from there. And the AI writes this very convincing essay on why recycling is actually not that good. This was very interesting. And also, one of the things that when I read that that came to my mind was that maybe it could potentially be that
[0:34:22] The idea is that we're not thinking about it from a first principles perspective. You should think about it from a first principles perspective of the way that you transfer goods and packages should not even have a component that goes into a recycling in the first place. That's right. It generates very good discourse. Now when I see a recycling bin, I'm like, why are we generating all this waste in the first place? The thing that's amazing about this model is that's an instance of transfer.
[0:34:49] It used this knowledge from this task of predict the next word, and it can suddenly apply it to a story, to an essay, to a debate. And here's where things get a little bit unfortunate, to generating fake news, to generating abusive content. And so this was the first model where we explicitly said, we're not sure that this should be released.
[0:35:16] Okay, this is an important point. There's periods of research and development that occur at companies when they realize that, like, okay, we may have invested lots of time and effort into something, but we can see that there is potentially more malevolence that could come out of this than benevolence. So what we need to do is we need to, how do we package that in a way that prevents other people from leveraging it to make bad acting decisions on the planet?
[0:35:45] That's right. And so the thing with GPT-2 is it's borderline. When we said this, it was very controversial. There were a bunch of people who said it was so obvious you should have just released it. There were a bunch of people who said it's so obvious you should not have released it. Which, given the fact that there are these two camps, I think definitionally it's correct to be a little bit hesitant and to wait a little bit and to make sure you're doing the right thing. Because taking an action that might be negative, it's kind of hard to roll back.
[0:36:11] And so I think that where we are with AI today, and this is one part of our charter as well, is saying that today we publish most things we do, but in the future we're going to have to decrease that due to safety and security concerns. And this was one concrete example of that. And if you look at the security community in computer security for hacking into systems, it took them a long time. Maybe there was a 10-year period, maybe longer, to really figure out a process for responsible disclosure.
[0:36:39] Right, so you're familiar with responsible disclosure, insecurity. Yeah, so it's a really interesting concept of, let's say that you're a hacker, you're not trying to do anything malicious, you're just kind of playing around with things, and you find a security vulnerability in a website. What do you do? Who do you call? You email them? Right, so you email them, and let's say that they ignore you. They don't fix it. What do you do now?
[0:37:05] Do we want to use the only 30,000 days we have to live to continuously invest days of time into trying to help them? Exactly, right? So this is the question. So maybe one reasonable thing to do is just to announce this exploit publicly. If you just say, here's this thing, they're going to fix it. Yeah, that's interesting. Okay, so where is this going? So where this is going is that there's real questions in the security community about...
[0:37:33] How do you, when you have something that's kind of malicious, you don't want bad things to happen, you want it to be fixed, but it's not being fixed, it's not being addressed, no one's helping. How do you have a process where you do the right thing? And what the community converged on is this idea of responsible disclosure, which is you email the company, if you don't get a response, if they're bad about it in various ways, then there's a time, there's some time window at which you can then go public. And this actually is accepted. You do that, the company gets upset at you, the whole community will say, nope, you did the right thing.
[0:38:01] This is the acceptable standard. And I think that developing community standards like that in AI, we haven't done it. The thing I just described for security took us a decade to do it. It's not obvious at all. For AI, we're developing these systems very, very quickly. Doesn't that also expose it to malevolent actors online? That's right, and you've got to be careful. But if you don't announce it, malevolent actors are going to find it anyway. And it's never going to get fixed. Interesting.
[0:38:31] So you hope that maybe the public can push an action faster. That's right. And just time and time again, this has turned out to be true. Interesting. And then this seems like a pressing aspect of AI safety and security when people discover these issues. And you want to have a convention in place before you build the model that you're sure, if it gets released, is going to be harmful. Interesting.
[0:39:01] Now, okay, we were kind of hinting at this earlier. I want to see if we can maybe understand this from a more technical perspective from you that this, and whatever you can teach us, like whatever you guys are comfortable with teaching about this process. So what happens when you start, you know, you were talking about this issue with transfer, with transferring intelligence from a narrow task to another narrow task.
[0:39:29] But then you realize that you guys could write out where you could write a whole story. And so you're starting to play around with creating an artificial general intelligence. What have been some of these wide awakening moments from you of really good?
[0:39:47] Steps to take to continue moving that process forward faster. Yep. Well, so again, in terms of how we structure ourselves, we have a number of different teams. One of the teams that I led is to build bots for the very popular video game Dota 2. Yeah, you guys are competing against the world champions in what is considered to be one of the hardest.
[0:40:13] Games to balance strategy, teamwork, resource management. When you're a gamer, you really get these things about how difficult it is for an archaeologist to be able to beat them. That's right. And famously, no one can program in AI for these games. Until... That's right. So we have a system that has learned how to play. And the way that it learns is that it doesn't know anything about the game when it starts. And a human sees the game as...
[0:40:43] A bunch of pixels. You have this image and you have all this knowledge about from our ancestral environments about animals and various things. You know that there are these heroes that are running around. You know that if you kind of hurt your opponents, that's good. There's all sorts of things like that. This AI starts out knowing nothing. And that it sees the world as just a big list of 20,000 numbers.
[0:41:09] That's all it sees. It just sees 20,000 numbers. So this is not a supervised model. This is a reinforcement model. That's right. So supervised learning, I think, then kind of bleeds into reinforcement learning. In reinforcement learning, you still have a label, but the label is you took a bunch of actions, and then you got a point. Now you've got to figure out, what did I do to get that point? What did I do that I really deserve that point for?
[0:41:35] And in the case of Dota, one thing that we do is we put in some knowledge about you should get a point when you successfully defeat an enemy unit, when you go and you get this kind of item, that there's some level of rewards, which kind of provides some high-level structure to what's happening. But we don't tell the agent what it's supposed to do. And the way that it learns what to do is through a very powerful idea called self-play.
[0:42:01] This is crazy and it can play against itself at millions and millions of times. That's right. And create, as it does, it can iterate on its, oh, why don't we have this version play against, it's just, this is, yeah. That's right. And so imagine if you play against a perfect copy of yourself. Yeah. It's actually the best thing for learning, right? Yeah, yeah. Because you try something new that you've never done before. It'd be like, imagine if Alan could sit across,
[0:42:31] From Alan. And Alan would be literally challenging me to become a better interviewer. Exactly. In all the different aspects of how to become a better interviewer. All the time. And I was literally sitting here 24-7 running this process. Yes. Yes. And it's exactly that. And so it's a really powerful idea. It builds these really amazing systems. And now, the product that we built, the end result of...
[0:42:58] This research of this two-year development phase of developing the algorithms, scaling them up, and all those things is a general purpose system, which we then took and applied to a different problem in robotics. And so we have this physical robot hand, which no one can program. No one can do the old-fashioned, like, write down all the rules. Like, these hands have existed for 20 years. When's the last time you saw a robot hand in operation? Intuitive surgical? But that's not even, I don't know if that's even a hand, right? That's right. And so it's just no one can use these.
[0:43:29] But we took this system that was developed for a video game and we successfully applied it to the robot hand to manipulate a block. That went through a process of reinforcement. It's reinforcement learning. To know how to have a strong dexterity when it comes to gripping something. And it develops these grasps. There's this whole taxonomy of grasps out there. And it develops these grasps that are very recognizable. It's got the pinch grasp. It's got a variety of other ones. And that's all learned.
[0:43:58] Totally from scratch. Well, yeah, that's very human for something that's not human. And here's where the answering stuff really is, right? It's like these questions of you build these systems that can learn. Where you set the goal, it figures out the process. It figures out the details. Like in some ways, that is the most empowering thing that we can hope for, right?
[0:44:24] Humans specify the goals, don't have to spend their time trying to figure out exactly how to program. If you're at this angle with this digit, then you've got to move here. And it lets us build systems that can do things like, think about elderly care robots. Those are going to have to be able to handle totally weird and unfamiliar situations that no one could possibly anticipate and program in. They've got to be really trustworthy, but they're also going to be so beneficial.
[0:44:47] You think about we've got this demographic time bomb that people talk about. We're all getting older, and how are we supposed to take care of you and me in 50 years? And I think that this kind of technology, it gives us a path. And so I think there's something amazing there. But the thing that's really important along the way is that the impacts that we see, that they help us be more human rather than feel threatened.
[0:45:12] And then, so one of the OpenAI teams, there's 100 of you now. That's right. And one of the teams is doing the...
[0:45:22] These reinforcement learning models for Dota 2 and then applying that to other, like you said, this robotic hand. Then the other aspects of OpenAI as well. The security, figuring out how to handle things on a governance and a geopolitical level. So you teach us about that one and the other ones. Yep. So it's something as a technologist just to focus on the technical problems. So we've got this hard technical problem of can we make systems that are smart at all?
[0:45:50] There's a second hard technical problem of how do we make these actually do what we wanted? We all have heard various sci-fi stuff, but I think that there's really three classes of risk that we're worried about. The first one is systems that pursue misspecified goals, a little bit of a careful what you wish for scenario. Second class of risk are systems that can be subverted by malicious humans. So that's if you can hack into an AI and make it do bad things, how do we make sure that that's not possible?
[0:46:19] But there's a third class of risk, which I think that people really underpay attention to, which is it's possible we solve those problems. We build these trustworthy systems that do what the operator wants. But somehow the resulting economy, the resulting world, is not one that results in broad benefit to human lives. That somehow we have these systems that are doing their thing, but it just hasn't made people's lives better. And that one, that's very pernicious. We don't want that. And so we have a safety team whose job it is
[0:46:48] To think about parts of these questions, the technical parts of these questions. But we also have a policy team whose job it is to think about not just how do we make sure it pursues the goals that it's given, but this question of what should those goals be? Who has a say? How do you make sure that these systems, that the world that we build, is representative? That it really has broad buy-in, that it actually is something that is able to
[0:47:11] Have government involved in the right way? That policymakers know what's going on and are able to put in the right regulations? And all of those questions, if you can solve the technical stuff, are going to become the most important questions to answer. Yeah, the nuance that you just explained safety and security risk mitigation with is very interesting. There's a lot of different ways to get into and to make sure that we're...
[0:47:40] Taking care of all the different possibilities. Yeah. And then, I mean, this is just taking us to the beginning where you just said that this is not a passing of a torch. This is we're trying to enhance humanity with this technology just like we have with every single other tech that we've built. It also does feel as though that there is a potential to build a
[0:48:10] What are your thoughts around the transhumanist movement towards that digital space and what you think is potentially best? I think it's really hard to predict.
[0:48:41] All right, you think about how can you even describe Uber to someone in 1900? Like, how would you do it? Yeah, yeah, yeah. Yeah, how would you explain airplanes to someone in 1500? You know, I guess that would make a little more sense. It's like we'd basically be the bird. That's right. That's right. Exactly, right? The mechanisms would make sense, and there's a lot of concepts there. But, you know, I think one thing that really distinguishes information technologies from a lot of the physical technologies that we're used to.
[0:49:13] It's almost this degree of it's so hard to anticipate and understand. Even the internet. The internet, in some ways, if you describe it to me, I'd be like, I don't really know why I need one of those. It's like a system that lets people connect. I don't know. We already have the post. What's the problem? But suddenly you get these capabilities, and they change the world in surprising ways. You get the capability of, I can basically instantly psychically communicate with anyone on the planet.
[0:49:43] It's called the text message. I have to type something with my little fingers. But it's not really any, if I actually literally had some telepathy built in, I don't know it would be any better. And so I think that the technology we've built has already given us superpowers in very, very surprising ways. And so I suspect that what's going to happen when we have even more powerful technology, it's just going to be hard to anticipate. I like how you give it a hefty dose of humility that's needed. There's so much.
[0:50:12] It's a conversation that's happening about what's happening in 10 or 20 or 50 years and all this type of stuff. It's also equally important to aim to extrapolate what we do have today with this cutting-edge neural network technology and see where it can be applied to maximize human potential incrementally along the way. The medical field is one of the big ones and preventative.
[0:50:38] Preventative healthcare is also with the medical imagery, making better decisions there. I'm curious to hear, what are your thoughts around the newest protocols with decentralization technologies being at play with artificial intelligence? Because you guys also do have a new aspect of OpenAI that's enabling you to kind of gain, it's called the LP.
[0:51:08] That's right, yeah. And so, you know, in some ways, the, so maybe two parts to that question, you know, what is OpenAI LP? And in some ways, OpenAI LP is an implementation detail of the OpenAI charter. Right, so we have this charter, it says, here's what we're about, we care about the broad benefit, we know we're going to have to raise huge amounts of resources, but we're going to make sure that that mission is served. And it has a cap, like you said. That's right. And the way that we think about this cap, right, so, you know, we kind of felt that, you know, so we started as a non-profit because
[0:51:37] We have this mission. It seemed like the most obvious implementation of the mission. How are you supposed to have a for-profit company that's going to benefit everyone? It just felt almost incompatible. And we spent really a long time trying to figure out, is there a legal structure that gives us what we need where our fiduciary duty, primary fiduciary duty, can be to the charter rather than to narrowly making money for investors? I don't think that there's anything wrong with...
[0:52:03] Returns for investors. I think that people who take a risk, there should be incentive for them to do it. But I think that this technology that we're talking about is just so powerful that we've got to make sure that we have a focus on making sure it goes well. And I don't think those things should come into conflict, but it's really important that your priorities are clear. An inclusive, stake-holding environment. That's right. And so we basically looked at every legal structure out there and concluded it wasn't quite the right one. And so we ended up custom-writing rules for an LP and defined what we call this cap profit.
[0:52:32] So the way that it works is that investors or employees have a similar stock option granted to them, have a capped return. So they put in some money today, they get a return. But if the company happens to generate more value than that cap, that value doesn't belong to any of the humans at OpenAI. It doesn't belong to the AGI either. It belongs to the nonprofit organization. That's very interesting.
[0:52:59] You wrote your own code for an LP because there was no available option to it for that before. That's right. And then from there, it's possible that the way that how should AGI benefit everyone isn't in the form of cheap or free AGI services, the AGI doctor. Maybe, right? And if that is, then we have the flexibility to do that.
[0:53:22] Or is it in the form of something that somehow generates capital and you return that to individuals in actual distributions? Also possible, right? But the point is we're not locked in to any of these outcomes. But the thing we are locked into, that we are committed to, is the mission that AGI will benefit everyone.
[0:53:41] Yeah, and this is where the decentralization component is kind of interesting, is that where do you also balance out a centralized OpenAI 100 employees with an open protocol for other people from around the world to help augment? Yeah, so I think that, first of all, I think that there's a lot of really interesting things going on in the blockchain space and in particular having these decentralized technologies.
[0:54:08] I'm actually on the board of Stellar, which is a nonprofit crypto system, which has the mission of giving financial access to the whole world. And that was a project that I helped launch in 2013, 2014. I've been really involved since. And there's some really interesting lessons from what we've seen at Stellar. In fact, when we initially launched, so there's this coin built into the network that we just gave away to people for free.
[0:54:34] That's part of the mission. We want to give everyone financial access, give everyone a stake in this network. And it was literally the case that if you signed up with a Facebook account, you'd get this stellar coin, and it was worth about $10 at the time that we launched, maybe $15. And so it was literally, you have a Facebook account, you go, you sign up, you get $10 to $15 worth of coin, and you can actually literally sell it the day that you got it. It's kind of this amazing experiment in a lot of ways. If you just let anyone in the world sign up for this thing, what happens?
[0:55:04] And we saw this real exponential growth. It grew faster than anything I've ever seen in terms of people claiming these coins, maybe not unsurprisingly. But the thing that was very surprising was we looked at where the sign-ups were coming from. And there was this very massive growing base of people in countries like Vietnam or the Philippines. We started to dig in to figure out, how are these people going to hear about it? What are they doing with Stellar? And we started to find these forum posts where it turns out that in a lot of these countries, there are these businesses that create.
[0:55:32] Legit-looking Facebook accounts. And so there's a lot of individuals who have a lot of Facebook accounts that are fake. And at first, I was furious. We were trying to give financial access to everyone in the world, and these people were stealing. They were taking that away from others. But then you start to read these forum posts, and you see these people would get their stellar coins, they would sell them, and they'd buy a goat for their village. Interesting.
[0:55:55] And you're like, this is amazing. This is almost a wealth transfer from these people, these crypto speculators in the first world to these people who just want to go for the village. Look at that impact. That's awesome. I didn't mean for that to happen, but kind of cool that it did. But then you fast forward about three or four years to the, do you remember the height of the crypto bubble of, I guess, early 2018, late 2017? Everything was skyrocketing. It was all very high at that peak.
[0:56:24] Those Stellar that we gave to people was $10, $15. It's worth about $4,500. Wow. And suddenly you realize that this system resulted in a goat for this village, but resulted in these cars for the crypto speculators. And the thing that I think is really interesting, I don't think there's any, it's just interesting to look at the system. I don't think that, it's less about kind of the...
[0:56:54] You know, the morality of it. But it's really more about the, like, there's this tendency for capital to accumulate more capital. The people who already had the capital were able to speculate, therefore were able to reap the benefits of this platform succeeding. And so I think that the same is important to look at with the future of AI, AGI. I think it's not enough to solve the technical problem of giving everyone access at a technical level, giving everyone a key, that sort of thing. Yeah.
[0:57:22] Malevolent actors can take advantage of that. Well, the thing about the Stellar story, so definitely malevolent, you've got to watch out for, right? But the thing about the Stellar story is no one's really malevolent, right? There are people in Vietnam who had these Facebook accounts that weren't legit, but I don't really view the transactions that happened to be illegitimate, right? The crypto speculators, it was like a fair transaction, right? They did get the goat, they took the risk, all those things. But somehow that system was set up in a way where it didn't result in the equality we were hoping for.
[0:57:52] And so with OpenAI, I think it's really important, and with AGI and these other technologies we're building, to learn from that and to think about how do you set up the system? Again, the technology is almost a servant to the impact. It's a vehicle. It's not the most important thing. The most important thing is what happens to human lives. And so that's the thing that I really focus on, and I think that's a really hard and really important problem.
[0:58:17] And what is a skill that children should learn as we enter into the automation AI age? I mean, so start with programming. Everyone should learn to program. Everyone. I think so. I think so. I think it's a really important way to understand how technology works and how these software systems that we all interact with, how they operate. Yeah. Yeah, interesting. Okay. Wow. This has been super enlightening. We have a couple quick questions about...
[0:58:46] That we usually ask on the show to ask you, right? Yeah, go right ahead. All right. First question is, are we alone in the cosmos? Not if we succeed at building AGI. And unpack that a little more for us? No, I think it actually stands alone. Yeah? Yeah. I think that I have real hopes for what happens if we succeed at building these really smart technologies. And I think that we can build systems that will enhance human lives, but these systems will also be different from us.
[0:59:20] In real meaningful ways. And so I think that there's something really amazing to discover there. So that AGI ends up helping us discover other? More so that AGI is its own. I think there's a chance. In some ways it will be an other. And in some ways it will be like us. And I think we get to control that. And then are we in a simulation? Hard to predict.
[0:59:50] I'm not sure what I would do differently if we were in one versus not. Yeah. You'd keep leveling up regardless. Yeah. And I think that I like to separate out the kind of concrete things that I can have scientific intuitions about. I think that for this question, in some ways, it feels like a non-scientific question in the same way that just like, is there a flying teapot out there between Mars and Jupiter? There might be.
[1:00:23] I have no way of disproving it, but I'm also not sure what changes my life. But that one is a little bit more, you're just kind of like throwing that, like a more of like maybe a meaningless question out. This one is probably a more meaningful question that can actually be probed with science. Potentially, but I think I would love for someone to try. Maybe in a couple decades with AGI we'll be able to run the simulation of a Big Bang and then see ourselves.
[1:00:51] 13.8 billion years later. It's hard to rule out anything. Yeah, yeah. Last question. Greg, what's the most beautiful thing in the world? My girlfriend. This has been so enlightening. Yeah. And we're very grateful that you've banded together some of the greatest minds out there to make sure that we do AGI right. And that helps.
[1:01:23] For our generation, also the older generations, but especially the younger generations to see your, you know, your charter, your principles, your first, your values that you're building this with and that helps them also want to be driven towards similar first principles and values and that's very powerful, very powerful.
[1:01:45] Thanks, Greg. Thank you. Thanks for coming on the show. I really appreciate it. We greatly appreciate you all for tuning in. Thank you very much. We'd love to hear your thoughts in the comments below about what was discussed here. Go and talk to more people about artificial general intelligence, what exactly goes into this, the power of neural networks. Everyone, get programming. That was Greg's takeaway. Get programming. Understand that this is a major part of our future and that everyone should be at least basically fluent in it.
[1:02:11] And huge shout out to Ron Vagus. Thank you very much for producing and directing. We love you very much. Everyone, support the artists and entrepreneurs that you believe in. All of OpenAI's links are below. Support them. Simulation, all our links are below. Support us. Support the ones in your community. Get them growing and flourishing. Go and build the future. Manifest your dreams into the world. We love you very much. Thank you for tuning in and we will see you soon, everyone. Peace.