[0:00:06] Six, seven years ago, you and Sam shared the slides of the open mission, and I was very blown away. Well, it's been very tough. There's a lot of challenges. One of my favorite quotes is this cycling quote of, it never gets easier. And I think that very much describes the OpenAI. I truly started to a plan. No, Stray and I were all there, amongst others, and start a lab. AGI play out better. I wanted to help. I was really that convinced. The next day, I was like, I'm full-time on this.
[0:01:09] We're going to make it happen. But it's one thing to get a bunch of great people together. It's another to really figure out, well, how do you, the technology, how do you try to sort of nudge things? And so the first thing we had to do was figure out, well, how do you get people to work together to material progress in AI? And we tried a bunch of things that didn't actually are Dota 2. That's right, yes. And the funny thing about that was that we started out saying, okay, we want to develop new algorithms. Come on, we're an AI research company.
[0:01:51] What is this really hard environment? No one's solved anything like it before. In order to develop something new, you've got to push all the old stuff as far as possible. So we took reinforcement learning, we just kept scaling it up, scaling it up. Every time I'd come to the office, our researchers, Jakob and Simone, who were still very close collaborators, would scale it by two times the amount of compute, and we'd have a two times better model. We kept doing that, and eventually we beat all the best humans, and we never got on to develop the new algorithm.
[0:02:25] And this to me was kind of this amazing sort of realization that there's so much more power in everything that we've developed. With computational power behind it, it can do things that you would not even dream of. And I think that that has been a theme that has clearly started to really permeate the future results. And I think everyone's talking about, the chairman's talking about all the work that SAA is doing along these lines. And I think this is a really global opportunity to really scale up this technology.
[0:02:55] There's always all these innovations that all kind of multiply together. We had our language results that people talk about GPT. But do you remember the unsupervised sentiment neuron? Yes, I do. Yes. So this was 2017. We actually had the very first one of these language model results where it learned semantics. And that was the moment where it was like, OK, if you scale this up by the same insight from Dota, then we're going to end up with something unprecedented. And looking forward, what do you think?
[0:03:27] You're going to be some of the main challenges or opportunities ahead in achieving these goals. Yeah, I mean, it's really across every facet, right? Our mission of trying to make this technology benefit all of humanity is about developing the technology. And that's hard, right? We set a goal of every year doing something. We've done that, I'd say, for the past seven years in succession. Every year we have something that kind of totally obsoletes the previous.
[0:04:00] But that's not it. We thought we were in the software business. It turns out there's this massive infrastructure behind everything we're doing. You need the energy, you need the chips, you need this whole supply chain. But there's also this impact of really benefiting everyone. And so that's a question of adoption. Where does society want this technology? Again, one thing I find very inspiring about Korea and the Korean approach to AI is...
[0:04:32] It's being very excited about the opportunities, thinking about where can we deploy this, how can this fit in, population, you know, all of those things. You have to engage with the risks, right? It's not all rosy. It's not all pure upside. Anything that's powerful, anything that causes... That actually reminds me of a story that you shared to me. I almost gave up on chat GPT many, many times. And for the audience, could you share that story? I was really inspired. The language is going to be important. It was just a question of when would it be...
[0:05:47] Team models, we kept scaling them up. And we had a project that I worked on at OpenAI. It was actually building our API because we had an image model, which was GP3 at the time. We had no idea what people wanted to do with it. And so we said, we're just going to make this available to developers, and they'll figure it out. And this, as a startup investor, you probably would not invest in a pitch like that because you're supposed to have a problem you want to solve, and no one cares about the technology behind it.
[0:06:18] And we had the exact opposite. But it still kind of worked. And that, for me, was this moment where you realize, OK, we actually can focus on technology development in this specific field. Because every business is language business. Right now, we're interacting in language. So if you can build a machine that can help a little bit with that, it will already slot in. People don't need to change problems. So adoption will be quite fast. And people will find how to use it. But now, in 2020, we've been developing GAPI for some time.
[0:06:51] We had a team that had been working on this, like, chat thing. And we were paying this contractor for people to talk to this model. And, you know, they maybe saw some chat feedback, but it was not anything that was really taking off. But in late 2020, we took GVD4, trained it with what was called a structure problem at the time, which is just you basically had a data set which was taught to write an answer. And I was like, well, what happens if you just write another question?
[0:07:34] It very happily generalizes, like, well, clear, I need to provide another answer, and it used the context previously, and you're just like, okay, you can talk to this thing, a real conversation, and it's good, and we didn't even try other tasks, and it just kind of generalized, and so, okay, this is clearing into the world, and we have this other infrastructure of this chat system that put the two together, and that's going to work. Actually, once, but who's going to want this old GP3.5 model? It's so bad, like, look at all the things GP4 can do.
[0:08:12] That was our mistake. It was not sort of realizing that we had updated. We had moved on. We had sort of started to see the flaws in everything with AI. One of the challenges is you have this sort of very non-human strength and weak factor. It doesn't really fit into how we turn AI into being created. And so we have these systems that are so smart in many ways. It just feels like GPT-4 is like what a smart human can do in like a split second. But you give GPT-4,
[0:09:28] A minute, 10 minutes to think hard about something that it didn't quite get right the first time, it's not going to come up with any better answer. This is clearly, maybe it's not the AGI, but it's clearly a remaining AGI. What we've been really trying to work on is to improve to double down on a specific problem, really one out of AI. We think of AI right now as this quick back and forth. You send a message, you get a response, you kind of throw it away. It's not really something that's a work of art. It's not going to win a Pulitzer Prize. It's not going to...
[0:10:05] How to cure a disease is not going to be a new technology. It's not going to be the idea for a new company, right? It's like, we're not there yet. But those are all tasks that humans do. And those are all things that, you know, that's what we set out when we started this company, to be able to solve tasks like that, to be able to help. And so that is ultimately what the reasoning sort of project is aiming at, is being able to spend lots of model thinking time on hard problems that really matter.
[0:10:38] And to me, this is very inspiring, right? It's like right now, before it runs for like, I don't know, 10 seconds, our new O1 reasoning model, which is the very first beginning of this new paradigm. You know, maybe you run it for a minute and get some great answers. You know, we're starting to, with enough computational power, be able to get a gold medal on things like the International Computing Olympiad. But what you want is you want to spend a month or you want to spend a year of one of these giant, you know, five gigawatt data centers.
[0:11:08] And as we move from chatbots to new reasoning models that you just described and even beyond, what use cases are you mostly in?
[0:11:43] The thing about AI is it's such a horizontal technology, right? It really touches every single image. And the applications that I'm most excited about, it's hard to pick in some ways, but healthcare, I think, is one that I think can really be transformed and is already starting to be transformed. You know, when you think about healthcare, it's one of these things, too, where the human baseline is actually, like, there's so much room for improvement.
[0:12:13] You know, it's a personal story. So my wife recently got diagnosed with a rare genetic condition. It's actually probably not as rare as it seems. As we do more research, I think it's one of those things that actually is just undiagnosed a lot of the time. And it's called hypermobile airless-downless syndrome. It affects every system in the body. And so that means that if you go to any specialist, they're looking at their little piece of the elephant. And they're like, okay, well, you have this.
[0:12:42] Let me ask you some questions related to the AI infrastructure.
[0:13:26] To support all these future applications that you just mentioned, robust industrial AI, industrial web advancements. Yeah, so AI, it's computational power, data, it's the AI system that we're talking about. Everyone likes to talk about the chips. That's very important. You need lots of compute. On the chips themselves, you need things like memory bandwidth bottlenecked. So this is, I love what SK Hynix is doing with next generation HBM. That's the kind of thing that...
[0:14:15] But then you also need to build these massive systems, right? That those require a lot of power. And so people are starting to talk about now, okay, how do we do gigawatt scale data centers, which is, that's an order of magnitude bigger than the kinds of data centers people classically build. And the thing about these things, you know, again, we thought of ourselves as in the software business, but really we're starting to see there's this massive construction business, this massive supply chain behind it. You know, we sort of talking and trying to really get...
[0:14:43] People across the world, governments across the world, excited about the opportunity here. It's very easy to say, oh yeah, a 5 gigawatt data center here and a gigawatt over there. But a 5 gigawatt data center, that's like 80,000 jobs to build that. 80,000 jobs to build this. And once it's up and running, you're talking many thousands of jobs to be made to be running. And that this system is something that can deliver these huge benefits. And there's many of them in the world. So I think they're really getting the world to understand.
[0:15:14] There's this massive increase in demand coming that we have been so compute bottlenecked in terms of our AI and the ability to get the AI out there, not just to develop it, but really to get it integrated into the world. And this is something where I think that as we have this continued exponential of AI progress, that there's going to be this equivalent exponential of demand for all this infrastructure. And we have to be a little bit careful not to overbuild. I think that's always a sort of question that people have. But I think in AI, it's going to be...
[0:15:45] Very hard to end up airing in that way. Then I must ask you this question. How important do you think a collaboration between governments and government is in building the AI infrastructure? Are there any instances where OpenAI actively works with governments to develop this? Yeah, so I think governments are a really key part of what we represent. And AI ultimately is for us about where... Permitting is an example of a government-related bottleneck that a lot of people...
[0:16:32] And just like what Chairman Chen mentioned earlier, SK aims to establish a strong foundation for the future AI ecosystem.
[0:17:57] Yeah, so we partner with SK, and I think that's something that's been really mutually beneficial. We work with companies across the world, and again, our goal is really to try to help push forward this technology and also make sure its benefits are available to everyone and lift everyone up. And this is one way we do it, is by really trying to work with companies and players across the world. I think it's a great choice, right? I think it's a great strategy to really say, let's go all in and really invest in...
[0:18:25] When I meet executives, I always ask what percent of your time is on AI. SK, I think that the answer I'd be getting is like 50%, which is kind of crazy. It's like something where it's just coming out of nowhere a couple years ago in some ways, and now it's really occupying this massive mindshare. I think that shows you the potential that everyone sees and really the commitment that is behind. It's not just buzzwords on a slide somewhere. It's really playing out.
[0:18:54] And all these projects and really across every aspect of the value chain. Oh, you know, let's shift gears again and move on to more of a general question for the audience. Do you think AI will surpass human intelligence anytime soon in some ways? And how will societies adopt it? Yeah, so I think, again, the thing about AGI, I think the answer is definitely yes, right? There is some point where that will happen. I think that people love predicting, you know,
[0:19:26] And so I think that it presents, there are challenges, but it presents an amazing opportunity too.
[0:20:00] Really think about how the world can be reorganized and changed and how we as society want to offer. Fundamentally, I think AI is both about helping us achieve more, but it's also about removing confusion. You look at, for example, software engineering right now. That's one area that people are adopting AI very rapidly. There's very good code level, like you have to know these details, specific library, build, and then you need to write down this very specific way. AI accelerates that so fast. And that the impact of it...
[0:20:38] It's not what you'd expect. Many of these industries, I think we'll see, are actually quite elastic, where if you make a software engineer 2x or 10x more productive, we're not quite there yet. But let's say you get there, you'll be 100x more programmers, because there's just so much to build, much impact costs. Now, that may not be true across every domain, right? But I think that there's just this sort of impact that's very different from what you just think off the top of your head. I must talk about a potential similar size, but the amount of people that...
[0:21:28] I think it will vary a lot depending on industry. For example, it has to make an impact through certainly pure cerebral technology. And I think that there's a lot of these industries that drive forward in the economy that will be harder. And to your point of, well, you can build companies with far fewer people, I think there's a great question of, well, how many more companies will be created, right, who really remove those barriers to entry?
[0:22:26] The whole thesis of things like Y Combinators, make it much easier for people to found companies and just all these things will happen that wouldn't otherwise. Same thesis at Stripe where we were, this is where I worked previously, we were trying to make it easier for people to access payments online, to accept credit cards online. And our thesis was we're going to create, we're going to increase the IP of the internet. So I think that there's these very interesting sort of impacts that are possible. But now as you...
[0:22:54] I think that the story of technology has always been some jobs go away, new jobs get created from the internet. Computers resulted in fewer file-in clerks, but now we have social media influencers. Very surprising kind of impacts that are hard to predict ahead of time. But AI feels like it might be different. If you really do have a system that is able to match humans across all these different dimensions, then what about the new jobs? What does that look like? And again, I think that there's...
[0:23:23] No one knows for certain. It's all new. It's nothing that's happened before. So it's all speculation. But I do think that there are ideas and reasons for optimism that are not obvious at first glance. For example, you have a world where you have these systems. For example, you're always going to have status competitions amongst humans. And it's already the case that there are many people who pursue passions.
[0:23:54] That are not for compensation, right? That they do it because they love it, they want to be great at it. And maybe not even the best. Maybe that there are many people who are better at that thing, but they want to do it anyway. And there are many other people who work at jobs that are not their passion. That they would love to be doing something else, but that they have to do this to survive. And both of those things change, right? That people can, we can have way more people pursue their passion, way fewer people have to work in a job. You know, people talk about the idea of, well, if you're in a world of abundance.
[0:24:21] Then there should be universal basic income so everyone gets the ability to survive without having to have a job. But one tech CEO I talked to recently said that undersells it. It really should be universal hyper income. And it really should be that everyone can live this amazing life by today's standards beyond anything that we can imagine. And I think that this to me is kind of the thing we should be thinking about. It's not what will the future be in a predetermined sense. What future do we want to build? And that is ultimately the reason
[0:24:52] We created OpenAI is to say we want to help steer things towards a positive future. I think this world of abundance that is available to everyone is like a pretty exciting one to get behind. So I want to make sure it's clear that it's not all just going to be easy. I think we have to work for it, but I think that there's a really positive future possible. And you know, there's a lot of conversation about countries needing to develop their own AI models.