[0:00:00] Where do you think the biggest bottleneck is today? We're heading to a world where energy is going to be a massive bottleneck. Do you ever consider some of the newer players like a Cerebrus or a Grock? So in 2017, we got super excited when we saw Cerebrus. 2017 OpenAI was, I guess, very different than today OpenAI. You'd be surprised people still don't always listen. Is my job in danger? AI.
[0:00:20] Is going to change a lot of jobs. Are there still developers? I think that we are going to change a lot of fundamentals of the social contract. Do you still have the same definition of AGI? I really used to think of it as this destination, but instead we really think of it as this continuous process. Let's say we were to 10X our compute supply right now. Would we 10X revenue? I'm not sure if we would 10X, but would we 5X? ChatGPT really makes you realize how unnatural it is to go to a static website to just read stuff.
[0:00:48] Do you think that software is going to be fully generated? I think so. I think it's going to be super cool. What is the conversation internally at OpenAI like? Pain and suffering. It's the real truth. All right. Greg, thank you so much for joining me today. Thank you for having me. It's good to be here. I have a bunch of questions for you. I wanted to start first with scaling and specifically Sora. So Sora 2 released last week.
[0:01:14] What is it like thinking about scaling a model like Sora and how is it different from a text or image model? Well, I guess the way I would think about it fundamentally is that, you know, at a broad level, everything is still just deep learning, same mechanics, same sort of underlying principles. You got to scale up with a massive amount of compute, a forward pass, a backward pass, gradient step. At a more detailed level, still a transformer, right? Which is actually quite amazing.
[0:01:43] Yes. And, you know, you train it in a different way. You're using, you know, sort of a different process, much more you think about things like diffusion or sort of a different way of thinking about how to, you know, sort of pour compute into these models. But fundamentally, the thing that I find so amazing is even though you're talking about text versus video, which are seemingly as different of modalities as you could expect, that you have this massive overlap in terms of the actual...
[0:02:11] Underlying computational process to learn and generate them. And so there's something like really, really deep about that fact. So are you pretty bullish that the transformer architecture is going to get us to really the next level, maybe full world models, which is, you know, Sora 2 is obviously a big step in that direction. Yeah, I think.
[0:02:28] Well, so two things. One is that I think there's a lot of questions around like, are we missing big ideas? Do we need another innovation on the level of a transformer? I think that there is lots of room for innovation. And we've seen it, right? That I think the algorithmic progress has kept a pace. We've done studies over the years to see exactly what the curves are. I don't see those stopping. The scaling curves also continue, right? The data curves also continue. And that's the thing that has driven this revolution is that there's...
[0:02:54] Multiple limiting reagents. In each one, you can just keep tuning and you'll just see the performance of the models just increase appropriately. So I think there's so much more to be built. I wouldn't be surprised if AGI looks kind of similar to the models we have today, but I would be shocked if it was exactly the same architecture. And then when you're looking at these different types of models, although they are all transformer-based, are they pretty significantly different in cost?
[0:03:21] And how do you measure the unit economics of the different types of models that you're serving today? Yeah, there's definitely different performance characteristics that sometimes we have a different inference stack associated, the optimizations are different, that different models are maybe amenable to different types of hardware, that you can really see that the exact balance between the memory and the compute and all those things, those can be different.
[0:03:43] So there's a lot of systems work that looks quite different at a detail level. And when you're really trying to extract screaming performance out of the hardware, that it pushes you in very different dimensions. But I think at the end of the day, we really view it as like the fundamental driver of all of this innovation and really bringing it to the world is compute. So you got to build as much as possible. And there's some specialization of different accelerators and those kinds of things. But if you zoom out, everything is just doing.
[0:04:08] Matrix multiplies, doing some attention mechanism, that kind of thing. And so we do a lot of juggling internally of capacity and sort of prioritize five different demands on one person's time who can go and optimize five different models. It's tough, but that's what we're signed up to do. All right, let's continue on hardware. A big announcement with AMD.
[0:04:33] So is it fundamentally different to build on top of AMD hardware? Is it just, okay, now we just have this increasingly massive pool of resources we can pull? Or is there like deep technical changes that need to be made? So we've actually been investing in AMD software in many ways for a long time, right? Because we build on top of Triton. That's something that a project that we've sponsored and really helped develop. And that a vast majority of our...
[0:05:02] GPUs are running Triton kernels effectively. And I'd say the way I'd look at it is that there's inference and there's training. There's a massive fixed cost to getting inference working. There's an even more massive fixed cost to getting training working. And we're at a point where we actually have been able to use with a very small amount of work.
[0:05:23] To be able to use AMD software this year and get good performance out of it. And so a lot of that is enabled through just like this partnership we've had for a very long time. We've provided a lot of feedback. And we're at a point now where from an inference perspective, we feel quite good about scaling. And there's different niches for different pieces of hardware. And that we're excited about the MI450 series. That there's a lot of good innovation in there. And that we will also scale up a ton of NVIDIA for inference and training as well.
[0:05:50] Do you ever consider some of the newer players like a Cerebrus or a Grok, like kind of the wafer scale computing? Do you ever consider those? Yeah. So in 2017, we got super excited when we saw Cerebrus because it was just this totally new paradigm. You looked at the numbers. You're like, wow, like if we have like a million of those things, we could build AGI, right? Just like you just realize it's a very different sort of platform. And I think that it's turned out that...
[0:06:20] Building non-GPU architectures has been way harder than we expected in 2017. But from the very beginning, we really mapped out the whole ecosystem. We tried to talk to all the different chip players and try to give them some advice, talk about here's what the shape of the workload is. And honestly, most of the companies wouldn't listen to us, right? It's like... 2017 OpenAI was, I guess, very different than today OpenAI. You'd be surprised. People still don't always listen. Okay. But I think it was...
[0:06:48] To some extent, it wasn't even that they thought we were wrong, but that if you have people who come from the chip world and have a particular way of looking at the problem and don't understand the workload, and you try to say, no, no, no, this perspective is backwards, that you really need to think about it in this other way, that it's going to be about big models, not small models, whatever, those kinds of design inputs, if you don't buy into it, it's very hard to then rebase your whole worldview on top of it. And so I think that what's really distinguished...
[0:07:17] The successful players in this space has been people who bring in people with a deep learning perspective or really try to pay attention to where the workloads are going. And when you're looking at the entire pipeline of building out compute all the way to serving inference, where do you think the biggest bottleneck is today? Well, I mean, I think we are heading to a world of absolute compute scarcity. And I think that we're heading to a world where energy...
[0:07:46] Certainly the US is going to be a massive bottleneck, but there's all sorts of parts of the supply chain right now that have not yet adapted to the demand shock that we see coming. And so that's what we've been really trying to, and back to us saying things continually over and over for many years now, we've been saying, hey, we need to build more compute. Yeah, and there's been rumors that OpenAI is developing its own chips, but do you ever see maybe investing in your own energy grids and systems?
[0:08:14] Inventing new things on that front. If you had talked, you know, fast forward or rewind 10 years back to, you know, 2015, 2016, me and ask what we're going to do. We're here to build AGI. And we thought of it very much as a software endeavor. We thought of it, in fact, as just a, we need to come up with some new ideas. We'll click it into place. AGI created. We started to find that actually it's a compute endeavor, that that is like this fundamental reagent.
[0:08:41] That is much easier to scale than many other things. That's why you press so hard on computing. You have to push it to the limit. And then you start realizing that actually there's this massive physical infrastructure build you have to do. And so we're now getting into that world. We're doing things like Stargate, starting to build our own data centers. And so I think that the where does it stop is really about what is the world willing to supply. And so if the world, if the market does wake up to the demand that we're really very loudly trying to say is coming, not just from us, but from the whole industry, then.
[0:09:10] Great. I would love not to have to go and figure out how to build energy ourselves, but we're here to do the mission. And so with the limited GPU, limited compute that you have now, you have a lot of conflicting needs. Consumer products, enterprise products, developer APIs, training. What is the conversation internally at OpenAI like when you're trying to decide where that compute investment belongs? Pain and suffering.
[0:09:37] It's the real truth. It's so hard because you see all these amazing things and someone comes and pitches another amazing thing and you're like, yes, that is amazing. And you guys are doing so much. We have a tiny company and we can't decide of the many things what to do. And so I can't even imagine that at OpenAI scale. Give me a little bit more about what that internal conversation is like. Yeah, the mechanics more...
[0:10:05] More specifically, you know, we've evolved them over the years, but right now there's, you know, Jakob, our chief scientist, and Mark, who runs research, together decide the compute allocations. But more broadly, actually, there's first a split between the research side and the applied side, and that usually gets adjudicated at, you know, sort of the, you know, SAM, you know, Fiji, like, you know, that set of people usually is making that call. And then, you know, within research, I just described how that gets...
[0:10:33] Allocated at a mechanical level, there's a bunch of people on my team who are really dedicated to this hard task of actually shuffling around the GPUs. So, you know, it's like it's really amazing to watch. So, you know, Kevin is an example. Kevin Park is someone on my team who just like you go to him and you're just like, OK, like we need like this many more GPUs for this project that just came up. And he's like, all right, there's like these five projects that are.
[0:10:58] Sort of winding down. And this one's got to finish at this point. And so we can sort of make the Tetris work. And it's just really amazing to see both the intention part of this, of which is like, where do we want the compute to go? But then the actual solver. And some of that is human. Some of it is spreadsheet. Some of it is like, I think it'll be very interesting to see if we get our models in there. But so I just say it's like, it's not an easy process. But I think it is just like, it's been very interesting to watch because I think that compute is such a driver.
[0:11:27] All right. Let's shift gears for a second. So you made an announcement. You're now bringing kind of lack of a better way to describe it, the web into ChatGPT. You showed off the Zillow example. As apps continue to migrate towards like a more native experience within ChatGPT, how...
[0:11:55] How are you thinking about this decoupling of the human to internet experience that seems to be happening? As agents continue to increasingly browse on our behalf, the time where an actual human is going on the internet browsing traditional websites seems to be decreasing. What do you think the next 18 months looks like? Just quickly, actually, I want to add one thing to the previous answer, which is also, I think where we're headed is a world where...
[0:12:25] Compute is the driver of economic productivity throughout the whole economy. And so this microcosm that we've seen within OpenAI that you said you see within your company, I think we're going to see everywhere. And so I really view this like we've got to build compute.
[0:12:38] As a way to alleviate this compute scarcity and this compute sort of clash over allocation. Before we move on to the next question, what do you think the ratio is between supply and demand right now? How far off are we? Oh, I think we're quite far off. I don't know. Orders of magnitude? So I was going to say, I don't know if we were, let's say we were at a 10X our compute supply right now. Would we 10X revenue? I'm not sure if we would 10X, but would we 5X?
[0:13:04] Yeah, maybe, right? Because we have so many products in the hopper that we cannot release. And you just see it very concretely, things like Pulse. It's like pro only, right? Pulse is such a great product. Yeah, we're going to talk about that. That is a really cool product. But we need more compute. We need more compute.
[0:13:22] I need to get you a sticker that says that. Okay, so talk to me about the decoupling of the internet because it just seems like the internet, the fundamental way that we browse the internet is changing drastically right before our eyes, especially with agents being able to browse on our behalf and now with bringing the kind of traditional website into ChatGPT. So what are your thoughts on that transition that we're seeing? Well, I feel like ChatGPT really makes you realize how...
[0:13:51] How unnatural it is to go to a static website to just read stuff, right? Just like static information, like a single fact that you're looking for, that you're kind of mining through a big page that is kind of not relevant to what you actually want. And I think we've sort of almost moved past that.
[0:14:06] Still happens, right? But that's not the dominant paradigm or the thing that people really want to do. And you just realize there was a bunch of time you were spending. It was not value-add time. It was like sifting through this haystack for a needle. And a machine should really do that for you. It really should. And I think what you're going to start to see with apps in ChatGPT, with these dynamic apps, is that going to a website to click a bunch of buttons to do something dynamic also just feels like this total backwards...
[0:14:34] Thing that we should have moved past a long time ago. And so I think that we're moving to a world where people are going to be much more protective of their time, right? Because there's sort of no excuse anymore for things that are not adding value, where the human's not thinking hard or providing creativity or somehow providing direction, feedback, something like that. If you're just sifting through a big list of stuff, that's what an AI is for.
[0:14:57] And how does that change monetization on the web, which is traditionally CPM-based, advertising-based? You give the website your eyeballs, and in exchange, they give you some free content as well as advertising. But when agents are browsing on your behalf, and especially as...
[0:15:11] You're bringing like a Zillow.com into ChatGPT, then there's all these conflicts of, okay, were they serving ads? And then what does that look like? And so like, how do you think about the changing monetization layer of the web as these changes occur? Well, the truth is, I think no one knows yet, right? But I think that we can see, I think we're gonna have to explore, we're gonna have to figure out what the right way to, you know, what the right new monetization paradigm, the right way to scale all of this. But I think that fundamentally,
[0:15:38] There's new pressure from these technologies to make sure that you're adding value to the user. And if you look at ChatGPT itself, right now it's a subscription-based product, right? We maybe would not have predicted that when we launched it three years ago, but people are willing to pay because it adds value. It adds value in your professional life, your personal life, across the board.
[0:16:01] I think that that's not to say that advertising doesn't have a place, right? But I think that advertising that is really about, you're just mindlessly scrolling through, you know, things to try to find the like sentence you care about. And it's just, you happen to be on that page and click a thing. It's just like that, that doesn't feel like quite the fundamental driver of value anymore. But I do think that there's going to be new revenue models. There's gonna be new ways to monetize. And I think that it's honestly like, I think probably the most exciting time to build yet.
[0:16:29] And so if you rewind a decade plus and you look at publishers during the mobile transition, a lot of them became beholden to Apple because they were in their app store. And what would you say to them on why this is different, where ChatGPT really becomes the first place, the homepage of your artificial intelligence experience? Well, I think the story is not yet written. And one observation that I have about...
[0:16:57] AI is, it seems to always play out in a surprising way, right? Just totally different from anything we've seen before. It has elements that are reminiscent, but I think there's no like one paradigm that you'd be like, this is exactly like the internet, or this is exactly like mobile, or this is exactly like the app store. I think it is something different. So what is the way that you want to interact with an AI? Is it that there's like one website that you go to that sort of mediates your interactions with everything else? Like, I'm not entirely sure because AI...
[0:17:25] The point in many ways is that it brings the machine closer to the human, right? Rather than you having to contort yourself to think about, okay, there's like a website with a URL and I have to go to this thing. It's just like, the machine should just do what you ask. And in fact, the machine should proactively think about what you might want and go do it for you. And I think that these shifts in the paradigm, I think probably will also shift how we think about what the entry point is, where the opportunity is. And so I just view it as there is so much surface area here to build that it is not obvious to me that it's even possible to have
[0:17:55] I want to actually continue on that for a moment.
[0:18:23] Starting to become much more of the focus. Or, you know, you give a small task and then the AI goes off and thinks for a day, for a week, for a month. And we have aspirations that we want to build AIs that can productively think for like a year, like for 10 years. I mean, like, like this is humans do this. With no human interruption during that year period? Well, I think maybe I'd think of it a little bit like, think about a human solving, like, like I think Andrew Weil solving for Maul's Last Theorem.
[0:18:54] I think he famously spent 10 years working on it basically by himself. Literally, there was no human interaction. He's probably thinking about sub-problems and he'd ask people about it. I think that that's what we want to achieve. We want AIs that can help us solve grand problems. And it's nice to have AIs that are able to go off and do productive work without us having to constantly micromanage. It's not very fun to micromanage humans, not very fun to micromanage AIs. But I think that this kind of world that we're heading towards is one where...
[0:19:22] You could micromanage if you want to, right? Whereas like for a productive human worker, if you micromanage them all the time, they're probably not going to be happy for very long. And so I think that it really is going to open up the surface of how you work and you'll really be able to choose where you want to spend your time. And I've seen a lot of highlights on like, hey, our new AI can think for X number of hours independently, autonomously.
[0:19:48] How do you think about the trade-off between the duration that an AI can think autonomously versus what it actually gets done in that period? Because if it takes 30 hours to do one plus one, it's a little different than solving cancer. So how do you think about the compression of intelligence within the given window and extending that window and the trade-off between the two? Yeah, I think it's a great question. It's very easy to have these benchmarks that are extremely misleading, to your point. I think that it's very clear to me that there are
[0:20:18] You can think of problems as almost having this computational complexity behind them or something, right? There's just some problems that are going to require more thought, more power, more compute behind them. And kind of what you want is you want an AI that can productively go and think for a day on one of those hard problems, but it would be nice if it would just solve it. Right, in 10 seconds. Yeah, that'd be great. And so I think the answer is these are two different dimensions, and it's important that we keep pushing on both.
[0:20:47] Okay. Well, with that, how long has GPT-5 thought for, Codex thought for with full autonomy? What's the record right now? I actually don't know what the record is. I think we've published it. I know that we've seen like a number of people have reported seeing like seven hours, but I'm not sure that that's actually the limit. So you can find it somewhere online. But I mean, we're at a point now where it's like we're starting to be able to spend quite a lot of compute on interesting problems. Let's talk about Sora 2.
[0:21:16] Really fun. Awesome. I think some of my team might be a little addicted to it, but that's okay. It is really like just a blast to use. As you were developing this, the new model, coming off of Sora 1, why did you decide to build it into this social experience versus just taking the path that Sora 1 took and releasing it for usage in kind of a more traditional way?
[0:21:44] Well, the way that we usually think about what surfaces to build is it really comes down to what is the capability of the model. And this is very much how we ended up with ChatGPT in many ways. Like I remember, you know, we've been kind of working on this infrastructure for doing chat stuff. And then we had GPT-4. And I remember we did the first post-train of it. And at the time, we were just doing instruction following, which is you have a data set of here's a question and train the AI to just provide an answer. And I remember...
[0:22:14] Well, what if you just provide another question that kind of depends on the context of the previous question and answer? Well, actually generalize to know that it should utilize that information. And it did. And you're like, wow, this model is smart. It's like capable of generalizing to this. It like wants to be a chat model. Like it just is like so clear that the technology is shaped such that you should release this as a chat system. And I think that for Sora 2, there's a little bit of that vibe to it, right? Where it's like, what?
[0:22:40] Are the strengths and weaknesses of the model? What are the things you could do with it? What is fundamentally new? And so I think that there were a lot of directions we could have gone, and there still are a lot of directions we can go. Like the model, the thing that to me on the inside is always a little sad about any interface, any post-training of the model, is you actually...
[0:22:56] Yes, yes.
[0:23:18] Talk a little bit about that. Sorry to cut you off there. No, I think this is actually something people don't really understand from the outside. And to me, it's a very sad thing because we used to release base models. GPT-3 was a base model. No post-training. Incredibly hard to use. Did you use GPT-3 back in the day with all the prompt engineering? Yeah. You had to provide six examples of solving a task. So that's a function of the model being a base model versus just it getting...
[0:23:45] Have they gotten better over multiple iterations? Yeah. So the specific way to think about it is that these base models, you know, we train them to just do next step prediction in they're almost sort of observing humanity's thoughts and feelings and, you know, all the public data available. And so it's just trying to say, like, given this prefix, what comes next? What comes next? And so that.
[0:24:08] At inference time, it's almost like you're dropping it in the middle of a document that it found somewhere out in the publicly available data, and you're asking it, what comes next? And so then you have to kind of think like, okay, how do I format my query in a way that could occur in this naturally occurring distribution? And so it turns out that just like a list of, if I have a question and an answer, and a question and answer, and a question and answer, and I have a question, probably what comes next is an answer. But if I only have a question, maybe what comes next is another question.
[0:24:35] And so it's like you're kind of trying to role play the AI into sort of thinking that it's in the middle of some sort of reasonable document that looks like it's training distribution. And so because this is so hard to use, it's a poor interface. It's not a good product. And it's also something where we have no control over the behaviors and the values that we'll express. It's a little bit like...
[0:24:59] You know, for a human who grows up observing the world, it's going to have knowledge of everything, right? And to some extent, like one analogy that I've heard that actually Alec Radford liked to use is that these base models are much more like training a humanity than a human, right? It's got everything in there. It's got every sort of value set. It's got every worldview. And so to the question of how we'll respond in a certain instance.
[0:25:24] It is almost anything that a human could respond there. You could set up the model so it would do so. But if you really want to narrow it down to a consistent set of values because you have some guardrails, you have, you know, we have a model spec that says how we're supposed to behave in certain cases, then we need some other step on top of it. And that's what post-training is, is really saying we're going to take this raw universe of this raw intelligence and we're going to refine it.
[0:25:46] To an almost consistent personality or consistent set of behaviors. Does that mean the decision to make it a more social product came before post-training? Or like, did you discover something? Oh, it kind of has this knack to do imitation really well. Like, well, yeah. What was the order of operations? It usually goes in a bit of an iterative loop, right? It's like you take the base model, you kind of see like, okay, like you kind of prompt it in a certain way.
[0:26:11] This is interesting. Actually, this is so cool. What if it was reliable at this thing? You didn't have to do all this work. And so it's a little bit like the base models of the world's best prototyping engine, but they are not reliable, right? Because it is so hard to like figure out the right prompt to get them to like do the task you really want. So it's almost a communication problem. And then the post-training is this communication. Is your cameo public? My cameo is not currently public. I put mine public. And I think Sam Altman mentioned this as well.
[0:26:38] It's actually surprisingly comfortable having people manipulate your likeness. It's pretty easy. It's pretty fun. Honestly, there's not that much thought behind my cameo status. Because I think in six months, no matter what we do, I think someone else is going to release a video model that lets you do cameos and is not restricted. And so I think we are heading to a world where all of our likenesses are going to be cameoed. OpenAI, I think part of what we...
[0:27:06] Yeah.
[0:27:20] So when you look at Sora 2, it's a world model. It's able to simulate the world. Yann LeCun has said like LLMs are not sufficient to reach AGI just because language alone is not sufficient to model the world. Do you agree with that? Why or why not? And are world models really the future of AI and reaching AGI? Well, look, I like to look at what things have we learned.
[0:27:50] Over the past five years, 10 years of AI progress, what are the things that now we have seen empirical evidence for one way or another? And I think that the language models don't have a world model. There's not enough information in written language to build a world model perspective. This is, by the way, a long standing debate. This isn't a 10 year thing. This is like, I don't know, like 50 years, 100 years, like a long time. I think that would have predicted that.
[0:28:19] You would not be able to do half the things that GPT-4 could do, right? Because it's like, you know, you ask it questions like, you know, like I put the water bottle on the table and then I take the bottle, the cap off the bottle, and then I put it under the table. Like, where's the cap? Like, I haven't tested that specific query, but like, do you think it's going to get that right?
[0:28:37] Probably. And yeah, I used to have a test I gave, you know, there's a marble in a ball you take, or excuse me, a marble in a cup, you pick the cup up off the table, where's the ball now? And it's still on the table. So I remember 3.5 didn't get it, 4 got it, and then 4.0 and beyond just nailed it after that. So spatial awareness is there. Exactly. So what does that tell you, right? Even if it's not perfectly reliable at your current super advanced task, but you had a super advanced task that was your test, and now it's conquered it.
[0:29:06] Right. It's like it shows you a trajectory. And so to me, like, I think that it's very easy to get sucked into semantic debates of like, what does it mean to understand? Like, are these models really understanding? Are they just simulating understanding? I'm like, what does that even mean? Like, I don't know what I don't know what those words mean. But I do know you show me an eval that sort of captures this task that you think should be impossible for the models and that we're seeing the models start to creep up into the right and then blow past it. Yeah. Saturate it. I'm like, I think it's got it. It's kind of like what Sam said earlier.
[0:29:35] Intelligence is really just prediction. Prediction is intelligence. And it seems like a similar argument to large language models actually will be able to be AGI. Okay, so I selfishly want to ask you, is my job in danger? You know, Mr. Beast said AI is a threat to content creators' livelihoods, and now that's my job. What do I have to be worried about? Am I worried? What do you think?
[0:30:04] Look, I think that it is definitely true that AI is going to change a lot of jobs, right? And there'll probably be some jobs that right now are something that many people do that will either just be totally changed in an unrecognizable fashion or just won't be there afterwards. There will be new jobs that we're not even thinking of that will be created. What will the balance be? What is the shape of these new jobs? How to think about it? And I think that fundamentally, the way that I think about it is that
[0:30:33] The shape, like one thing I do think is true about this AI revolution is that I think that we are going to change a lot of fundamentals of the social contract. I think we are going to go to a world that is one of abundance, right? And I think one where, you know, that you should have like this amazing quality of life, even if you are not economically working because there's just so much, but that if you...
[0:30:58] I just started my job, so I'd like to keep it.
[0:31:27] Like to be hard for AI to change is actually things about human connection, right? It's like that's something very interesting. I also think for what it's worth, just like skilled trades like plumbers and electricians, like I think they're already in short supply. And I think it's going to be really difficult for us to go and build AIs that make dense and, you know, that is even able to add value in those domains.
[0:31:50] Let's talk about Codex and other products that OpenAI releases. We're at a developer event. We have a room full of developers. You announced AgentKit. How should developers building on top of OpenAI think about potential platform risk? And I'm sure you think about this internally. I'm sure you've gotten this question before.
[0:32:13] I think the meme is every time OpenAI has a dev day, there's a thousand startups that died. I don't believe that, but I wanted to get your thoughts on what does that conversation internally look like? Where is that line being drawn between this is what we're building and this is what we provide a platform for others to build? Yeah, we get this question a lot. We think about it a lot. And it's also very important to us. We want ultimately to help transition the world to this AI-first economy and have that be something that uplifts everyone. And we can't do that alone.
[0:32:42] We just absolutely cannot, right? We really work with developers. We need people building on top of our platform, figuring out how to connect this technology to the real world. And we have to pick, right? Because we are a company. We're a few thousand people now. Sounds like a lot of people. But if you look at the size of the whole economy, it's tiny, right? You look at the number of domains, the amount of expertise required to do a good job in each one. So we have to be very choosy. And what we really try to think about is what are domains that synergize?
[0:33:11] And ones that we either have expertise or that we can really see, you know, example coding, right? That's something where we know a lot about coding. And also if we do a good job of coding, it speeds up our own work, right? Nice synergy right there. And so I'd say that we try to like really think about how to amplify as many people as possible and then go deep in specific domains that we feel like we can really add value to. Do you think code is the language of AGI? That's a great question.
[0:33:39] I honestly think that just natural language is going to be the language of AGI. Like I think that they'll talk to each other, might be like slightly more optimized English or something. If you look at, for example, some of the math proofs that we produced for the IMO this year, where we got the gold medal, you can see it's like the proofs are actually quite readable. But they're like very terse and it's just like an interesting dialect that the AI has discovered.
[0:34:10] Are humans going to be in the loop for a while? I think as these models get better, I'm seeing kind of the middle to middle push outwards. But there still is room for humans currently on the beginning, prompting in the end, verifying. How long is that going to be the case? Is it going to be the case forever? How do you see that playing out? Well, I do think that the fundamental purpose of this technology is to benefit humans, right? And really not just humans, right?
[0:34:38] Animals, just like all living beings that can sort of experience pleasure and joy. And I think that that is something where AI should be something that uplifts everyone. And so the question is, well, what does that mean? I don't think that we want to be in a world where it's humans have to like.
[0:34:58] Craft a prompt and write the code for context engineering. These are the mechanical details that to me feel legacy. They feel like what computers were, not what computers are supposed to be. And so what I want and what I think the world should want is AI tools that make the machine move closer to the human and understand what is it that are your goals and help you achieve your goals. And so I think that that to me is the real key. I think that we want to make sure that...
[0:35:27] We are uplifting humanity. That's the whole mission of OpenAI and trying to move the technology in a direction that achieves that. Okay. So for someone who thinks about coding often, you obviously spend a lot of time in Codex and thinking about building out what is the natural language, coding language.
[0:35:48] I actually asked you this in person a few months ago when I met you. Do you think that software is going to be fully generated at a certain point all the way down to the operating system level? Every single pixel on the screen that you see is going to be generated in real time, assuming we can solve for consistency? I think so. I think it's going to be super cool. It's actually a little bit mind-bending to think about what a fully generative UI would look like, right? Just like a real-time Sora-like thing that you're like...
[0:36:15] Doing stuff like, are there buttons? Are there not buttons? Like, what's the most natural thing? Like, it makes you start to realize that probably a lot of the interfaces we build are around the proclivities or like sort of the way our operating systems actually work right now. But if you could reimagine it from scratch, there's no legacy code. There's no concept of like files and folders and whatevers. What would it look like? And I don't feel like I actually know the answer. But again, I guarantee it's going to be something like totally surprising. Okay, let's envision that future a little bit.
[0:36:43] Are there still developers? Are there still applications in that world? It just seems like probably not, but what am I missing? Well, take a look at something like Sora, right? And by the way, Sora is something that's really interesting for me because...
[0:36:58] I remember watching one of the promo videos that we did where Bill's driving around on a snowmobile and he takes his helmet off. And I was like, wow, Bill's so good at snowmobiles. And I'm like, wait a second. He did not do this. And you realize that it's exactly how the human is involved. It's just very different. It's very different from a movie where he actually was snowmobiling. But he's still involved. He was thinking about the creative process of this. It's his cameo. There's something about him that is present within this video. And you make a...
[0:37:28] I saw a video with a cameo of someone. You're sharing it. You're excited about it. And the fact that you are excited about it actually is something that makes me excited about it. And we actually had this lesson from earlier this year when our image gen went viral. Everyone was generating these portraits of themselves and their family. And the thing is, we realized that if you just make a generated image, just some...
[0:37:53] Image that has no grounding in like being a picture of your dog now turned into, you know, some cool anime style. No one cares. It's boring, right? Just like, it's like cool, whatever. But as soon as there's some humanity in it, as soon as there's some element of like the grounds it with this, this connection, then people are really interested. And I think you can have this amplification of like, it was like a picture of your, you know, of your, of your kid. And now there's like a ton of AI interesting stuff that happened. And that that artifact is something that people connect to. But I guess that.
[0:38:23] To bring it back to software, I wonder if that will be the case, too, that people are going to build apps because it'll be this, like, you imagine how some sort of, you know, dynamic system could work or, you know, whatever the future of apps are. Wouldn't it be cool if you had something that you share in a certain way? And then, yeah, the AI is your developer and you outsource to them and they produce this, like, massive piece of code or, you know, fully generative UI. But then you share on the ChatGPT app store, perhaps. It really sounds like curating.
[0:38:49] A great human experience, or even a shorter word, taste, is going to be critically important in the future, more so than any of these more hard skills like being able to actually develop the app. It's you just curating this experience. Is that what you believe? I think I believe a shape of that. Yeah, I think that's a good higher bit. I do think there are a bunch of mechanical skills that I think will transfer, and we see that from generation to generation of models of people who really try to go and explore what the model's capable of, somehow tend to get the best results.
[0:39:18] But yeah, fundamentally, knowing what you want, having good judgment and taste, I think those are critical.
[0:39:48] The thing about this field is there are no new ideas. All of these ideas are things that people have thought about, we've thought about many times. The thing that's new is models that are capable enough to actually make good use of them. And you can see this with plugins, right? We did plugins a couple of years ago. The models just were not ready for it, right? That's just like we couldn't have more than like three plugins active at a time because if you have too many functions, then the model gets confused, doesn't know how to call them. And just like today's models.
[0:40:13] Okay. Okay. Let's talk about the future for a second. I mean, I guess that's all we're talking about right now, but so much has happened from last...
[0:40:43] Cool. Tough questions. I mean, I do think next year we will have some incredible models. The milestone I am most excited about is really having models that can solve hard problems, right? And the analogy I like to make is think back to AlphaGo.
[0:41:13] You know, 2016, Move 37, right? It's just like, you know, change people's understanding of the game. And imagine that in coding. Imagine that in material science. Imagine that in medicine. And so I think having real breakthroughs that are either potentially the AI by itself, but I think the AI assisted with top humans. And I think we'll start seeing that. So what would be the utility of that for developers? I mean, it's almost...
[0:41:37] Indescribable. It's like for any sort of domain, it's like you want to help someone in finance and you can build like the most amazing application that can go solve their hardest finance problem. Like, you know, maybe we won't be at the top finance problem, but I think we're going to be at like really hard problems. And by the way, it's going to be a lot of compute. So that's one thing is like, we got to make sure that these tasks that people apply these things to are economically valuable because otherwise no one's going to be willing to fund that compute.
[0:42:01] Maybe there will be some that we'll be able to do because they're something that people want to do for the world. And that's something that I think would be very aligned with our mission. But I think it'll just be like something that we're going to have to really think about how to bring this technology and this kind of breakthrough machine to life. 2030, I think, is really hard to predict. I think that's within well past many people's AGI time horizons at this point.
[0:42:26] AGI time horizon. I have to ask. Yeah, of course. Of course. And I mean, you know, it's all the usual like the fuzzy definition, whatever. But I think we're in the like one to three year time horizon. I think maybe more at the three than the one. But like I would be kind of surprised. I feel like something kind of went wrong if we were not there by 2030. Do you still have the same definition of AGI where I can do most economically viable human work? What is your current definition? Has it changed? I do think the fundamental thing that has changed is that I really used to think of it as it's.
[0:42:56] This destination. We're just building OpenAI to complete the mission. But instead, we really think of it, and I think I've sort of grown in maturity in how we think about it, as this continuous process. And there's a particular point, which is AI that can match humans at economically valuable work, whatever the definition that we used in 2018 is. And it's an important milestone, but it's not the end.
[0:43:20] And I think that this is really important because it's really about that follow through, right? It's really thinking about, you can see people starting to shift from, okay, talking about AGI to super intelligence or people who reject all of these words altogether. To me, that's not the most important thing. To me, the important thing is saying, can you make AI progress? Can you uplift the whole economy? Can you actually deliver these benefits to people? And you need to think about.
[0:43:40] What that means, right? You think about something like Sora, right? You think about how we're approaching safety and chat GBT. These are important topics. They're very core to our mission. And so we really try to think about the whole end-to-end. But yeah, I think that there's going to be a point there that we'll look back and say that that was basically what we were talking about in 2018. And I think it's not that far away. All right. Everybody, Greg Brockman, thank you so much. Appreciate it. Appreciate it.