[0:00:00] Hi, guys. Happy South by. I feel like what a way to kick this off. One of the things I love about South by Southwest is I've been coming for the last decade, and we're always talking about what's the next big thing in tech. And I would say artificial intelligence and ChatGPT couldn't be more relevant. So glad to be sitting here with you. How many folks in the audience have used ChatGPT?
[0:00:26] Okay, so it feels like this is an audience that we can... That's good. I can be very specific on this stuff. And remember, you guys ask questions. I'm going to leave 15 minutes at the end to get to it. So I want to get to OpenAI, and I want to talk about the company behind ChatGPT. But I would love to start with ChatGPT. So let's go. It's November 22nd. You guys released ChatGPT. This is an AI chatbot that's developed...
[0:00:52] By OpenAI, it's built on top of large language models, a large language model called GPT-3. You release it November 2022. Over 100 million users in two months. This becomes the fastest growing application in history. Just for some perspective, it took Facebook meta 4.5 years to reach 100 million users, took TikTok nine months. Why was ChatGPT the killer app?
[0:01:22] I actually think about this question a lot. Because for us, we actually had the technology behind it, the model behind it, created almost a year prior. So it wasn't new technology. But the thing that we really did differently is that we did a little bit of extra work to make it more aligned. So it really, you could talk to it. It would do what you wanted. But secondly, we made it accessible. We built an interface that was super simple. It was kind of the simplest interface we could think of.
[0:01:52] We made it available for free to anyone. And I think that the thing that was very interesting was as this app really took off and people started using it, we could see the gap between what people thought was possible and what actually had been possible for quite some time. And I think to me, this is actually maybe the biggest takeaway, is that I really want us as a company and as a field to be informing people to make sure that they know what's possible, what's not.
[0:02:20] Kind of what the forefront is going to look like and where things are going. Because I think that's actually really important to figure out how to absorb this in society. Like how do we actually get all the positives and how do we mitigate the negatives? Like in the past, I mean, should we talk about Tay? We won't talk too much about Tay. But like chatbots are hard to put out there. But there was something about what you put out there and you talk about that gap, right? That it didn't implode, right? It learned a lot. And all of a sudden it's almost...
[0:02:47] Spurred this whole new era of everyone saying, could we do this? Could we do this? Could we do this? Why now? Yes. So as we were preparing ChatGPT for release, the thing I kept telling to the team was the most important thing.
[0:03:01] We can be overly conservative in terms of refusing to do anything that seems even a little bit sketchy. That's fine. Most important thing is that we don't have to turn it off in three days. Because that is- Are you worried when you pressed publish on this? You have to be worried. How could you not? We've been doing lots of testing. We have our own internal red teams. We'd had beta testers on it, hundreds of beta testers for many, many months. But it's very different from exposing it to the full diversity and adversarial.
[0:03:26] And sort of beautiful force of the world and where people are going to apply it. And so for us, I think that we have been doing iterative deployment for a very long time. We've been ever since 2020 June or so is when we first released a product in API so people could use these language models.
[0:03:45] We've been making them more capable, getting them into more people's hands. But we kind of knew this was going to be just a different dimension. And it was our first time building a consumer-facing app. And so we definitely were nervous. But I think that the team really rose to the occasion. Yeah. Well, I want to look. I definitely want to talk about the future of ChatGPT because I know a lot of folks, especially we have a lot of users in the audience, are curious about it. But let's look. I want to go to the past, right? Because the company behind ChatGPT, Dolly,
[0:04:12] And this is, it's interesting because in the Silicon Valley world, you have like a sexy company, it comes out, everyone's talking about it. OpenAI was just kind of the opposite. It just was kind of like hanging out in the background until this thing came out, until you put out these products that could shift culture and start all these questions. And so let's go back. It's 2015, July, and you're in Menlo Park at a fancy hotel called the Rosewood. I don't know if anyone here has been to the Rosewood. It's certainly a scene.
[0:04:42] So you're sitting there. Who's there? What are we eating? Why are we there? What's the topic of conversation? And I promise I'm going somewhere with this. Well, I couldn't tell you what was on the menu that night. I just want to know what Elon Musk was eating during this conversation. Okay, sorry. I got ahead of it. Go ahead. So we were having a dinner to discuss AI in the future and kind of just what might be possible and whether we could do something positive to affect it. And so my co-founders at OpenAI, so that's Elon, Sam, Ilya, and other people were all there.
[0:05:11] And kind of the question was, is it too late to start a lab with a bunch of the best people at it? We all kind of saw that AI feels like it's going to happen. It feels like AGI, really building human-level machines, will be achievable. And what can we do as technologists, as just people who care about this problem, to try to steer in a positive direction? And kind of the conclusion from the dinner was, it's not obviously impossible to do something here.
[0:05:37] You felt a sense of urgency. I did. Why? I think the thing that is easy to miss here is I think now people see ChatGPT and they say, wow, suddenly you feel the possibilities. And you both see what's possible, not science fiction anymore, actually usable today. But it's still hard to extrapolate, to really follow the exponential, to think what might be possible tomorrow. And I think that the mode that I have been in for
[0:06:06] A long time has been really thinking about that exponential. I remember reading Alan Turing's 1950 paper on the Turing test. And the thing that really stuck out to me, and this was right after high school, was he said, look, you're never going to program a machine to solve this problem. Instead, you need a machine that can actually learn how to do it. And that, for me, was the aha moment. The idea that you could have a machine that could solve problems that I could not, that no human could figure out how to solve.
[0:06:34] That so clearly could be so transformational. There's all these challenges, global warming, medicine for everyone, all these things that are kind of out of reach. I don't know how we're going to do it. But if you could use machines to aid in that process, we want to. And so I think we all kind of felt like, okay, the technology is starting to happen. Deep learning is an overnight success that took 70 years. It's like 2012, there was a big breakthrough on image recognition, but it really took another decade to start to get to the point that we're at now.
[0:07:02] But we could all see that exponential, and I think we really wanted to really push it along and really steer it. And I mean, you at the time, so before you were the CTO of Stripe, this little company called Stripe. And you really felt that you, Sam Altman at the time, Elon Musk, we can get into all this later, but that you guys could build something better, and you guys could build something that was pro-humanity and not...
[0:07:25] Anti-humanity, which there's always that fine line in technology, which I think the last decade has kind of taught us. Yeah, and I would quibble a little bit with, you know, I don't know that, at least for me personally, that I viewed it as we would build something better, you know, in the sense of like, you know, there's lots of other people who are in this field doing great work too.
[0:07:43] But I wanted to contribute. And I think one thing that's actually very important about AI and something that's very core to our values and our mission is that we think this really should be an endeavor of humanity. If we're all thinking about, well, what's my part of it? What do I get to own? I think that is actually one place where the danger really lies.
[0:07:59] And so tell me about how the company was and is structured because now that was seven years ago. So take us behind the curtain. I saw something Sam Altman wrote. He said, we've attempted to set up our structure in a way that aligns our incentives with a good outcome. What does that even mean? Yeah, so we are a weird looking company. In what sense? So we started as a nonprofit because we had this grand mission, but we did not know how to operationalize it, right? We know that we want to have AGI benefit all of humanity.
[0:08:28] But what does that mean? What are you supposed to do? And so we started as a research lab. We hired some PhDs. We did some research. We open sourced some code. And our original plan was open source everything. You think about how you can have a good impact. Maybe if you just make everything available to anyone that can make any changes they want, then if there's one bad actor, well, you've got 7 billion good actors who can keep them in check. And I think that this plan was a good place to start.
[0:08:56] Ilya and I, we were really the ones running the company in the early days, spent a lot of time really thinking about how do you turn this into the kind of impact that we think is possible, into something that really can make a difference in terms of just how beneficial AGI ends up being. And I think that we found two important pieces. One was simply a question of scale.
[0:09:25] All the results that we were getting that were impressive and really pushing things forward were requiring bigger and bigger computers. And we kind of realized that, OK, well,
[0:09:35] You're just going to need to raise billions of dollars to build these supercomputers. And we actually tried really hard to raise that money as a nonprofit. I remember sitting in a room during one of these fundraisers and looking in the eyes of a well-known Silicon Valley investor. Who is that? I wouldn't share the name, but he was like, $100 million, which is what we're trying to raise. He's like, that's a staggering amount for a nonprofit. Right.
[0:10:04] And we looked at each other. We were like, it is. Yeah. And we actually succeeded. We actually raised the money. But we realized that 10x that. That was not going to happen. I mean, if anyone in this audience knows how to do that as a nonprofit, please, we will hire you in an instant. But we realized that if we wanted to actually achieve the mission, that we needed a vehicle that could get us there.
[0:10:31] And we're not anti-capitalists. That's not why we started OpenAI as a nonprofit. Actually, capitalism is a very good mechanism within the bounds that it's designed for. But if you do build the most powerful technology ever in a single company, and that thing becomes just way more valuable or powerful than any company we have today, capitalism is not really designed for that. So we ended up sort of...
[0:10:54] We're designing this custom bespoke structure. It's super weird. We have this limited partnership with all custom docs. If you're a legal nerd, it's the kind of thing that is actually really, really fun to dig into. But the way we design things is that we actually have the nonprofit as the governing body. So there's a board of a nonprofit that kind of owns everything. It owns this limited partnership that actually has profit interest, but they're capped.
[0:11:18] So there's only a fixed amount that investors and shareholders are able to get. And that there's a very careful balance in a lot of these details in terms of having the board have a majority of people who don't have profit interests, all these things, in order to really try to change the incentive and make it so that the way that we operate the company comports with the mission.
[0:11:40] And so I think that this kind of approach of really trying to figure out how do you balance, how do you approach the mission, but how do you make it practical? How do you operationalize it? That is something that has come up again and again in our history. And if we look back at the history of artificial intelligence, this is nothing new, obviously. So what is it about now that feels like a watershed moment? Why now are all companies putting money into this? Why now is this the thing that we all...
[0:12:05] What is it about the technology now? Yeah. Well, I think the fundamental thing here is really about exponentials, right? It's like no matter how many times you hear it, it is still hard to impossible to internalize. And when I look back, like we've done these studies on the growth of compute power in the field and we see this nice exponential.
[0:12:28] With a doubling period of like every 3.5 months, you know, as opposed to 18 months for Moore's Law. It's been going on for the past 10 years or so. But we actually extrapolated back even further. And you can see that this exponential continues all the way. Slightly smaller slope. It used to be Moore's Law. But over the past 10 years, basically, people have been being like, well, you could go faster than Moore's Law by just spending more money.
[0:12:46] And I think that what's been happening is we've been having this accumulated value, this slow roll. Rather than trying to do a flash in the pan, like just get rich quick kind of thing that maybe other fields have been accused of, AI, I think, has been a much more steady, incremental build of value. And I think that the thing that's so interesting is normally if you have a technology in search of a problem, adoption is hard. It's a new technology. Everyone has to change their business. They don't know where it fits in. For AI, for language in particular,
[0:13:14] Every business is already a language business. Every flow is a language flow. And so if you can add a little bit of value, then everyone wants it. And I think that is the fundamental thing that really has driven the adoption and the excitement is that it just fits into what everyone already wants to do. Well, and also in 2017, you know, a model called Transformers, right? These large language models and this idea that you could treat everything as a language, music and code and speech and image.
[0:13:42] The entire world almost looks like a sequence of tokens, right? If we could put a language behind it. That was really an accelerant for a lot of what you're building too. Yeah, I think that the way they think about the progress, like the technological driver behind this, is that it's very easy to latch onto any one piece of it, right? Transformer, definitely a really important thing. But where the transformer came from was really trying to figure out how do you get good compute utilization out of...
[0:14:09] The compute hardware that we use, these GPUs. And the GPUs themselves are really impressive feat of engineering that has required just huge amounts of investment to get there. And the software stack on top of them. And so it's kind of each of these pieces. And each one kind of has its time. One thing that's super interesting to me, looking from the inside, was that we were working on language models that looked very similar to what we do today, starting 2016.
[0:14:34] We had one person, Alec Radford, who was really excited about language. And he just was kind of working on building these little chat bots. And we really liked Alec. And so we were just very supportive of him doing whatever he wanted. And meanwhile, we were off investing in serious projects and stuff. And we were just like, whatever Alec needs, we'll make sure he gets. And 2017, we had a first really interesting result, which was that we had a model that was trained on Amazon reviews.
[0:15:03] That it was just predicting the next character, the next character, just what letter comes next. And it actually learned a state-of-the-art sentiment analysis classifier. You could give it a sentence and it would say, like, this is positive or negative. It may not sound very impressive, but this was the moment where we kind of knew it was going to work. It's so clear that you had transcended just syntax, where the commas go, and you'd move to semantics. And so we just knew we had to push and push and push.
[0:15:29] I mean, it always comes to Amazon reviews. Who knew that this is the real story behind it? Exactly, exactly. You always start small. You know, every day there's a new headline on how this technology is being adapted. I just literally was Googling it yesterday. It's like the latest headlines are companies are harnessing the power of a chatbot to write and automate emails with a little bit of personalization. Another headline, how ChatGPT can help abuse survivors represent themselves in court if they can't afford otherwise. We obviously know about Microsoft's being in a disrupting search. From the seat that you're sitting,
[0:15:58] What, for you, and if you could be as specific as possible, what do you think are the most interesting and disruptive use cases for generative AI? Yeah, well, you know, I actually first want to just tell a personal anecdote of the kind of thing that I am very hopeful for. So, you know, medicine is definitely a very high stakes area. We're very cautious with, you know, how people should use this kind of technology there. But even today, I want to talk about a place where I have just been like, I really want
[0:16:28] For my own use. So, you know, my wife, a number of years ago, had a mysterious ailment that she had this pulsating pain right here on her abdomen, bottom right side. And it wasn't appendicitis. You know, we went to the first doctor and the doctor was like, oh, I know what this is. And prescribed some antibiotic.
[0:16:46] Nothing happened. Went to a second doctor who said, oh, it's a super rare bacterial infection. You need this other super powerful antibiotic. Took that. And over the course of three months, we went to four different doctors until finally someone just did an ultrasound and found what it was. And I kid you not, I just typed in a couple sentences of description that I just gave here into chat GPT. It said, number one, make sure it's not appendicitis. Number two, ruptured ovarian cyst. And that is, in fact, what it was. Wow.
[0:17:16] And so the kind of thing that I want is I personally in the medical field want something that I don't rely on. I don't want it to replace a doctor. I don't want it to tell me like, oh, go take this, you know, super rare, you know, antibiotic. I don't want a doctor telling me that either. Also, Chachi Petit sometimes confidently says the exact wrong thing. It's kind of like a drunk frat guy. Exactly. You got to be careful. Something we're working on. Yeah, right. It's actually interesting. We're actually, just quick aside, we're actually finding that
[0:17:44] Our models actually are much more calibrated than we realize and can say when they're right or wrong. But we currently destroy that information in some of the training processes we do. So more to say there. But yeah, I think this suggests, give you ideas really in writing.
[0:18:00] It's like the blank page problem. But I think this, for me, is where generative AI can really shine, right? It's really about sort of unblocking you, giving you ideas, and just giving you an assistant that is willing to do whatever you want 24-7. And so you've now, the ChatGPT has been deployed to millions. Has there been anything that's really shocked you or surprised you in how people have been utilizing it? I mean, of course. Yeah, I mean, I do think that for me,
[0:18:29] The overall most interesting thing has just been seeing just how many people engage with it for so many just sort of surprising aspects of life, right? Like what? Well, you know, I think that knowledge work is maybe the area that I kind of see as most important for us to really focus on. And, you know, we see people within OpenAI who don't have, who aren't native English speakers use it to improve their writing.
[0:18:57] And at first, there was someone with OpenAI who is suddenly his, you could just tell the writing style of everything changed. And it was just like way more fluid and just also just like, honestly, just like way more understandable. And at first, we're like, what just happened? And he literally at one point had hired someone to
[0:19:16] But that was actually really hard. It was just like a lot of overhead and he wasn't able to get the points across. But with ChatGPT, he really was able to. And I think that that for me is just like so interesting to see that people just use it as this aid, this cognitive aid to think just more clearly and to communicate with others. Well, you always know you have disruptive technology when you put it out there and people misuse it. I remember...
[0:19:38] A decade ago doing like a story on like pimps recruiting women on Facebook, right? Which is like, okay, you know, if someone's using your technology in a bad way, like you have something that's hitting mainstream. So like, can you tell us like what, how are people using it in ways that it's not designed for? What have you learned from putting this out there? And what have you learned from how people are misusing it? Yep.
[0:19:56] Well, misuse is definitely also very core to what we think about. Part of why we wanted to put this out there was to get feedback, to see how people use it for good and for bad, and to continually tune. And honestly, one of the biggest things that we've seen, we always anticipate all the different things that might go wrong. For GPT-3, we really focused on misinformation. And actually, the most common people
[0:20:19] The most common abuse vector was generating spam for drugs, for various medicines. And so you don't necessarily see the problems around the corner. For ChatGPT, one thing we've just seen is people just creating thousands or hundreds of thousands of accounts in order to just be able to use it much more. Some people generating lots of spam. It's clear that people are using it for all sorts of different things. I think for individuals, there's definitely
[0:20:47] I think actually, I would say this is an interesting category to your point where it says something that is confidently wrong. My drunk frat guy point. Exactly, yeah, that over-reliance, right? And thinking, oh, because it said that, it must be true. And that's not true for humans. It's not quite true for AIs. I think we will get there at one point, but I think that it's going to be a process and something we all need to participate in. Right, and so, I mean...
[0:21:12] I would love to get into kind of what we can predict in the future with AI, but before we leave ChatGPT, this isn't really ChatGPT, but I feel like we have to talk about Sydney for a moment. People in the audience, who read Kevin Russo's article in the New York Times? Right, so just a little background. You guys put ChatGPT out there, Microsoft, Google racing to get search products out there.
[0:21:40] Microsoft releases its own AI-powered search, a Bing chatbot, and all of a sudden, Kevin Roos, great writer at the New York Times, is playing with it, with the Bing chatbot. It reveals that its shadow name is Sydney, and also tells Kevin, when prompted a certain way, I want to be alive, and tried to persuade him to leave his wife. Obviously, that's an awkward conversation. To be clear, Microsoft's an investor and partner. This isn't something that...
[0:22:08] OpenAI specifically put out there. But I do think it's an interesting point of saying you put this stuff out there, the next thing you know, I don't know, Sydney's trying to make you leave your wife. So what are the guardrails that need to be put in? What have you learned over the last couple months where you've seen the misuse? And what can you put in to make sure that we're not all trying to leave our significant others because bots are telling us to? I mean, look, I think this is actually a great question, right? And I think that even the most,
[0:22:38] High order bid, right? The most important thing in my mind is this question of when. When do you want to release? Right. And my point earlier of, well, there was this overhang in terms of this gap between people's expectations, what they were prepared for, and what was actually possible. And I think that's actually where a lot of the danger lies. You know, we can kind of...
[0:22:59] Joke about or laugh about this article because it wasn't very convincing. You know, just like chatbot saying, you know, leave your wife. Sydney was pretty spicy. Yeah, it was very spicy, right? But did not actually have an impact, you know? And I think that is actually, in my mind, the most important thing is trying to surface these things as early in the process as possible, right? Before you have some system that is much more persuasive or capable or able to operate in more subtle ways because we want to build trust.
[0:23:27] And figure out where we can't trust yet. Figure out where we put the guardrails in. So that to me, this is the process. This is the pain of the learning. And that we've seen this across the board. We've seen places where people try really hard to get the model to do something. And it says, sorry, nope, can't do that. We've seen places where people use it for positive things. And we've seen cases where people
[0:23:48] Have outcomes like this. And so I think that my answer is that we have a team that works really hard on these problems. We have people who build on top of us, who customize the technology in different ways. But fundamentally, I think that we're all very aligned in terms of trying to make this technology more trustworthy and usable. And we do a lot of red teaming internally. We hire experts in different domains. We hire lots of people to try to break the models.
[0:24:18] When we actually released it, we knew we'd kind of cleared a bar, we felt, in terms of just how hard it was to get it to go off the rails. But we knew it wasn't perfect. We knew that we had come up with some ways to get around it with sufficient effort, and we knew that other people would find more too. But we've been feeding all that back in. We've been learning from what we see in practice. And so I think that this sort of loop of there being failures, I think that's important, because if not, it means you're kind of holding it
[0:24:48] Well, I think it's also, we'll get to a little bit later, but an important segue too to talk about the future of misinformation and how we can prep now for what's coming with this innovation. Before we get to it, I mean, I...
[0:25:12] I think one of the most interesting things to me is the ability for this technology to synthesize information and make predictions and identify patterns. So can you tell me what you think the most interesting future use cases of what artificial intelligence will be able to predict will be? Like predict disease, predict stock market, predict if you're going to get a, not you, if someone's going to get a divorce, you know, like what could this predict? Paint the image of the future. Well, I think that the real...
[0:25:40] My story here in my mind is amplification of what humans can do. And I think that that will be true on knowledge work. I think that it will just be that we're all, it's kind of like if you hire six assistants who are all like, you know, they're not perfect, they need to be trained up a little bit, they don't quite know exactly what you want always, but they're so eager, they never sleep, they're there to help you. They're willing to do the drudge work. And you get to be the director. And I think that that is going to be
[0:26:09] What writing will look like. I think that's what coding will look like. I think that's what business communication will look like. But I also think that is what entertainment will look like. You think about today where everyone watches the same TV show and maybe people are still upset about the last season of Game of Thrones. But imagine if you could ask your AI to make a new ending that goes a different way.
[0:26:36] And maybe even put yourself in there as a main character or something. Having interactive experiences. And so I think it's just going to be every aspect of life is going to be sort of amplified by this technology. And I'm sure there are some aspects or people or companies that will say, I don't want that. And that's okay. I think it's really going to be a tool just like the cell phone in your pocket that is going to be available when it makes sense. I think we think a lot at...
[0:27:06] At my company, we're knee-deep in exploring how artificial intelligence can personalize content, develop closer relationships with the audience, which is a wide open space and an interesting space. But also, there's so many ethics that come up with that, so we're developing a lot of these ethical frameworks around it. I'm curious, when you talk about Game of Thrones and personalized media and being able to put yourself in it, when we look at the future of...
[0:27:29] Media and entertainment, would you say this is a new frontier for personalized media? Yeah, I think for sure. I mean, I kind of think it's a new frontier for most areas. It may not be great yet at some domains, but I think that we are just going to see just like way more creative action happening. And to me, actually, the thing that's I think most sort of encouraging is I think it will be the barriers to entry decrease. And this is...
[0:27:58] By the way, how we thought about things at Stripe. Decrease the barrier to people making payments online, integrating them into their services. Way more activity happens, things you would never think of. And I think we'll see this in content. Individuals who have a creative idea that they want to see realized, they now have a whole creative studio at their disposal. But also the pros, the people who really want to make something good, will make just something way better than any of the amateurs could. And we've seen this with Dolly. There's literally these 100-page books that people write.
[0:28:27] And so I think that skill doesn't go away. I think it's this multiplicative effect. I mean, but there will also be all these murky questions around identity and attribution as these models go mainstream. So it's not perfectly clear what the data sets are used to train. So when we take a step back, and this is a more fundamental question, should an artist's style with models trained on their work, should it be available to folks?
[0:28:53] To anyone without use of attribution, what are you guys thinking about when it comes to these ethical? Yeah, so we engage very closely with policymakers, and I think this is a really important conversation to have. Fundamentally, we as a company want to provide information and to show just like.
[0:29:10] Kind of what's possible and let there be a public conversation about these topics. Like, I don't think that we have all the answers, but we think it's really important to be talking about. So take for me, take me for example, right? I like to put myself in the, I'm like the beta test. I'll put myself in the driver's seat. So let's say someone took all the footage of me interviewing folks like you, Zuckerberg, whatever, throughout the years. And they included my voice, my body. They trained this as like a Lori, Lori model.
[0:29:36] I've already named it. I don't know. Please don't do it, guys. And then I don't know why I'm inviting this. But then they launched a podcast using my likeness, my style, my voice. Hopefully you'd have fabulous style. That would be all I'd ask. But could they do it? Should they get it cut?
[0:29:54] Should I get a say in it? As a content creator, as someone who's sat at the center of these ethical questions about the future, what does that look like? Again, I think this is a great question and I think it would be kind of hubristic of me to say that I have all the answers, but I can tell you a little bit about how we think about it. As a company, our mission to build AGI that benefits all of humanity, we've kind of built with this cap profit structure and I really think that
[0:30:24] In answer on this question, but more broadly, how do you make sure that all of humanity are kind of stakeholders in what gets built? And everyone benefits if it's access to these services, if it's that you're able to kind of have your AI personality or this AI that you build up that represents you and sort of build a business with that. I think all of this is on the table. And I do think that we need some sort of like, I think that society as a whole needs to adapt.
[0:30:54] There's no question that something is changing. And I think that we need to lean into that question. Do you think, to get a little black mirror, but why not? Do you see a future where we verify our own AI identities and we can license them out? So I could license out my likeness to some degree? Yeah. Again, I think kind of everything is on the table. I think actually this, to your earlier question too, of why now, what's happening now, is I think everyone kind of senses it. That we're building almost this like,
[0:31:23] New kind of internet or something like that. In what sense? Well, I think that where content comes from, you know, in good and bad ways, right? How it's created, like what an application is. You know, there's Web 1.0 and 2.0 or something. And, you know, I'm not going to talk about Web 3.0. Is it too soon? There you go. Yeah, no, I've never, yeah. More to say there. But I think that...
[0:31:51] Where we're going is what an application is will be very different. Right now, you think of this content that was written by someone that's very static. You can't really interact with it. But we're clearly moving to a world where it's alive. You can talk to it, and it understands you and helps you. Honestly, every time I go through some menu and I keep trying to find where I'm supposed to click, I'm like, why is this still here? And I think in the future, it will not.
[0:32:17] Going back to kind of the next iteration of ChatGPT, and it was built on GPT-3, correct? 3.5. Okay, 3.5. How much powerful is the current technology you're building? Well, you know, we are continuing to make, I say, significant progress. But like blink twice if it's 10 times more powerful. Or, okay.
[0:32:44] Three times. There we go. I guess all I can say is that I can't comment on unreleased work, but I can say that we work really hard both on the capability side and on the safety side, and that there's been a lot of rumors swirling around about what we're going to be releasing and what's coming out. And what I can definitely say is that we do not release until we feel good about the safety and the risk mitigations.
[0:33:09] And I mean, you guys have the ability to turn up the dial, turn down the dial. And we've seen, I joke about ChatGPT confidently, it does so many fascinating things. And it sometimes confidently says the wrong thing. Like I was asking at my bio and it confidently said three out of four things that were correct, right? So can you give any insight, maybe speaking like, I don't know, we could speak around it kind of about what future versions are going to look like? Will it be more cautious, more creative? Yep.
[0:33:37] And let me give you a mental model for kind of how we build these systems. So there's a first step in the process of training what we call the base model. And the base model is just trained to predict the next word. You just give it a bunch of text. You give it all the good stuff and all the bad stuff. It sees true facts. It sees math problems with good answers and sort of incorrect answers that no one tells it's incorrect answers. It sees everything.
[0:34:04] And it learns to predict. It learns to just, given some document, it's supposed to predict what comes next. And it has to think through everything of like, okay, I see some math problem, but is this maybe written by a student who doesn't really know that much? Was this written by Terence Tao? It has to kind of infer all these contextual things to figure out just what's the next word. So that model, it has every bias. It has every ideology. It has every idea that has been almost expressed.
[0:34:31] In this system kind of compressed and learned in a real way. And then we do a second step of reinforcement learning from human preferences of what we call post-training. And here you move from this like giant sort of sea of data of everything to really trying to hint to the model, okay, you kind of know all this stuff, but here's what you really should do. And here I think there's something that's very important, very fraught, right? This question of, well, what should the AI do? Who should pick that?
[0:34:58] And that, I think, is also a whole different conversation and something that we're really trying to get some legitimacy around. But that second step is where these sort of behaviors come from. And I alluded to earlier that we found that the base model itself is actually very calibrated on its own certainty. That if it spits out like, yeah, there's like a 10% chance this is right, 10% of the time, that thing will be right with quite good precision. But our current post-training process,
[0:35:27] This sort of next step that we do to really say, no, no, no, this is what you're supposed to do. We don't really include any of that calibration in there. The model really learns like, you know what, just go for it. And that, I think, is sort of a engineering challenge for us to address. And so you should expect that even with the current ChatGP team, we've released like four or five different versions since December.
[0:35:50] And they've gotten a lot better. Factuality improves. Hallucinations are a problem people talk about. Those have improved. A lot of the jailbreaks that used to work don't work anymore. And that is because of the post-training process. And so I would expect that we will have systems that are much more calibrated, that are able to check their own work, that are able to be much more calibrated on when they should refuse, when they should help you.
[0:36:15] But also that are able to help you solve more ambitious tasks. Like what? Well, you know, I think that the kinds of things that I want as a programmer is that, you know, right now, we started with a program called Copilot, which can do sort of, you know, just like autocomplete a line. And it was very useful if you don't really know the programming language that you're in, or you don't know specific library functions, that kind of stuff. So it's basically like, you know, being able to skip the dictionary lookup, and it just does it for you right there in your text editor.
[0:36:44] With ChatGPT, you can start being more ambitious. You can start asking to write whole functions for you. Or like, oh, you write the skeleton of writing a bot in this way. And I think that where we're going to go is towards systems that could help you
[0:36:56] Be much more like a manager, where you can really be like, OK, I want a software system that's architected in this way. And the system goes, and it writes a lot of the pieces, and it actually tests them and runs them. And I think this kind of giving everyone a promotion, making you into more of the bumping up a couple pay grades, I think.
[0:37:17] Literally and figuratively. I think that's like the kind of thing that they would do. So the future of chat TVTs, we're all getting a promotion. I think so. I think so. It's not too bad if we achieve it. I think there's obviously a lot of fear around the future of artificial intelligence, right? People say AI is coming for our jobs. Be honest with all of our friends here. What jobs are most at risk? Yeah.
[0:37:41] Well, the funny thing is the way I think everyone used to think about this, certainly that I did, was it's very clear the AI is coming for the jobs, just a question of what order. And clearly the ones that are menial or just require physical work or something like that, oh, the robots will come for that first. And in reality, it's been very different, that actually we've made great strides on cognitive labor,
[0:38:11] Think about writing poems or anything like that. And we have not made very much progress on physical things. And I think that this amplification is kind of showing a very different character from what was expected. But it's also the case that we haven't really automated a whole job. I think the lesson from that is that humans, I think, are much more capable than we give ourselves credit for to actually
[0:38:39] Do your job to do what you're doing right now. Well, I asked ChatGPT. These aren't the ChatGPT's questions. I had to follow up and say, can you be more hard-hitting? There you go. Well, thank you. Are these the hard-hitting ones? No, they're coming. Here we go. We're about to go into the future of truth right after this. There we go. Perfect. But ChatGPT, it's not up here on stage with me. There's a personal relationship aspect. There's this judgment aspect. There's so many details that are what you want from the person in charge.
[0:39:06] But the writing of the actual copy, I mean, who cares about the specific questions? So ChatGPT cannot replace me because it won't do the follow-up. Well, it probably will do the follow-up question. My follow-up question is, so give us a couple jobs most at risk. Yeah. Well, I'll tell you the one that I think is actually content moderator. So jobs, what I've really seen is jobs that you kind of didn't want human judgment there in the first place. You really just wanted a set of rules that could be followed, and you kind of wanted a computer to do it.
[0:39:34] Content moderation, I think, is just a difficult thing. I think we've all read about people having to read these pretty horrible posts and decide, is this thing sufficiently horrible or just slightly not sufficiently horrible to be disallowed? And that's something I already see this technology impacting. So that might be a good segue into the future of truth, right? Because I think we're entering this really fascinating, exciting, and scary era of the rise of...
[0:40:00] Deepfakes, these automated chatbots that could have the ability to persuade someone one way or the other. What happens to truth in an era where AI just makes fiction so believable? Well, I have a slightly spicy take here, which is that I think technology has not been kind in a lot of ways to journalism. And I think that AI and this particular problem might actually be something that is quite kind.
[0:40:31] And actually really reinforces the need for authoritative sources that can tell you this is real. We actually went out, had humans investigate this. We looked at all the different sides of this thing. And these are authenticated videos or whatever it is that can tell you what happened and what the facts are. And so I think that where we're going to go is away from a world where because
[0:40:57] Certainly, you saw some text somewhere that you can trust it's true. It's never really been the case. Humans have always been very good at writing fake text. Images, doctored images, those have existed since the invention of photography. But this gives us the ability to do this at viral speed. 100%. All the bad things that happened over the last decade, if we're not careful.
[0:41:19] This will amplify. Yes. To me, I agree with this. I think this is the crux is that the fact of being able to do these things at all, not new. The fact of being able to do it with a much lower barrier to entry, that's new. That will, I think, spark the need for new solutions. We've never had real answers for chain of custody of
[0:41:43] Information online. We've never really had verified identities. All these things people talked about since the beginning of the internet. But I think there was never really a need for it. And I think the need will come. Yeah. I was at an event for the folks for the Center for Humane Technology. They're the folks who did also like the Social Dilemma, which in my opinion, Social Dilemma was great, but it's like we've been having these conversations for 10 years before Netflix puts out a doc and asks these questions, right? So we're at the beginning of an interesting era and we should ask these questions before,
[0:42:13] We have to do a sexy doc on it in 10 years. So there was something that was said there that I thought was really important. They said that 2024 will be the last human election, meaning by 2028, we will see synthesized ads, viral information powered by artificial intelligence. Someone releases a Biden-Trump filter. Tens of millions of videos are going out there. We don't know who's saying what.
[0:42:33] What can be built now? What has to happen now, in your opinion, to get ahead of what will be the inevitable downside of this? Yeah, so I think this is a great question. And I think this is maybe also going to be a tip of an iceberg kind of problem, where it's the most visible one. It's clearly extremely impactful. It's one that has been very topical for a long time. But I think that we're going to see the same questions appearing across all sorts of human endeavor, of just as there's more
[0:43:01] How do you sift through for good creation? How do you actually find what is true or find what is high quality? How do you make sense of it? I think some of this is really going to be about people building good tools. We've seen this within, I think, the social media space. Even, for example, people building tools for cyber harassment to make it so that people can easily block.
[0:43:28] You know, various efforts and things like that. And I think that we need lots of tools to be built here that are really tackling this problem. And so that's one reason that we don't just build ChatGPT, the app. Actually, our main focus is building a platform. So we release an API. Anyone can use this to build applications. And I think that you have an opportunity, some using traditional technology, some using the AI technology itself.
[0:43:56] In order to actually sift through and figure out what is high quality curated and people want to put their stamp of approval on it. I remember the move fast and break things era of meta Facebook. Remember they used to have the signs that said move fast and break things. I know OpenAI puts these things out there in an iterative way and has a philosophy about limiting growth to some degree and getting feedback.
[0:44:20] I would say because of what's launched, there's this AI race with the biggest companies throwing in money, investing, and we both know that the economic incentives don't always align with what's best for society. What do you think we've learned from the last decade of tech innovation that we must use as we enter into this new era where the stakes, you could argue, are even higher? Yeah, we think about this a lot. I have spent a lot of time really trying to understand
[0:44:49] For each of the big tech companies, what did they do wrong? And right, but to the extent that mistakes were made, what are they? What can we learn? And actually, one thing I found very interesting is that there's not really consensus on that answer. I wish there was a clean narrative that everyone knew, and it's just like, just don't do this thing. Well, I could give an opinion. Please, I would love it. I've interviewed Mark Zuckerberg many times, and I would say, just having been across from some of those folks, I think...
[0:45:16] The biggest mistake is you is not understanding humans. In a nutshell, right? So how I think like...
[0:45:24] We've got the stamp of approval on it. Great. And the audience. So I think it's you. I mean, it sounds like you guys have done a lot of thinking into how you put this out there and how you build out these APIs that other people can build on. Who are the people that need to build out for these solutions? Who can you guys, now that you have a seat in Silicon Valley and you're at this really powerful place, who do you guys bring in that's different, diverse, and interesting? Yeah. So we do quite a lot of outreach. And I actually think this is one of the things that's going to be most important. Like even, for example,
[0:45:53] on how we make decisions on the limits of what the AI should say.
[0:46:00] We've written a blog post about this. But we think that this is something that really needs legitimacy. It can't just be a company in Silicon Valley. It can't just be us who's making these decisions. It has to be collective. And so we're actually, and we'll have more to share soon in terms of exactly what we're doing, but we're really trying to scale up efforts to get input to actually be able to help make collective decisions. And this kind of like question of global governance is something that has been really core to our goals from the beginning. And so I think that the...
[0:46:27] It's just so clear that you do need everyone to have a seat at the table here. And that's something we're very committed to. And then talking like regulation, I think it's OpenAI talks about moving at a bit of a slower pace, but these tools are being deployed to millions. So the FDA doesn't allow a drug to go out to the market unless it's safe. So what is the right regulation look like for artificial intelligence and what's happening? So yeah, this is, again, something we've been engaging with policymakers since day one, really. I did a couple of...
[0:46:54] Congressional testimonies back in 2016, 2017. It was so interesting to see that policymakers were already quite smart on these issues and already starting to engage. And I think that one thing we think is really important is really about focusing regulation on regulating harms. That it's very tempting to regulate the means. And we're actually seeing this right now with the EU AI Act. That's kind of a question of exactly how to operationalize some of these issues.
[0:47:22] The thing you really want is to really say, let's think about the stakes and really parse apart what are high stakes areas, what are low stakes areas. What does it mean to do a good job? How do you know? And these sort of measurements and evaluations, those are really, really critical. And so we think the government, it's a key part of the issue, right? This question of how do you get everyone involved? The answer is we have institutions that are meant for that. Right. Should there be a new regulatory body for artificial intelligence? Because often, remember when Zuckerberg went to Congress and they asked how Facebook made ads?
[0:47:52] I'm sorry, I have Facebook made money and the answer is like we sell ads. So really understanding because it certainly seems like there's going to be all these new issues. Should there be a new regulatory body for this? Again, I think it's on the table. I think more likely what I see happening is like I think that AI is just going to be so baked into so many different pieces and honestly so helpful in so many different areas that you kind of can't have the FDA not know about AI. You can't have any of these institutions be like, eh, someone else has got it, it's all good. And so I think that you do need some cohesive strategy.
[0:48:21] But I think that every organization, government or otherwise, is going to have to understand AI and really figure it out. I know we have to wrap soon because I want to get to questions, but I thought we could do a little lightning round. I love a good lightning round. Okay, AI will be sentient when? A long time from now. Like how long? This kind of question I prefer not to comment on. It's hard to answer. Most interesting future use cases for DALI?
[0:48:51] I think it's going to be just making your dreams come to life. In what sense? Sorry, that's not part of the lightning round. You hook up your brain machine interface, and then you do a nice rendering, and you'll get great visions of your dreams. Spiciest take on the future of AI that you're generally not allowed to say publicly. Oh, man. I think that...
[0:49:22] I think we're going to figure it out. I think it's going to go well. You're optimistic. I'm optimistic. I consider myself an optimistic realist. I think it's not going to go well by default, but I think humanity can rise to this challenge. Elon Musk no longer really deeply involved with OpenAI, building potentially what's called anti-woke AI. Success or failure? Well, I think a failure on our part for sure. In what sense? Well, I think we were not fast enough to address biases in chat GPT, and we did not intend them to be there.
[0:49:52] That our goal really was to have a system that would kind of be sort of egalitarian, sort of treat all the sort of mainstream sides equally. And we actually made a lot of improvements on this over the past month and we'll have more to share soon. But yeah, I think that people were right to criticize us. And I think that we really sort of responded to that. It's one of the pieces of feedback that I think is most valuable. Fill in the blank, a world powered by AI in 2050 is? Unimaginable. Okay.
[0:50:22] I like that. Single most important ethical issue we're facing when it comes to the future of AI and humans. This one's hard. I think it's the whole package, honestly. I think it's this question of how the values get in there, who's in control, how do the benefits get distributed, how do you make sure that the technology itself is safe and kind of used in the right ways, and sort of the emergent risks that are going to appear at some point with very capable systems don't...
[0:51:00] End up overwhelming the positives that we're going to get. And so, yeah, I think that it's the whole thing. And at some point to your first question, the sentience question, at what point do the systems have moral value? And the answer today is definitely not. But I am not, I don't know, we need to engage the moral philosophers to help answer some of these questions. Are you guys going to hire philosophers? We're going to hire, I think, kind of everyone across the board. I think that this is not a...
[0:51:29] This is one key thing to get across. I think that within AI, I've definitely seen this fallacy of people thinking this is a technology problem or just saying, look, there's the alignment problem of how to make the AI not go off the rails. But the society thing, that's the hard part. I'm not going to worry about that. And I think you can't do that. I think that it really has to be that you engage with the whole package. And that, I think, is going to require everyone.
[0:51:53] I think I like the understanding of understanding the people behind the code that transforms society. And so I've just met you in person today, but we've spoken a little bit about some of the ethical stuff too. You're at the helm of one of the most important technological advances of our time. What do you want people here to know about you? Well, I love my wife.
[0:52:17] Not going to listen to the chat bot. I met her. She is fabulous. She's not being replaced. Sydney cannot break up that marriage. And we were talking about this last night. She was asking, why do I do it? Because I work a lot. I think we give up a lot of time together as a result of just how much I really try to focus on the work and trying to move the company forward. And I hadn't really thought about that question for a while.
[0:52:46] And I thought about it, and my true answer was, because it's the right thing to do. I think that this technology really can help everyone, can help the world. I think these problems that we just see coming down the pipe, climate change, again, being one of them, I think we have a way out. And if I can move the needle on that, and I'm grateful to be in the position that I am, but honestly, when we started the company, what I cared about most was I was just like,
[0:53:15] I'm happy to do anything. First day, two people were arguing about something. They didn't have a whiteboard. I was like, great, I'll go get the whiteboard. And I think that this problem is just so important. It transcends each of us individually. It transcends our own position in it. And I think it is really about trying to get to that good future. Great. Well, thank you. I'm going to get to some questions because people have some great questions. Do you believe that there's a risk of a decline in human intelligence as we start to outsource our cognition to AI?
[0:53:44] Yeah, this is definitely something that keeps me up at night, although it's interesting to see this trend across all previous technologies, you know, radio, television. You know, I've talked to some esteemed statespeople who have said, like, the politicians these days, nothing compared to Teddy Roosevelt. Like, read, you know, all of Teddy Roosevelt's, like, great, great thoughts, and people just, like, don't read enough anymore, and so they just, like, don't think as well. It's so unclear to me, like, you know, I feel, like, is this true or is it not? But I think that what is,
[0:54:15] Definitely important as we see this new technology coming is figuring out how to have it be an intelligence multiplier. So that sometimes, yeah, you do need to solve the problem yourself. But what you really want is you want a great tutor. You want someone who breaks down the problem to you, really understands what motivates you, and if you have a different learning style. And so I think there's an opportunity here. If you're just blindly not thinking anymore, yeah, you're probably not going to learn to think. But if you have something that actually is figuring out the how do I
[0:54:42] Help you fish. How do I help you learn to fish? I think you could go way further. What is your opinion on, this one was upvoted a lot, so I'm being true to the audience. They have a good question. All right. What is your opinion on intellectual property rights for AI-generated content trained on the work of a particular artist? We talked a little bit about this, but the people want more. The people want more. I mean, honestly, this is, I think, like... Important question. Exactly. I think this is like asking a question about exactly how...
[0:55:11] Copyright should work, you know, right at the creation of the Gutenberg Press, right? Where it's like, we are going to need to have an answer. We're engaging with the copyright office. We're engaging with lots of different areas. And I don't personally know what exactly the answer should be, but I do think that, like, one thing that I do want to say, you know, not to kind of hedge everything here, is that I do think that the content creators should be sort of, you know, it should be a more prestigious, a more compensated, a more just like good thing for people to pursue now than ever.
[0:55:40] And I think if we don't achieve that in some way, then I think that something has gone wrong. Will there be new laws that didn't exist? Oh, for sure. I mean, there should be. What do you think they will be? Well, again, I don't want to speak out of turn. I just don't want to speak out of turn on these issues. But I think that, to me, the process...
[0:56:00] That's happening right now is really important. There's a lot of just conversation about these things. People really care, and they should. And that we are trying to figure out mechanisms just within our own slice of how we implement things and how we work with different partners. For Dolly, for example, the very first people that we invited to use it were artists, because we really wanted to figure out how do we make this be a tool that you are excited about and that you feel like, yes, I want this. I want there to be more of this in the world.
[0:56:29] Someone had the question, what should I teach my one-year-old daughter so she can have a job 20 years from now? I think that the most important thing is really going to be these higher-level skills, right? Judgment, really figuring out is this good, is this bad? Do I like this? Do I not? Knowing when to sort of, you know, sort of dig more into the details. And really, I think today, just even playing with these systems. Like, I think that it will be the case that we're going to make...
[0:56:57] The next generations of the dollies and these other systems just be, you don't even have to know language, right? They should become much more child accessible. And I think that children being sort of AI native users, I think you're going to find that you're going to figure out how to just use these in totally unimaginable ways. Let's see. Sorry, this one's not working. I'm going to this one.
[0:57:19] Okay, how can we maintain the integrity of AI models like ChatGPT when capital from corporates has entered the space, monetizing a tool run by a nonprofit? And you've, I mean, a lot of folks, actually, this is what ChatGPT also.
[0:57:33] Ask me to ask you, which is interesting. It's very topical. I like that. This is good. And so if you could give us a little more insight, because obviously we're very far from when you guys sat at that dinner and said we want to change things. And now there's money, there's profit, there's all these other things. So how do you guys maintain that? Well, I think that our answer to this question, and you should hold us accountable, by the way, is really about structure.
[0:58:00] That we've really set up our structure in a very specific way, which, by the way, has turned off a lot of investors. We have this big purple box at the top of all of our investment docs that say the mission comes first, that we may have to, if there's a conflict with achieving the mission, cancel all of your profit interests, which sends many traditional investors running through the hills. And I think that there's a part of the frame of the question that I...
[0:58:27] And so I think that the important thing for us has been to really figure out
[0:58:52] How do you set up the incentives that are on yourself so that you do, as much as possible, get the best people to join? You can build the massive supercomputers. You can actually build these tools and get them out there. But at the same time, if you do succeed massively and wildly beyond anything that's happened, how do you make sure that you don't, once you've kind of gotten to everything, you don't have to then 2x everything? And I think that these kinds of very subtle choices make a huge difference in terms of outcome. I want to end with...
[0:59:21] A quote from your co-founder, Sam Altman. He wrote, a misaligned super intelligent AGI could cause grievous harm to the world. An autocratic regime with a...
[0:59:31] Superintelligence could lead to that too. Successfully transitioning to a world where superintelligence is perhaps the most important and hopeful and scary project in human, is perhaps the most, sorry, I'm really messing this up, is the most important, hopeful, and scary project in human history. Success is far from guaranteed, and the stakes, boundless downside and boundless upside, are there to hopefully unite us all. So last question to you, Greg, is,
[1:00:01] Do you think we're heading towards boundless upside? And if so, what is the one thing that we can do right now to make sure we tip the scales in that favor? Yeah, so I think that what we're seeing is looking very consistent with a slow roll. You know, people kind of have thought that maybe what's going to happen is kind of nothing and then boom. You know, like nothing and you get either the great outcome or the terrible outcome. And I think what we're seeing is much more of a gradual integration and that...
[1:00:28] It's scarier because it's much harder to solve that problem in the lab and in your head. It's not a math problem. It's not a code problem. It's a human problem. And I think that this is the key. And so I think that by really engaging with these technologies, all these questions that we don't know the answers to yet, that's the responsibility not just of us. That's the responsibility of everyone, not just in this room, but really in this world. And I think it's going to be
[1:00:55] It's going to be a project of decades to really go from where we are to the kinds of systems that we're talking about there. And all along the way, there's going to be surprising things. There's going to be great things that happen. There's going to be causes for joy, causes for grief. And I think that they all happen in small ways now. And I think in the future, maybe they'll happen bigger and bigger ways. And I think that just really engaging with this process, just really everyone educating themselves as much as possible, trying the tools to understand what is possible.
[1:01:24] And figuring out, how can this fit in? I love the question about, what should I teach my one-year-old? Because that is a hope for the future kind of question. And I think that I am very optimistic. Again, I think I consider myself this realistic optimist, that you really have to be calibrated. You can't just blindly think it's all going to work out. But you have to engage with all the problems. But I think it is possible that we will end up in this world of abundance and sort of all the
[1:01:52] The real good future. I think it won't be perfect. I think there will be problems. And there will certainly be many problems along the way. But I think we can rise to the occasion. You have children? Not yet. Not yet. Yeah, working on convincing my wife, though. Okay. So I was going to say, do you believe that kids of your friends, if you end up having children, will grow up in a better world? I do think so. I think we have a shot at it, right? And again, it's not guaranteed. I do not consider myself to think that any of this is for certain.
[1:02:22] And I think the moment you think that it is, that's when things go wrong. And so I think we all have to be sort of constantly asking, what can go wrong and what can we do to prevent that? Right. Greg Rockman, thank you so much. Thank you. Thanks, guys. I appreciate it.