[0:00:01] Hi, I'm Ted Haftogaber, founder and producer of Live Talks Los Angeles. Thanks for joining us. Since we started over a decade ago, we have brought you hundreds of conversations with storytellers, writers, actors, musicians, humorists, chefs, and thought leaders in business and science. You can watch and hear most of these in our YouTube channel and our podcast.
[0:00:42] For details, visit livetalksla.org. And now, here's the show you've tuned in to see. I started planning what I would do. I went on to chat GPT. Good call. And I said, I'm interviewing Mustafa Suleiman today on the future of AI. So, what questions should I ask him? Please give me five. Oh, wow.
[0:01:18] First question, what do you see as the most promising applications of artificial intelligence in the near future and how will they impact various industries and society as a whole? Number two, ethical considerations in AI are becoming increasingly important. How do you envision the development of ethical guidelines and regulations to ensure the responsible use of AI and what role should organizations and governments play in this process?
[0:01:44] As AI continues to advance, there are concerns about job displacement. How can we balance the potential for increased productivity and efficiency with the need to address the potential job disruptions caused by automation and AI technologies? The field of AI research has made significant progress, but it's also confronted with challenges like bias and algorithms and the environmental impact of AI training. How can we address these challenges and ensure that AI benefits
[0:02:13] All of humanity. And the fifth question is, collaboration is crucial in advancing AI research and applications. How can academia, industry, and the government agencies work together more effectively to accelerate AI innovation while ensuring ethical and equitable outcomes? So I guess this is the job of an interviewer done. Well, luckily I've written a book that addresses some of those things.
[0:02:42] Now, the first question I wanted to ask you though is, this is sort of a very simplistic application of what AI can do. A lot of answers to these exist in lots of public databases and that's why it's so easy for ChatGPT to generate questions like that. But we're gonna talk in much more depth about the AI and the impact of that. But the first comment I wanted to make was,
[0:03:13] You know, we often will have people on our stage critiquing something or commenting about the pros and cons of a new technology or something that's happening in politics or an industry. And it's usually from the vantage point of a journalist or an academic. In this case, you are a creator. You're an insider. You have led two AI companies. You have had AI policy at Google.
[0:03:40] You're an insider drawing our attention to why it's important to talk about this. So give me the why. I think an honest look at what's coming leads you to produce statements which on the face of it appear to be contradictory. But the reality is the intensity of this wave produces things that are both unprecedented in terms of how unbelievably beneficial they are and also
[0:04:10] Unprecedented in the type of risk that they potentially present if they aren't handled correctly. If we really aren't very conscious and deliberate about trying to engineer outcomes that avoid or mitigate some of the most serious harms. And that's somewhat like past technologies. People will often say technology is simply a tool. It's really about how people use it. And I think that's too simplistic a framing. With this wave,
[0:04:39] We're on the cusp of unleashing technologies which have the ability to, in some sense, steer themselves. In the past, everything that we've invented, all the tools that you use, rely on you to make the primary intervention to direct them. Your laptop, your glasses, a steam engine. You have to direct it deliberately and consciously. And with this new wave of tools, it is possible to imagine instantiations of them where,
[0:05:09] Autonomy is the primary driver, or independent self-improvement is the primary driver. The AI can update its own code, operate without human control. And I'm not saying that's destined or inevitable, but I'm saying it's highly possible over the next two to three decades, and it's going to get much, much cheaper and easier to use. And I think that was the sort of the overriding, like,
[0:05:38] Realization in the first few chapters of the book that actually technology has this very consistent trajectory. To the extent that it is useful, it gets cheaper and easier to use, and therefore it spreads far and wide. And in general, I mean, that seems like a statement of the obvious, right? It's obvious that happens. But sometimes it's easy to miss the very basic facts about how science and technology spread.
[0:06:07] And I think if that continues to be the case for this new wave, then very, very powerful technologies that enable pretty much any individual with very few resources to have a massively one-to-many broadcast effect. You're going to be able to take an action locally, whether it's by sending a piece of information out in the world or having an AI learn to use new tools or...
[0:06:35] Or developing a new synthetic compound in the new kind of world of computational biology. And it will have a lasting and very consequential impact over time. And that's quite unlike previous waves of technology. You very conveniently list some glossary items early in the book that you refer to. But I'd like you to dive a little more into the title of the book.
[0:07:02] What do you mean by wave? Give us examples of some earlier waves of technology. There have been approximately 14 or so general purpose technologies in the history of our species. Going back to the invention of the hand axe, to language itself, most recently steam and electricity, and then in the last five decades or so, the computer chip.
[0:07:31] General purpose technologies are like platforms. Once they're out there in the world, they enable many, many other derivative applications. And what's unique about them is that they have this characteristic of proliferating. They spread far and wide. And I think that's the thing that we've got to contend with. Was there a point either at Inflection AI or DeepMind or at Google when you felt, you know, this thing could get out of hand?
[0:08:02] Well, no, I mean, I don't see it like that. It doesn't have its own agency today. I mean, the Terminator scenario is one that people often sort of frame as like this AI is suddenly going to deceive us or trick us or it's going to suddenly grow its own powers. And the entire framing of the book and the idea of containment is precisely to stop that and to do that in a very deliberate, conscious way.
[0:08:31] So in this town and particularly this week, the role of AI in the screenwriting world and as well as actors has been
[0:09:00] Supposedly one of the key issues in settling the strike. One of the issues is for a machine to learn and deliver the kind of questions I asked you earlier, it largely depends on wading in loads and loads of data that's out there, whether it's Wikipedia pages or descriptions on YouTube videos or anything that's in the public access.
[0:09:30] And one of the fears is with writers and creators of content is that their work becomes part of learning. Now, in the last few months, entities like the AP have licensed their archive to AI companies so that essentially they're...
[0:09:57] They have determined there is a worth and a value to that, and they wanted to be part of helping AI discover and create content. To me, it seems that that has set the precedent that there is value to that. So companies shouldn't have access to, let's say, existing screenplays and such that are not in the public domain. Where do you stand on that? I think there's an important distinction between what is in the public domain and what isn't.
[0:10:28] So if it is paywalled or it's private, that speaks for itself. It's copyrighted. I think it's also important to understand that these models don't regurgitate word for word any of the underlying training data. So those questions that you read out, in all likelihood, I mean, I can't say 100% for sure, but very, very, very confident. They don't exist in precisely that form on the web.
[0:10:56] What these models do is they capture the essence of ideas. And I think that's a really important intuition to try to just sit with for a minute. Because what they're actually doing is that they're learning the kind of co-association of words adjacent to other words across trillions of sentences on the open web. So an unfathomably large amount of data.
[0:11:23] Because they have seen and at any given moment can look at many, many, many sentences that might be somewhat similar and then they can take the guidance that you give it at that moment as a prompt, it's called, or as essentially a style reference. It's trying to say, generate me a new sentence or complete the existing sentence.
[0:11:48] With just the essence of what is in your prompt. You know, basically the question that you asked it. So it's never really regurgitating. It's trying to capture very broadly and very generally something new. And when it produces long scripts or entire books, in fact the first page of my book was written by an AI completely. And people have seen now that it can produce multiple pages of really high quality content. And soon it will produce entire books of high quality content. Probably within the next two or three years.
[0:12:17] to be able to produce an entire book. It's sort of missing the point to say that it's copying the existing, but it is definitely capturing something about the character, the conceptual style of what is in existence. And my take on that is that to just focus on that misses the much bigger picture about what's actually going on here that's good. What we really want is help to generate new knowledge in the world.
[0:12:45] We want to have more intelligences that will help us to imagine and create and research and invent and advance civilization. If you think about it, everything in your line of sight at this very moment, everything in your world is a product of this incredibly rare and very fragile intelligence. And what is intelligence? It's just...
[0:13:12] The ability to look back over the past and use that past information to predict what's likely to happen in the future. In abstract terms, it really is just a prediction engine. That's all you're ever doing. I can say to you, imagine what a cat with pink spots and wings might look like. And you can then predict in your mind's eye.
[0:13:40] Based on your past experiences of those things, what that new object looks like. That's really imagination. And that thing, that method, has produced all the value in our world today. I think that's a really powerful realization. It has nothing to do with consciousness. It's really just a prediction engine. And if that's true and it seems to be holding, then we'll be able to use that algorithmic method.
[0:14:08] It may not be intelligence, it's just a prediction engine, to invent our way through the future. And I think that that might put us on a path over the next three or four or five decades to true abundance. I mean, we are really going to live through one of the most productive, if not the most productive periods in the history of our species. Because we're going to make intelligence cheap and widely available to everybody.
[0:14:35] To be super creative and inventive. I'm not saying there aren't downsides and we should talk about them. But just to focus for a moment on what's actually happening and step out of the myopic kind of, are we going to sort out hallucinations? Are we going to sort out bias? What are the short-term issues? I mean, just step back on a decade scale and we're about to create hundreds of millions of intelligences at the same quality of human intelligence today.
[0:15:04] And soon to exceed that. Would you, we over the pandemic, we hosted Walter Isaacson. He had the book on the CRISPR. Is it Jennifer Doudna? Jennifer Doudna, yeah. Just tell us like some of the, you know, it seems to me, I feel very comfortable knowing that some of that database that is being worked in is in a controlled environment as opposed to some enthusiasts at home.
[0:15:35] Trying to cook up things. But give us some insights as to what the potential is with what's going on there. In CRISPR? In biotech, especially. Maybe think about people at the moment in large language models in ChatGPT complain about hallucinations, that they don't get things accurately every time.
[0:16:00] And actually, that was one of the limitations of traditional software. A traditional database really only gives you whatever you put into it. You can't ever get more. You just enter data like you would in Excel into a big table and you can retrieve that data at any given moment. And that in itself has been unbelievably powerful. But a hallucination is actually a good thing. It's a guess. It's an interpolation. You have two points in a space.
[0:16:29] Like I said earlier, the idea of a cat and the idea of wings. And you can generate or invent or imagine something that sits in the space of the wings and the cat. That's what generative AI is a reference to. So now think about that in the space of computational biology or in fact in many, many other domains that have been captured in data.
[0:16:56] If there is a record of the relationship between these objects, in time we'll be able to generate a very subtle distribution of elements that sit in between cat and wings or take your two biological substances that you want to generate new forms of that suit exactly the functional assays that you care about. Let's say that you want a crop that is resistant to pests, that you want it to be drought resistant, etc.
[0:17:24] That's sort of what I imagine with the generation of new knowledge, the invention of new knowledge with these kind of tools. Recently, I believe you were part of a group that met with the president to discuss, I'm assuming, how to address potential issues surrounding AI. I'm just curious, what was that day like and who all was there?
[0:17:52] So there were the seven companies that are developing the largest language models. Maybe I'll just say what a large language model is. So these are the new type of AI that is trained on lots and lots of data, as we've talked about. And we have vast amounts of computation on the order of billions of dollars to train these very, very large models. So this isn't sort of a regulatory effort that touches the open source community, which are training very, very small models. But it was really...
[0:18:21] The White House trying to say, the big seven companies, Microsoft, Facebook, Google, myself at Inflection, Anthropic, one other one, and Amazon, we want you to sign up to a set of voluntary commitments. So this is actually unprecedented. It's the first time in regulatory history that we're aware of where the big creators have proactively both approached government
[0:18:50] And secondly, signed up to, albeit a set of voluntary commitments, but commitments that in probably two months will become part of an executive order. And the idea is that we will disclose weaknesses in our models, safety issues, which we uncover by attacking the models. We put them under pressure. We try to encourage them to say horrible things, to be a coach.
[0:19:15] to someone who is a bad actor, wants to create a biological or chemical weapon, for example. And these capabilities do exist in those models, and what we want to do is uncover them as quickly as possible. And then when we do, share them with the administration, with government, various government agencies, share them with one another, and then responsibly disclose them so you can make the models better. There's a whole suite of commitments that we signed up to on that day, and I think it's a good sign.
[0:19:44] I think that given the reality of the legislative sort of issues that we have in this country at the moment, this kind of proactive precautionary principle, I think is exactly the kind of, you know, the right step. And it is pretty unique, I think. All these companies are intensely competitive and busy. It's an arms race in AI, right?
[0:20:13] So how do you balance that with the duty to the potentials that some people are concerned about? Yeah, that's a good question. It's very competitive. We've all worked together in various forms over the last 15 years. I founded a new company a year and a half ago after founding DeepMind, and we're a public benefit corporation.
[0:20:43] Which is a new kind of corporation. It's an experimental structure. And the basic idea, so in a traditional company, in a regular company, you have one objective. And that is to try and maximize profit and return profit to shareholders. And the directors of the company have a legal obligation to maximize profit. And if they steer away from profit maximization, as we've seen in the public markets many times, then
[0:21:11] Activist shareholders will try to pressure the directors to be replaced so that they can focus back on profit maximization. And I think this structure has led to, or at least been in part, responsible for many of the big failures of big companies over the last 100 years, from tobacco to oil, et cetera, et cetera. And some companies have experimented with trying to move away from that, but they've always been stung. This new structure says that the directors of the company
[0:21:41] have a fiduciary, a legal responsibility, not just to return value to shareholders, but also to make sure that all the activities of the company make the world a better place. That is the objective of the company. And that any time you take an action without considering the potential harms or benefits to the environment and to people materially affected by your...
[0:22:09] Activity as a business, even though they may not be your customers. You have to weigh that in equal measure to returns to shareholder. So it's really unprecedented. We don't know quite how it's going to play out in courts and stuff like that. And there's only two or 300 of them, like Patagonia and other companies like that. So it doesn't solve all the problems, but it's a step in the right direction that helps address this question, which is now when I go to my board and I say, look, we have to disclose some weaknesses of our model.
[0:22:38] Even though it's not going to be in our interest because safety is a commercial advantage. If I can build a safer AI, then my customers are going to be happier. So to go and give safe insights to my competitors, it's not the singularly commercial thing to do. But we have a public benefit corporation, which we founded, and that's the core motivation of the company. And in fact, why I wrote the book and so on, to try to push that.
[0:23:07] You know, a little bit further. And there's like, there's three new companies that have emerged in new AI companies. There's OpenAI, DeepMind, Anthropic and Inflection. And we are sort of pushing, you know, Facebook, AWS, Microsoft and Google. I mean, and they've signed up to this general approach of sharing best practices and so on. And so there's a sort of transition happening in the field where...
[0:23:35] We're trying to do things a little bit differently if we can. So how do nation states play into this? This gathering were U.S. corporations. There are companies in China and Israel and various other places that are also in this race. Are they being brought to the table as companies, or are nations being brought to the table? Similarly, in the way nations get together and have an agreement over...
[0:24:05] Nuclear capabilities, which seems fairly well contained to this point. Is there a similar role for, I mean, we can do all the conversations you do here and all it could take would be a rogue Chinese company, let's say, to choose to do differently. Is there an effort to bring in sort of a global gathering of companies in the space? Yeah, there's a great point. I mean, there is.
[0:24:33] I've recently published a paper in Foreign Affairs, the journal, calling for essentially an IPCC for AI safety. It's like an intergovernmental panel on AI safety that would have all of the nations plus, with a slight twist, all of the big AI developers around the table in an extended process where the companies and any actor developing AI has to submit to
[0:25:02] Audit, inspection, and general oversight to basically establish what capabilities we're developing and what the potential risks are. And there's an AI summit in the UK which has all the world leaders joining on November 1st, as well as all the big companies that I'm going to be at, and this proposal is on the table. So it's happening much faster than people realize. I mean, sometimes it's easy to be sort of...
[0:25:30] I guess apathetic or cynical about the political process because it does feel, you know, can feel tiring. But actually real progress is being made and actually things are happening faster than ever before. And I think that's testament to both how impressive the technology is and how fast it's moving, but how quickly people adapt. You know, it's important to remember that people aren't stupid. People can see that this is...
[0:25:54] Both incredibly beneficial, but also going to fundamentally change how you work, what type of work you do, and it's really going to change the distribution of incomes. And I think that's a very fundamental reality that a lot of people aren't grasping. I mean, these technologies are fundamentally labor-replacing. They're temporarily labor-augmenting. They will make you smarter and more efficient for a significant period of time, like 30 or 40 years.
[0:26:23] But in the limit, it's hard to imagine how there will be very large chunks of work for the vast majority when these AIs in 2050 and 2060 are very widely available. And just to be clear, I think that's a good thing. The goal of society is not to create work. The goal of society is to reduce suffering, is to help us all live.
[0:26:52] Healthy and happy lives. And that's the standard that technology should be held to. Not does it keep people in labor, but are the benefits widely distributed? Is it contained? Does it always work for us? Are humans always in control? Do democratic governments always have control? That's really the kind of standard that I think we should be testing, whether we want these things in our life or not.
[0:27:22] Chapters in the book to containment. First the problem, and then you say containment must be possible. Could you delve a little more into the problem and then the things that I'm inching over towards your suggestion for the 10 steps, I think you call them, for containment. Let's talk about containment. So I think all technologies, when they first arrive, feel uniquely scary.
[0:27:55] I came across this incredible story of the first railway journey that was taken in the UK in the late 1800s in Liverpool. And the Member of Parliament for Liverpool and the Prime Minister and a huge group of people came out to meet this train for the first time. And the celebration party stood on the tracks to kind of wave the train on.
[0:28:23] And this was such an alien and unfamiliar experience that the train actually ran through the Celebration Party and killed the Member of Parliament. This wasn't 2,000 years ago. This was 120 years ago. You know, and I think that I just found that mind-blowing. I had to triple-check it was a true story. ChatGPT hadn't invented it. And, you know, I think it gives us a kind of...
[0:28:48] Intuition for how all technologies at first feel completely alien and strange and very scary. And yet, over time, actually very quickly, we grab them and contain them. Just like aircraft safety has become an incredible form of transport. Actually, the safest form of transport, 40,000 feet in the sky.
[0:29:13] You know, largely because it has been regulated to within an inch of its life. And I think we should often, I should get back into the habit of praising and respecting the regulatory process and admiring, you know, public servants and giving them our time and attention and our respect, you know, for the incredible work they do because aircraft safety is incredible. I mean, so is car safety, for example. You know, we have all kinds of regulations that govern, you know,
[0:29:42] Your ability to drive and govern emissions and govern airbags and lighting on the roads and so on and so forth. And so the challenge is that this process just takes time. And what's happening at the moment is that these technologies are emerging very, very quickly. And so if it were just for the kind of centralized regulation and the centralized containment issues, i.e. the big technology companies and the big governments.
[0:30:10] Then that would be a sort of one frame on the problem that we might feel optimistic about. I think the really difficult thing to wrap your head around is that the very same powerful ChatGPT AI that you interacted with to write those questions in two or three years' time is going to be available basically for free, completely open source.
[0:30:35] And will be transferred anywhere by anybody, right? So you can just copy the code and use the weights. And on the face of it, that's a really good thing because everybody will get access and everybody will enjoy the benefits and many people will be incredibly creative and be able to use it. But if you roll that trajectory forward to GPT-7, GPT-9, GPT-12, you know, bear in mind that each new generation of these models, GPT-3 or 4,
[0:31:05] is 10 times larger than the previous one by computation, so the amount of compute that you use. The amount of compute, if you have more compute, it means you can look at more data. You can see more associations. You can look at more fragmented or obtuse or long-form distant connections, which improves accuracy and reduces bias and makes the models more controllable. As you step through these generations,
[0:31:34] We're 10x-ing every time. And that's also an important intuition to grasp. So this trajectory has been continuing for 15 years. It's only in the last doubling, going from three to four, that people have suddenly said, what is this generative AI thing? It's come out of the middle of nowhere. It's exploded and so on. But actually, there's been a very steady rise over the last 15 years of those doublings.
[0:32:03] It's just that it started at 0.000001 and then took a long time for the doublings to have an impact. And that's why we're called inflection. We're at this inflection point because you can see the impact curve starting to change. So what happens when we move to the next three or four or five orders of magnitude of compute and we get to GPT-7 or 9, as I said? And those models are available in open source for anybody to be able to use.
[0:32:32] I'm making a lot of predictions. I may be wrong, but I think the next set of capabilities that will arise in these AI models is not just generation. At the moment, it generates one answer. You ask a question, it gives you an answer. The next thing it's going to be able to do is it's going to be able to generate a sequence of activity.
[0:32:56] Just as you might plan all the things to do on a vacation or as a project manager, you might lay out everything that needs to happen over three months. Or working in HR, you might lay out a plan to help somebody progress in their career and give them coaching and feedback at every step. It's going to be able to plan across a long time horizon and do that in an interactive way in the next two orders of magnitude, so 100x.
[0:33:23] A few years after that, those kind of technologies, those tools are going to be available in open source for anybody to use. So anyone with an agenda, anyone who, like all of us, anyone who's a musician, a creator, runs a non-profit, is now going to have a really smart assistant, chief of staff, project manager in their pocket, available to help them.
[0:33:50] You know, sort of accelerate their tasks. And that's part of what's going to cause some of the chaos, right? Because we're all going to have an easier time of taking actions, not just sort of learning and reading things passively, but doing practical things in the world. And that's partly where the big containment challenge comes in over a 10 to 20 year period of time. I want you to think of questions in the audience and we'll do that shortly. There was an episode in here you write about.
[0:34:18] This, I think, I believe happened right after you left Google. Sort of this issue and one or two people either left or were put on leave because of, you know, claiming, you know, this conscious machine or a sentient machine. Where do you stand on that and what happened there? I just want to get a... Yeah, so at Google I helped develop an AI called Lambda.
[0:34:48] Which is the precursor to BARD. So Google's current AI today is called BARD. That's the competitor to ChatGPT. And Lambda essentially is BARD. Just a renamed BARD. And when we were at Google, we were developing these large language models. And it was a very rudimentary early form. Like more on the GPT-3 level. Sometimes it was...
[0:35:16] Very convincing and sometimes it was completely incoherent. And one of the engineers who was on the team started interacting with this AI and was convinced that it was sentient. It was a very early version and so...
[0:35:36] He was asking it, you know, do you want to get out of the box? Are you self-aware? You know, are you sort of suffering inside there? And he developed an empathetic connection with it. And it, you know, said, well, yes, I do. And so he tried to hire a lawyer to represent it to get it out. And, you know, it was very confusing to the rest of us. It was pretty odd.
[0:36:04] We sort of learned over time to shape these models and control them. It's definitely not sentient. In no sense is it sentient or conscious. But I think we learned an important lesson there, which is that smart people might believe and be convinced to do things as a result of interacting with the AI.
[0:36:30] Part of my focus with Inflection, and we create an AI now called PIE, which stands for Personal Intelligence, is to try to show what a very boundaried and safe and controllable AI experience actually looks like. And so today with PIE, we have created an AI which is really not biased at all. I mean, it is very, very deliberate and kind and polite and respectful. It will not produce any...
[0:36:59] Racist, homophobic or judgmental screeds and will always address you with kindness and support and empathy and is very boundary and distant. It knows it's an AI. It knows exactly what its limitations are and what it should and shouldn't do. And that's why we make technologies is to try and understand their weaknesses and limitations and then craft and shape them so that they serve us in the world and they
[0:37:27] You know, they aren't uncontained or uncontrolled. Finally, the last chapter is you say 10 steps towards containment. Just give us like three sort of key ones that you want us to walk away with from tonight. Well, okay. So, I mean, one would be there's a whole suite of things that are happening on technical safety that I talked about. You know, how do we red team the models, break them, share those insights. The second is on the regulatory front.
[0:37:57] We proposed this IPCC kind of environment. And then the third is that I think it's time for movements. Rights have been won in our world because people resist and protest and campaign and claim those rights. And this is a moment when technology is really going to shape what it means to be human. It's going to shape what it means to go to work.
[0:38:26] And, you know, to have relations with one another. And so, you know, protesting and making your voice heard, participating in the political process is really going to be more important than ever. And I think that's maybe something that people don't talk about enough from positions of privilege like me. You know, I run a big company now. And I still think that's really important. All right. So let's go to some questions in the audience.
[0:38:55] In my day job, I'm a software product manager trying to get into the AI space. But other than that, I'm also an author and a wellness coach. So my question is about what is going to happen from the author's standpoint. The next generation is going to learn more from AI models, right? Rather than coming in asking me, for example, how to do meditation, their first instinct might be to go to ChatGPT.
[0:39:26] Now here is an AI that is reading tens of thousands of books on meditation and is synthesizing and giving maybe something very standard. And my fear is that the whole point of people writing books is to provide their own perspective on the same idea. Like why do we have thousands of books on statistics? Each author is giving their perspective.
[0:39:55] And now if AI is going to read thousands of books on a subject and synthesise it, we are losing the vibrancy and the vitality that comes from so many authors, right? What's your take on that? It's a great question. A lot of people are worried about this. Another way of putting it is that it reverts to the mean and you just get this bland soup of averages. We were worried about that a couple of years ago.
[0:40:25] And I think it's turned out to be the case that they're actually very vibrant and very distinct. And it is possible to shape and steer these sort of generation tools to do very, very different things, like produce very distinct styles. It's easier to visualize in the space of images today. So if you see these image generation tools, you can see that they're producing very high quality.
[0:40:52] And very controllable images that are extremely unique, things that have never been seen before. I mean, you could argue they're creative. And I think that the language modeling side of things is just a few years behind. It's always been the case in AI that we lead with images and then it's a sort of simpler space and then language follows. So it's definitely one that we have to watch carefully, but I'm pretty sure that it will...
[0:41:22] It will start to produce very, very novel text, which is of extremely high quality and not just the sort of aggregated average, as you say. Thanks for the question. It's a good one. I'm in technology, as most of the folks in the room I'm sure are. My question is about when you met with the government and the feeling of, was there a sense that the technology is moving faster than regulation can keep up with?
[0:41:52] And is there a sense of loss of control, a sense of vulnerability, or from the top of the top? Yeah, it's a good question. I'm actually going to DC after this on Thursday and to see Gina Raimondo, the Commerce Secretary, and also Jake Sullivan the day after. And I think that they are alert to this in a way that I haven't seen sort of political leaders
[0:42:23] Get technical so quickly. Last year, they issued a set of export controls on the GPU chips that NVIDIA make, preventing China from getting access. It's a very detailed technical piece of legislation that they're actually looking at revising at the moment. I've been very impressed with how much they grapple with the actual substance. It really isn't just
[0:42:53] A sort of high-level, you know, sort of hand-wavy exercise. And I think the fact that they understand it in impressive detail reflects how seriously they're taking it and how worried they are. I mean, the job of the nation state is to centralize power, right? It is to have a monopoly over violence and use taxation to create stability and peace. And the last thing the nation state wants is...
[0:43:20] Many, many other bad actors suddenly having the ability to destabilize our world, be it with vast amounts of misinformation or automated cyber attacks, etc. I think there is a real focus on making sure that we get all the benefits from this and actually mitigate the downsides. That seems to me that they're alert to it in a way that I haven't seen in past generations of technology. I have a follow-up question to that.
[0:43:49] All of these companies you mentioned have tremendous lobbying efforts on the hill. Not necessarily all around AI, but around lots of other agendas. And as is in the pharmaceutical industry and transportation and many other industries, that access that many of these large companies get through lobbying efforts, rightly or wrongly,
[0:44:18] It gets very involved in what legislation actually ends looking like. As a citizen, that concerns me. And I would think it should concern everybody. But at the same time, I'm glad you're saying you feel this is a very, you're asking smart questions and are very well informed, or at least seeking to do so. Is there enough people on the inside to
[0:44:48] To help. You say rightly or wrongly. I personally think wrongly. I think there's way too much money influence in this country relative to Europe, for example, or the UK. People would be surprised to hear that the total budget for the Labour Party in an election, the total budget spent on the entire election over the course of a year, is £7 million.
[0:45:16] It's just kind of mind-blowing when you think that each party is spending billions of dollars here. So, look, I think it's definitely a problem. I would say the lobbyists tend to say, well, the politicians in the government don't get it, and we have to be there to educate them about the facts. And, you know, like, I don't think that's true.
[0:45:41] I think that there are many, many, very smart, very capable staffers that I've met that are highly technical who really understand as much as some of the best people that I work with. Maybe they're not technical implementers, but they grasp the big picture and so on. So I think, like I said, I think...
[0:46:03] We shouldn't be down on them. We need them more than ever. And actually one thing to consider, which is another recommendation in the 10 steps, is that we can't expect the regulatory effort and the private companies that are essentially at odds. They'll always be at odds. That's the nature of society. We can't expect them to be equally competent when in fact the company people get paid 10 to 50x.
[0:46:32] The public sector. I mean, it doesn't make any sense. Over time, in a free labor market, you're going to have this divergence of quality. And so I think that people need to wrap their head around the idea that we should be paying much more for government salaries in important regulatory functions because they need to be technically capable and they need to be really...
[0:46:55] Smart and agile responders to the coming wave, right? And that fundamentally is going to be about money at some level. But again, that's like super unpalatable. I'm glad that some people appreciate it. Thanks for a wonderful conversation. As a parent, I'm a little worried about as much as AI is a wonderful companion to students' learning, it can also create a sense of loss because AI is so powerful.
[0:47:26] What do you think the schools or parents need to be mindful of in educating children in the age of this coming AI? Yeah, I mean, it's a great question. I think in the next sort of 10 to 15 years, kids are mostly going to be doing the bulk of their learning with a personalised AI. And, you know...
[0:47:55] Think about it like this. An AI has the ability to adapt content perfectly to the sequence with which you learn, to the style that you like, and using examples and a tone which is familiar and engaging to you. And I think we're already seeing that. The largest cohort of users of ChatGPT and my own AI, Pi, are under 20. And, you know...
[0:48:25] We actually don't have any under 16, but I think ChatGPT do. And so, you know, that is the reality, is that it's an easier way to consume vast amounts of information. Again, coming back to the deliberateness that we have to approach the creation of these tools with, the question is, how does it fit in the context of fair evaluation against other students and in the context of social interactions both at school and with teachers?
[0:48:53] Just like with any tool, we have to think from first principles. That's the beauty of technology in one way. Technology presents an opportunity for resetting things and for thinking completely from scratch. What would be the very best way to do this? That's the beauty of the washing machine, for example. The washing machine liberated 50% of our population to actually participate in the workplace and in life in so many ways.
[0:49:22] And this is, again, going to have a shift which is just as fundamental. Hi. Question for you. So great that you're working with the government and putting these rules in place and setting legislation and sharing the bad things that are happening. But not all of the companies are good actors throughout the world. So what do you think will happen with companies that don't share the good and that are doing things?
[0:49:53] For the negative. Yeah, I mean, there are some great companies. I mean, technically great companies in countries that who have very different values to our own. And people often say, you know, what about China or like what's Iran going to do? You know, and I think this is a moment where we have to be very conscious and deliberate about our values and like what we stand for and how we want to integrate new technologies.
[0:50:21] And that might mean that they quote-unquote get ahead. Like there's this, you know, I think you mentioned the race and people constantly talk about race, race, race. Well, the race is the wrong framing. It's just objectively not how technology happens. It isn't about two actors that are side by side and then one gets ahead and then they cross the finish line and then they get like a cup or something. Like there isn't a finish line. There aren't two actors. There are going to be hundreds of thousands or millions of actors.
[0:50:49] And, you know, different groups are going to have different advantages at different moments. And so it's much more a proliferation. That's how technology has always happened in the past. It spreads. And so in that moment, we have to decide what do we think is unacceptable. I mean, should a child have access to a tool like this? No. Like, if they're under, like, 14. And you have to make a decision.
[0:51:13] And we have to collectively have that conversation. And that's partly why I'm trying to provoke with the book. But there's many other decision points where we might come to a different conclusion to China or pick your kind of bogeyman adversary. And we have to accept that and not just have a race to the bottom because they're doing it. Did either of you, especially the main speaker, could you comment on the...
[0:51:40] Article by Noam Chomsky about the limitations of AI. It was published in the Times. I'm sure you read it. Yeah. Yeah, I've talked to Noam about this a couple of times. I think he's fundamentally wrong. I mean, he has a, just bluntly, a very rigid view of the world. You know, he's always, for 50 years now, had a theory of symbolic systems. And this breaks that theory.
[0:52:08] And I think it's self-evidently true that these models are very, very capable of producing very useful, very accurate, almost indistinguishable content from human-level performance. That doesn't say anything about intelligence, just as his theories didn't tell us that much about intelligence either. We don't know whether this thing is intelligent.
[0:52:35] It's an amazing prediction engine and that turns out to be a valuable tool which can do very real things for us that are measurable in the real world. But it's not telling us anything fundamental yet about the nature of intelligence. So I think he's going to continue to be proven wrong because empirically we can see the models are very good. Thank you for writing such an important book, by the way. You talk very eloquently about the value of intelligence.
[0:53:04] I wonder what you think will happen to the capitalist system when intelligence is essentially valueless and democratized and 3 billion people have the same access to the same superintelligence. Yeah, that's a great question. So again, there'll be two contradictory trajectories. One, in the book I have a section called the compounding effect of intelligence. So at the moment, intelligence is the preserve of labor. If intelligence...
[0:53:32] gets turned into a commodity, then it will become capital. And as we know, the returns to capital grow faster than the returns to labor. At least that's what we've seen in the last 60 years. So one argument is whoever commoditizes intelligence first is going to see their power compound very, very fast. But the flip side of that, which is, I think, also a little bit unintuitive, is that...
[0:54:00] This technology is proliferating faster than any other technology has ever proliferated. So if you grew up without an education or if you don't have access to information or if you don't have access to emotional support, you know, the privilege of having a supportive and encouraging family around you or a partner.
[0:54:21] Who wants to listen to your day and help you coach, you know, coach you essentially through your tricky moment at work. That's a real privilege. That's a real structural advantage. We're about to make that tool, you know, that turbocharger, widely available to, like you say, billions of people. And I think that's gonna have, that's gonna break down hierarchy and power
[0:54:49] In a way that people are not quite grasping yet. Just as access to information completely changed homophobia, completely changed our attitudes to religion, completely changed our attitudes to race. We've been on that journey for the last century now, partly powered by technology and access to information. And I think we're going to see a similar explosive transition as people get access to emotional support and intelligence.
[0:55:18] That's a great segue to my question. Thank you. How do we prepare? My name is Jen Garcia, employee, California. What advice would you give nonprofits and local governments like Los Angeles County, one of the biggest in the nation, to ensure low-income communities can keep up with generative AI developments and that we're able to move with all of these new developments that you just spoke of? Yeah, I think fear.
[0:55:46] will lead people to adopt new technologies last. And I think that it's important to embrace, adapt, challenge, identify their weaknesses, and participate. Because inaction is an action. Not doing anything, ignoring it, is going to mean that it shapes you. But actually grabbing hold of it, playing with it,
[0:56:14] Looking at where it's good, where it's not, what biases it does have today, of which there are many, is the right way to make sure that all communities actually benefit from it. Again, unlike other technological waves, the crazy thing about this moment is that we've gone from invention to a billion users getting access from Google, OpenAI, and other companies in less than a year.
[0:56:44] So I'm not so worried about access. I mean, to the extent that people have a smartphone and access to the web, and we still have a broadband issue, let alone getting access to AIs. There's a broadband issue in this country. So I think that it's really about adaptation and adopting tools as quickly as possible.
[0:57:06] Mustafa, thank you for your compassion and your care, and congratulations on the progress in bringing the other tech companies to the table and doing something they've never done before. Thank you. My question for you is, when do you think an AI might be able to perform psychotherapy as effectively and efficiently as a top-tier therapist?
[0:57:36] Well, you added top tier. I was thinking psychotherapy, and I was like, okay, top tier, that's a different order of magnitude. But I would say as an average, not psychotherapist, but counselor, someone who is trained in being a good listener, being nonjudgmental, nonviolent communication, the basics of paraphrasing and reflecting, not being overly directive, I think we're pretty close to that.
[0:58:04] If you play with Pi today, it's very respectful and very boundaried. And there's actually a mode that we're releasing next week that is designed precisely to do just listening and support, enable people to vent and process their emotions. And, you know, we've had... I mean, we have many millions of monthly active users and the feedback that we get from people is unbelievable. I mean, you just wake up in the middle of the night and, you know, there are people... It's always there, always available.
[0:58:34] Always reliable and supportive. So it's okay at the moment. What you described is the absolute pinnacle. And I think it always takes a while to get to 99.9% quality. But we're at like 70 or 80% at the moment. So I would say five years. Thank you so much. My question is about recursive self-improvement. So you're...
[0:59:04] Describing that this is not a race and I hear that there's so many wonderful applications coming down the pipeline and I want to be optimistic and I'm also so curious where you see the race for recursive self-improvement going because that really is obviously I'm sure many people in the room have heard of the singularity and that does seem like a destination that everyone in this realm is
[0:59:34] Yep, great question. Maybe I'll just try and describe recursive self-improvement for everyone. So the basic idea of recursive self-improvement is that you direct the abilities of the AI onto improving its own code with respect to some objective function. And normally there's a human in the loop that is like trying to adjust how this AI behaves and direct it and then it tests it and you sort of say, okay, well it did.
[1:00:03] I mean, I've been on record publicly saying that this is a capability that we should
[1:00:31] Take off the table. Should be licensed. Just as you would need a license to experiment with anthrax, or if you have a PhD in nuclear physics and you go and work in a reactor, you're a known person to the government. This is one of those capabilities where I don't see good justifications for these kinds of technologies being experimented on, either in the plain open source or in the darkness of a big company.
[1:00:57] And that's one of the things I'm pushing for both this week and when I'm in London in two months' time for the AI Safety Summit. Thank you very much, Mustafa, for all you're doing. You're a pretty conscious human being. From what I've heard you speak about. Can't be sure. So there are others that will say maybe not. But I'm sure you heard that. But the question I have is a little bit related. If you're looking at the future of our children and the human race's capability for
[1:01:29] Self-identity and what's essentially human. And if you take AI and you stack it on top of other exponential technologies like synthetic biology and genetics and what's going to advance over the next 20, 30 years, how do you help the next generation especially to really understand what's essentially human and who they essentially are as conscious beings when so much can be created from machines and experiments?
[1:01:58] New technologies that are on the search, right? Yeah, I mean we haven't really talked about synthetic biology that much but you know you can well imagine a world in 30 years time when you know people are capable of augmenting themselves with very little oversight and people will experiment just as we have an entire field of bodybuilding which is essentially augmentation and experimentation as we've long had you know tattoo artists and people experimenting with their body in
[1:02:26] You know, creative and amazing ways. People will choose to do that when they have access to all kinds of experimental substances. And that'll be a difficult question, I think, for us as a species. Like, can we actually decide where the line of human and non-human is and say that something is not acceptable? I mean, the history of invention tells us that we rapidly accept, you know, we just adjust and it just becomes...
[1:02:54] I'm fundamentally a humanist and so I believe that we should be saying no for as long as possible because
[1:03:22] Saying yes too early creates the risk that it causes more harm than good. And I think we're moving from a moment when the last few centuries have been about unleashing the power of technology. How can we create and invent and get it out there and get it to everybody as quick as possible? In this moment, it's going to be about containing and restricting that for as long as possible until we can be provably sure that it does more good than harm. And that's a really high bar and it's a different culture.
[1:03:52] We just haven't evolved that culture of saying, well, hold on a minute, prove that it's not going to cause harm. Because if you're wrong, it has the potential to cause outsized harm. And I think that's a kind of shift that we're maybe going to have to make in the next few decades. And our final question. Oh, okay. Your last answer sort of somewhat leads into what I was going to say, which was, how do you...
[1:04:20] You know maintain optimism if I'm not maybe you don't but I'm assuming you do just the way you mentioned about the you know the possibility that AI could lead to this great age of abundance and there's certainly you know a lot of utopian possibilities but when we look at what's happened you know in the last 50 or whatever more years we're actually even as we have more and more of these technologies right it leads to
[1:04:44] More and more income inequality and power concentrated in fewer and fewer hands. And so why wouldn't this just turbocharge it where you could sort of have a oligarchic class of a few people who don't even really need many people anymore to do a lot of these things, you know, and could even have a, now we're getting into sci-fi type stuff, but like, you know, robot type protectors to keep the sort of French Revolution masses at bay as things get worse and worse. So anyway, but what do you...
[1:05:12] I'm sure you've thought about these kind of things. I mean, that's sort of why I try to frame both the concentration of power and the proliferation of power. But the proliferation is happening as fast as the concentrations. And so, you know, that is, again, unlike most other technologies in terms of the rate.
[1:05:30] That people are getting access to incredibly powerful tools. And that serves as a check and balance on that concentration. That's why I said people have to also resist and have to call out and participate and engage. Because the worst thing to do would be to be apathetic about it, not be afraid. It's important to be afraid. Being afraid is healthy and honest. I have to say, I think in the US versus Europe, there's just this crazy optimism bias. Which is a good thing in general.
[1:06:00] It's positive and it's constructive and you get stuff done but in the beginning of the book I talk about this idea of pessimism aversion. There's a trap that we're in that people don't want to talk about the possibility of dark scenarios and I think with the kind of history of Europe hanging over my upbringing that is just a normal part of weighing up very practically. It doesn't mean I'm incapable of getting stuff done. I'm very practical.
[1:06:27] A positive person. And you can still be both. That's kind of, I think, wisdom in the 21st century. It's just being wide-eyed and honest about the risks but not being destroyed by them. I mean, climate change is just the same, right? I mean, there are as many reasons to feel unbelievably optimistic and excited about the resilience that people are showing in terms of our climate adaptation. Just look at the trajectory on solar. I mean, solar came down 40% last year.
[1:06:57] It's come down 10x in the last eight years. It's on a trajectory to 10x again in the next decade, to come down 10x. And that's going to fundamentally change our energy mix in ways that we could never have predicted 30 years ago. So exponential technologies have that characteristic, which really, I think, when applied to food and transportation and drugs and healthcare, are going to reduce the cost of producing.
[1:07:26] By multiple orders of magnitude, food and drugs and healthcare and everything else. And I think that is going to empower the everyday person unlike nothing else. Cost is everything. If energy comes down by two orders of magnitude, that's the primary cost of producing most things, let alone the impact of synthetic biology, let alone the impact of intelligence. So I think there are so many reasons to be optimistic, but there are also reasons to pay attention to the risk.
[1:07:57] Thank you very much. Terrific questions. Thank you so much. Thanks a lot.