[0:00:00] Hello, everybody. It's great to see, my gosh, so many friends, actually, and indeed my husband, which is a bit alarming. He never turns up to any event I do. But it is great to be here to talk about literally one of the hottest topics of the moment with someone who has written one of the best books about it. How many of you have used ChatGPT? Just a show of hands. Virtually everybody.
[0:00:26] I think you'd probably then agree that ChatGPT-Watt came November last year and it was only then that most people realized that artificial intelligence, generative AI models in particular, were about to change the world. And suddenly there was a kind of collective global, oh my god, this capability is extraordinary. And it's been reflected in endless numbers of...
[0:00:49] Editorials, hand-wringing, politicians. And I think I'm right in saying the main focus has been on the downsides. Everyone has their pet view of what the odds are of existential risk. Are we all going to kill ourselves? It's all terrible. And Mustafa comes into this as a man with considerable credibility. He is a man who has co-founded not just one, but two successful AI companies. And he's a man who in this book takes a sober, realistic,
[0:01:18] And actually a very compelling look at what lies ahead of us. And so that's why you really should read it. It's great. I've read it twice. You should read it. Mustafa, just to give you some... He doesn't need much introduction, I don't think, to this group, but he was a co-founder of DeepMind back in 2010.
[0:01:35] He then was a co-founder of Inflection AI with Reid Hoffman. Reid Hoffman, his co-founder, has, with the help of JackGPT, written an extremely upbeat view of the potential of this technology, so I'd love to know the debates between the two of you. He got a CBE a few years ago for his visionary services and influence in the UK technology sector. He is also on the board of The Economist, so I get to see Mustafa working up close.
[0:02:03] He's a friend of The Economist and a great figure in British technology. And the place to start with this book, and the book is called The Coming Wave. If you've turned on your TV or listened to a podcast recently, you will know that, never mind The Coming Wave, there is already a wave of publicity and people being impressed with this book. I believe you've had, you told me, 60 appearances of various sorts. So consider yourselves lucky, you're 61 on this list.
[0:02:30] But understandably, the book has had a tremendous impact because it is very interesting, very thoughtful, and it's on the hottest topic of the moment. So we want to talk most of the time about the book, but I do want to, for those of you who don't know Mustafa, to get a little bit of background. And the first is that Mustafa is actually not a computer geek. You didn't study computer code, right? You studied philosophy and theology at Oxford. So can you just give us the kind of potted history about how a man who studied philosophy and theology comes to be the co-founder of...
[0:02:59] Two tech companies. What are you doing? Well, I've always found philosophy a systems thinking tool. It enables me to be rigorous and clear about what I think. And, you know, right from the very outset, I think when I was 19, I actually dropped out of my philosophy degree. I didn't know that. Yeah, I didn't finish. And I was really motivated by the impact that I could have in the world. I left to help start a charity.
[0:03:27] At the time, it was a telephone counselling service called Muslim Youth Helpline. And it was a secular... I was an atheist, even though I'd grown up with a Muslim background. It was a secular service that was designed to provide faith and culturally sensitive support to young British Muslims. This was in 2003. And, you know, I found myself at Oxford studying this very theoretical...
[0:03:55] Esoteric set of ideas. And I wanted to put real things into practice in terms of my ethics. And that was why I went to start the Helpline and worked on that as a volunteer for three years. I soon got frustrated about the scale of impact in our nonprofit. And I worked briefly for the mayor of London at the time, Ken Livingstone, as a human rights policy officer.
[0:04:19] And that was inspiring, but I was also struggling with the scale of impact. I realized that if I didn't capture what really makes us organized and effective as a species, the profit incentive, then I was going to miss one of the most important things to happen in my lifetime. And at the time, I saw the rise of Facebook. This was sort of around 2007, 2008.
[0:04:45] And it had grown in the space of two years to 100 million monthly active users. And I was totally blown away at how quickly this was growing out of seemingly nowhere, something completely new to me. And so I set about on a quest to find anyone and everyone that would speak to me.
[0:05:02] To teach me about technology. I had started a bunch of businesses before that. Two different businesses. One, actually a technology company selling electronic point of sale systems. Actually around here in Notting Hill. In restaurants. Trying to put Wi-Fi infrastructure in there and so on. That was unsuccessful. That was ahead of its time. And so I was looking for people who I could, you know...
[0:05:24] Form a new partnership with and figure out how to take advantage of technology. And that's where I met my friend and co-founder of DeepMind, Demis Hassabis, because he was the brother of my best friend at the time from school. And he was just finishing his PhD in neuroscience at UCL and we got together and, you know, the rest is history. And at that time, you know, back in 2010, you had between you and there was another co-founder, right, Shane, like the three of you had the ambition that you were going to create
[0:05:52] An artificial intelligence that was capable of replicating human intelligence or even succeeding it. So just think, this was 13 years ago. The rest of us didn't even know this stuff was really going on. You're in your, where is it, in Regent Square somewhere. Did you imagine that by 2023 the world would have what we have now? I mean, in a way, yes. It was difficult for us to imagine exactly how it would unfold. But we made a very big bet on deep learning.
[0:06:22] Which is one of the primary tools that is powering this new revolution before anybody was involved in deep learning. So the current chief scientist and co-founder of OpenAI, the creators of ChatGPT, was one of our interns back in 2011. Jeffrey Hinton, who subsequently became one of the heads of AI at Google and is known now as the godfather of AI, recently in the press,
[0:06:51] Worried about the consequences. He was our first advisor, our paid advisor. I think his salary was £25,000 a year to advise us. So I think three of the six co-founders of OpenAI at some point passed through DeepMind either to give talks or were actually members of the team. So really it was incredibly about timing. We got the timing absolutely right. We were way ahead of the curve at that moment and somehow we managed to hang on.
[0:07:18] So you were there for a while, and then let's fast forward a bit. You can read the rest of this in the book. You now have co-founded and run Inflection AI, and you are creating an AI called Pi, which you can interact with if you like. Tell us what Pi does. So Pi stands for personal intelligence, and I believe that over the next few years, everybody is going to have their own personal AI. There are going to be...
[0:07:47] Hundreds of thousands of AIs in the world. They'll represent businesses, they'll represent brands, every government will have its own AI, every non-profit, every musician, artist, record label. Everything that is now represented by a website or an app is soon going to be represented by an interactive conversational
[0:08:10] Intelligent service that represents the brand values and the ideas of whatever organization is out there. And we believe that at the same time, everybody will want their own personal AI, one that is on your side, in your corner, helping you to be more organized, helping you to make sense of the world. It really is going to function as almost like a chief of staff, prioritizing, planning, teaching, supporting you.
[0:08:40] So that sounds great. What does it actually mean, though, in practice? Because so often this conversation about AI, it's at this point then it turns into the apocalyptic. We're going to end up, you know, wiping ourselves out because there'll be some rogue person, you know, sitting in a garage somewhere who will, you know, unleash a virus that will kill us all. So before we get to all of that stuff, in, let's say, I don't know, five years, I mean, you've said within the next three to five years, you think AI will reach human level capability across a...
[0:09:09] Variety of tasks. Perhaps not everything, but a variety. So paint a picture for us of what life will be like in five years, in 2028. First of all, will it be you and me here, or will there be the kind of Mustafa AI and the Zany bot? Okay, let me just go back 10 years just to give you a sense for what has already happened and why the predictions that I'll make, I think, are plausible.
[0:09:37] So the deep learning revolution enabled us to make sense of raw, messy data. So we could use AIs to interpret the content of images, classify whether an image contains dogs or cats, what those pixels actually mean. We can use it to understand speech. So when you dictate into your phone and it transcribes it and records perfect text. We can use it to do language translation.
[0:10:05] All of these are classification tasks. We're essentially teaching the models to understand the messy, complicated world of raw input data well enough to understand the objects inside that data. That was the classification.
[0:10:20] The first stage is understanding and classifying, if you like. The second stage, having done that well enough, you can then ask the AI to say, given that you understand, you know,
[0:10:45] What a dog looks like. Now generate me a dog with your idea of pink, with your idea of yellow spots or whatever. And that is an interpolation. It's a prediction of the space between two or three or four concepts. And that's what's produced this generative AI revolution in all of the modalities. As we apply more computation to this process, so we're basically stacking much, much larger AI models, and we're stacking much, much larger data,
[0:11:15] The accuracy and the quality of these generative AIs gets much, much better. So just to give you a sense of the trajectory we're on with respect to computation. Over the last 10 years, every single year, the amount of compute that we have used for the cutting edge AI models has grown by 10x. So 10x, 10x, 10x.
[0:11:39] 10X, 10 times in a row. And that is unprecedented in technology history. Nowhere else have we seen a trajectory anything like that. Over the next five years, we'll add probably three or four orders of magnitude, basically another thousand times the compute that you see used today to produce GPT-4 or the chat model that you might interact with. And it's really important to understand that. That might be a technical detail or something, but it's important to...
[0:12:06] Like sort of grasp that because when people talk about GPT-3 or GPT-3.5 or GPT-4, the distance between those models is in fact 10 times compute. It's not incremental, it's exponential. And so the difference between GPT-4 and GPT-2 is in fact 100 times worth of compute.
[0:12:29] The largest compute infrastructures in the world, basically to learn all the relationships between all the inputs of all of this raw data. What does that entail, enable them to do? In the next phase, we'll go from being able to perfectly generate, so speech will be perfect, video generation will be perfect, image generation will be perfect, language generation will be perfect, to now being able to plan across multiple...
[0:12:56] Time horizon. So at the moment, you can only say to a model, give me a poem in the style of X. Give me a new image that matches these two styles. It's a sort of one-shot prediction. Next, you'll be able to say, generate me a new product. In order to do that, you would need to have the AI go off and do research to look at the market and see what was potentially going to sell. What are people talking about at the moment? It would then need to
[0:13:25] It generate a new image of what that product might look like compared to other images so that it was different and unique. It would then need to go and contact a manufacturer and say, here's the blueprint. This is what I want you to make. It might negotiate with that manufacturer to get the best possible price and then go and market it and sell it. Those are the capabilities that are going to arrive approximately in the next five years. It won't be able to do each of those automatically, independently. There will be no autonomy in that system.
[0:13:54] But certainly those individual tasks are likely to emerge. So that means that presumably the process of innovation becomes much, much more efficient. The process of managing things becomes much more efficient. What does that mean? And let's stick with the upside for the moment. I promise you we'll get to all the downsides, of which there are many. But what is that going to enable us to do? I mean, people talk about...
[0:14:19] AI will help us solve climate change. AI will lead to tremendous improvements in healthcare. Just talk us through what some of those things might be so we can see the upside. Intelligence has been the engine of creation. Everything that you see around you here is the product of us interacting with some environment to make a more efficient, a cheaper table, for example, or a new iPad.
[0:14:44] If you look back at history, today we're able to produce a kilo of grain with just 2% of the labor that was required to produce that same one kilo of grain 100 years ago. So the trajectory of technologies and scientific invention in general means that things are getting cheaper and easier to make.
[0:15:07] And that means huge productivity gains, right? The insights, the intelligence that goes into all of the improvements in agriculture, which give us more with less, are the same tools that we're now inventing with respect to intelligence. So, for example, to stay on the theme of agriculture, it should mean that we're able to produce new crops that are drought resistant.
[0:15:28] that are pest resistant, that are in general more resilient, we should be able to tackle, for example, climate change. And we've seen many applications of AI where we're optimizing existing industrial systems. We're taking the same big cooling infrastructure, for example, and we're making it much more efficient. Again, we're doing more with less. So in every area from healthcare to education to transportation,
[0:15:52] We're very likely over the next two to three decades to see massive efficiencies. Invention. Think of it as the interpolation I described with respect to the images. The AI is guessing the space between the dog, the pink color, and the yellow spots. It's imagining something it's never seen before. And that's exactly what we want from AI. We want...
[0:16:17] to discover new knowledge. We wanted to invent new types of science, new solutions to problems. And I think that's really what we're likely to get. I believe that if we can get that right, we're headed towards an era of radical abundance. Imagine every great scientist, every entrepreneur, every person having the best possible aid, scientific advisor, research assistant, chief of staff, tutor.
[0:16:47] Coach, Confidant, each of those roles that are today the exclusive preserve of the wealthy and the educated and those of us who live in peaceful civilized societies. Those roles, those capabilities, that intelligence is going to be widely available to everybody in the world.
[0:17:06] Just as today, no matter whether you are a millionaire or you earn a regular salary, we all get exactly the same access to the best smartphone and the best laptop. That's an incredibly meritocratic story which we kind of have to internalize. The best hardware in the world, no matter how rich you are, is available to at least the top two billion people. And that is, I think,
[0:17:30] That is going to be the story that we see with respect to intelligence. All right, enough upbeat stuff. We've had 20 minutes of upbeat, which is more than you've had in most of the interviews you've done. But you didn't call your book, you know, The Coming Nirvana. You called it The Coming Wave. And I'm told that you were thinking the original title was going to be Containment Is Not Possible. I'm glad you didn't call it that. It wouldn't have sold so well. But explain, the argument you're making is not actually,
[0:18:00] Nirvana is around the corner. In fact, it's a much, much more subtle argument than that. So tell us what the downsides are and what it is that your book, the focus on containment in the book is about. Yeah, I mean, I think I'm pretty wide-eyed and honest about the potential risks. And, you know, if you take the trajectory that I predicted, that more powerful models are going to get smaller, cheaper, and easier to use.
[0:18:27] which is the history of the transistor, which is the history of every technology and value, basically, that we've created in the world. If it's useful, then it tends to get cheaper, and therefore it spreads far and wide. And in general so far, that has delivered immense benefits to everybody in the world, and it's something to be celebrated. Proliferation, so far, has been a really, really good thing. But the flip side is that if these are really powerful tools,
[0:18:55] They could ultimately empower a vast array of bad actors to destabilize our world. Everybody has an agenda, has a set of political beliefs, religious beliefs, cultural ideas, and they're now going to have an easier time of advocating for it. So at the extreme end of this spectrum, there are certain aspects of these models which provide really good coaching on how to manufacture biological and chemical weapons. It's one of the capabilities that...
[0:19:23] All of us developing large language models over the last year have observed. They've been trained on all of the data on the internet, and much of that information contains potentially harmful things. That's a relatively easy thing to control and take out of the model, at least when you're using a model that is manufactured by one of the big companies. They want to abide by the law. They don't want to cause harm. So we basically exclude them from the training data, and we prevent those capabilities.
[0:19:53] is that everybody wants to get access to these models, and so they're widely available in open source. You can actually download the code to run, albeit smaller versions of Pi or ChatGPT for no cost. And if that trajectory continues over 10 years, you get much, much more powerful models that are much smaller and more transferable.
[0:20:18] People then who want to use them to cause harm have an easier time of it. I think that's a really important distinction, that there are the leading companies, you, Google DeepMind, OpenAI, who have the biggest models now, and there are a relatively small number of these ones, and they are bigger and more powerful, but not far behind are a whole bunch of open source ones. And so the question is then for your containment, can you prevent...
[0:20:47] The open source ones, which will potentially be available to the angry teenager in his garage or her garage, can those ones be controlled or not? The darker side of my prediction is that these are fundamentally ideas. They're intellectual property. It's knowledge and know-how. An algorithm is something that can largely be expressed on three sheets of paper and actually is readily understandable to most people.
[0:21:15] It's a little bit abstract, but you can wrap your head around it. The implementation mechanism requires access to vast amounts of compute today. But if in time you remove that constraint and you can actually run it on a phone.
[0:21:29] Which you ultimately will be able to do in a decade. Then that's where the containment challenge comes into view. And I think that there are also risks of the centralized question. This is clearly going to confer power on those who are building these models and running them. My own company included, Google and the other big tech providers. So we don't eliminate risk simply by addressing the open source community. We also have to figure out what the relationship is between these super powerful
[0:21:57] Tech companies that have lots of resources and the nation state itself, which is ultimately responsible for holding us accountable. So let's go through some of the most frequently cited risks or indeed negative consequences. And the one that you hear a lot is as AIs become equivalent to or exceed human intelligence across a wide range of tasks, there won't be any jobs for any of us.
[0:22:23] Why would you employ a human if you could have an AI? So history suggests that that's bunkum. We've never yet run out of jobs. And being a good paid-up economist, I think it's a lump of labor fallacy. But lots and lots and lots of people say this. What's going to happen to the jobs? Where are you on that? Well, let's just describe the lump of labor fallacy, because I think it's important to sit with that, because that is the historical trend so far. What it basically means is when we have...
[0:22:47] When we automate things and we make things more efficient, we create more time for people to invent new things, and we create more health and wealth, and that in itself creates more demand. And then we end up creating new goods and services to satisfy that demand. And so we'll continually just keep creating new jobs and roles. And you can see that in the last couple decades. There are many, many roles that couldn't even have been conceived of 30 years ago, from app designer all the way through to the present day.
[0:23:15] Prompt engineer of a large language model. So that's one trajectory that is likely. I think the question about what happens with jobs depends on your time horizon. So over the next two decades, I think it's highly unlikely that we will see structural disemployment where people want to contribute their labor to the market and they just can't compete. I think that's pretty unlikely. There's certainly no evidence of it in the statistics today.
[0:23:44] I think it's possible that many people won't be able to, even with an AI, produce things that are of sufficient value that the market wants them and their AI jointly in the system. I mean, AIs are increasingly more accurate than humans. They are more reliable. They can work 24-7. They're more stable. And so I think that that's definitely a risk. And I think that we should...
[0:24:11] Lean into that and be honest with ourselves that that is actually maybe an interesting and important destination. I mean, work isn't the goal of society. Sometimes I think we've just forgotten that actually society and life and civilization is about well-being and peace and prosperity. It's about creating more efficient ways to keep us productive and healthy. Many people, you know,
[0:24:37] Probably in this room and including us, enjoy our work. We love our work and we're lucky enough and we're privileged enough to have the opportunity to do exactly the work that we want.
[0:24:47] I think it's super important to remember that many, many people don't have that luxury and many people do jobs that they would never do if they didn't have to work. And so to me, the goal of society is a quest for radical abundance. How can we create more with radically less and liberate people from the obligation to work? And that means that we have to figure out the question of redistribution.
[0:25:08] And obviously that is an incredibly hard one, and obviously I address it in the book, but that's the thing that we have to focus on. What does taxation look like in this new regime? How do we capture the value that is created, make sure that it's actually converted into dollars rather than just a sort of value add to GDP? So we're going to get on to redistribution and the role of government in just a second. But first, to remind you, and I should have said this at the beginning.
[0:25:30] Mustafa and I are going to talk for perhaps another 15, 20 minutes, but then we're going to open it up to questions. And for those of you who are watching on the live stream, feel free to start asking them now, because if this little AI that I have here is telling me that calls and notifications will be silenced, that's not very helpful. Yeah, now I've got it. I do see the questions there. So please start writing in the questions and we will get to them in about 15 minutes. But okay, role of government. You need to have, you will in this world need more radical redistribution. But one of the concerns,
[0:26:00] Is that AI and the rise of AI makes actually the functioning of democracy ever harder. We're already seeing lots of concerns about deep fakes wrecking the 2024 elections. Four billion people live in countries that will have elections next year. People are worrying about 2024, never mind 28 or 34. And Mustafa and I just had a conversation with Yuval Harari, who is as pessimistic as you are.
[0:26:28] Thoughtfully optimistic, who basically said it was the end of democracy. I'm not sure that either you and I agreed, but what is the consequence for liberal democracy in the coming decades in this world of AI? I think the first thing to say is that the state we're in is pretty bleak. I mean, trust in governments and in politicians and the political process is as low as it has ever been. You know, in fact...
[0:26:53] 35% of people interviewed in a Pew study in the US think that army rule would be a good thing. So we're already in a very fragile and anxious state. And I think that to sort of empathize with Yuval for a moment, the argument would be that these new technologies allow us to produce new forms of synthetic media that are persuasive and manipulative, that are highly personalized, and they exacerbate underlying fears.
[0:27:22] That is a real risk. We have to accept that it's going to be much easier and cheaper to produce fake news. We have an appetite, an insatiable, addictive, dopamine-hitting appetite for untruth. It sells quicker, it spreads faster, and that's a foundational question that we have to address. I'm not sure that it's a new risk that AI imposes.
[0:27:46] It's something that AI and other technologies accelerate, and that's the challenge of AI. That is a good lens for understanding the impact that AI has in general. It is going to amplify the very best of us, and it's also going to amplify the very worst of us. And what about the fact that this is developing in a world which geopolitically is split in a way that it hasn't been?
[0:28:09] At least in the last couple of days, in the post-cold war world at all. So we have the tensions between the US and China. We have essentially a sort of race for global dominance between these two regimes. In that kind of a world, how can you achieve the sort of governance structures that you write about in your book that are needed to try and perhaps prevent the most extreme downsides of AI?
[0:28:34] Yeah, I mean, much as I've been accused of being an optimist about it, I've also been accused of being a utopian about the interventions that we have to make. And I think that, unfortunately, that's just a statement of fact. What's required is good functioning governance and oversight. I mean, the companies...
[0:28:52] are open and willing to expose themselves to audit and to oversight. And I think that is a unique moment relative to past generations of tech CEOs and inventors and creators. Across the board, we're being very clear
[0:29:08] That the precautionary principle is probably needed. And that's a moment when we have to go a little bit slower, be a little bit more careful, and maybe leave some of the benefits on the tree for a moment before we pick that fruit in order to avoid harms. I think that's a pretty novel setup as it is, but it requires really good governance. It requires functioning democracies. It requires good oversight.
[0:29:34] I think that we do actually have that in Europe. I think that the EU AI Act, which has been in draft now for three and a half years, is super thorough and very robust and pretty sensible. And so in general, I've been a fan of it and kind of endorsing it. But people often say, well, if we get it right in the UK or if we get it right in Europe and the US, what about China? I mean, I hear this question over and over again. What about China? And I think that's a really dangerous line of reasoning.
[0:30:02] It sort of demonizes China as though China has this sort of like maniacal, suicidal mission to at all costs, at any cost, you know, sort of take over the world and, you know, be the next dominant global power. I mean, so far, I don't see any evidence of that. I mean, you know, I'm not ruling it out. I'm not, you know, sympathizer. But I think we should just be wide eyed about the actions they're actually taking at the moment. They have a self-preservation instinct, just as we do. And the more that we can
[0:30:32] I think the second thing is it's dangerous to sort of point the finger at China because actually we can't just have a race to the bottom on values. We have to decide what we stand behind.
[0:30:59] I mean, I'm a believer that we shouldn't have a large scale state surveillance apparatus enabled by AI. We shouldn't do that just because China are doing it. We shouldn't get into an arms race and take risks just because they're taking those risks. And that's difficult for some people to accept because they might be hyper pragmatic. And I think that that only leads to an inevitable self-fulfilling prophecy that we both end up taking terrible risks which are unnecessary.
[0:31:27] Government, or let's be concrete, what should this government, we're in the UK and presumably most people here are from London, the British government wants to be the superpower of AI, an AI superpower, and is having an AI conference on AI safety in November, there's a big focus here. What should this government, or indeed other governments, be doing concretely to minimise the risks? Is there stuff that should be banned now? Is there rules of the road that should be put in place?
[0:31:57] So the first thing is that governments have to build technology.
[0:32:01] You know, we've got into this habit of outsourcing and commissioning third parties to create technology. And I think it's really difficult to be able to control what you don't understand. And unless you build it, you don't deeply understand it. So I think that's just the first thing, which in itself is very controversial. When I propose that in government, people sort of throw up their hands and there's a lack of will, there's a lack of self-confidence, there's a lack of belief that government can be a creator and a maker, especially on the technology front.
[0:32:30] To do that, I think the second thing is that we have to have deeply technical and engineering people as well as, you know, technologists more generally in cabinet positions and at the heads of every government department. You know, it's pretty crazy to me that we don't have a CTO, a chief technology officer in cabinet, you know, running our big institutions. All of that is outsourced. The challenge is to be able to do that, you just have to pay...
[0:32:57] Close to private sector salaries. Again, another highly sensitive topic that no one wants to talk about, should never earn more than the Prime Minister. To me, this makes no sense. How can we have an open labour market where on the one hand we're saying to people, you know, go work for whoever you like, and on the one hand, you know, people are being paid 10x, and on the other we're saying, well, take this huge sacrifice in the name of public service.
[0:33:21] The practical reality is that if that happens over many decades, the net effect is that you have quality of one type over here and another type over there. And that's really what we're facing. We have to confront that reality. It's very difficult for people to accept that we should be paying super large salaries. It creates other issues around how we hold those kinds of people accountable given how much of the public purse they might be earning, etc.
[0:33:48] But fundamentally, those two things enable a third thing, which is governments have to take risks with regulation. There is a fear that governments act too aggressively or too experimentally and upset the big companies. And as someone who's on the receiving end of this quite a lot and have been in the past where mistakes have been made, I still think the right thing to do is to give governments a break.
[0:34:16] Let them make mistakes. Let them make investments that don't work. Praise the experimental governance structures. Have faith in the political process. Participate. Encourage it. Because otherwise, you know, there's just this spiral of decline, this sort of lack of confidence that we can actually do the right thing, that we should do the right thing. And then that ultimately leads to the self-fulfilling prophecy, much like with China. And do you think that your view is the exception in your industry? I mean, the stereotype is a
[0:34:44] A bunch of 30-year-old tech bros who, you know, think the government is useless and who are going to kind of change the world with AI and, you know, we're going to do this. Is that an accurate stereotype? Are you the exception? I mean, there is a, you know, should we worry about the hubris of people in your industry? I think, you know, we have polarization everywhere. So the stereotype is probably true, but the counter is that, you know...
[0:35:09] We can do it without technology. And I think that's totally wrong. Technology is an absolutely necessary but not sufficient part of the process. And I think that some people in Silicon Valley does have a tendency to be much more techno-libertarian. There's no question about that. The government is the problem. The objective is to eradicate the state and run it completely independently. And I'll be honest, there are some very, very influential, very powerful people who have that objective, are building towards that objective with both their companies.
[0:35:38] And that's what shapes a lot of the public fear about this, that you have a bunch of hyper-powerful people who are shaping this without much, with a kind of disdain for the state and the democratic process. Two quick questions for me, which I know someone would ask otherwise, and then we're going to audience questions. The first one is the whole question of the singularity.
[0:36:06] We can't have a conversation about AI without the singularity. Will it happen? When will it happen? I honestly think it's a very unhelpful framing of what's to come. And people jump to this framing because it's easy to point to Terminator and Skynet. But it's almost like leaping to the moon before we've even invented the transistor. I mean, it's hundreds of years away.
[0:36:29] Really unhelpful. There are many practical near-term operational capabilities that you can predict, just as I've tried to describe, and you can then use those to wrestle with what are the consequences for the nation-state. How does this change our businesses? What does this mean for our governments? So in general, I don't make those predictions. I'm very skeptical that the superintelligence framing is useful to us. What about the other one that, you know, backyard?
[0:36:56] Wannabe AI commentators are always talking about, which is the odds of existential catastrophe. What are the odds that we will wipe ourselves out with this? Again, I mean, I think very, very low. I really think... What's very low? I think infinitesimally small such that it's not worth putting... The reason I ask you that is because I asked one of your... Someone somewhat similar to you what this was. Oh, very low, they said. And I said, what's very low? Oh, about 5%. I was like, what? Yeah, yeah, yeah.
[0:37:24] So you think it's infinitesimally negative? Negative zero. Okay, well, that's a good place to end on. All right, we're going to open now to your questions and questions from the online audience. This is a good question from Kitty Haddock, who asks, what will be the impact of all that computer power on our carbon emissions? Or will AI be able to enhance productivity so we reduce carbon elsewhere? Yeah, another hot take on this. Very low and really inconsequential.
[0:37:50] The amount of carbon that we spend on our data centers is genuinely minuscule, relatively speaking. Secondly, most of that happens in completely renewable data centers. Google and Microsoft are both entirely 100% renewable. Google actually owns the largest wind farm.
[0:38:10] The largest set of wind farms in the world. One of the projects that I worked on whilst I was at DeepMind was making the entire wind farm fleet 20% more efficient. So, you know, right from the outset, they have been focused on this. I'm not saying there aren't other environmental consequences like the use of, you know, galenium and cobalt in the actual chip manufacturer and so on.
[0:38:33] I honestly think that relative to the benefits that we're seeing and with respect to the absolute cost of carbon per unit of computation, it's very, very small. And just a follow up to that, because an argument I have often heard is that the cost of electricity and the access to power will be a constraint on the development of these AIs and their proliferation. Do you also think that's not true? No, I think that's not true. I mean, I think that's not true. I think that some data centers
[0:39:03] We'll be at the 100 megawatt scale, which is maybe a single digit percentage of a small city's electricity consumption. But we're talking about a very small number at the 100 megawatt scale. I mean, that really is enormous. Nothing like that exists today. So don't worry about the carbon consequences of the actual AIs. From the audience, questions here? Yes, lady here in the second row.
[0:39:28] I'm not quite sure what that is. Do you get a microphone? Does it work that way? Yeah, it's on its way down. On its way down here. Right here. Maybe in the second row there. Thank you. Sherry Coutu from a number of education companies that use AI. My question to you is, if you think about two industries, say healthcare and education, and you think about...
[0:40:02] The applications that AI has. Could you choose between the two which you would hold the most hope for? And how should they be thinking about it? Should they be thinking about procuring it? And how do you safely or procure it well? Or again, as you said, you could produce it, but some of those organizations...
[0:40:25] May not be in a position to produce it anytime soon. So if you're a procurer, how do you do that? Well, and what are some of the frameworks that should be used for that? Yeah, thank you. That's a great question. I mean, on the I'm probably most excited in terms of the immediate near term impact about education. I mean, these models are already being used. I think the primary use case of chat GPT is in fact homework help. And people often think
[0:40:51] Oh, my kids are copying and copy-pasting. But actually, if you actually watch the way they're using these models, and many people use our models, our Pi, for exactly this reason, it's a conversational interaction, much like an enthusiastic teacher might speak to a child about the interest that they have. So the child, or the learner in general, gets to phrase the question in exactly their style.
[0:41:18] Picking on exactly the thing that they're interested in, asking the odd, obscure, poorly phrased, you know, not complete picture type question. And of course, the AI is infinitely patient, provides really detailed, mostly factual information. I mean, it's not always perfect, but it will be perfect. And I think that's an unbelievable meritocratic gain for everybody. And I think we need to picture a world in five years time where
[0:41:46] The best education in the world, completely personalized, entirely factually accurate, is available to absolutely everybody who wants it on the planet pretty much for free. Which sounds amazing. How do you go from where we are now to that world? I think the beauty of these models is that they have an inherent tendency to proliferate and get smaller.
[0:42:13] I mean, this is the upside of proliferation. They spread because everybody wants access. Everybody wants to integrate them. You know, there are so many competing models now. The cost of buying a model per word. So if you're building an app, for example, you'll go to one of the three or four big model creators and you pay per word. That cost has come down 70x since January.
[0:42:41] Because we're all competing with each other, right? So that means that you can now take a regular app that you might have been developing for years in its current instantiation and add a conversational widget. In fact, we're doing this at The Economist with the EconoBot secret project underway. Clearly not so secret anymore. Thank you. Sorry. And you integrate the conversational
[0:43:10] So you should be able to ask any question in the style and the theme of your brand about the specific content that you have. And it will be like a widget. It's like a plug and play widget that you can put anywhere in the app. And that's what I mean about proliferation. Obviously, everybody finds that useful. And you'll be able to use that tool in a low code or no code environment.
[0:43:35] The image generation models are being integrated into Adobe today. If you're already a user of Adobe, you're using the absolute cutting edge AI models in a drag and drop way. No training required. If you're building a new website today, it's drag and drop. You just grab a little widget and plop it over here and suddenly you have a YouTube player with your video and suddenly you have a conversational interaction with a language model that is conditioned over all your data.
[0:44:04] I think it's important to wrap your head around the idea that this is going to be widely available to everybody. There isn't going to be an access issue. And the risk and harm comes from mitigating the downsides of the bad actors who might use it for nefarious purposes. But the upsides are incredible. Let's get a go. Lots and lots of hands. Yes, lady there, but I'm going to get one from online while you get your microphone. And the one from online is a question from Sao Paulo. Gosh, your audience is...
[0:44:33] It's going from a long way. Rene de Paula Jr. asks, when you say we will solve this and that, who is this we? Humanity, corporations, the UN, or Elon Musk? I definitely hope it's not Elon Musk. I think of it as the kind of...
[0:44:56] The community of researchers, inventors, and creators. There's this sort of dialogue. Sometimes you see snippets of it on Twitter. Sometimes you see it in the research papers that academics publish. You see it in the blogs and the products that big companies produce. There is this sort of...
[0:45:13] Unfolding, you know, evolving mold of an ecosystem which is referencing each other, creating and evolving. And so when I say we, I certainly don't mean me at Inflection, my current company. I just mean the ecosystem of humanity, like we're trending collectively in a direction of invention and creation. Just one time. Does that ecosystem include Chinese scientists? So ten years ago, Chinese scientists were
[0:45:43] Not really part of the conversation. They weren't really very relevant. Over the last 10 years, they have launched onto the scene producing very high quality research, creative research. You know, the old stereotype was that they can only copy and steal. Again, I think a demonization, partly by Elon Musk, actually, who was a big proponent of this idea that they were just robbing our intellectual property. And there was some of that. But largely, they were just as creative as us.
[0:46:11] And they wanted to get access to these tools to build their own businesses and provide new products and services for their own citizens for the same reason as we do. And so if you start from that assumption, then of course they're participating in this ecosystem. Of course they're creating incredible models. They have their own constraints with respect to censorship, and that has slowed them down by a little bit.
[0:46:33] They're actually not going to be that far behind now. I mean, there are some issues with the export controls and they don't have access to cutting edge models, but I don't think that's going to hold them back for very long.
[0:46:44] Thanks, this is excellent. My question is about AI ideas and the people needed to think of them. And if you take someone like Steve Jobs, for instance, he had a very specific person, very specific interests and skills and talent to be able to develop not only technology, but the brand and a point of view on the world that came with that. Do you think AI would be capable of coming up, let's say, with the version of the Apple idea now? Will it be in the future or will it simply be a machination of past information?
[0:47:13] So I think people have often characterized these AIs as regurgitating their training data or reproducing whatever they have seen previously. And I think that's a kind of misunderstanding of what they do.
[0:47:27] They're almost always doing interpolation. The thing I described earlier is predicting the space between two ideas. They're saying, let me mash together these two concepts, just like the dog and the yellow spots and whatever. Take your pick of any combination. And that's creativity. Fundamentally, when I invent something,
[0:47:47] I'm really being inspired by a huge range of different experiences and ideas, and I'm using those to then produce a novel prediction or generation at any given moment, and I'm testing it out and seeing if it's useful or if it makes sense or if it catches on. And then it has a life of its own, and it's sort of independent of me. So I think for the next couple of decades, these AIs are going to...
[0:48:12] Aid the human in that process of creation and invention and discovery. They're not going to wander off and have their own agency and do their own thing. I mean, it's just not possible. The capabilities just aren't there and won't be there in the near term to do that, right? And so I think it's going to be the human-AI combo for a good time to come that does the creation. Exactly. It's more of the assistant, exactly. The brilliant assistant. Right. Let's go further back. Yes, gentleman there, four rows back. Let's get you up.
[0:48:50] Are you seriously trying to suggest that the AI companies are able to self-regulate and didn't the banks prove that that is an impossible concept?
[0:49:05] But the banks are highly, highly regulated and so not just by themselves. But look, I'm absolutely not proposing self-regulation. I mean, if that came across, then I apologize, I'm wrong. I mean, in the book, I really don't say that. I go to great lengths.
[0:49:21] To say that independent, external, technical expertise is required to do governance properly. I think the practical challenge, as Zannie pushed back on me earlier today when we were talking with Yuval, is where are these competent regulators who get the technical aspects? Where is this democratic process that gives us confidence that we can appoint people to conduct that kind of oversight?
[0:49:45] Some pessimism that they're capable of doing that. That should not mean that we sit around and do nothing in the process. You know, for example, I visited President Biden six weeks ago now at the White House with the other six AI companies, Microsoft, Meta, Google, DeepMind, et cetera, et cetera. And we signed up to voluntary commitments.
[0:50:06] That are precursor to regulation, which the White House designed because they realized they can't pass new primary regulation anytime soon. But the voluntary commitments are very material. We basically have said publicly, we expose our models to expert independent scrutiny, to red team or stress test, find weaknesses in our own models. Once we identify those weaknesses, we share them with each other and we share them publicly. So in transparency and the open light of day.
[0:50:34] And we know that that framework, the voluntary commitments, are a precursor to an executive order, which is coming from the president sometime in the next few months. They're also the basis for the prime minister, Rishi Sunak's AI summit in November in Bletchley Park.
[0:50:51] Where many world leaders and all the big tech companies are coming. And those voluntary commitments are going to form the basis of the discussions for what becomes binding, not just in the UK, but hopefully worldwide. So I'm totally with you that we're not going for a self-regulatory approach. But you don't think there's a conflict of interest? Well, I mean, there's definitely a conflict of interest. Of course there's a conflict of interest. I mean, we are a profitable, a for-profit company. In fact...
[0:51:20] I'm a public benefit corporation, so I think it's kind of an important clarification. It's a new type of company, closer to a B Corp.
[0:51:28] Which is a hybrid for-profit, non-profit mission. It means that our directors have a legal obligation to factor in the impact of our activities on the wider world, both the environment and people materially affected by what we do who aren't just our customers. And that doesn't solve all the issues with for-profit businesses and the conflict that you described. But it's a first step in the right direction. And I believe that that's how change happens, taking small steps in the right direction.
[0:51:57] Let's take a question from over there. Yes, gentlemen, quite near the back with the white T-shirt. Yeah, right there. Hello? Yeah? Okay. My question to you as an electronics engineer is, should we now focus on the hardware part of it, considering there's a monopoly going on and the concentration of chips to a certain country? The hardware part of it is raising a very big question. We saw it in COVID.
[0:52:24] Things are really bad when hardware supply goes down. So is this a great time to focus on hardware, considering we are good with software part for now? That's a great question. I mean, we didn't really talk about that too much here. But just for everyone's benefit, these AI models are trained on GPUs, graphics processing units.
[0:52:44] Chips that were previously used for gaming, for representing graphics and computers. And we take each one of these chips and we daisy-chain them together thousands and thousands of times. We have a computer at Inflection which is the size of four football pitches and has 25,000 of these chips daisy-chained together. An enormous cluster. It's cost about a billion and a half dollars. Now, all of these chips are manufactured by one company, Nvidia.
[0:53:14] Who, I'm sure people have heard, have seen their share price go up by 350% since January. Their chips are manufactured entirely in one factory called TSMC, Taiwan Semiconductor Manufacturing Corporation, which is obviously in Taiwan. The key component of their chips, of their fabrication facility, are manufactured by one company called ASML, a Dutch company.
[0:53:43] So the supply chain is, I mean, we can talk about how this happened over 30 years, but extremely narrow. There really are no competing providers that are material at any of those three stages. As a result, the good news is that that means that there are choke points that can be used by regulators to monitor who has access to the critical chips that enable the training of the models.
[0:54:10] And of course, restrict access to certain people. So I think I loosely alluded to the export controls a minute ago, which is a new piece of legislation or a rule that the US administration imposed on China last year, which prevents China, anyone in China, any manufacturer in China from getting access to the latest version of these chips, which means that they won't be able to train the GPT-5 level model.
[0:54:38] A number of people have referred to this as a declaration of economic war on China. And so I think that we have to be very cognizant of that denying them access to that is likely to deliver a significant counterattack on the West. We are hugely dependent on their supply chain in many, many respects. So yeah, chips are absolutely at the heart of this, both in good and bad ways. So if you're focused on a chip company,
[0:55:07] It's a big bet. It takes a long time to mature, but it has the potential to be the critical component here. Just a follow-up question. Yes. Do you think that open-source hardware will help in creating a better setup right now, considering very few companies are focusing on creating the hardware and all of them are completely for-profit? So something like open-source hardware focusing more, will it help us create better computers,
[0:55:36] Better models with less power. Yeah, so I think...
[0:55:40] Open source hardware is a serious effort. And just to clarify, I mean, open source elements of hardware design are used in many, many areas. So Open RAN, for example, is a hardware design for 5G masts, which ensures they're interoperable. It means that the software that runs your telephone networks actually can run on any type of hardware because the interface is standardized, which is a great thing for competition. There isn't a lock-in between hardware, the builder of the masts, and the software, the people who run the...
[0:56:09] The downside of it is that it has tended to be a bit more flaky than the fully integrated side of things. So I think you should be wide-eyed about it. It isn't going to be the panacea to solve all of our problems anytime soon. Let's take another question there. Yes, lady in the fourth row. Hi, thank you both. I'm Javaya Raskari. I lead digital regulation work at Tech UK, which is the digital tech trade body in the UK.
[0:56:41] Over a thousand members, ranging from big tech, DeepMind, Google, Meta, all the way through to cybersecurity providers, SMEs. Many of our members are harnessing the really positive impacts of synthetic media, but many are becoming increasingly concerned with the rising malicious use of deep fakes. So everything from revenge pornography, undermining digital ID verification, fraud, which is a big one.
[0:57:08] In your opinion, what should companies do now to address the rising kind of problem of deepfakes? I know you mentioned voluntary charters, which we already do with things like fraud, but what should we do now? Yeah, it's a great question. I mean, I think the first thing to say is that political parties and political campaigns shouldn't be allowed to use...
[0:57:30] AI generators for their content. I think we should just start by taking that off the table. That's a precautionary principle. There are potentially some downsides to that, but it feels like a safer and sensible thing to do. The second thing to say is that we shouldn't allow the big tech platforms, so Facebook or Twitter or anywhere where there's a broadcast of information, to have digital people, counterfeit digital people. So if you
[0:57:59] You know, have a handle Zannie on Twitter, for example. Only Zannie should be allowed to represent as Zannie on Twitter. I shouldn't be able to come along, create a perfect synthetic fake of Zannie and have that, you know, imitate her language. Now, I think that's a reasonably straightforward, sensible thing that all the big tech platforms will commit to. It doesn't address other platforms, right, outside of, you know, the big...
[0:58:25] Big providers. And those tools and techniques are going to be widely available. Again, it's a proliferation question. It's going to be really difficult to say to somebody, well, you know, you're using synthetic media to generate a new product design or a new fashion outfit or all these other good uses. You're not allowed to have it because there's a risk that you're going to be able to generate some, you know, deep fake.
[0:58:47] I think we should also be wide-eyed about how quickly we adjust to the risks. Back in 20-odd years ago, people were like, well, we'll never be able to do financial transactions on the internet because there's so much fraud.
[0:59:01] We're going to be inundated with fraudulent activity. We do tens of trillions of dollars of transactions. It's completely transformed our world. And we have a minuscule amount of fraud. And it's a constant back and forth. Likewise with spam detection. Everyone thought we were going to be inundated with spam. We're going to produce all this automated content. Increasingly, the next threat is that older people are being tricked by AIs that can imitate the voice of, say, your daughter or child.
[0:59:30] Who might be asking you for a loan or something. There's this con man scam type thing, which is now more possible and more capable. Of course, that's a new threat vector that causes real harm. On the flip side, spreading knowledge and information about it, there's a very, very simple defense, which is just to say, never provide access to my account over the phone. I'll never call you out of the blue asking for that. So we adjust.
[0:59:56] We adapt. And it doesn't mean that we can eliminate all of the harms, but it means that, like, net-net, we just have to be more resilient and more focused on adaptation. Gosh, lots of questions. Yes, gentleman there. Four rows back. And then I'm going back to this side. Yeah, right there. Thank you. Tom Czarnecki, Sustances, Think Tank on AI. I look at the intelligence in the name of your company, it's Intelligence Square, and that reminds us...
[1:00:33] That it is not just a new type of technology. It's a new type of intelligence. So I agree with you entirely. I also agree with your view of the world of abundance. Absolutely superb. I'm also an optimist. Where there is an area of contention is about superintelligence and about the existential risk. I must say that I've been shocked hearing what you are saying. And I just challenge you.
[1:01:02] On that, on AIGI, Artificial General Intelligence, which many people think may emerge within the next five years or so. Apart from the definition of what it is, let's make it very simple, that it will be smarter than humans. And if it is smarter than humans, then of course it can outsmart us, set its own goals, and exponentially increase its power.
[1:01:31] And the end extension, Fred, to us. Yep. Fair question. And I certainly hear this a lot. I think that there's a risk of anthropomorphic projection. We see a model that is capable of generating images or generating text. And we assume that, therefore, it is going to emerge the capability to have its own goals.
[1:01:53] Or it's going to emerge the capability to be able to update its own code. Or somehow it's going to sort of naturally learn to operate autonomously and then deceive us and get out of the box. And my belief, and I may be wrong, but my firm belief from all of my years of working in this field is that those are capabilities that we would choose to design into the model, that we would be able to observe. And if someone does
[1:02:22] If you choose to create those models, those capabilities, then yes, there are risks that they could get out of the box and they could be uncontrollable. That is really the program of containment. It's basically saying that it is conceivable that these models could be used to do really bad things over a couple of decades and that they have to be restricted.
[1:02:52] AGI's and each of them may be smarter than humans and some of them won't be controlled by us and therefore the risk is there I think. Thank you. I'm going to leave it at that because you can imagine huge number of things and there's a lot of actual hands gone up with lots of questions. So yes, lady here in the third row. Hi, I think we said we were going to talk about redistribution. I just want to know what you make of the kind of growing disparity between
[1:03:22] The individuals, the people that provide the raw data that make the realization of these kinds of technologies possible and those that obviously control these technologies to whom the lion's share of the wealth flows to. How are we going to redress that kind of growing disparity and how are we going to compensate people for what they give to these systems ultimately? It's a good question. The way that these models are trained today is that they have scraped
[1:03:49] So far, anything that you put up on the web, a blog or a website, the cultural and legal consensus over the last 25 years has been that it is fair game. It's open to anybody to read it, to use it, provided you don't regurgitate it word for word. So if you copy an entire paragraph, that is copyright.
[1:04:17] People in the big tech companies and myself included are making is that we're capturing the essence of these models. We're learning the style. We're learning the tone of text, never reproducing the underlying content. And practically speaking, I don't think it is possible to capture the dollar value and return 0.001 cents to a creator of a website. Go ahead.
[1:04:44] If that's going to automate people's labor, then we need to find some way of redistributing that. You said that there was no... That I agree with. You kind of commented saying that there was no displacement that we were kind of feeling. What about the SAG strikes? Or what about, you know, arguably the RMT are striking because they are being automated to some degree. Their jobs are being automated. We are seeing to some degree...
[1:05:06] Do you think that that is going to be a growing part of labor relations going forward? Is the directors and actors strike the beginning of something that you're going to see?
[1:05:29] Is this going to be a real fight? There's clearly a fight for copyright already in my industry. We're furious that you just sucked up all of our data without telling us. Yeah, yeah, totally. The transition is going to be painful, for sure. So at the moment, taxation in the US is on average 25% for labor. So anyone who's working on average 25%. The tax on software is only 5%. And tax is a tool for incentivization.
[1:05:57] So we should think about it as a tool for adding friction in the areas that we want to go slower and speeding up the things that we want to go faster. It's pretty much as simple as that. If you add a huge taxation burden, then yeah, you're going to slow down innovation, but it is going to keep people at SAG in work for longer. Those are rules that we get to make. They're choices that we get to make. And I think that's exactly the discussion that we should have.
[1:06:24] There's a question from online which is related to this, and it's a sort of question about the big picture outcome. Will AI make inequality better or worse? So the extremes of inequality are going to continue. So those who currently have access to power and resources are going to be the first to adopt new technologies. We've already seen that.
[1:06:51] Tech companies that have vast cash reserves, that can hire the best people, that can acquire the most amount of data and compute, are moving faster than ever before. On the flip side, this revolution is one that is also being led by the open source movement. I mean, that's kind of incredible. Today, you can get an absolute cutting-edge model that is, say, 18 months behind, that costs...
[1:07:20] Less than $2,000 to train. And it'll be as good as GPT-3. So that was the cutting edge 18, 24 months ago. That trajectory is going to continue. So I think that the open source movement is always going to be 18 to 24 months behind, at least for the next five years until the models get really, really big. And that's an amazing story for inequality. I mean, it's a very meritocratic moment that...
[1:07:48] Whatever your job, whatever your app that you're developing, you'll be able to integrate these tools into your own workflow very, very cheaply and easily. I mean, as I said, the cost of even using one of the best models in the world has dropped 70x in the last year. So I think that on the face of it, that does good things for equality of access. What you can't stop is that the very top people race away the fastest. And I don't know what that trajectory ends up looking like. Thank you. Yes, gentleman there.
[1:08:18] Five rows back. I'm going to try and get through everyone's questions. Thank you for the talk. It's been very, very interesting and also great participation. I want to bring it a bit back to the human side. We talk a lot about tech. So how do you think about solitude and having a positive relationship with technology? And I'm thinking, I tried Pi. It's amazing. What if it becomes better than all of my friends together? What if it gives me all of the best ideas? I'm like, why should I hang out?
[1:08:45] Or come to this event or do things in real life? That is a really, really good question. So for those who don't know, I mean, we've designed Pi to be an amazing conversational friend. So if you use ChatGPT, you can say generate me a business plan or a marketing strategy or a poem or a travel itinerary. Pi is much more like talking to a best friend or a confidant.
[1:09:12] It's super fluent and high EQ. It asks you clarifying questions. It rephrases what you've said. It reflects back what you've said. It's very relaxed and supportive. It's extremely non-judgmental. No matter how awful your racist diatribe or the vent that you need to get out about your horrible colleague at work, it's very patient and supportive.
[1:09:36] And I think that's an amazing contribution to the world, providing people with a supportive companion. But we've also designed Pi to encourage you to talk about your friends and to get out. It is explicitly trying to help you have a place to simulate and practice if you're feeling anxious, but reconnect with other friends.
[1:09:58] The values that we bake into these models and what we actually mean in practice by quote-unquote safety first, that's the key conversation that we have to have today. Because the mental model that we've got to accept is that we're not going to struggle with hallucinations, which everyone talks about. We're not going to struggle with bias. Focus on the challenges that we have in the world when we have perfection. And that's exactly what I think you're getting at, which is what do we do if it really is...
[1:10:27] A place that does make me, you know, a relationship that makes me smarter, that makes me feel calmer, that makes me feel more kind, more optimistic, more respectful of myself.
[1:10:38] The reality is that's the trajectory that we're on. But if we're on that trajectory, to follow up with a reference to an incident that I'm sure many of you will know about, that New York Times journalist who had a fairly weird interaction with an early prototype of ChatGPT, which said, you know, why aren't you leaving your wife? I know you don't really love her. Is that a risk with Pi? No. So Pi is...
[1:11:02] Currently the safest AI in the world today. None of those provocations work. So Pi knows that it is an it. It knows it's an AI. If you try to flirt with Pi, if you try to have a romantic relationship with Pi, it's extremely clear and resistant. Again, it doesn't judge you.
[1:11:22] It'll tease you. It tells you, it pops you off without upsetting you. It'll keep you at a distance. It'll keep you at a distance, say, look, I'm not designed to do that. I can't do that for you. These are boundaries. I am an AI. I am not designed. You are trying to flirt with me. I can't go there. Boundaries are critical. Boundaries are what give us the feeling that we have control and establish trust between us. So it's a very important part of Pi's design, and it doesn't suffer that. We read teams that a great deal.
[1:11:50] Okay, thank you. Yes, lady here in the fifth row. Yeah, go ahead. Just picking up on your comment about politics. If it was up to you, at what point would you allow politicians or political parties to use AI? And how could AI help politics be the best of itself, best it could be? It's a great question. I mean, and I hope...
[1:12:20] That an AI like Pi not only makes you more kind and respectful to yourself, more forgiving of yourself, but also in doing so makes you more kind and respectful to other people. I mean, I think we've just become so overwhelmed by an adversarial politics and social media and celebrity culture. I think, you know, I hope that these AIs can help you to imagine and model, simulate and practice.
[1:12:46] You know, those kinds of more respectful and pro-social behaviors. I'm not itching to give politicians access to AIs as decision makers yet. I think we're a long way from that. And I think people often imagine that it could be the ultimate strategist or it could have the ultimate policy insight. And I think for now, I think I'm much more focused on, you know, the emotional intelligence that it can all give us. There's a question up here. Can we get the microphones up there? There's a gentleman there on the balcony.
[1:13:17] I'm sorry, I hadn't seen you up there. Hello, I'm Faisal. I'm a data scientist and a government contractor. I've worked on simulating COVID and developing autonomous weapons. So today we've spoken about the recent AI wave of large language models. So my question is to you, what's your kind of take on the kind of feeling within the large AI houses today
[1:13:47] Can we still keep the progress of AI alive with just this current paradigm of large language models? And what is the desire to push forward and explore new model classes, new paradigms of AI? Yeah, so I mean, I don't think there's any risk of progress in AI slowing down anytime soon. I mean, I think some people have been afraid that we're going to sort of regulate the progress out of the system.
[1:14:19] I personally think that that is extremely unlikely at this point. I mean, the challenge of the last century has been in inventing and creating new technologies and powers. And I think that the challenge has now flipped. The challenge of the next few decades is going to be in containing and shaping those powers so that they always work for us. I think that large language models and deep learning itself, large language models are a version of deep learning.
[1:14:48] You know, ecosystem of invention has already been opened. It's set on its course. And I don't think we're lacking any fundamental algorithms or that, you know, another, I mean, I'm not sure you were referencing about other methods, but I'm not convinced that we need other methods to make progress. I was kind of meaning it in the sense of continued progress now leading to a general form of intelligence. Do you think the large language model paradigm is enough?
[1:15:16] Because from my point of view, it's still an open question. Right. I think that most of the capabilities that would be involved in a properly general intelligence like the AGI that we described would be engineering decisions. There would be ways that you organize the current set of tools to do, for example, recursive self-improvement, to do self-supervision, self-goal definition.
[1:15:40] Those are, I think, engineering capabilities which we can choose to do or not in the next 10 years. Awesome. Thank you so much. Thank you. The lady back there on the... Yes. Two rows from the back. We've probably got time for two or three more questions. Thank you. Congratulations, Mustafa. You've achieved many things.
[1:16:03] Incredible that you've written a book. I actually wanted to ask about you. What motivated you to actually write this book whilst you were starting this company? And also, thank you for unpacking. Yeah, I often ask myself that same thing. Yeah. Well, also, like, exposing yourself to answering every single question on the future of humanity and progress. Great question. Go ahead. Why did you write this book? And why are you here tonight answering these questions? I couldn't help but write it.
[1:16:32] I wanted to be on record making a prediction about how I think things are going to unfold in order to sort of look back in a decade and calibrate and see, you know, was I, as Zannie sometimes thinks, too much on the catastrophe side of things, too dark about my predictions? Do I have my own, like, sort of, am I over-obsessed with pessimism or indeed the reverse? Like, am I dismissing existential threat?
[1:16:59] you know risks too much but maybe that's real and I think the rigor of putting something out that other people can critique and really researching I mean I really really did research a lot of the historical trends as well was just a very kind of satisfying thing to do so from a selfish perspective it was really about sort of
[1:17:19] Trying to be articulate and clear about my predictions so that I can validate them in the future. And then have an excuse to spend time every morning, you know, before I start work, writing and reading and researching and trying to, you know, see what history has to teach us about this. I mean, the first three or four chapters are mostly about the historical basis for proliferation and general purpose technologies. Over here, one more question over here. Yes, there, the three rows in the back.
[1:17:50] Thank you, Mustafa. I thought that was really interesting. You've laid out quite a compelling case for the next five years and the likely trends for AI. But I'd be quite curious to know what's taken you by surprise in the past couple of years. Has there been any developments in AI which you perhaps didn't manage to predict or anticipate? I'd like to know what surprised you the most recently. So initially, there was a fear that we would never be able to
[1:18:20] Control the quality of the output. So two years ago, we thought that bias was going to be the big challenge. All we were talking about was bad training data in producing toxic generations. This thing is going to constantly make things up. It's going to constantly hallucinate. And what we have found, empirically observed, is that with each order of magnitude more investment in compute,
[1:18:50] The models get easier to control. We can create very precise and detailed behaviors. You know, the tone of Pi that I just described to you, and I hope people will try it. I mean, you can actually phone Pi. You can speak to Pi in fluent natural language, just as you would on a normal phone call, and it will speak back to you in one of five different voices. What's the choice of voices? They're called V1, 2, 3, 4, and 5, so you'll have to guess.
[1:19:20] We deliberately designed it to be age neutral, gender neutral, and hopefully race and accent neutral. So it actually is quite varied. Sometimes it sounds a little Australian, sometimes a little English. It's very subtle. It's not like annoyingly all over the place. But we try to capture the essence of what is AI like. You know, we spend so much time thinking about what is human like, what are human like capabilities.
[1:19:46] I wanted Pi to be true to what it is to be an AI. And an AI is a product of all the training data and all the people that it has interacted with. So we tried to not make it too much like one character in our world. So that was the surprising thing, is that the models got easier to control. And it's almost like having a new clay. It's this new design material that you can
[1:20:16] You know, shape into almost a personality. You know, it's a really precise, yeah, clay. And I think that's been super exciting and very, very creative. And I'm just so excited because in the next few years, you know, we've been one of the first in the world to get access to it because we're lucky in our position and everything. But in the next few years, many, many people are going to have access to the same tools.
[1:20:40] In just natural language, you're just able to give an instruction or low code, no code environments where you drag and drop and plug and play. And I think that's going to be a really, really amazing time to see what people do with it. I think we have time for one more question from the floor. And I'm sorry that there are lots that we haven't got to. Let's go to the back there. I know it's hard to get to. Hi. Thank you. So I just want to ask a question in regards to the governance and also...
[1:21:16] Who's going to benefit most and who's going to be left behind when we think about emerging markets and developed economies? You mentioned that, for example, Pi can respond in five different tones, but in terms of languages or parts of the world where English is not that proficient, how will the impact be felt there? It's a great question. I mean, Pi already speaks about 25 languages.
[1:21:43] It's really good in the major languages, Spanish, French, German, and so on. It's much less good in Japanese, Mandarin, Arabic, et cetera. It's certainly not good in the long tail of languages. I think it's kind of remarkable that these models have arrived with so many capabilities in terms of their languages simultaneously, but I think it's going to take us
[1:22:07] A few years before we add the full suite of languages. And it's not just languages, but it's actually the training data that reflects the cultures of people outside of the Western world. I mean, English has dominated over the last few centuries, and most of our culture is documented in English. And that is obviously a subset of all cultures. So there's a clear representation question there, which I think is going to be challenging when it comes to smaller communities.
[1:22:36] So I want to conclude by seeing what impact you've had on the crowd. I should have asked this at the beginning, but let's at the end, put up your hand if you think that having heard all of this, the net impact of AI for humanity is going to be positive. Okay, and just as a check, those of you who think it'll be negative. Well, it's not overwhelming, but the positives clearly win out, Mustafa. I think after you've read Mustafa's book, you will have a...
[1:23:06] I think a very clear and sober sense, both of the potential benefits, but also of the risks. You're not a wild-eyed, you know, Panglossian about this. It's very serious. It's a really excellent book. I recommend it. Mustafa, thank you for joining us. Thank you so much. Thank you. Thanks a lot. Well done.