[0:00:00] The very best founders I know are brilliant at building systems. They connect teams, they remove bottlenecks, and they eliminate single points of failure. And yet, when it comes to their own wealth, most are running a disconnected stack. A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them. Creative planning was built to fix exactly that.
[0:00:25] One integrated team of tax professionals, estate planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together. Proactive tax efficiency, estate strategy, investments, all under one roof. Creative planning where wealth works together. Learn more at creativeplanning.com slash masters of scale.
[0:00:53] We've all spent money on something that seemed incredible but didn't live up to the hype. A kitchen gadget, a car. When it doesn't deliver, the disappointment is real. Marketers suffer from this too. Chasing metrics that look great but don't translate into revenue. LinkedIn calls that bull spend. Invest in what matters and reach the right buyers with LinkedIn ads. You can target by company.
[0:01:18] AI is full of possibilities. Enterprises run on realities.
[0:01:45] That's why Pega combines the world's most powerful AI with the governance, transparency, and control enterprises demand. So you can move faster, adapt faster, and scale AI with confidence. All the power of AI, none of the uncertainty. Learn more at pega.com slash AI. A lot of the cyber issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering.
[0:02:20] The human's very much still in the loop. And the next phase, which I think carries the most risk, is closed RSI, closed loop, without the human reviewer process. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI. I do think there are some things that generally all the leading labs do agree on, like...
[0:02:39] The idea of third party auditors or evaluators coming in, obviously there's a question about which third parties. And I think we need a healthy kind of ecosystem of that. You could have good startups and companies doing that. There's enough revenue there for them, given the amount of investment going into AI as a whole. This is Masters of Scale. I'm Jeff Berman, your host.
[0:03:02] Thanks for having me. We've known each other for...
[0:03:42] I think well over 10 years now, maybe approaching 12 or something like that. It was an honor when you asked me to be on the AI ethics board for Google for our various funny meetings. So let's start, because one of the things we've been trying to do at the summit is to be envisioning the world as we'd want to make it. The theme has been agency.
[0:04:09] I have been a leader in and knew well before most people were thinking about AI that one of the key things for thinking about AI for human benefit is medicine and medical. And so say a little bit about how you envision this, how this helps people say, oh, if I could be getting that, I understand why you AI guys are actually trying to build something that creates a longer human experience, a more quality of experience. Call it anti-X risk.
[0:04:38] Yeah, for sure. I mean, if we take a step back for a moment, it's like big picture. And you know this, knowing us from quite early in our DeepMind journey, that the whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to advance science and medicine and our understanding of the world around us, but also improving human health.
[0:04:57] I've always said that the number one thing we should use these amazing tools that we've now built and we were dreaming about 20 years ago is to improve the human condition and improve human health. And I've tried to do that certainly with my own...
[0:05:11] More personal projects, obviously things like AlphaFold, but also the spin-out drug discovery company, Isomorphic. I've tried to push it really hard in that direction, as I know you have too more recently. And I think it's exactly the sorts of things we need to be showing the world that AI can do. I think what better thing than to use AI to help us cure terrible diseases. So some of the protein folding was this kind of...
[0:05:37] We have really good data. We could use scale learning because the whole learning, the cognitive revolution from learning, starting with...
[0:05:47] Even Pong and self-play and all the rest. It's like, okay, this would be a really good way of doing it. How does AlphaFold lead to medicines? Because one of the things is a question I get frequently is like, yeah, you're talking about that, but is it true? Is it going to happen anytime soon? So I'm passing along the question I get to you. Yes, sure. Well, biology is the most complex systems.
[0:06:09] So that's why it's so hard because of the emergent behavior, the interactions, and proteins are the building blocks of life. That's how I've always seen them. And the protein folding problem, which is what we tried to solve with AlphaFold, was considered to be like a 50-year grand challenge in biology. It's sometimes described to me as like Fermat's last theorem, but for biology.
[0:06:32] Thank you. Thank you. Thank you.
[0:06:57] And so it was always a critical, but not obviously, only still a small part of the entire drug discovery process. But it's an absolutely critical part of that. And we were able to, with AlphaFold, accelerate it by orders of magnitude by building AlphaFold. And interestingly, at the time, and if you think about it, we finished AlphaFold 2 back in 2020, which is six plus years ago now. And it's still probably the best example of...
[0:07:25] Concrete example of AI applied to a really hard scientific problem that was thought to be sort of uncrackable. And so that just shows you how far ahead of its time it was. And now, you know, we're really proud of the impact it's made. Like 4 million researchers around the world use it. Pretty much every biology and medical researcher in the world, almost every drug in future will have made use of AlphaFold, all the structures. But it's not enough. We want to expand it to biochemistry and...
[0:07:51] Also predicting things like toxicity or ADME properties. All of these other things that you need to know about before you can have a drug that's safe to use and that's efficient in whatever it is you're trying to tackle. So speaking of accelerations, let's go to one of the hot topics of the day, recursive self-improvement. What is your kind of theory of the game? And I say this because you...
[0:08:14] Or a master games person. So I use that as kind of deliberate. And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia. So like RSI, opportunities, concerns. Well look, RSI, it's obviously an extremely useful technique to make progress. It mostly works best in what I call verifiable domains.
[0:08:39] What's called variable domain. So math is one, which is why we're seeing so much advance there. Coding is another. But originally it was games.
[0:08:48] So, actually, one of the first sort of famous recursive self-improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which was the kind of, you know, famous program that beat the world champion at Go back in, you know, 10 years ago now, 2016, and kind of was the beginning, I think, of the modern AI era. And then we built a second version that was in some ways more consequential, AlphaZero.
[0:09:13] that didn't need to use any human data anymore and it just basically played against itself. Started at random and then...
[0:09:20] After around a dozen to two dozen different epochs, it played like a million games against itself. It would then train a new version of itself that was slightly better than the old version. And then that would play a million games against itself and so on. And for example, and then it could play any game, not just Go. It could play any two-player perfect information game. And for me, it was especially visceral as a chess player because I remember watching it play chess. And within, you know,
[0:09:49] 12 hours, basically, or less than 12 hours. It would start from random play in the morning to, you know, you'd have your coffee break at 11 a.m. and I could still just about beat it. Then by lunchtime, it was grandmaster level. And then by tea time, it was better than any chess playing entity. Tea time is dinner time for the U.S. folks. Yeah, computer time. Or computer, you know, kind of handcrafted chess computer is better than all of them. So we sort of saw the power of that recursive self-improvement.
[0:10:18] And obviously the thing that all these domains have in common is you can verify the answer. Is it a correct answer? Is it an optimal answer? Is the equation correct? Does the code run? Did you win the game? And the important thing about a verifiable domain is you can verify the answer so you know when you've hit the right point. But also that means you can create almost infinite synthetic data.
[0:10:41] Because you can just use the current best versions you have to set new problems or make new moves and then just sort of verify whether the synthetic data was correct. The problem with synthetic data a lot of the time is you can't verify that it's right from the right distribution. But in these cases you can.
[0:10:59] So we've seen the enormous progress with maths and with coding, and coding's obviously become enormously valuable, and everyone is having to do that now. But I do worry about it because in the limit, so there's different forms of RSI. Right now we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is it's making a human engineer maybe three, four, five times more productive.
[0:11:28] But the human's very much still in the loop and reviewing code and architecting things and writing code still. But it's all just enhanced. The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure.
[0:11:51] We'll obviously be faster, but I think it's also a very dangerous way to get to full AGI because we're likely for two reasons. It sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know how do you control, keep control of such a system, such a process.
[0:12:10] And then the end result will likely be quite alien. It's going to be like an alien intelligence. There's no reason you to think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for us to understand it and maybe for it to understand us. So I think we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that, but there is a sort of race to this sort of intense,
[0:12:39] It's obviously a competitive race to improve your coding systems. So presumably the incentives to be in a race continue, and you and I, some people really know this, have tried to figure out if there's various ways you can get global collaboration to have less.
[0:12:57] You've dedicated a huge amount of energy to that, so thank you. It's a hard game, as you know. But presume that the competitive pressure for continuing to do RSI and coding continues. In that environment, what are the best ways that we can potentially navigate that? Because we have to see where RSI is going, what it's pointed at, what are the kinds of things that the leading labs, such as your own and the others...
[0:13:24] What should they be doing to try to navigate this environment? Well, what we're doing and I think other leading labs are trying to do is you've got to, so there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of your general models that are fine-tuned for cyber defense.
[0:13:51] And so you keep the defenders ahead of the attackers. It's a pretty tricky problem to do in general. Also, building much more hardened sandboxes. I think a lot of the cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. I think you need really amazing monitoring systems. You know, this shouldn't be possible for agents to be running for weeks without anyone realizing. You know, these kinds of things that we've seen some of our competitors and others
[0:14:20] Maybe it's just the beginning of this happening, so I think everyone will get rapidly better, but they have to as well, because otherwise something bad is going to happen inadvertently. Some kind of incident, I sometimes call it like an oil spill, is going to happen on the current trajectory if we don't get it collectively, if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems. Eventually, you kind of want...
[0:14:43] formal guarantees, maybe some bounds around what these systems can do and how robust, test how robust the things like the guardrails are. This is actually something I encourage a lot of top mathematicians that I know well who are maybe pretty sad about the state of pure maths, but their skills would be incredible if they pivoted to these types of domains, more like complexity theory, game theory. And of course, we're going to have to start dealing with swarms of these agents, potentially collaborating in some way.
[0:15:12] How does one keep track of all of that? Should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, mutual accountability. Yeah, I mean, there's been lots of proposals and I've been working on these things a lot. And you saw my essay over the summer, maybe some of you read it about...
[0:15:39] Potential for an SRO industry-run standards body with some kind of government oversight over it. I think something like that is going to be needed, and then ideally it becomes international. It's US-led, and then obviously the state of geopolitics today makes it, and our international institutions are very, very weak, as we all know, just at the time when we need them to be strongest, in a way. So that's a bit unfortunate, but we've got to start somewhere. By the way, British, a bit unfortunate is, holy f***.
[0:16:07] Just to translate. Just to translate. Another translation. Yes, like tea time and dinner time. Yes, exactly. A bit unfortunate. And so we're trying to, you know, various of us are trying to do things about that. And that essay did have an impact in the White House and in the administration and with our colleagues. And eventually, you know, I think some of that energy became the accord that the famous letter everyone signed, which is...
[0:16:34] Which is better than nothing. I don't think it goes far enough, but it's a start. And at least it acknowledges from all of the main players the issues that are there and that they need to be addressed. It doesn't say too much about what to do about them, but that has to come next. And at least it gives air cover, I think, for the teams to work together on what sorts of benchmarks, what are best practices on safety, what are you seeing in terms of analysis of what things could happen with these types of systems. And I do think there are some things that generally all the leading labs do agree on.
[0:17:03] The idea of third-party auditors or evaluators coming in, obviously there's a question about which third parties, and I think we need a healthy kind of ecosystem of that that actually, you know, you could have...
[0:17:15] Good startups and companies doing that, there's enough revenue there for them. And there should be, given the amount of investment going into AI as a whole. A bit like there are auditors in the finance industry or in any of these industries. So I could see that happening. And some of the big players are moving into this, like Accenture. We're working and other leaders are working with. I think they're a little bit ahead of the game because they've acquired a good startup, a good friend of ours, Mark Warner Startup. So they have some expertise internally. But I think we need more of that. And then other things like the leading labs, maybe red teaming each other.
[0:17:44] When you've built substantial wealth through your business, it's often tied up in a single equity position. The upside is real, but so is the risk, and knowing when to act isn't always obvious. Creative planning works with business owners to build a strategy around concentrated equity, when to diversify, how to manage tax risk, and how to protect what you've spent years building. Creative planning, where wealth works together.
[0:18:20] Learn more at creativeplanning.com slash masters of scale. We all know the disappointment that comes when a grand gesture falls flat. Something that seemed impressive in the moment, but didn't deliver the results you expected. Marketing budgets too often fall into that trap, aiming for big impressions, big reach, but without the revenue to back it. LinkedIn has a name for it, bullspend.
[0:18:45] LinkedIn ads generate the highest ROAS of all major ad networks. They target by company, industry, job title, and more. Spend $250 on your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to linkedin.com. That's linkedin.com. Terms and conditions apply.
[0:19:08] Global hiring is such a headache when you're building fast, but don't let payroll compliance or onboarding slow you down. Companies like Eleven Labs and Mistral are scaling globally and effortlessly with deal. Eleven Labs quadrupled their headcount in 2025, and Mistral pays 364 employees across 15 countries, all from one system.
[0:19:31] Build your global team with Deal. Visit deal.com slash MOS and start expanding your business today. That's D-E-E-L dot com slash MOS. So let's move from that to a little bit more of the fantastical.
[0:19:51] It started actually a few years back. I think it was a Google engineer who said, I asked Gemini if it was conscious and it said yes, so therefore it's conscious, which seems like a little bit of a weak inference. But that's now gaining some motion with some of the things, especially Anthropic is doing and the way that it...
[0:20:10] Thank you.
[0:20:34] What do you think, I mean, because part of what I think maybe many people don't realize is you also come to this from a depth of expertise in brain science. And, you know, one of the things is not just a, oh, did the chatbot say it was conscious or not, is not a way to make a decision on this kind of thing. What do you think some of the ways to think about this is and also how we should be deliberately trying to shape it? Yeah, well, look, I think there's quite a few things to say about this. First of all, we haven't...
[0:20:58] Despite thousands of years of thinking about this, there isn't an agreed-upon definition of consciousness, so that's one issue. I think, and that's one issue with labelling things conscious, especially the hard problem of consciousness, you know, is it really feeling this internally like the way we seem to?
[0:21:13] Well, I don't know about you, but... Yes, exactly. But I used to argue with Daniel Dennett about this, that there's an additional thing is we're not just behaving in ways like we'd imagine conscious beings behave, but we also share the same substrate. We have the same squishy brains, and so then it's most parsimonious to think you're feeling what I'm feeling. But, of course, with an AI, that's not the case. You don't have the substrate equivalence. So I think it's going to be very hard to conclude that.
[0:21:41] I think there are dangers. Look, there's two reasons why I think we should be focusing on building intelligent tools today, not conscious entities. I think for one thing is a safety argument and technical safety and risk, which is that, you know, there's a direct trade-off between creating something, let's say, with self-volition, right, or self-awareness, even prior to consciousness. These are the prerequisites, right, to a conscious entity, and that and instruction following.
[0:22:08] Or sometimes called corrigibility. So this idea of a system actually following instructions and if it's going wrong, you telling it, you as the user or the human user telling it, stop doing that. Or switch yourself off or whatever it is that you're going to change course. So if you are training your system to be able to reject commands based on its own internal workings, then you're setting up a situation where...
[0:22:34] If you've got your constitution wrong or your rules wrong, and this is what Asimov was always warning about with his three laws, that was a very simple form of a constitution. And his books are all warnings that you're never going to get that right in every corner case, in every edge case. And I think his books show that very well. And yet we're sort of, I think, a bit hubristically thinking that that, some people thinking that, you know, sometimes called the misalignment problem is sort of under control.
[0:23:02] Just because today's systems, the good news is they seem more malleable than we might have imagined 15, 20 years ago, which is great news in terms of their steerability. But that doesn't mean we've solved the misalignment problem in total as we get closer to a GI and superintelligence and so on. And I just think it's dangerous to think one has. So that's the first thing. There's a technical problem, especially if you combine it with something like an RSI. Yes, exactly. That's the reason why we're going in this direction.
[0:23:29] Self-volition is sort of doubly dangerous.
[0:23:33] So that's one thing. Second thing is a societal, more of a societal issue, which is that I don't think we are ready yet as a society. We haven't had the debate with a wider society, philosophers, theologians, artists, humanities, not just technologists, as to, you know, and from many different cultures. Like, do we want to cross that second Rubicon? Are we ready to have the discussion about do these systems have moral standing? Do we need to worry about what are we doing with them in this box? You know, are they slaves of some sort?
[0:24:02] All these questions come up that are complex questions, and I don't think we're equipped to even begin to start dealing with those questions. So my view is we should cross the first Rubicon on its own, which is build these intelligent tools, and then actually use these intelligent tools to do more neuroscience, to go back and do more neuroscience to actually understand what the neural correlates of consciousness really are.
[0:24:27] Finally, understand that. And then, you know, that'll probably take five, ten years. And then cross the second Rubicon. Well, to have a discussion about should we cross the second Rubicon? And then if so, how? Of the conscious entity type. And how would you, because, you know, you engaged in hours of these discussions, how would you answer the folks who say, look, it's the only way we get the intelligent correlate.
[0:24:49] to be powerful enough is it has, as it were, the ephemeral accidental. Yes. It actually has this cognitive state and so on. Right. So we should talk about this a lot. We discussed this, you know, even 10 plus years ago, and I used to discuss it in my PhD back 20 plus years ago. Look, I felt, and I still feel, intelligence and consciousness are double dissociable, which means that you can have an intelligence system that's very smart on IQ and so on, kind of like the systems we have today, which I personally don't feel are at all conscious yet.
[0:25:18] Everyone probably has their own view, but I don't feel yet. And then you can have conscious entities that are not that smart. For example, my pet dog, it's lovely, and he's pretty conscious, I think. He dreams and everything, but he's not that smart. And that's provable. So I think you can have double dissociation here. Many of the higher animals, dolphins, monkeys, and so on, elephants, I think most of us would view that they have...
[0:25:44] Some, you know, they're conscious beings, and they exhibit all of that sort of behavior. And of course, they're also made of carbons and the same neurons and types of things we're made of. So I feel like it's pretty obvious they have some of those qualities, but they're not necessarily that smart. So it feels like you can dissociate these things. And I think to the extent that you can, I feel like we should. You know, we're coming up close to the end of our session and the end of the summit, sad.
[0:26:13] We think about, in terms of the whole notion of AI sovereignty, because obviously one of the things that is kind of going around in a lot of countries right now is, oh, is this really important? And one of our earliest conversations was with you trying to solve this problem. So I wanted to share some thinking, and obviously it's updated for whatever that was 13, 14 years ago. Yeah. Well, look, I think first of all, first thing to say is, it looks to me like there's going to be...
[0:26:40] Half a dozen, six to eight, if you include Chinese companies, offerings at the frontier or near the frontier.
[0:26:47] And it feels like there's going to be leapfrogging and maybe something's like three to six months ahead. But it's going to be relatively close. So there's a lot of options. It depends if you're talking about an enterprise or a country, right? So I think as an enterprise, you have a lot of options to arbitrage between these different things. Some of them are better at certain things, you know, multimodal understanding versus coding. It's very hard to be better on all vectors of importance at once.
[0:27:13] So I think that's going to happen. And then open source, which also has its risks, because how do you, as we've discussed for many, many years, how do you restrict bad actor?
[0:27:22] Using that whilst enabling, of course, as many good actors, scientists and others to use these fantastic tools. So that's an unresolved problem. But open sources, you know, maybe only lagging six months to a year behind. Obviously, we need kind of Western open source solutions. But I think at this point, if we're talking about countries as well, all companies that need their data to be private or they want to fine tune further on their own, you know, sensitive data, again, be that enterprise data or sovereign data. I think that's going to be one solution.
[0:27:51] There's also looking at what are the downstream things that can be done with these tools, big companies and big sectors that can benefit from these tools like healthcare, that you can just use this almost like commodities and then build the value on top of it. I wouldn't recommend every country, for example, trying to build a frontier model. That just doesn't make sense to me. They're going to have to find some other way of shortcutting that. Yes, and also collaborating.
[0:28:20] Working out arrangements with multinational companies like your own. I think one of the reasons I wanted to end on that question is that part of the theme of the summit is agency. And agency is not just a question of individual agency, like how do you take agency, but it's also a collective, how do we take agency.
[0:28:39] And I think part of the thing that's actually important in the AI age is that we take agency not just as any one country, but as countries together. And it's part of how we have a we are in this together and we are building for the benefit of humanity. And I think that's the kind of arc of we start with Sarah Fryer. And thank you for joining us because ending endemic has been.
[0:28:59] With such intensity that I was so grateful that you set up the agenda for me thinking about how to help in this from our very first discussion at King's Cross. So let's give Demis a hand. Thank you. Thanks again to Demis Hassabis for joining us at the 2026 Masters of Scale Summit. This conversation was recorded live on stage in San Francisco at the amazing Presidio Theater. I'm Jeff Berman. Thanks for listening.
[0:29:40] When you are ready to incorporate your business, Stripe is ready for what's next with Stripe Atlas. Start your company with incorporation, founder equity setup and startup legal documents all in one guided process. And once your company is formed, which for most people happens in just one day, you're ready to begin accepting payments on day one.
[0:30:03] More than one in four new Delaware corporations is formed with Stripe Atlas. Learn more at stripe.com slash atlas.
[0:30:34] Intuit Enterprise Suite is the AI native ERP solution that's powerful, painless, and proven. Built for businesses scaling fast without the complexity that usually comes with it. From the makers of QuickBooks, learn more at intuit.com slash ERP. Humans will never be more intelligent than AI. There's going to be two types of companies. Those are great at AI and those that went out of business because they weren't. How do we build a future?
[0:31:05] That is human-centered. I'm Rana El-Khayoubi, and on my podcast, Pioneers of AI, we answer that question and so many more. As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone. Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives. Find Pioneers of AI wherever you tune in.
[0:31:37] Masters of Scale is a Wait What original. Our executive producer is Eve Tro. Senior supervising producer is Trisha Bobita. Associate producer is Masha Makatanina. Video editor is MG Frederick. Senior talent executive is Stephanie Stern. Mixing and mastering by the audio boys themselves, Aaron Bessanelli and Brian Pugh with original music by The Remarkable.
[0:32:01] Ryan Holiday. Visit mastersofscale.com to find the transcript for this episode and to subscribe to our newsletter and be sure to subscribe to our YouTube channel.
[0:32:20] Hey, listeners, Bob here. If you listen to Rapid Response on Masters of Scale, you may be missing half the show because every Friday we release a second Rapid Response exclusively in the Rapid Response feed. The guests and topics are just as compelling and timely from Ford CEO to NASA's administrator to the lessons from the Devil Wears Prada. It takes about 10 seconds to find. Just search Rapid Response wherever you listen to podcasts.
[0:32:50] I hope to see you there.