[0:00:00] Support for the show comes from Comcast Business. With advances in AI, cybersecurity is getting more powerful, but so are attacks. In the past 12 months, Comcast Business detected 24.5 billion phishing attempts designed to exploit user behavior, steal credentials and compromise devices. AI is forcing a shift in how organizations defend themselves, demanding not only faster responses, but continuous intelligence.
[0:00:25] To survive in the current climate, leaders must build a security-first culture to equip their team with the tools they need to stay protected. With Comcast Business, connectivity, intelligence, and response work as one orchestrated system. Get better equipped to handle today's cyber threat landscape with the 2026 Comcast Business Cybersecurity Threat Report. Read it today at comcastbusiness.com slash cybersecurity. Cyber confidence starts with Comcast Business. This episode is brought to you by Google Chrome.
[0:00:57] You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses, setup required, compatibility and availability varies 18 plus. Support for the show comes from CrowdStrike. It's not a huge stretch to say that AI is the next major computing platform.
[0:01:27] Every platform shift changes the way we build software, and it changes security too. CrowdStrike is defining cybersecurity in the AI era with AI detection and response, or AIDR, a solution for seeing, monitoring, and securing AI across the enterprise. Learn more about how leading companies are turning to CrowdStrike to secure AI and secure their business at CrowdStrike.com slash decoder.
[0:01:56] Hello and welcome to Decoder. I'm Nilay Patel, Editor-in-Chief of The Verge, and Decoder is my show about big ideas and other problems. Today I'm talking with Mustafa Suleiman, the CEO of Microsoft AI.
[0:02:07] He was one of the co-founders of DeepMind, which got acquired by Google in 2014 and is now known as Google DeepMind, the center of that company's AI efforts. He was a Google VP for several years before leaving to found another AI startup called Inflection in 2022. Then earlier this year, Inflection cut a deal with Microsoft to license its core technology in a weird and kind of controversial, not quite acquisition that sent Mustafa, his co-founder, and a majority of their employees into Microsoft.
[0:02:35] and made Mustafa CEO of Microsoft AI. As CEO of Microsoft AI, Mustafa now oversees all of Microsoft's consumer AI products, including the Copilot app, Bing, and even the Edge browser and MSN, two core components of a lot of people's web experience that feel like they're radically changing in the world of AI. That's a lot, and it's a lot of decoder bait. I am continually fascinated by Microsoft's org chart and all of the little CEOs that report up to Satya Nadella.
[0:03:02] And of course, I'm also obsessed with what AI might do to the web at large. I also took the opportunity to ask Mustafa to compare and contrast working at Microsoft and Google, since he has direct experience at both. And I would say his answer is pretty revealing. It was also important to me to ask Mustafa about AI training and the data it requires. He's caught some heat for describing content on the web as, quote, freeware before, and Microsoft and its partner OpenAI are in the middle of major copyright lawsuits about training data.
[0:03:30] I'm generally curious about how AI companies are thinking about the risky and seemingly uncertain legal foundations of their work, and I wanted to know how Mustafa was thinking about it now. But before we got into all of that, I needed to ask about AGI, or Artificial General Intelligence. That's the idea that these AI systems will be able to handle tasks as well as a human, or even better in some cases. Sam Altman at OpenAI, which, again, is a huge partner with Microsoft, has said he thinks AGI is achievable on our current computing hardware,
[0:04:00] And his most recent comments at the Dealbook conference seem to lower the bar for how he defines AGI entirely, which makes it easier to argue that it will arrive sooner than we think. On top of that, there's a lot of reporting that says OpenAI can get out of its Microsoft deal when it says it's achieved AGI. So Altman has a lot of incentives to say it's happening. So I asked Mustafa straight out if he agrees with Sam Altman and if AGI is achievable on current hardware.
[0:04:26] Because if the answer is yes, maybe a bunch of org chart questions are a little secondary to what should humanity do to deal with AGI? You'll hear Mustafa be optimistic about achieving AGI in a much more diffuse time frame than Sam Altman, but you will also hear him pull away from the idea of AGI being a superintelligence, which feels like that same kind of redefinition. There's a lot going on in this episode, including a discussion of what I have started calling the DoorDash problem. You'll see what I mean.
[0:04:55] Okay, Microsoft AI CEO, Mustafa Suleiman. Here we go. Mustafa Suleiman, you are the CEO of Microsoft AI. Welcome to Decoder. Neelai, great to be with you. Yeah, I'm very excited to talk to you. I have a lot of questions for you about how Microsoft AI is structured within Microsoft, about what it means to be the CEO of Microsoft AI at a company that appears to be all about AI lately, about how you make decisions, all the Decoder stuff. But I want to start hot.
[0:05:33] I hope you're ready for this, because I realize that the answer to this question from you is, if you answer one way, this whole interview goes a different way. So I'm just going to ask right out the gate. Very recently, Sam Altman said in a Reddit AMA that he thinks we can achieve AGI on current hardware. Do you think that's possible? What does current hardware mean? Within one or two generations of what we have now, I would say.
[0:06:01] I don't think it can be done on GB200s. I do think it is going to be plausible at some point in the next two to five generations. And I don't want to say, I think it's high probability that it's two years, but I think within the next five to seven years, which is, you know, each generation takes 18 to 24 months now.
[0:06:26] So five generations could well be up to even 10 years, depending on how things go. We really are facing increasingly tough challenges with these chips. I don't think it's going to be as linear in terms of its progress, cost per dollar and so on, as we've seen in the past. But things are accelerating very fast. So I agree with that sentiment. So between two and 10 years, you think? I think that's the right...
[0:06:53] The uncertainty around this is so high that any categorical declarations just feel sort of ungrounded to me and over the top. Yeah. You and I have spoken several times in the past about a lot of things, and I want to follow up on a lot of those ideas. But it just occurs to me that if we think between two and ten years, very much in the spans of our lifetimes, there will be AGI, maybe we shouldn't be working on anything else?
[0:07:21] That seems like it will complete, that's the paradigm shift, right? We're through the singularity, now there's AGI, everything will be different on the other end of it. How do you approach that and then also think about, well, I need to launch the co-pilot app on the iPhone? Well, so that depends on your definition of AGI, right? AGI isn't the singularity. The singularity is an exponentially recursive self-improving system that very rapidly accelerates far beyond anything that might look like human intelligence.
[0:07:52] To me, AGI is a general purpose learning system that can perform well across all human level training environments. So knowledge work, by the way, that includes physical labor, the definition. So a lot of my skepticism has to do with the progress and the complexity of getting things done in robotics. But yes, I can well imagine that we have a system that
[0:08:17] can learn without a great deal of handcrafted prior prompting to perform well in a very wide range of environments. And I think that is not necessarily going to be AGI, nor does that lead to the singularity. But it means that most human knowledge work in the next five to 10 years could likely be performed by...
[0:08:43] One of the AI systems that we develop. And I think the reason why I shy away from the language around singularity, artificial superintelligence, I think they're very different things. And the challenge with AGI is that it's become so dramatized that we sort of...
[0:08:58] End up not focusing on the specific capabilities of what the system can do. And that's what I care about with respect to building AI companions, getting them to be useful to you as a human, work for you as a human, be on your side, in your corner, on your team. Like that's my motivation. And that's what I have control and influence over is to try and create systems that are accountable and useful to humans rather than pursuing the purely kind of theoretical super intelligence quest. Yeah.
[0:09:27] One of the reasons I'm particularly curious about this is the notion that all human knowledge work can be performed either with the assistance of a very capable general AI or by the AI itself. It sort of implies that we will build a new kind of AI system, one that will be able to be as creative as a human knowledge worker at the 99th percentile. And I don't see that in our systems now. The way an LLM works, they don't necessarily come up with a bunch of...
[0:09:54] Individually creative thoughts. You can prompt them to do surprising things, but that turn, I have not experienced it. Do you think that the way that the current LLMs are built and trained and deployed is a linear path to the kind of AGI you're describing, or is there another kind of thing we need to build? It's funny because two or three years ago, people would often say, well, these systems are destined to regurgitate the training data that they were trained on.
[0:10:21] And that there is some one-to-one mapping between query training data and output. And it's pretty clear today that they're actually not doing that. The interpolation of the space between multiple n-dimensional elements of their training data is in itself the creative process, right? It's picking some point in this massively complex space to produce or generate a novel form.
[0:10:47] of the response to the question that it has never seen before. We've never seen that specific answer produced in that specific way. To me, that is the beginnings of creativity. It's the kind of glimmer of a truly novel invention, which is obviously what we're trying to produce here, right? Just to level set here, intelligence is the very thing that has driven all of our progress in the world in history. This power to synthesize vast amounts of information.
[0:11:14] aggregate it into conceptual representations that help us reason more efficiently in complex space, make predictions about how the world is likely to unfold, and then take actions on the basis of those predictions. And whether you're making a table or you're playing baseball with your friend, every single one of those environments that you experience has those characteristics. And so if we can distill those sort of moments, if you like, into
[0:11:43] An algorithmic construct, then of course there is huge value there. And what I think we see in this little mini moment in the last three or four years is the glimmers that they really can be creative, that they can exert real judgment, that they can produce novel ideas, these LLMs. And your point about can they do that proactively?
[0:12:04] I think is a good one. Like, can they do that unprompted? Can they do it independently? Can they do it with very subtle or nuanced or lightweight guidance? I think that's kind of an open question. I feel very optimistic about that myself. Much of the infrastructure to sort of ensure that they can do that is kind of an engineering issue now. So stateful memory and meta reasoning about the current context of a model.
[0:12:34] are things that we know how to do in software today. We know how to introduce a second or a third system to observe the working state of an LLM in its activity and use that to then steer or re-steer a prompt that it is operating to. And if you can do that kind of asynchronous meta-reasoning, which is what the initial quote-unquote chain of thought methods seem to show in the last sort of six to 12 months or so,
[0:13:02] Then you can imagine that it could string together actions in these continuous environments and then orchestrate and coordinate with other parts of its working memory, other parts of its system, some of which are designed to do more short-term things, some to draw from long-term memory, some to be a bit more creative, some to be more adherent to the behavior policy or the safety policy that you're designing to.
[0:13:25] So it's obviously not done and dusted, but there are very, very clear signs that we're on the right path, I think. Those orchestration systems are fascinating to me because the models themselves are never going to produce the same output twice. A lot of the things we want computers to do are insanely deterministic. We definitely want them to do the same thing twice over and over and over again in a variety of situations where an AI might be really helpful.
[0:13:52] If you want to just do tax preparation, you want the AI to be very helpful and understand all the inputs, but you also want it to follow the rules 100% of the time. And it seems like connecting our more traditional logical computer systems to control the non-deterministic AI systems is a big pathway here, more so than making the AI more capable. And that feels like a new way of talking about it that I've only just recently seen.
[0:14:21] Feel like the kinds of products you need to build? Are you still focused on the capability of the model itself? It's a good framing, but let's tease apart what you mean by determinism, right? So determinism operates at layers of abstraction. So at the very lowest layer, each token is being generated non-deterministically. But as those outputs become more recognizable with respect to a behavior policy or a heuristic or a known objective, like in your case, filling out the tax form,
[0:14:49] Then that knowledge can be stored in representations which are more stable and deterministic, right? And this is exactly how humans operate today. No matter how well you might memorize something, if I ask you to do it a hundred times over, you're most likely going to have some variation in the output. We don't really store things deterministically.
[0:15:10] Co-occurring conceptual representations, which are quite fluid and abstract, which we then reproduce and fit into a schema of words and language in order for us to be able to communicate with one another. These models are actually very similar to that architecture. They can store stable information that can then be retrieved in quite deterministic ways. And like you said, integrate with existing computer systems and knowledge bases.
[0:15:40] The models are going to get way more capable. And the methods for retrieval and information access and the use of existing databases or making function calls to third-party APIs to integrate that information, those are going to advance simultaneously. And by the way, we're going to open up a third front, like a whole new front of capability, which is that now these LLMs can speak natural language. They're going to be able to go and query in real time.
[0:16:09] Other humans and other AIs. So that's like a third paradigm for retrieving information or verifying that information or accessing new knowledge or checking state on something. So that in itself is going to drive huge gains in addition to straight up model capabilities and integration with existing systems.
[0:16:31] I want to talk about the agent component of that at length, because that seems to be where so many companies are focused, including to some extent Microsoft. And it raises a million questions about how our computer systems and our networks should work. But now that I've just level set on, we think we're headed towards AGI between two and 10 years. We think we can do it with an increase in model capability, but also some novel approaches to how we use those models.
[0:16:55] I want to talk about how you're actually doing it at Microsoft. But it occurred to me from the jump, if we didn't agree on what the goals were, the structure conversation would be kind of ungrounded from reality. So those are the goals. Those are huge goals. At Microsoft AI, how are you structured to accomplish those goals? First and foremost, my organization is focused on the consumer AI part. So it is about Bing, Edge, MSN, and Copilot.
[0:17:25] So for consumer-facing products that have many, many hundreds of millions of DAO, lots of user data, and lots of direct commercial surfaces where we can deploy into production, get feedback, drive large-scale experimentation. So for me, that's mission critical because...
[0:17:46] Five years ago, we were in a state with LLMs and AI more generally, where we were still relying on the benchmarks to drive progress. Everything was happening. Evaluation was taking place in basically academic environments, albeit in commercial engineering labs. The models weren't good enough to actually put them into production and collect feedback from the real world. That has completely shifted now, where...
[0:18:10] All of the innovation is happening by optimization and hill climbing in production. So I think that's the first thing to say. Second thing to say is, obviously, our Azure business and the immense number of customers that we have that are using M365 Copilot every day provides another huge experimentation framework, which is very different to the consumer experimentation framework. And it's actually a great opportunity for me because I'm learning a lot from how many, many businesses are integrating
[0:18:38] True AI agents in their workflow today, because they have more visibility and control of their internal data, and in many cases they have tens or even hundreds of thousands of employees, they're able to introduce novel...
[0:18:52] Co-pilots into their workflows, be it for training sales agents, upskilling underperforming sales agents, providing marketing feedback. I've seen HR co-pilots. There's all kinds of customer service co-pilots happening. So that gives me a sort of window into all the different flavors of testing and pushing the limits of these AI models in third-party production environments in the enterprise context.
[0:19:20] The third arena, of course, is our collaboration with OpenAI, our great partners. I think this is going to turn out to be one of the most successful partnerships in computer history. Five years old, that partnership now, and many, many years to run. And we get models from them, we get IP, and they get compute and obviously funding, and it's obviously a huge source of support for us. And then the fourth area is that we've just spawned, since I arrived eight or nine months ago now,
[0:19:50] Our own core effort to develop these models at scale inside of Microsoft AI. And so we have some of the best AI researchers and scientists who are pushing the frontier of post-training and pre-training for our weight class, which is, you know, so we're choosing a flops match target that really suits the kind of use cases that we care about and making sure that we have absolutely world-class frontier models that can do that.
[0:20:17] Let me just unpack some of the vocabulary there. You said weight class. Does that just mean giant corporation or do you mean something more specific by weight class? Oh, sorry. Weight class is the way that we refer to comparing frontier models with one another. So obviously your flops need to be matched to your competitor model that you're evaluating yourself against. So size is really by far the overriding.
[0:20:44] predictor of capability performance in these models. So you can't compare yourself to something that's 10x larger by flops. You sort of have to treat them as weight classes or flops classes, if you like. Yeah, that makes sense to me. And then you said you want to target it towards the applications you're using, right? So you're making many models that are geared towards specific Microsoft products? That's right. So if you think about it, Copilot under the hood is a whole collection of different models, different sizes that adapt to different contexts.
[0:21:12] If you're in a speech setting, it's a different type of model. If you're on desktop, if you're actually in the native apps on Mac or on Windows, they're all slightly different models. And then there's different models for search, reasoning, safety. And I think that that is going to get even more heterogeneous as we go. And then I just want to be very clear about this. It sounds like you're developing a frontier model that can compete with Gemini, with GPT-4 or 5, whatever it is. Are you working on that as well?
[0:21:41] For the current weight class, yes. So at the GPT-4, GPT-4-0 scale. But it depends on how things turn out over the next few years because each order of magnitude increase is really a phenomenal piece of physical infrastructure. You're talking about hundreds of megawatts, soon gigawatts of capacity. There will really only be three or four labs in the world that have the resources to be able to train at that scale.
[0:22:10] By the time that we get 10 to the 27 flops for a single training run. So we won't duplicate that between us and OpenAI. OpenAI is our pre-training frontier model partner for those things. And hopefully that continues for a long time to come. So you're not going to compete with the next generation of model size, right? Right. You're going to let OpenAI do that. The reason I ask that is because Microsoft runs the data centers, right? That is a partnership. It is ongoing.
[0:22:37] But Amazon runs its own data centers and Google runs its own data centers. And it seems like there is just a core tension here, regardless of how good the partnership is, between we are going to build these data centers and restart nuclear power plants in the United States to supply power to some of these data centers. And maybe it's better to sell that to someone else versus build the models ourselves. Do you feel that tension?
[0:23:01] You know, every partnership has tension. It's healthy and natural. I mean, they're a completely different business to us. They operate independently and partnerships evolve over time, right? So, you know, back in 2019, when Satya put a billion dollars into OpenAI, I mean, it seemed pretty crazy. I didn't think it was crazy, but I think a lot of people thought it was crazy. And, you know, now that has paid off and both companies have massively benefited from the partnership. And so...
[0:23:27] You know, partnerships evolve and they have to adapt to what works at the time. So we'll see how that changes over the next few years. Do you have a backup plan if OpenAI declares AGI and walks away from the Microsoft deal? There's some credible reporting that says if they declare AGI, they can walk away from the deal. No. So, look, I mean, it's very unclear. That's why I said to you, it's very unclear exactly what the definition of AGI is. I mean, we have inside of Microsoft AI one of the strongest AI research teams in the world. If you look at the pedigree of our crew,
[0:23:57] My own co-founder, Karen Simonian, for example, led the deep learning scaling team at DeepMind for eight years, was behind many of the major breakthroughs. Nando DeFritis has just joined us. He previously ran audio-video generation at DeepMind for 10 years. So we have really an exceptional team, and we'll make sure that whatever happens, we'll be in a position to train the best models in the world.
[0:24:21] It does seem like you have some uncertainty there. You've said whatever happened several times now in the context of the OpenAI deal. Does that feel like something that you can rely on? It definitely does. Look, they're an exceptional company. They're on a tear. There aren't many companies in the world that have grown as fast as they have. And so, you know, during that kind of meteoric rise, things are going to be brittle and some of the bits and pieces are going to fall off occasionally. And that's what we've seen in the last 12 months. So that doesn't...
[0:24:50] Really changed their trajectory. They're going to be incredibly successful and we're going to do everything we can to help them be successful because they've helped make us successful. That's genuinely what's going on here. And that's naturally in any partnership, there's little tensions here and there, but fundamentally we will win together. Yeah. I want to come back to the cooperation competition dynamic there when we actually talk about products, but I want to stay focused on Microsoft AI inside of Microsoft for one more turn. You obviously started Inflection, Microsoft.
[0:25:19] You know, Microsoft, so I've sort of known Satya for a very long time. He's been sort of trying to get me to come and be part of the Microsoft crew for...
[0:25:38] For a good long while, as far back as 2017, when we first started hanging out. And, you know, I've always been inspired by his leadership particularly. And I think the company is actually in an incredibly strong position. The investments that we're making in compute, like I said, the distribution that we have with so many enterprise partners now deploying M365 Copilot. And what you can learn from that is a real game changer. You know, a lot of people are talking about these actions, right?
[0:26:05] Clearly, you want your consumer co-pilot experience to have these seamless interactions with brands, businesses, opportunities for getting stuff done, buying things, booking, planning, and so on. And so having that kind of protocol built in-house and available to the consumer side is super important. The thing I realized about where we were at with Pi and Inflection, I mean, we had really unbelievable engagement with Pi. Very high intensity DAO. The average session,
[0:26:35] of the voice interaction lasted 33 minutes a day. It was pretty remarkable. So it was well up there. But I think the challenge is the competition is going to invest for years and years and years and basically keep it free, if not reduce it to like nothing and basically make it widely available to hundreds of millions of people.
[0:26:56] And so from a consumer perspective is a very, very, very competitive landscape. And look, when Satya made me the offer to come and run all the consumer stuff here, it was just an offer that we couldn't refuse. It's sort of enabled us to pursue our long-term vision of actually creating a true.
[0:27:15] AI companion that has a lasting relationship with hundreds of millions of consumers that is really useful to you. And to me, that's going to shape the future. That is really the thing that is going to shape our long-term trajectory. So I couldn't turn that down. We have to take a short break. When we come back, we'll get into all the juicy questions about structure.
[0:27:52] But you're really stuck doing all the work by hand, chasing screenshots, validating controls, and playing catch up when audit season comes around again. Vanta works differently. Their agentic trust platform is built to scale with you, not slow you down. With over 1,400 automated tests across more than 400 integrations, Vanta collects evidence and monitors your controls year-round. Plus, you can access the Vanta agent everywhere you work, whether your team is in Vanta every day,
[0:28:21] We're living in tools like Claude or Cursor. They work around the clock from full program context, so nothing slips through the cracks. The numbers back it up. Vanta delivers 526% ROI over three years and pays for itself in three months. Over 16,000 fast-growing companies like Ramp, Rider, and Harvey save time and resources while scaling fast with Vanta. Learn more at vanta.com slash decoder.
[0:28:47] Spelled V-A-N-T-A dot com slash decoder. This is advertiser content from Comcast Business.
[0:29:20] Change the cybersecurity landscape for you and your team at the attack level. The reality is it has had a dramatic impact on everything we do. The bad guys leverage AI for malicious and bad purposes. So the only way to fight fire is with fire. And so we now use AI just as the attackers are using it to break in more quickly.
[0:29:49] We have to pivot and use it to defend more quickly at scale. Comcast business specifically detected 25.4 billion phishing attempts and 21.9 billion drive-by compromise events over one year. Talk to me a little bit about that data. Phishing is so popular because it's so easy to do. And AI has accelerated that. Every business should have.
[0:30:15] Software in place to detect phishing attacks, to train their users, and also that now extends to collaboration software, not just email, such as Zoom, such as Slack, such as Teams, because that's becoming increasingly prevalent. Everybody expects to get breached now, and that is not the attitude we use at Comcast. We are protecting our customer data extremely seriously.
[0:30:46] Support for the show comes from Granola. Have you ever left a meeting thinking that was productive? And then 10 minutes later, when it's time to take next steps, you're at a loss. Or worse, you've been heads down the entire meeting, furiously typing notes instead of actually listening. And now your brain is fried. Meetings are a mess. Granola fixes that.
[0:31:10] Granola is an AI-powered notepad that makes your meeting notes useful. It captures what happens in your meetings and turns it into clean, structured notes with decisions and action items pulled out and made easy to find. And the best part? Granola works through your device's audio, which means it integrates seamlessly into the video conferencing tools you already use. No setup, no awkward bots. It's just your normal meeting with superpowers.
[0:31:35] You get to actually listen instead of frantically typing every word and still walk away knowing exactly what was decided, who's doing what, and what comes next. Once you try it on your first meeting, it's hard to go without. You could try it totally free. Just head to granola.ai slash decoder. That's granola.ai slash decoder to get your time back. Go to granola.ai slash decoder and try it on your next call.
[0:32:08] Welcome back. I'm talking with Mustafa Suleiman, CEO of Microsoft AI, about Microsoft's somewhat unusual leadership structure. You are the CEO of Microsoft AI. Microsoft is an interesting company in that it has a CEO and then several other CEOs. Phil Sensor is the CEO of Microsoft Gaming. Ryan Roslansky is the CEO of LinkedIn. We just had Thomas Domke from GitHub on. He's the CEO of GitHub. What does it mean to you to be the CEO of Microsoft AI?
[0:32:36] Look, Microsoft's an enormous organization, a quarter of a trillion dollars in revenue, about 280,000 employees. So the logic of making single individuals accountable for our own P&L, I think is very rational. There's about 10,000 or so people in my org. We have full integration from...
[0:32:57] Training the models, building the infrastructure, running the ads platform, managing all the sales leaders, making sure that our content is high quality, getting that integrated across four platforms. So it just creates accountability. That's the logic here. And that's very much how Satya runs it. Extreme accountability. One thing that strikes me here is GitHub is a product. LinkedIn is a product. It has a beginning and an end. It's very tangible. People can understand it. Microsoft AI is the company.
[0:33:24] There's just a lot of AI at Microsoft that is infusing into all of these products. I think Satya has agreed that AI feels like a platform change. There's enormous opportunity inside of a platform change. You've obviously got your core products in Bing and Edge and MSN and all that. But when you think about the relationship to the rest of the AI efforts at Microsoft, where does the line begin and end for you?
[0:33:46] That's a good question. Right now, the company is so focused on winning on Azure OpenAI, for example, getting our models into production and getting them into the hands of hundreds of thousands, millions of businesses. I'm involved in a lot of the reviews on the enterprise side, but really I play a role as an advisor and support and our MAI internal models so far haven't really been focused on those enterprise use cases. My logic is that
[0:34:15] We have to create something that works extremely well for the consumer and really optimize for our use case. So we have vast amounts of very predictive and very useful data on the ad side, on consumer telemetry and so on. And so my focus is on building models that really work for the consumer companion. That's a product-focused structure, it sounds like. Have you reorganized Microsoft AI to be a more product-driven team? I think the business was focused on the product before. What we've done is bring...
[0:34:43] The kind of AI sensibility into the heart of each one of our products. So, you know, we have a lot of ranking. We have increasingly conversational and interactive surfaces. We're trying to bring the voice of Copilot to Bing, to MSN. You know, we want to make it a core part of the search experience so that your first thought is...
[0:35:05] Let me just ask my AI. What does my AI think about that? My AI can remember that for me. It can save it. It can organize it. And so making sure that it shows up in deeply integrated ways that really support the surface rather than sort of just are an adjacent kind of add-on or an afterthought, that's the craft that we're kind of working towards.
[0:35:23] You are a unique person to have on the show because you also founded DeepMind and you worked at Google. We've had Dennis, the CEO of DeepMind on the show before. Google is a challenging place to work at. He is the CEO of Google DeepMind. Google doesn't have CEOs like the way that Microsoft has CEOs in that particular way. Can you compare and contrast these two companies? You worked at one huge company. You were at a startup for a minute. Now you work at the other huge company.
[0:35:49] They are very different culturally and structurally. Do you think Microsoft has advantages over Google's approach? I do. I think that at Microsoft, there is a lot of discipline around revenue and P&L. And I think that is a very healthy attitude because it really focuses the mind on what is a consumer going to find truly valuable and be prepared to pay for.
[0:36:14] Second, I think there's a real long-term thinking in terms of like, where does this platform shift take us? And what does the five to 10 year horizon look like? So there's a kind of planning attitude, which at least during my time at Google felt more instinctive. I mean, their instincts are really good. It's a very incredibly creative company and many times they've made long-term bets.
[0:36:38] But it was kind of reactive, instinctively reactive, whereas I think there's a lot more thought in the kind of scenario planning and really kind of thorough deliberation. I would say the third thing I guess I'd say is like, you know, Friday's senior leadership team meeting with Satya is just like a phenomenal experience. I mean, it runs basically 8.30 till 2.30 in the office in Redmond, and everyone's there, all the leaders.
[0:37:05] And, you know, we sort of review all the big businesses or all the big strategic initiatives in detail. And the SLT cross-functionally is in the weeds. And that is a pretty remarkable thing because they're sort of reviewing these things week after week, like security, huge priority, genuinely like a number one focus for the company, AI and infrastructure, then reviewing all of the businesses.
[0:37:29] It's very cool to see that other leaders ask the questions or probe. And I kind of see the world through their eyes, which is sort of slightly different. You know, so although there's sort of lots of CEOs, everyone's looking at everyone else's businesses and giving advice and feedback. So it's quite an intellectually diverse group. And then the other thing I would say is that because there's obviously an enterprise DNA to the company, there's a real focus on what does the customer want.
[0:37:57] But Google is like, what would be a cool technology for us to build? Whereas Microsoft's like, how would this actually help the customer? And what are they asking for? I think both of those strategies have their own benefits. But if you swing one way or the other to an extreme way, there's sort of real problems. And so I've certainly enjoyed learning from the fact that Microsoft is very much like, what does the consumer want? What does the consumer want? What does the customer need, et cetera? You mentioned security at Microsoft.
[0:38:27] The renewed focus on security is because there were a bunch of lapses earlier this year, right? This has been an issue. Can you just explain, you have an outsider perspective. You're building a lot of products that might go out into the world and do things for people. You're building a lot of products that require a lot of customer data to be maximally useful. As you go into these meetings and you talk about Microsoft's renewed effort on security because there were some problems in the past, how that's affected your...
[0:38:55] Yeah, I mean, I definitely think that the company culture is security first. But that's now. I just want to be very clear to the audience. Sachi has started saying that now, but it's because there were these enormous security lapses just in the past year. That's true. That is very true. I'm just saying since I've got there.
[0:39:16] So, culturally...
[0:39:36] As far as I've known it, it is the central priority, which has been good for me too. I mean, for my businesses are also like mission critical that we preserve consumer trust, right? And trust means that people just expect us to be able to store and manage and use their data in ways that singularly benefit them and are in their interests. That is a central part of the culture. And you're right, maybe that's a refocusing.
[0:40:02] That stuff hasn't paid off yet.
[0:40:33] That's true. That's true. The AI stuff hasn't paid off yet. I think it's fair to say. But remember, I spend over half my time focused on the Bing business. And Bing business is doing incredibly well. I mean, we grew 18% last quarter, and we actually took gains from Google, which means we're growing faster than Google.
[0:40:52] Which makes everybody feel happy. And that's kind of the main goal. So the product is deeply integrated AI. There's generative search results in the context of your search experience. There's increasing conversational experiences there. And just the general quality that we've been able to level up with LLMs has been very impressive. And I think that's translating into revenue improvements as well. So in that sense...
[0:41:18] There's so many different ways that AI is showing up. I think you've got to think of it more as like a new...
[0:41:46] High bar in terms of the table stakes of the features that we offer. The part where the LLMs are integrated into a bunch of products like Bing or Edge, are they driving more revenue from those products or are they just taking share away from Google? The way I think about it is like it's improving the quality of ads that we show, improving the relevance of those ads. And so it's making the experience more useful for the consumer.
[0:42:15] And that is obviously the overall pie is growing, which is that's the nature of the growth, because obviously Google's growing too. So the entire market is continuing to grow. The point is that we're growing faster than Google for this quarter. And I think that's a huge achievement. The team's done an amazing job. And it's not about me, by the way. That's a product of them. Many years of them investing in quality and relevance and just generally doing a great job. Famously, when Bing with...
[0:42:43] Co-pilot was introduced. I sat down with Satya. He said, I want to make Google dance. And then I went and asked Sundar about that. He said, he just gave you that quote so that people would run that quote. And that was kind of his response. Sundar was very calm in that way. You came into it after that whole situation, and now you run the products that are directly competitive with Google. Do you think that you are growing faster than Google in some places? Do you think that you are actually posing a competitive threat to Google in either Bing with Search or Edge with Chrome?
[0:43:12] I think one of the things that I've realized as I've got a bit more experience than mature over the years is that you have to be very humble about how the landscape changes. On the one hand, this really is an opportunity to re-litigate some of the battles of the past. The chips are going to fall in a completely different configuration in the next two or three years. At the same time, that's a very challenging thing to do. Habits die hard and so on.
[0:43:41] Our goal with this completely new interface is to make it 10 times easier for people to access information, advice, support in a truly conversational way and to do things that our competitors won't do that are truly useful to everyday consumers. And I think that's actually going to be one of the differentiators. It's like, what is the personality and the tone and the emotional intelligence of an AI companion? Because remember, most people do love information.
[0:44:11] And they like getting accurate and reliable information, but that's going to be commoditized. All of these models are going to have that. And despite what we like to think in Silicon Valley, surrounded as we are by nerds and information obsessives who read all the content that you can get access to, most people really connect to brands, really connect to ideas in a social way. They connect to it because it is...
[0:44:39] Friendly, kind, supportive, emotionally reassuring. And I think that's going to form a big part of the way these models actually turn out to be successful in a few years' time. I need to ask you the core decoder question, but then I want to come back to the idea that the information will be commoditized. You've described a lot of change, right? You were at one company, you were at a startup, you're at Microsoft, you're learning how Microsoft works. You have big decisions to make about how to deploy these products. What is your framework for making decisions? How do you make them?
[0:45:09] is in a six week rhythm. So I have a six week cycle and then we have a one week meetup for reflection, retrospectives, planning, brainstorming, being in person. The reality post COVID is that people work from all kinds of places and they like that flexibility. So my rhythm is to keep people in person two to three days a week and then really come together for that seventh week of retrospectives.
[0:45:36] So my general framework is to try to be as in the weeds as possible. Like I really spend a lot of time in our tools, tracking telemetry, hearing feedback from people, and then creating this very tight operating rhythm where in the context of a cycle, six, seven week process, we have a very falsifiable mission. Every single team can express in a sentence exactly what it is they're going to deliver.
[0:46:03] And it'll be very falsifiable at the end of that, so we'll know. And then when we observe whether or not that happened, that's a moment for retrospective and reflection. And I like to really write. I'm a writer. I think by writing. And I like to broadcast my writing. So every week, I write a newsletter to the team that is just like a reflection on what I've seen, what I've learned, what's changing, what's important.
[0:46:28] And then I document that over time and use that to kind of track and like steer where we're going. That's kind of the basics of how I practically implement my process for reflection and stuff like that. But in terms of the framework, so one thing is to really tune into the fact that no matter what product you invent, no matter how clever your business model is, we're all surfing these exponential waves.
[0:46:55] And the goal is to predict which capabilities fall out of the next large training model. And if you overthink that and assume that actually there's some genius, you know, new ecosystem incentive or new business model or new UI style, all of that is super important. But if you think that it's really only going to be that or that that's going to be the overwhelming driver, I think that's a mistake.
[0:47:25] trying to build these models. You know, remember at DeepMind 2014 to 2020, I was banging my head against the table trying to ship machine learning models, ship CNNs in the early days, find classifiers, do re-ranking, try and predict what to watch next on YouTube, trying to do activity classification on your wearables, trying to ship crash detection algorithms inside of Waymo. Like every single...
[0:47:49] Applied practical machine learning objective I had explored there. And now we actually have the tools to be able to do those things and do them really, really well. They're really working. So we're basically surfing those tides. The goal is to really nail those waves because we already have models which are giving us more than we can extract and apply into products.
[0:48:15] That's quite a profound state that we're in. We actually haven't yet completely extracted all the gains from the current class of frontier language models. Every week, there's still some new capability or some new trick or people have crafted or sculpted them in post-training in a new way. And I think that that is going to continue for the next few years to come, many years to come, in fact. So in terms of the decision-making framework,
[0:48:42] The goal is to be very focused on model development and scaling those models, getting them to be practical and useful, really aligning them and getting them to behave in the way that you need for your product. Well, let me ask you about that because model development and we need to get more of the models we have now. There's a little bit of attention there, right? There's a notion that the scaling laws are going to run out, that the next class of models is not significantly outperforming the models we have now. And I think you can track that in just the way we're talking about the products.
[0:49:12] A couple of years ago, it was AI is an existential risk. We have to stop it so we can make sure it's aligned before we kill everyone. And now we're kind of at like, well, we got to get more out of the models we have now and actually ship some products and make some money, hopefully, and figure out what it's all good for and how to best use it. Because it doesn't seem like the next generation models is actually running away as fast as we think they might. Is that your view, that the sort of frontier models are not getting better as fast as we thought they might? And so we have to get more out of what we have?
[0:49:42] No, I don't think that's true. I think that they're going to continue to deliver the same seismic gains that we've seen in the previous generations. Remember that they are more costly and they're more fragile and they take longer to train this time around. So we're not going to see them happen in the same sort of 12 to 18 month timeframe. It's going to shift to 18 to 24 months and then a bit longer. But I don't see any sort of sign that there's this like structural slowdown.
[0:50:10] I kind of see the opposite is that there's huge gains to extract from where we are today, but it's very clear to me that there's huge gains to extract from the next two orders of magnitude of training as well. We have to take another quick break. We'll be back in a minute. This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web.
[0:50:45] Like restoring a vintage motorcycle from a 50-page restoration block or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses, setup required, compatibility and availability varies 18 plus. Support for the show comes from Framer. The future of website building is seamlessly blending all the tools we have available. And you could use a platform that has it all and stays useful even after you launch.
[0:51:15] That's what Framer is. Framer is the AI website builder that helps creators, teams, and businesses ship production ready sites faster than ever while getting every detail right. You're able to prompt, inspect, edit, and publish in one place at a whole new pace. On Framer, agents bring speed and scale while people bring taste, judgment, and control. And that all takes place in the same workspace where the real site is designed, managed, reviewed, and published.
[0:51:43] Everyone's ideas land on the canvas, stay editable, and can be published when the team is ready. Thousands of businesses from early stage startups to Fortune 500s are choosing to build their websites in Framer, where changes take minutes instead of days. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com slash decoder for 30% off a Framer Pro annual plan. That's framer.com slash decoder.
[0:52:12] Rules and restrictions may apply. Support for the show comes from OutShift, Cisco's incubation engine for frontier technology. You can find multi-agent AI in every enterprise roadmap right now. But here's the problem everyone's facing. Agents can pass messages, but can't think together. These systems silently underperform through cognitive failures, misreadings, unverified claims, false consensus, and conventional monitoring can't detect them.
[0:52:44] OutShift by Cisco is building the fix with their new comprehensive resource, the Internet of Cognition. It's an open source guide that's a foundation for building multi-agent systems from design to production. The goal is to give agents shared context, shared memory, and guardrails, all to drive the results we actually expect from AI. You can lean on this resource to understand agentic AI's evolution and how it's shifted from isolated agents to orchestrated superintelligence.
[0:53:12] You can gain insights for how to future-proof your business by tackling complex challenges with multi-agent systems. You can read the paper, experience a demo, grab the code from outshift.com. That's O-U-T-S-H-I-F-T dot com. Welcome back. I'm talking with Microsoft AI CEO Mustafa Suleiman. Right before the break, we were talking about extracting gains from current AI models.
[0:53:42] Mustafa mentioned the idea of commodifying information, and I definitely wanted to dig into that a little bit more. I want to make sure we talk about the thing you mentioned, the commodification of information, and then I want to make sure we talk about agents very quickly to bring this all around to the products to come. The commodification of information is, I think, the big story of the internet that we have today, the platform internet, for lack of a better word. You go to Google, you ask it a question, now it might spit out an AI-generated answer. You go to MSN, you ask it...
[0:54:10] And it might algorithmically or with AI sort a bunch of news and summarize that news for you. Everyone's headed in this way. We've been talking about this for a long time. To train the next generation of models, we need even more information. You've gotten yourself into some trouble, I would say, saying that the information on the internet is quote-unquote freeware and the expectations that you can use it to train. There are a lot of lawsuits, including several pointed at Microsoft. Where do you think that next body of information comes from?
[0:54:39] Before we sort out the copyright implications of using all this stuff to train. So one way of thinking about it is that the more computation you have, the more time these models can spend attending to the various relational components of all that training data. So think of flops as a way to spend understanding time.
[0:55:02] Learning the relationship between all these various sort of training inputs. So first of all, you can still gain more from just having more computation to learn over all of the existing data. Second thing is we learn a vast amount from interaction data. Users tell us implicitly and explicitly how they feel about an output. Is it high quality? Is it used? Is it ignored?
[0:55:26] Third is we're generating vast amounts of synthetic data. That synthetic data is increasingly high quality. When you ask an AI teacher or a rater to compare two or three different examples of the synthetically generated output and the human written output, it's extremely difficult to detect those precise nuances. So the synthetic data is increasingly high quality and used in a whole bunch of different settings.
[0:55:51] Then the fourth is I can imagine AIs talking to other AIs, asking for feedback, AIs that have been primed for different areas of expertise or different styles, prompted in different ways. You can imagine those interactions producing valuable new knowledge, either because they're grounded in different sources or just because of their stylistic output, they're actually producing novel interactions.
[0:56:17] So I don't necessarily see data being the limitation anytime soon. I think that there's still huge benefits to come from scale for a foreseeable future. So that's all new data, right? You're going to get a bunch of interaction data. Maybe the synthetic data will be high enough quality to train the next generation models. But the original data sets were the web, right? It was a bunch of web content. It was the entire internet. Maybe it was...
[0:56:39] to video platforms to some extent from some of the model providers. The quote I have from you in June, I think you were speaking to Andrew Ostorkin. Here's the quote. You said, I think that with respect to content that's already on the open web, the social contract of that content since the 90s is that it's fair use. Anyone can copy it, recreate with it, reproduce with it. That has been, quote, freeware, if you like. That's the understanding. I'm curious, right? You said that. That was the understanding for search.
[0:57:06] Right. And there was a lot of litigation around search and Google image search and Google books that led there. Do you think that that is still stable enough for you in the age of AI with all of the lawsuits outstanding? I mean, what I was describing in that setting was the way that the world had perceived things up to that point. And I think that my take is that just as anyone can read the news and content on the web to increase their knowledge.
[0:57:30] Under fair use, so can an AI. Because an AI is basically a tool that will help humans to learn from publicly available material, and all the material that has been used for generating or training our models has been scraped from publicly available material. But publicly available and copyrighted are very different things on the internet, right? Publicly available does not mean free of copyright restriction.
[0:57:57] Oh, yeah. I mean, look, obviously, we respect the content providers, right? So that's not, you know, that's an important distinction. But, you know, I guess what I'm trying to say is from our perspective, there's certain types of content, like, for example, in our Copilot Daily or MSN Daily, that is paywall publisher content that we pay for directly.
[0:58:18] is what MSN has been doing since the beginning of time. It's what we've decided to do with Copilot Daily for high quality content because we want those publishers to create an information ecosystem that really works for everybody. And I just think this is one of those things where...
[0:58:32] Things will play themselves out in the courts. At any time when there's a new piece of technology, it changes the social contract as it is at the moment. There's clearly a gray area in terms of what constitutes fair use and whether an AI can have the same fair use as a human. And we'll just have to play it out over the next few years. And I think we'll have some perspective over that in the next few years as things land. One of the reasons that I asked that as directly as I'm asking it is,
[0:59:01] The cost of training the next generation of models is very, very high. But that cost is built on a foundation of, well, the training data is free. And if a couple of court decisions go a couple of ways, the cost of the training data might skyrocket, right? If a court says it's not fair use to use the New York Times as content or it's not fair use to use these books from these authors, suddenly you might have to pay a lot of money for that data as well. Do you think that that is something that the business can sustain?
[0:59:32] Yeah, I mean, we already pay for books at huge scale, right? So, you know, and if it's a copyrighted book, that's not, we're not sort of, you know, taking that, hoovering that up from the internet. Well, Microsoft might not be, but there's a very big lawsuit from a bunch of publishers who say that, for example, OpenAI is, right? And that's the model that you are reliant on. So it just seems like there's a, maybe legally.
[0:59:56] We'll see what the answer is. But economically, there's also a lot of uncertainty here, right? Because of the costs of the underlying data. Yeah, that's true. And I think our focus has been to make sure that we pay for the really high quality copyrighted material from publishers, both news publishers and book publishers and others. And I think that's going to continue. And that's definitely what we're committed to. Who decides what's high quality? That's actually an interesting question. Quality is...
[1:00:24] Actually, something that we really can measure, we want to make sure that the content, especially from a nonfiction perspective, so we're particularly interested in academic journals, academic textbooks, we can verify the source and citations for that knowledge. And that is one of the big measures that we consider to be high quality. But, you know, the visual artists, the nonfiction artists, visual effects artists, the movie industry.
[1:00:53] Right. They're saying, hey, we're going to get pushed out of work because we are not compensated for any of the work that's going into these models. Like, how do you think this plays out for that? Because, again, I agree that the law here is deeply uncertain. These cases are going to play out. But I'm looking back at what you're describing as the social contract of the web. And what I see is, oh, Google litigated a million of these lawsuits. We didn't just all wake up one day and decide this is how it's going to work. Google went to court 15 times.
[1:01:20] And they were a bunch of kids who had slides in the office. And they had just made Google. And they were positioned as a company in a moment. And they had a product that was so obviously useful in so many different ways that they kind of got away with it. And I don't know that the tech industry is in that position anymore. I don't know that the products are so obviously useful the way that putting Google on the internet for the first time ever was so obviously useful. And I certainly don't know that the feelings from...
[1:01:50] Particularly one set of creators are as mixed to positive as they were for Google back in the 90s and early 2000s. And that to me feels like, I mean, you're on the board of The Economist, right? That to me feels like the people that make the work are having the most mixed emotions of all. Because yes, I think a lot of us can see the value of the products, but we also see the value transfer to the big tech companies, not the upstarts, not the cute kids with the slides in the office.
[1:02:18] I think that this is going to be more useful and valuable than search. I think search is completely broken. And I think it's a total pain in the butt. And we've just kind of got used to using a terrible experience. Typing a query, just sort of think about what a query is. We had to invent the word query effectively to describe this really weird, restricted way that you express a sentence or a question.
[1:02:46] Into a search engine because of the weakness of the search engine. And then you get 10 blue links. And then those things are vaguely related to what you're looking for. You click on one and then you have to go and refine your query. I mean, it is a painful and slow experience. I think that if we can get this right, if we can really reduce hallucinations to the minimus amount, I think we've already demonstrated that they don't have to be toxic and biased and offensive and all the rest of it.
[1:03:10] It's pretty good. It's not perfect, but it's getting much, much better. And I think it's only going to get better with more stylistic control. Then these conversational interactions are going to become the future of the web. It's quite simple. I mean, this is the next browser. This is the next search engine. It is going to be 100 times easier for me to just turn by voice to my co-pilot and say, hey, co-pilot, what's the answer to this? I already do it five times a day. It is my go-to. It's made my bottom right-hand app in my iPhone.
[1:03:39] My thumb instantly goes to it. I use the power button to open, you know, my favorite app like I did with Pi. I mean, it is clearly...
[1:03:47] The future, that conversational interaction. So to me, the utility is phenomenal. And I think that is going to weigh into the cases as they make themselves, make their way through the court. So that leads us, I think, directly to agents, right? Where you're just going to ask some app on your phone or some part of the operating system on your computer to do something and it will go off and do it. It will bring you the information back or it will accomplish some task on your behalf and bring you the result. You and I have talked about this before in various ways.
[1:04:16] That commodifies a lot of the service providers themselves, right? You say, I want a sandwich, and now I don't know if it's DoorDash or Uber Eats or Seamless or whoever is going to bring me the sandwich. My AI is going to go out and talk to them. That implies that they will allow that to happen, that they will allow the agents to use their services.
[1:04:36] In the best case, they would provide APIs for you to do it. In the worst case, they let people click around on their websites, which is a thing that we've seen other companies do. In the medium case, they develop some sort of AI-to-AI conversation. Not quite an API. We're just literally clicking around on a website and pretending to be human, but our AIs are going to have some conversation. What is the incentive for those companies to build all of those systems or allow that to happen to become disintermediated in that way?
[1:05:04] I mean, people often ask that when there's a new technological or scientific revolution and we're going through it and it's causing a massive amount of disruption. And people are curious. It's like, well, why would someone do that in 10 years? And then if you look back for centuries, it's always the case that if it is useful, it gets cheaper and easier to use. It proliferates. It becomes the default.
[1:05:30] And then the next revolution comes along and completely turns everything on its head. My bet is that every browser and every search engine and every app is going to get represented by some kind of conversational interface, some kind of generative interface. The UI that you experience is going to be automatically produced by an LLM in three years, in five years, and that is going to be the default.
[1:06:00] And they'll be representing both the brands, businesses, influencers, celebrities, academics, activists, organizations, just as each one of those stakeholders in society, you know, ended up getting a podcast, you know, getting a website, writing a blog, you know, maybe building an app, like using the telephone back in the day, you know.
[1:06:25] The technological revolution produces a new interface which completely shuffles the way that things are distributed. And some organizations adapt really fast and they jump on board and it kind of transforms their businesses and their organizations and some don't. And so there will be a kind of an adjustment. We'll look back by 2030 and be like, oh, that really was the kind of moment when there was this true inflection point because...
[1:06:54] These conversational AIs really are the primary way that we have these interactions. And so you're absolutely right. A brand and a business is going to use that AI to talk to your personal companion AI. Because I don't really like doing that kind of shopping. And some people do, and they'll do that kind of browsing experience direct to consumer. And many people don't. And it's actually super frustrating and hard and slow. And so increasingly, you'll come to...
[1:07:19] And that'll just be a much more efficient protocol because AIs can talk to AIs in super real time. And by the way,
[1:07:35] Let's not kind of fool ourselves. We do already have this on the open web today, right? We have behind the scenes this real-time negotiation between buyers and sellers of ad space or between search ranking algorithms. So there's already that kind of marketplace of AIs. It's just not explicitly manifested in language. It's operating in vector space. That's the part I'm really curious about. The idea that natural language...
[1:07:58] Is the platform shift? Is the paradigm shift? I think it's very powerful. I don't think it has been expressed very clearly, but the notion that actually the next form of computing is inherently based in natural language, that I'm just going to talk to the computer, it's going to go off and do some stuff because it understands me, is very powerful. I buy it. How that actually plays out on the back end is the part that, to me, still feels up in the air. I'm going to ask for a sandwich. That necessitates there to be companies that are in the business of bringing me a sandwich.
[1:08:27] And how they talk to my AI and how they stay in business seems very challenging. Right now, those companies, they're in business because they can sell ad space on my phone to the other companies that actually make the sandwiches. They have upsells. There's a million different ways these companies make money. If they abstract themselves down to their AI talks to my AI and says, okay, here's a sandwich, and I take away all of their other revenue opportunities.
[1:08:54] I'm not sure that that ecosystem can stay relevant or even alive.
[1:09:21] Sandwich-making AI shows up in the context of your personal AI context in maybe a sponsored way too. So, you know, there'll still be that core framework of keyword bidding, paying for presence, paying for awareness. There's still going to be ranking. That is still going to be relevant to some extent. It's just that...
[1:09:41] You are going to be represented by a personal AI companion that is going to be that interlocutor and negotiator. And those two AIs are going to have an exchange in natural language, which is what we would want. We would want to be able to go back and audit that negotiation and check where the error came from and see if it really was a good price in hindsight and all the rest of it. As you start to build these products in Copilot, have you had these negotiations with these other providers? Have they started to say what they would want? We've talked.
[1:10:10] I wouldn't describe them as negotiations. I think lots of brands and businesses are building their own AIs. Today, they're characterized as customer support AIs that pop up in your website. But tomorrow, in two or three years' time, they're going to be fully animated, conversational, rich, clever, smart, digital co-pilots that live in social media. They're going to appear on TikTok.
[1:10:36] They're going to be part of the cultural space. So I think that there's not so much a negotiation to happen there. I think it's just this inevitable tide of the arrival of these co-pilot agents. You run MSN. You obviously have peers at Microsoft who run other kinds of social networks, other kinds of information products. I see a flood of AI slop choking out some of these networks. I've searched Facebook for Spaghetti Jesus.
[1:11:04] I have seen the other side of the singularity, my friend. We already had one conversation about determining high quality, and the answer is sort of I know it when I see it. But if you run these networks and you're faced with a bunch of agent AIs who are talking or AI influencers on TikTok, can you label that stuff effectively? Can you make it so that users can only see things from other people?
[1:11:28] You certainly can. It would require a shift in the identity management system of the platform, which has a lot of pros and cons. You can certainly tell which accounts come from a human and which are AI generated. To some extent, I think there can be digital watermarking and signing for verified human content or verified AI content from a specific source. And then to some extent, there can be detection of synthetically generated content because that does have a specific signature.
[1:11:58] Long term, I don't think that's a defense. I think it is going to be perfectly photorealistic and very high quality. And it's going to be a game of cat and mouse, just as it has been in security and privacy and information for decades and centuries, actually. So I expect that to continue. It is going to get harder and more nuanced, but this is the sort of natural trajectory of things. Do the people who run LinkedIn or do your folks at MSN say, this is a problem that we can't stop?
[1:12:27] We need to make sure we don't have too much AI content here because right now it's not good enough. I can see it a mile away. I see those bullet points. I think someone made this with ChatGPT. I don't even want to read it. Is that a problem that you're facing right now? Is there a problem to come? I think it's a problem to come. But the thing I would say is we humans are behaviorists. We observe the output of other humans and we evaluate and we decipher trust based on
[1:12:56] The quality of information with respect to our own assessment. Is it accurate? Is it reliable? Is that person consistently doing what they said they would do? And so we can observe their actions and rather than sort of introspecting, why did this happen? Why did this neural network generate this output? Why did this person come to this conclusion? And that's actually an important distinction because I think a lot of...
[1:13:20] Purists are kind of fixated on the causal explanation for why an output has been produced, rather than the kind of more observational assessment of like, was it useful? Does it do the same thing over and over again? That's what drives trust.
[1:13:34] So I do think that poor quality content will be detectable in that sense, or sort of AI content that is deliberately misrepresentative or misinforming will be detectable because I think we'll have better models and we're getting them all the time for ranking down and deprioritizing certain types of content. One of the things that I've been thinking about a lot throughout this conversation, you're in charge of Microsoft's consumer business. Microsoft's consumer business, I think famously right now in 2024,
[1:14:04] is built around not making the iPhone. That's the thing that Microsoft missed in Consumer famously. It has nothing to do with you, but the iPhone happened. Microsoft pivoted to being an enterprise business. It's now slowly coming back because I think the company rightfully sees a platform shift, a paradigm shift in computing. Apple still exists. The iPhone still exists. You said, I've got this icon on my iPhone. It made it onto the home screen and it's in this preferred position, the position everybody wants in the bottom corner. Apple has a pretty big distribution advantage here. They have a deal with...
[1:14:34] Can you make products so good that you overcome the iPhone's distribution advantage? They're bundling into the operating system? It's a great question. I mean, distribution and defaults really matter. And so from our perspective, obviously we're focused on distribution deals. But fundamentally, we're also focused on building something that's truly differentiated. And to me, that AI companion really is going to be differentiated. The tone and the style of that interaction matters.
[1:15:02] Thank you so much for having me.
[1:15:17] Having your companion reach out, be supportive is a differentiator. And that's how a lot of people make their decisions and it's how a lot of people seek support. So I think that's a really big opportunity to spread a good vibe and spread kindness. And I think most apps and most kind of product thinking in Silicon Valley doesn't really engage with that kind of emotional plane in the way that the advertising industry in New York.
[1:15:43] Just thinks of that as second nature, for example. And I think that's a big shift that we're making as an industry. And it's certainly one of the areas that we're going to be focused on in Copilot. And we have to build something that is really beautiful and differentiated. It's going to be a real challenge. It's not easy. Do you think this is an opportunity to build consumer hardware again? Not a phone, but whatever comes after the phone? I am open-minded about that. The voice-first experiences are going to be transformational and they do represent a new platform.
[1:16:11] And I think we're increasingly tired of our screens. I'm frankly sick of looking at a grid of multicolored icons on my iPhone. I think many people are. You sort of feel trapped down this sort of, you know, structured, fixed kind of unit of tapping these things. And I don't know. I think people are looking for more opportunities to go hands-free and to be away from keyboards and screens. Leave your phone at home.
[1:16:38] So I think there's a lot of opportunity there. I'm very, very interested in that space. Have you played with the products that are out now, the Humains, the Rabbits? I have. I've played with all of them, yeah. And I've actually just come back from an extended trip to China where I visited all the big manufacturing companies, got to know those guys. Very impressive what they're doing out there, moving at light speed. It's very, very interesting to see. Should we expect hardware from you? No, not anytime soon. But I think we're a huge company. We're keeping an open mind about lots of things.
[1:17:09] And, you know, we'll see how things go. Very good. Well, Mustafa, we're going to have to have you back very soon. I have a million questions here I didn't get a chance to ask you. This was great. Thank you so much for being on the show. This has been a lot of fun. Thanks, Eli. I really appreciate it.
[1:17:21] I'd like to thank Mustafa for taking the time to join me on Decoder. We spoke over the Thanksgiving break, so I appreciate it. I'd also like to thank you for listening. I hope you enjoyed it. If you'd like to let us know what you thought about this episode or really anything else, drop us a line. You can email us at decoderattheverge.com. We really do read all the emails. And for our last episode of the year, I'll be answering some listener questions. So now's the best time to send some in if you want to ask anything. You can also hit me up directly on BlueSky or on the threads. I'm at reckless1280 on threads and at reckless.bsky.social on BlueSky.
[1:17:51] We also have a TikTok. Check it out. It's at DecoderPod. It's a lot of fun. If you like Decoder, please share it with your friends and subscribe wherever you get your podcasts. Decoder is a production of The Verge and part of the Vox Media Podcast Network. Our producers are Kate Cox and Nick Statt. This episode was edited by Travis Larchuk and Callie Wright. Our supervising producer is Liam James. The Decoder music is by Breakmaster Cylinder. We'll see you next time. The secret society of semi-homemade? Listen up. Sam's Club has pre-made dishes for a Thanksgiving feast that's ready in two hours.
[1:18:25] All while tasting suspiciously homemade. Come join us. Sam's Club. Individual oven times may vary. Your sausage McMuffin with egg didn't change. Your receipt did. The sausage McMuffin with egg extra value meal includes a hash brown and a small coffee for just $5. Only at McDonald's for a limited time. Price and participation may vary. Promotion pricing may be lower than meal pricing.