[0:01:55] Ladies and gentlemen, please be seated. Our program is about to begin. Ladies and gentlemen, please be seated. Our program is about to begin. Please welcome Bennett Richardson, General Manager and Global Head of Public Affairs for Semaphore. Good afternoon. Thank you all so much for being here. Come on, grab a seat. We still got a couple down front here.
[0:07:42] But thank you all so much for being here, seriously. I'm Bennett Richardson, Semaphore's general manager and global head of public affairs. Wonderful to see you all here in person. Thanks to all of the AI agents on the live stream. We appreciate you tuning in. But for our first annual Semaphore Tech Summit, I think most of you are very familiar, but Semaphore is a young, ambitious global news brand. We launched in October 2022, so just two and a half years old. But we're already spanning the globe.
[0:08:12] We have a simple vision, which is to build the most independent, transparent, and intelligent
[0:08:40] Global News Service for leaders like yourselves across the world. And at a time like today where technology's impact on society and the stakes for that have never been higher, we're really excited just to have such timely conversations. And I think especially this year, seeing for both consumers and for companies, advancements in AI moving from that early hype into real.
[0:09:04] Real-world application and figuring out those implications as well, but really unlocking that full potential, looking at the restraints on infrastructure and resources, regulation, responsible deployment. We're going to cover all of that in some fascinating newsy interviews.
[0:09:19] We'll dive into conversations around AI, not just as a powerful technology, but also as a force reshaping the ways that we live and the ways that we work. In a few minutes, you'll hear from a series of top AI leaders on interdisciplinary voices on what's happening next. We're really grateful to our partners for helping us bring today together. A special thank you to our founding partner, the Singapore Economic Development Board, our presenting partner, Booking.com.
[0:09:47] And our supporting partner, Stanford University's human-centered artificial intelligence institute for all of their support of the tech summit and of our live journalism. So thank you all for joining us and thrilled to have you with us as we start to explore AI's next chapter. With that, please join me in welcoming our star of the show, Semaphore tech editor, Reid Albergati. He's gonna kick things off.
[0:10:17] Hi, thank you all for being here. This is Semaphore Tech's first live event and Semaphore's first live event on the West Coast. The goal here is to take the journalism and the scoops that we do in the Semaphore Tech newsletter to life. And shameless plug, I hope you'll all, if you haven't already, go and subscribe to our newsletter. And what better topic than AI? It's something that's really changing the way we live, the way we work, what we pressure our kids to major in in college.
[0:10:47] And this is a really great group. I mean, in one way or another, everyone who's going to talk today has influenced the course of AI history. And we're going to talk to them today about the present and the future of AI. And my hope is that you'll come away with some new insights and new questions, new conversations. So with that, let's get started. Thank you. Round of applause for Joe. All right.
[0:11:23] So Jill Chase, we didn't get the AI voice of God. I don't know what happened. It must have been a hallucination. But Jill Chase with a capital G. So you're the sort of growth stage investors for Google. They're the biggest LP. And with that, you get this amazing ability to go in and talk to people there. We saw all the announcements yesterday at Google I.O. There's just incredible amounts and
[0:11:53] Capabilities that they've announced. What is that like? What insights have you been able to gain from those conversations? Thank you so much for having me. Thrilled to be here. There we go. Got it on.
[0:12:06] Thank you so much for having me. Yeah, so I work at Capital G, which is Alphabet's independent growth fund, where I lead all of our AI investments, which has made me very popular in recent years. Capital G is Alphabet's independent growth fund, so independent meaning we invest purely for financial return. We're just like any other venture fund, but our only LP happens to be Alphabet.
[0:12:28] We typically engage with folks on the Google side in sort of two ways. The first is what I would call pre-investment, where we're looking at categories or spaces or companies, and we want to leverage the expertise of folks within Google or Alphabet more broadly to really more deeply understand that category and technology. And so that's where it is so compelling to be able to ping Jeff Dean and say,
[0:12:51] Hey, I'm working on AI coding. Could you tell me a little bit more about how do we think about this? How have you seen it evolve over the past five years? What do you think will happen in the next 10 years? Things like that is just so incredible, and I feel so fortunate to work with folks on the Google DeepMind team, GCP team, et cetera. The other way in which we interact with Google is on the post-investment or portfolio support side. And so oftentimes on either product or go-to-market, our portfolio companies will love to engage with folks on the Gemini team to
[0:13:20] Think about how to integrate Gemini into the products that they're building. Or on the go-to-market side with GCP, how do we think about working with the co-sell agreement to get GCP reps out, retiring quota for our portfolio companies? So those are the types of ways we work with Google.
[0:13:35] Yeah, it's so interesting. You were a YC founder not that long ago, but it seems like the AI world that's a very long time ago, things have really changed. Yes. People are vibe coding now. I'm just wondering if you could sort of, as you now look at these startups from a different angle as an investor, what's changed for the founders now?
[0:13:53] Yeah, well, I think, as many of you know, sort of a big tenet of YC is move fast, break things, build as fast as you can, get to as many customers as you can, sort of continue to ship and iterate. And I think never before has that been more relevant than today. The challenge, though, is everybody can, when that sort of used to be, okay, certain sets of founders and exceptional engineers can ship and iterate in the best way possible and sort of get out into markets that deserve disruption. Now it's more like a lot of people can ship.
[0:14:22] Using the tools that exist across vibe coding, et cetera. Repl.it does an amazing job at increasing the capacity for engineering. And so you're sort of like, all right.
[0:14:32] That really increases the competitive landscape for you as a founder. And you really have to, that's sort of like ship and iterate is table stakes, but you really have to think about driving durability and how you're going to embed yourself in a workflow or aggregate data or do something from a brand perspective that's differentiated that allows you to land in a bunch of customers very quickly and use that as a compelling wedge to build from. And I think that was much less of an emphasis when I went through YZ.
[0:15:01] Investing in the companies that you invest in, they're these bigger bets, right? And that means more time. Sometimes it takes more time. I think of like dev agents, for instance, the fascinating company building these interconnects between AI agents. But meanwhile, everything is moving, as you said, so fast. I mean, how do you pick these bets when...
[0:15:21] It's so hard to see around the corner. Yeah, I was just talking about this. It's really fun. It's really hard. The thing I love about investing is you sort of have multiple learning curves simultaneously where you're always trying to get.
[0:15:36] To be a better investor, that's one learning curve. But then you also have what's going on with drug discovery, what's going on with site reliability engineering, et cetera. And now you're layering on these almost vertical learning curves in the AI innovation space, and it's quite hard. The way we think about it as a team is you kind of have to look at whatever new technology comes out, whatever new demo comes out.
[0:15:58] And just assume it works. It might not work perfectly today, but let's take it to its logical conclusion and assume that technology works well. In that state of the world, what would we invest in? So a good example of that would be like, all right, when all of this sort of AI mania started, the typical investment structure that you would sort of use to break up the category was models, infrastructure, applications. If you assume at that time sort of the models worked okay, not great.
[0:16:28] If you assume the models work really well, then maybe the types of infrastructure you invest in are gonna be different than the types that are getting traction today. Because the types of infrastructure that are getting traction today are the ones that are making the only okay models more usable. When instead, you might wanna invest in infrastructure, like to your point, this is what led us to dev agents, where you say, okay, the models are gonna work really well, agents are gonna be ubiquitous. In that state of the world,
[0:16:57] There's this infrastructure challenge, which is how do the agents engage with one another? Where is it appropriate for humans to come in and say, I want my agent to do this, but not this? We need someone solving that problem, not the problem of how do we elongate context within this version of the model, because let's just assume that gets resolved. So that's sort of the way we think about it. And then you have to back founders who either see that end state vision and are living in the sort of like skip to that point,
[0:17:25] Or are so compelling at finding a really nice wedge and then always on what's happening with the technology and sort of shifting into what works. So what is it? You're talking to these people with this vision. Yes. What are you seeing out there? What are the crazy ideas that are not possible today with these capabilities that will be? Yeah, so I think I'll name a few. One I've been spending the past year on has been drug discovery, which is just, first of all, the most heartwarming.
[0:17:53] It's a great place to spend time. It's so incredible to see what will be possible in the next even year to five years.
[0:18:01] Think about the drug discovery process. There's the like, how do you come up with ideas for drugs? And then how do you maximize the probability that those get through clinical trials? And there's really interesting things happening from an AI perspective on maximizing coming up with those ideas and widening the top of the funnel. And then also increasing the probability that those ideas even get through clinical trials in a much hastier way, which I think will touch everyone in this room's lives in some way, shape, or form in the next decade.
[0:18:29] That is awesome. That's one sort of, I guess, crazy but really compelling idea. We're also spending quite a bit of time on what will the world look like when agents exist? So things like dev agents, like you said, where I led the investment in November. But even sort of getting to the point where you're like, okay.
[0:18:46] How will the web need to be re-architected if agents are the ones that are browsing the internet and not humans? Is the information organized appropriately even just on a web page? Or is there a different way that you would architect a web page when an agent is the one going and sort of screening for things and pulling out the information? And so we're spending quite a bit of time on like, wow, is the entire way the internet is architected likely to change? Wow. On the drug discovery thing, I mean,
[0:19:14] Is there something that is sort of not working today, like a bottleneck there that you see changing? Yeah, so I think right now the technology has been amazing at that first part of what I described, which is historically we've been limited in drug discovery by things that are sort of either trial and error or things that happen naturally. So for example, penicillin was...
[0:19:37] A result of somebody leaving out an orange and randomly they found discovered penicillin and we realized it could be applicable in a whole variety of ways. Now using models like AlphaFold or Tide Discovery and things like Isomorphic Labs, there is a much, much better way and a much more sort of scalable way to come up with ideas for what's called protein binders or candidates for things that could be interesting ultimate drugs.
[0:20:03] The bottleneck, like what we haven't made as much progress is that's easier to do when it's just, hey, we have a protein and we want to predict whether another protein will bind to it. It's really hard to then think.
[0:20:18] Once we've predicted that, we put it in the human body and imagine all of the chaos that exists in our bodies. Like, who knows what's going to happen to that thing? Who knows what will happen with toxicity? Things like that. And that's the part that will really move the needle on clinical trials and both speed and increasing probability. And I would say the technology doesn't quite work there yet because it's such a massive problem. It's fascinating. We could talk all day about that. But I want to also ask you, how are you using AI at Capital G?
[0:20:48] What are the cool, you have access to so many cool things, and give us some of the top secret ones, please. Yeah, of course, happily. On a mic in front of a crew of 150, perfect. So I'd say we use it in really three major ways. The first is from a data perspective. Obviously my job is figuring out what are the fastest growing, hottest, you know, buzziest AI companies and making sure that we're in front of them.
[0:21:12] And so part of that is just making sure that we're out there meeting with a bunch of founders as much as we can and talking to earlier stage investors. Part of that, though, can be utilizing data that exists like LinkedIn headcount growth. If a company is growing quite quickly on LinkedIn,
[0:21:25] It's probably a good indication that it's one we should keep an eye on. So that's just one example. We have many more proprietary examples that I won't share, but things like sort of collecting that data to better predict what companies might be at that inflection point to make an investment. That's sort of one category. The second category is in things like sort of the deep research assets that everyone is using and using that to really accelerate our thinking and understanding of different categories.
[0:21:53] The one thing I feel strongly about there is those technologies today are really good at sort of setting up a framework for how to think about, if I said like, I know nothing about drug discovery, write me a research report on drug discovery. It'll come back with things that are interesting. It's much better if you can say, I'm interested in drug discovery. Here's the initial work I've done on it.
[0:22:16] I have a question around this sort of like probability of getting through clinical trials. Giving it more guidance sort of gets you to a much more interesting answer. So that's how we use the deep research stuff. And then the last piece is on just figuring out ways to get ourselves prepared for meetings in a much more efficient way. We used to do a ton of desk research in anticipation of a meeting. Now we can just give a quick prompt and say like, hey, I'm meeting with Reed.
[0:22:42] Tell me what I should know about this guy who lives in Mill Valley, things like that. Right. Does it ever get something wrong, though? Do you worry about that? Yes, I worry about that quite a bit. We back check everything, and we would never, for example, with the deep research stuff, we only turn to it after we've done sufficient amount of work that we feel like we are the experts to be able to back check thing or know when something feels off.
[0:23:03] Are you doing, I know you're not, you were not a coder, right, by trade as a founder. I was not. Are you doing it now? Are you vibe coding? I vibe code a little bit. Yeah, I vibe coded a couple times to sort of get in front of companies and show either I'm using their tools or, you know, here's an interesting way to sort of show something interesting that I know the company is excited about. I will say a funny story about getting it wrong. I'm a big March Madness fan. I'm a big basketball fan. And I tried to vibe code myself like, hey, give me all the live scores.
[0:23:32] So I can duck out of my work meeting and watch the end of it.
[0:23:39] And it did that, but it was 100% wrong. And it's just alerting me to like, Duke's playing UNC and they're down by 20. And I'm like, that's just not even accurate at all. One of these teams isn't even in March Madness. And so it's kind of a funny example of like, all right, these tools are awesome and exciting and there are some limitations. But according to your logic, next year it will work, right? Next year it will work, exactly. No, I do, I mean, I think the reality is like you, as an investor, you have to live in that space because otherwise you run the risk of-
[0:24:08] Investing in categories that are, you know, if I take the sort of model infrastructure application example I was using earlier, infrastructure categories that are just not relevant in a longer period of time or applications that are limited by the technology today but not what the promise of the technology could be. You mentioned talking to Jeff Dean about coding and you did one of these early AI coding investments with Magic. Yeah.
[0:24:35] How, you know, was that too early? Has that worked out, do you think? Or do you think it's like, I mean, it's a crowded field? Yeah, so I mean, it's actually quite a good example of skating to the puck or sort of thinking like, hey, if this technology works, where would I invest? And so, yeah, I led an investment in a company called Magic.dev in...
[0:24:54] Early November 2022, which is relevant because it was before ChatGPT came out, which when ChatGPT came out, I was like, ooh, this either makes me very smart or not so smart. And we'll find out. I think jury's still out on that one. But the concept of the investment was.
[0:25:09] I saw this technology because of our work with Google. I was chatting with Jeff Dean and Noam Chazir, who pioneered Lambda and much of the early technology within Google. And I was like, gosh, coding is such an obvious application for this. It's a massive market. It's hamstrung by the number of engineers. It's like a very compelling use case for this technology. So I started to meet with all of the coding companies. And what I felt like at the time was there are co-pilots.
[0:25:36] And then there's autonomous software engineers. And Copilots to me, and again, this was pre-ChatGPT, so now we've obviously seen all of this play out, but it felt like that would be a point in time exciting trend, but so long as there would be a future with autonomous software engineering, I couldn't really see why Copilots would be around in that future. And so my options were kind of like, okay, I could invest in sort of that part of the...
[0:26:04] The trend where it's more of a co-pilot today. Or I can just jump to the future and make a bet on autonomous software engineering. And I met the magic team. And they had a technology, have a technology that allows for near infinite context windows. I think what is publicly available is 100 million token context windows.
[0:26:21] As I'm sure folks know, that is extraordinarily important for autonomous software engineering and making it a reality. And so there's some exciting stuff coming out later this year for Magic, actually. But I do think it was on the right side of history in terms of like, yeah, agentic coding is sort of where we will end up. And Copilot, I think, was a really awesome way to bridge the gap from nothing to agentic software engineers, but everyone's sort of moving in that direction. I mean, you sound like you're, I mean...
[0:26:47] Very gung-ho about this technology, but then are there things that, and I always say you overestimate the short term and underestimate the long term. Are there things that you would not invest in? Are there places you're very skeptical of where this is gonna go? Yeah, I actually think, maybe just to hit your point, I'm almost underestimating the short term.
[0:27:08] I feel like I am a bull on the long term, and I think the short term we just kind of have to live through the kinks. I would say categories where I'm a bit more skeptical, again, is on this sort of infrastructure around what models can do today, just because I do think it is a safe bet that these models will get better. And so anything that's sort of around the fringes of making okay models great feels a little bit risky to me. Making okay models great.
[0:27:38] Interesting. What about like physical AI, for instance? I mean, there's so much in robotics right now that, you know, including at Google. Totally. Yeah. I mean, I think the physical AI stuff is so interesting. That one is so hard to time. There is a very clear market demand for it, obviously. But when the technology works sort of depends on two things.
[0:28:00] One, the generalized foundation model has to work. And so teams, I spent a ton of time with Lockie and the team at Physical Intelligence. They came out of Google and Stanford. Exceptional team and exceptional early results. But you sort of like, that has to come to fruition. You have to bet on that model working. And then you have to bet on these types of models sort of integrating well with whatever physical use case we want to use it for. And if you were to look at robotics historically, that's where things like
[0:28:27] Things have been held up on sort of that integration point. And so I think I am very bullish about the future of that. We haven't yet made an investment because I do think we want to wait a little bit more to sort of see the technology pan out and then think about how we would apply it to the physical world. Do you think that's, is that a data problem? I just, I went to their office and got all these people just.
[0:28:49] Basically creating data with these robotic arms. I did it too, although I think I ruined their data by folding laundry in the wrong way. But is that a data problem or is it something else? I think it's a data problem. It's why there's a lot of discussion around...
[0:29:03] How many foundation models should there be? And is it right to have a specific foundation model for voice and for image? Or should they all be in one? Is it right to have a different foundation model for drug discovery and for robotics? And I think the robotics thing, it's very clearly this data question is a big question. And so if you can aggregate in the short term a mass amount of data, it's quite compelling. I will say, and I know Fei-Fei is up next, I think. I don't know.
[0:29:32] She'll talk about it here, but what they're doing at World Labs is truly mind-blowing. I was in their office doing a demo, and my jaw was on the floor. And she and her team have an incredibly innovative way to think about data, which if you have that sort of advantage in that end market, it can be quite compelling. What a great segue. Thank you so much. Yeah, of course. Two seconds left. I might as well. I was like, wow, I couldn't have planned it better. Thank you so much. That was amazing. Of course. Awesome.
[0:30:02] Please welcome Dr. Fei-Fei Li, founding co-director of the Stanford Institute for Human-Centered AI, and CEO and co-founder of World Labs, and Reid Albagotti, technology editor for Semaphore. Great to see you. Great to be here. Now I just want to go right into like, okay, so what are you building? It sounds amazing. World Labs, it's...
[0:30:40] I know it's, I've gone to your website and seen these amazing, like you can go into a Van Gogh painting and it turns into a 3D world. But tell me how this is leading to this broader vision of yours to build these world, these giant models. Yeah, so World Labs is the startup that I co-founded with a few of the...
[0:31:05] Really incredible technologists. And the goal of this company is really to bring 3D space to life and create foundation models of 3D space that people can use to generate 3D space, to reason with it, to interact with it. Why is this significant? It turns out whether we're talking about creators or game developers or...
[0:31:33] Design professionals, whether it's architecture or art or marketing, or we're talking about the future of many experiences, whether it's movies or tourism or even XR, AR, VR.
[0:31:53] For robotic simulation and all that. All these different use cases boil down to a key piece of technology, which is to create and reason with 3D worlds or interact with 3D worlds. And it's hard to make them, right? Nature makes them. We live in one. But to lift what nature makes.
[0:32:19] Into the digital space is really hard. And to then imagine newer ones for all these use cases are even harder. So that's what our labs is making. And Jill mentioned data.
[0:32:33] You have a history of being very creative on getting data sets. That was the ImageNet, which is one of the reasons you're called the godmother of AI, right? You figured out how to go and get all these photos labeled using Mechanical Turk at the time. And that was, I mean, it just blew away the whole field. So what was Jill talking about? What's the data? What are you doing now, if you can say?
[0:33:00] Yeah, well, first of all, I totally agree with Jill. Data is still fundamental to AI. Every AI effort is also a data effort. And what I did many years ago now with ImageNet was really the first project ever in AI that brought big data into the thinking of making AI systems. And it turned out it's one of the three
[0:33:28] Crucial ingredients of making AI happen, the other two being GPUs as well as neural network algorithms. So for a company like World Labs, we have to solve our own data problem. And we work with pixels. So everything we work with are pixels, voxels, however you name them. So we have to go about
[0:33:54] Creating, sourcing, processing a very heterogeneous pot of large amount of data, whether it's images, or videos, or 3D, or processed information, or synthetic. So we're giving a very hybrid approach to data efforts in World Labs.
[0:34:19] Okay. It sounds like there's some trade secrets there, so I won't push you. I got to be honest. I think there is always going to be some trade secrets when it comes to data and algorithms for AI companies. Do you have a commercial sort of a way to make money in the short term, or are you thinking more long term? You're going to eventually figure that out. Well, we definitely want to make money.
[0:34:44] Really, what it boils down to, create products that people will love and use. And we do think right now that creativity and creator industry, like I said, it's already a horizontal enough industry between artists, designers, developers. We think that industry or that market would benefit a lot from having a
[0:35:10] The ability to control and create and interact with their own 3D models. And that's what we are aiming at in the first phase. That's interesting. So as someone who has made these huge breakthroughs in AI research over the years, what do you look for today as you hire researchers or work with them at Stanford? What's a good AI researcher today?
[0:35:37] That's a great question. I think a great AI researcher is similar to a great artist or a great doctor or a great lawyer. You start with someone who is curious, who is open-minded to learn. Because no matter what you know, there is a lot more to learn. I also love people, especially hiring
[0:36:08] Both at Stanford, for my students, for my lab, as well as hiring at World Labs, I look for fearlessness, courage, the fearlessness of pursuing problems that no one has solved, the fearlessness of failing and just do it again. This kind of audacity to try and grit
[0:36:38] Persevere, even against multiple failures. So interesting time in history right now, geopolitically. There's so much talk about, OK, so students who are from out of the country, are they going to be able to stay here? Are we going to lose talent, essentially? Do you have thoughts on how do we attract and keep talented researchers in the US right now?
[0:37:07] Yeah, well, US is an immigration country. And I think throughout my career as a professor at major universities, we benefit a lot from talents all over the world. I think it's still important that we remain the magnet of talents.
[0:37:28] Because many of these talents end up staying and contributing to the workforce, to innovation and all that. So I do hope that we keep that going. Do you have thoughts on how we can't? What do we need to do to do that?
[0:37:49] Well, where I stand, because I work in the tech field, right, and most of my students are computer scientists and technologists, I generally hope that they can get visa and work visa and then have a path of immigration. This has been, you know, of course, visa or the, I don't know how many of you have heard of quotas for visa and all that has been, you know.
[0:38:17] A challenge for many talents to stay, and I hope we can continue to address that.
[0:38:25] Another sort of one of these big topics right now is just funding for research. I mean, a lot of the work you did was funded by government grants, essentially, right? Actually, no, that was rejected by government grants many times. Right, but a lot of the university research is paid for through NSF or various agencies, and that's sort of being threatened right now. And I just wonder what...
[0:38:54] What are the risks there in taking that path, do you think? Yeah, it's a complex topic. I think this is a nuanced topic. I think there are areas of opportunity to actually update the way that we use grants, universities use grants. But in the meantime,
[0:39:23] I absolutely believe that the public sector, especially academia and higher education, has been a pivotal central part of America's innovation ecosystem. If you look at the past 100 years, or especially since World War II, that higher education's innovation has been a critical part of our economic growth. To be honest,
[0:39:53] If you look at today's AI, so much, almost everything that we now know as classic knowledge of AI came from academic research, whether it's algorithms or data-driven methods or microprocessor, early research in microprocessor. So I think continuing to nourish our higher education, our
[0:40:21] Our public sector, for this kind of innovative blue sky curiosity driven research is critical for the health of our ecosystem. It's not only important for continued discovery and knowledge expansion, it's also important for training the next generation talents.
[0:40:44] So another big topic, a big challenge in AI right now is observability or trying to figure out how these models actually work. It's the black box problem. And I'm just curious, and I think it ties into a lot of the work you do at the Stanford Human-Centered AI Department, trying to make these models safe. Do you think that's a problem that's solvable, figuring out how these work on a neuron level?
[0:41:10] That's a great question. So I think you're referring to Stanford HAI is an institute of many different disciplines and areas. And one of the major area that we have been supporting is the research for foundation models, especially large language models. And we created this center called CRFM.
[0:41:35] Many years ago, pre-chat GPT. And some of the topic here is explainability, observability, and also generally just responsible way of using these models. Do I think this is solvable, or is it ever going to be solved? I do think it will be solved one day. Maybe this is just the optimism of being a scientist.
[0:42:02] You're not afraid of asking these questions and forever hold the hope that these questions will be answered. There's a lot of great research that's coming now, both in industry as well as in academia, on the question of observability. And in terms of responsible use, there is also so many layers. There is the fundamental science of understanding
[0:42:32] The nuance of the model, including bias and hallucination and repeatability and all that. But there is also the application of use these models for good, whether it's for health care or it's for environment or for economics and for history, for art. These are all responsible ways of using these big models.
[0:42:57] So this is kind of an off-the-wall question, but I'm really curious. You started in neuroscience. That was one of your first areas, in addition to physics. And we're now seeing these BCIs that pop up, right? Are you interested at all in what we can actually learn from so much of AI is inspired by the brain, but we don't actually know what's going on? Do you think we'll learn something from what we get from these individual neurons? Oh, definitely.
[0:43:27] That's a great question. There's a little bit of a two-question pack there. One is, can we learn from the brain? The other one is BCI, specifically. So can we learn from the brain? Absolutely.
[0:43:40] If you think about how neurons are communicating and are connected together, it's very different from today's AI models. On one hand, some part of the AI model has already surpassed the human brain, the number of neurons and all that and chips.
[0:43:58] Compute a lot faster than neurons. On the other hand, there's still so much mystery, right? For example, neuron to neuron communication is not just through electrical signals of the synapse. There is a lot of chemical communication, a lot of dendritic communication, a lot of different. There's just a lot that we still don't know that I think if we learn more about the brain.
[0:44:26] We'll become smarter in our AI algorithms. And vice versa is also true, actually. Today's AI is helping to uncover some of these brain science questions and helping with scientific discovery. And the question of BCI, which stands for Brain Computer Interface, that's a really interesting question.
[0:44:47] BCI is intended to be more augmentative to humans, especially for clinical usages. It is not, per se, a method to read out individual neuron and what they're doing, unless it's part of the loop for clinical usage.
[0:45:11] I do think it's a very intriguing up and coming technology. In my lab, we have been doing human robotic interaction through non-invasive brain reading, mostly electrical signals from the skull, not the invasive ones. So I have a little bit of a bias towards non-invasive method. If you check our web page, we show that humans can
[0:45:40] Wear an electric cap, and then it can give instructional thoughts to a robotic arm. And our robotic arm is able to cook a Japanese meal, actually, using electrical thoughts. So for me, that is a future I would love to see, which is using this kind of brain computer, brain robotics interface to help patients and those severely paralyzed people.
[0:46:09] That is so fascinating. Another thing I wanted to ask you about was I saw this Elon Musk conversation with Satya Nadella the other day. I think it was on Monday. I did not read the news. It was a very simple point that he made or claimed, which was that his AI model, Grok, is grounded in physics, in the laws of physics, and that would somehow make it safer, more grounded to truth.
[0:46:38] It seems to contradict sort of, I mean, your view is like they need to understand the world, right? And that's sort of the grounding, if I understand it correctly. But do you think, as someone who started off in physics, is there anything to that, or do you think that's sort of? Yeah, so I have no idea what Elon and Satya were talking about, so I'm not going to be able to, because I thought Grok is a language model, so I don't know what he means by physics here.
[0:47:09] Sorry. I honestly just don't know what he's talking about. I'm creating world models, 3D models. I do think physics plays a role.
[0:47:26] The world generally obeys the law of gravity. And the way trees are growing, the way balls are falling, and everything we do, the dynamics have a lot to do with physics. So this is why I'm so excited by building world model, because the next frontier of AI modeling goes beyond language, goes into understanding this world and modeling this world and being able to replicate what this world is and follow the dynamics and physics of the world.
[0:47:55] Thank you. So you've gone back and forth from academia to big tech to startups now. I know you've had one in the past. What is the difference between the research that is done in academia compared to the private sector? That's a great question. I get this question a lot these days. There's a part of me that wants to say that, especially in AI,
[0:48:22] They're not that different because whether we're in the labs in the university setting or in my startup is a deep tech startup. So we are really going after these.
[0:48:33] Big, audacious, deep tech problem that people have not solved yet. Having said that, there are some differences. For example, university is a training ground for the next generation researchers. We care a lot about students having the opportunity to do blue sky research, not only to get trained as
[0:48:56] To work and think like a researcher, but have that freedom to pursue purely curiosity-driven research. Also, in universities, there is a lot more opportunity for interdisciplinary research, especially on a campus like Stanford. We've got the world's best medical school, law school, education school, sustainability school, business school, and engineering school. So we can cross-pollinate a lot of ideas.
[0:49:25] That are not easily done in industry. In the meantime, where I am as a deep tech startup, it's also exhilarating that a group of the most talented, fearless, young technologists can come together and identify a North Star problem that is so hard and everybody works towards that. We still have to do trial and error and experiment on different ideas, but that sense of teamwork
[0:49:55] I think the work you've done on the national AI research with basically providing a compute to academia has been really important. So do you have anything to say on that at this moment in time? Yeah, so I think really you're referring to since about more than five years ago.
[0:50:24] Leadership at Stanford Human Center AI Institute, we recognize not only Stanford as a university has this resourcing problem in terms of chips and data. I think it's a problem nationwide that our higher education and public sector are lacking access to the critical ingredients of AI. So we work with actually multiple White House starting in 2019. And then...
[0:50:53] In the first Trump White House and then the Biden White House to try to establish a national research AI research cloud as well as data repository. We made a lot of head of progress in terms of getting a task force set up, getting a plan, putting a bill on the floor of Congress and Senate. It is not passed yet right now.
[0:51:21] Don't know where that is going. But I think there is more recognition now that resourcing, not just the industry, but resourcing public sector and higher education for compute and data so that students and researchers can do good research in AI is very critical for our country and our economy. Fei-Fei, thank you so much for the time. Thank you. It was so interesting. Yeah.
[0:51:53] Please welcome Glenn Fogel, CEO of Booking Holdings and Booking.com, and Bennett Richardson, GM and Global Head of Public Affairs for Semaphore. Glenn, thank you so much for joining us. Well, thanks for having me. Excited to dive into AI in the real world with consumers and travel. But first, I want to go off script for a second, give you some street cred with our San Francisco crowd loves evaluation.
[0:52:28] I'm going to quote your interview with our colleague Andrew Edgecliff-Johnson earlier this year. Booking has grown more than 800 times from your smallest valuation to today closed market cap over $172 billion. And you've beat the market more than 100% across the past five years. Just an amazing run with the company. And obviously AI, huge transformational force across the entire industry.
[0:52:56] Taking a look at the industry as a whole, I guess, both travel and for consumer-facing businesses like yours, how is AI transforming these industries? And is there sort of proving grounds since there seems to be some early adoption here? Well, I think part of it is you have to start with, when we say AI, what do we mean by that? Obviously, different forms, different things. I mean, you mentioned the valuation, over $170 billion now.
[0:53:23] That goes from when I first joined this company 25 years ago in the beginning of 2000. Our stock was down to like $6 a share, and now we're over 5,000. And it was driven by technology. And one of the things was being very early, using machine learning, creating models that would help predict what we needed to do and whether it was how we would bid on Google or what we should present to a consumer.
[0:53:51] And of course now with generative AI, it's just so much better. So many things we're going to be able to do now that we dreamed about before and that will hopefully propel us because I'm trying to keep up. We have a 30% compound average growth rate for 25 years. That's going to be hard to keep up. But hopefully with using GenAI, we'll be able to provide.
[0:54:15] A service to consumers that is so much better that we can continue the growth. We'll see. I feel like executives like yourself at all sorts of companies are all thinking the same thing right now, which is the pace of innovation, the pace of advancement is so fast. What does it take to innovate at speed so that you can continue to stay ahead? No, it is so hard. One of the benefits when you're large is you can afford.
[0:54:42] To hire a lot of great people, a lot of really smart, driven people, and you can take chances in different areas that if you don't have that capital, if you don't have the scale, you may not be able to do that. So it's good to be really big. The problem with being really big, though, is that can slow you down. A small startup can be very focused in one area, but of course, that's one shot. We get the benefit of having many different shots, and that's a benefit to us.
[0:55:12] So I know most of our ideas are going to fail. That's okay. I just need a couple to be really great. Well, I want to double-click on that because having those ideas fail, that's not always what an investor might want to hear. How do you navigate this constant disruption while also keeping an eye on the bottom? No, it's extremely hard because you're right. The investor community wants to see a certain return.
[0:55:38] And you have to live up to what you've promised them. They have put their money into the company. And whether it's a private company or a public company, they're expecting a return. And they're expecting a certain margin. And I may say, gee, should we invest even more now? Because I believe that if we do that, we'll be that much better off 10 years down the road. And that's the way we usually think about we want to do things for the long term. But you have to balance.
[0:56:06] Because people are expecting a near-term return, too. So it's always a balance. And sometimes I wishfully think of the pleasure of being a private company, and you see a great new Gen AI idea. And you can just go full bore, and you don't care the fact that you're losing a fortune, because the VC guys knew that was what it was going to be for the first few years. We would not be permitted that.
[0:56:31] You look at the analyst reports. They're putting out and they're saying, we're going to do $9 billion in EBITDA. That's what the analysts are putting out for this year. And they're expectations. So you cannot just throw all the money into new things. You have to maintain what the current business is. Fortunately, again, the size we are, it affords us the ability to make investments in new areas. And there's so many new things coming in using Gen AI to make the travel business.
[0:57:00] For our business, at least, so much better. Well, excited to hear from you about how those investments translate to that consumer experience for all the travelers that are using all of your sites. How is AI changing that customer experience, that traveler looking to book a table, travel in all of your different platforms? How is that changing the experience?
[0:57:23] I'm sure there's probably a bar moving in terms of what a great experience looks like, as well as value. How is that changing? Well, OK. So let's just state the obvious right now. Travel is one of the most wonderful things to do, but it's also one of the most frustrating things to do. I mean, every single person in this room has had a horrible travel experience. And you have wondered, why is it like this? Why can't it be better?
[0:57:50] Using AI, it will be much, much better. And we're seeing it already. Examples. So this is something like, you know, you have a problem and you call customer service and then you wait and then you wait some more. And when you finally get in touch with somebody, they can't solve the problem. They got to send you to somebody else and then you wait some more. Absurd. Using GenAI, we are producing and are able to open table right now.
[0:58:17] When you have a problem with a restaurant and you're trying to talk to somebody, you may be talking to AI because we have a partnership with a couple of the voice AI players and the restaurants together. We've all come together. So that is answered immediately and it is solved more easily. So that's a simple example. And we're doing that in the travel side too. In our Booking.com company, when people are having an issue and we're responding.
[0:58:46] It usually is a chat. We're slowly starting up the voice stuff that'll be coming down the road. But that's a simple example. Even more so is the problems or the frustrations of choice, of trying. You're just not sure what you want. And the permutations are almost infinite. And think about things. I'll give a little example. So I don't surf at all. So anything I say that's really stupid surfing, it's because I don't surf, but my wife surfs.
[0:59:15] So imagine, you know, we want to go on a holiday and she's saying, you know, I want to do some surfing where we're going to go. And I'd like it to be, you know, probably California, but could be maybe Latin America or maybe Rincon in Puerto Rico. And, you know, I'm not that good. So I want to have things that are just, you know, small waves, you know, maybe nothing more than a five foot face or something. And I want to have a left break because I'm goofy and all that. Well, theory is absolutely no.
[0:59:42] There is no travel service that you can put that in right now and get a whole bunch of, here's some choices, here's some things to do. You can't do that. But very shortly you will. And we right now at Booking.com, you can have a natural language conversation without, you don't have to use those filters, you know, click the box, I want, you know, luxury. No, no, you just go and say what I just said. Type that in and it'll be coming back. And then take it to the next level, which is...
[1:00:10] I want to have refundable hotels so that I can change in case the weather doesn't look good or the waves aren't so good. And then think of this in AI because obviously it's very easy to bring all the information in.
[1:00:27] So you have a real-time agent who's looking out for you and comes back to you three days before the trip and says, I think you should change this. Instead, you are going to go down to Rincon. I think actually La Jolla would be better for you right now. And here are the hotels. Think of that. And so many of the things in your life that have been frustrated will be solved. I mean, it sounds like the best travel agent you've ever had, right? It is. It's going to be incredible.
[1:00:54] I mean, when you have something going on, travel, and the weather was bad, and then you're trying to go all the trouble of changing what you were going to do that day on that. Imagine your agent, your booking.com agent, it's doing it all for you. And it's permitting you to have agency, meaning you have the choice, yes or no. It's coming back and forth. It's like having the best travel agent in the world who lives in your pocket and is always looking out for you.
[1:01:24] It will be solving problems before you even know there's a problem. It will know that plane is going to be delayed and is offering you choices to change before you leave your house. And that is what we are building. And that requires a lot of people, a lot of work, but we are on the path of doing this. Now, it's so exciting to hear. And as you're building out, like you said, a lot of people, a lot of work.
[1:01:52] There's clearly a lot of safety and responsibility with building out those tools. I'm curious as you think about that balance. Lots of benefits, as you just described, from those Gen AI-powered experiences. What are you all thinking about in terms of the safeguards that need to be put in place? One, hedge against any potential drawbacks. Even maybe more importantly, continuing to build.
[1:02:15] The great trust that you've built with your... Yeah, no, trust matters a lot. And so one thing that's interesting, you hear people say, well, aren't you going to get disintermediated by somebody else? Because I've been hearing that for 25 years. I heard that at the beginning, well, aren't the hotels going to disintermediate you as soon as they can learn how to do a website? And then it was like, as soon as they can learn to do a mobile site. And then, of course, Google, they were saying, oh, Google's going to disintermediate you. Yet every year...
[1:02:41] More and more people come to us directly. In our last earnings call, I talked about mid 60% of the people who do a transaction with us, they came to us directly, discounting, not counting our B2B, where we put that, I'm saying B2C, consumer comes.
[1:03:00] About mid 60% came to us directly. They didn't type into Google or anything like that. Why? Because they trust us. And trust is incredibly important. And you can ruin it very quickly by a bad experience. I assure you, though, that happens a lot more with human beings than it does with systems. That's the first thing. But people, people are willing to give other people a break. They're not willing to give systems a break as much, which is an interesting thing about human beings. Because that safety thing is interesting.
[1:03:29] So if somebody is using AI to determine whether or not there's a tumor, and the AI system does not catch that tumor and the person has a terrible experience or whatever, we'll see the headlines on the front pages everywhere about that. I assure you, radiologists are missing that tumor all the time. And we accept that. We accept that. It's OK. The same thing with autonomous vehicles.
[1:03:56] 40, 50,000 people die on our highways every year. Eh, it's the cost of doing business, right? One autonomous vehicle has an accident. Oh, God. It's headlines. So I agree with you. We have to make sure that we do things that are proper, trust, safety, et cetera. But I think anything that is new is going to be, some people are going to expect a higher level. So we'll be sure to try and do that. Fortunately for us.
[1:04:23] You have a lousy holiday. We'll lose a customer, but we're not going to lose a life. So for us, the stakes are a lot lower. Definitely. And probably less likely these agents are going to hallucinate some new hotel that doesn't exist. Actually, no, because we do have a bunch of these things out there. We have at Booking.com, it's our AI trip plan, or at Priceline, it's called Penny is the name of it. So we've got a bunch of these out there, and I'm playing around with it, and I've gotten some things I know are not true.
[1:04:47] I know that hotel does not exist. I'll be perfectly honest. It's beta. It happens occasionally. You couldn't book it, though. That's the great thing. It'll describe it, and you'll think, oh, this is great. Try and book it. It doesn't exist. So it doesn't happen much. But again, the issue is not as important in other areas where AI is really important. I mean, we want our aircraft, our systems in the sky to be absolutely perfectly safe. That's an area where...
[1:05:16] You know, it's a no-fall zone. You cannot make mistakes in that. Absolutely. And just coming back to something you said a minute ago, I feel like we're entering another one of those moments, like the hotel website, like the hotel app. This week has been a lot of news about AI agents, Google I.O., and, you know, the topic coming up. Seems like that's another...
[1:05:35] Wave coming of these general purpose AI agents and start to be able to book travel for people and things like that. How are you thinking about staying relevant in this next moment that's coming? Yeah, and this is an interesting issue too about the value chain and who's going to have more value in all this. So one of the things everybody's concerned about is when you are creating agents that are talking to other agents.
[1:06:00] Because we're a business. We're a for-profit business. And how is the money going to flow? And of course, one of the things we're always concerned about is you have proprietary IP, and you don't necessarily want to just be giving it away, and then other people can use it. So it's very important. So we have agreements right now. We were one of the co-founders, co-partners to start off with the operator system with OpenAI.
[1:06:25] We have an agreement over doing the Alexa Plus. We got an agreement with Microsoft. We're dealing deals with everybody that we are willing to talk with anybody because we have some very special things that other people do not have and very hard to create. The data we have is something that is a great mesh with other people who have horizontal models, et cetera, to create a great experience. What's really important, though, is setting it up.
[1:06:52] Write ground rules so each can benefit fairly together and one isn't taken advantage of. So that is one of these in-process things that we're doing right now. Absolutely. Well, being able to bring all of those together. Bring it all together and create a better experience. Absolutely. Well, that's all the time we have. Thank you so much for joining us. We need to do a demo next time so we can find that surf lodge. It'd be great. Perfect. Thank you. Thank you. Thanks for inviting me. Please welcome Jack Clark.
[1:07:21] co-founder of Anthropic, and Reid Albaghotti, technology editor for Semaphore. Thank you so much for being here. Absolutely. Happy to be here. Let's get right into it. We don't have much time. So easy question right off the bat. You go to Washington a lot. You did in the previous administration, during this administration too. You meet with these officials in the Trump White House. Do you have a sense of, is the strategy coherent?
[1:07:58] Because it seems to me there are contradictions. But do you have a sense of what the AI strategy is in the White House? So I think the challenge of AI is that it touches kind of increasingly everything. It touches electricity generation. It touches export controls. It touches aspects of how you do the structuring of US science and research.
[1:08:20] And this administration has gone through the pattern of coming in and saying, oh, wow, we've got this cool technology. It's called AI. And then they discover very quickly that this technology touches every single thing they care about. So what you're seeing is the administration wake up to the importance of AI and stand up initiatives in energy and in some cases in export controls and in aspects of research. And it will eventually form into.
[1:08:45] My hope is a larger, kind of a more coherent strategy, but this is some of what happens when you're dealing with a technology that touches everything, and you might have come in thinking it was less significant than it is. So I was like, oh, first time? Well, I've been doing this for a while. You have advocated, your CEO, Dario, as a company, I think you've advocated against allowing these chips to just flow to China. We recently had this deal with,
[1:09:15] In the UAE, where I think something like 500,000 NVIDIA chips a year are going to go over there, and people are worried. Are they going to end up being diverted and going to China? What do you think? Are you worried at all about that deal? I think there are two things here that are in real tension, and it's going to be increasingly visible in the coming years. On the one side, you want to build out...
[1:09:41] A US-led AI platform around the world, and you want to build that on US chips, and you want to have data centers stood up in many different countries. You can think of this as us building an AI economy. On the other side, if the things come true about AI, which Dario says, or the CEOs of the other AI labs say, then this technology will become an incredibly powerful dual-use technology.
[1:10:08] And all of those computers all around the world are going to become equivalent to factories that can turn out both cars and tanks. And you're going to see which way it goes depending on what the alliances look like at that point in time, which is in the future. So what we're going to see in the coming years is as the technology gets better, we are going to need to think about how we apply really good security to these factories that we've now built all around the world.
[1:10:37] And in places where they're existing in countries that haven't been previously as close to the U.S., you're going to have to take a really close look at the security of those factories and how you know what they're being used for. So talking with Fei-Fei, we talked about interpretability. Dario, the CEO of Anthropic, wrote this fascinating blog post on this subject saying we can do it.
[1:11:04] And he calls it an MRI for AI models. And we actually have to do it because this is our competitive advantage. If we can't do this, then we can't use these things for anything important. Where is that effort? Do you think that we're getting somewhere?
[1:11:24] We're getting somewhere. I think at Anthropic, we like to say this statement that sounds almost a little silly, but is very hard to do, which is computers can be understood. And it's our like motivational statement because AI is wildly complicated and these systems are incredibly complex. But if we want to actually trust them in the case of healthcare or making decisions about our own finances, you need to be able to look inside that system and see how it's thinking.
[1:11:54] And what it's doing. Today, we've been able to do what we call interpretability studies up to some of our smallest production models. So to some extent, we can understand through the equivalent of an fMRI some of the smallest systems that we sell today. And we're trying to scale up to the entirety of them.
[1:12:15] The one problem is we keep on creating more work for ourselves because we keep on creating much better and more complicated systems, and then our interpretability team needs to catch up. But in the limit, if these systems become as capable as we think, the public will demand some level of transparency about how they're thinking and what they're thinking about. So our bet is that in a few years, you will need to understand these systems to be allowed to use them.
[1:12:43] So the next big model in Anthropic is going to be trained on Rainier, right, which is the big Amazon cluster. Apparently, it's not all in one place. It's connected in multiple buildings. You've talked about the risks of distributed compute when we get to this point where, I don't know, like all of our GPUs at home could be used to train one big model. Is that a sign that that's like we're getting there, that we're heading in that direction?
[1:13:13] Yeah, basically for a while now, people have been training AI systems that are so large that they no longer fit in just a single data center, which is kind of wild. It's like, yeah, we now can't fit this thing in a football field worth of computers, and we need multiple ones just to give you a sense of the scale. Google, one of their Gemini models, was trained in multiple data centers at once. They disclosed in one of their papers, and we and the others are doing the same. So you're going to see these.
[1:13:41] Data center campuses that start to get used. Eventually, this might take the form of people here being able to train AI systems on your own computers in the form of a kind of federated network of people. But I think for a while, it's going to be these multiple football fields that you connect with essentially a very big pipe to let you train things. And then when that happens, it kind of reminds me of the SETI at home project for finding artificial, you know,
[1:14:09] Extraterrestrial intelligence or something. What are the risks, though, when you can, like, what do you think happens? Well, I mean, I think the risks are, if we truly don't understand these systems, and they're sort of coming out of, like, everyone's networked phones and toasters, then that sounds, like, quite sci-fi-like and worrying. On the other hand, the onus is on the frontier labs to figure out the safety things first, because we're going to...
[1:14:36] Train these large-scale systems before other people because we've made these investments in the computers. But eventually, everything becomes doable on smaller and smaller amounts of compute or compute that's networked. So we'll live in an interesting world in a few years.
[1:14:51] It's true. You're a former journalist. You're a writer. I've read in your newsletter, which everybody should also subscribe to. It's great. You write these little short stories that are kind of sci-fi about AI. Have you ever thought about writing a big sci-fi book or a movie? What would it be about?
[1:15:12] What would the plot be? Yeah, I'm like a frustrated writer that wants to write sci-fi and keeps on working at AI companies, developing AI systems, and then being like, oh, my day job is just doing the sci-fi thing, and then I go home and try and write the stories. I think about writing the book, but to some extent, the book is being created right now and trying to build these AI systems themselves. I do think that the interesting stories we don't really deal with enough are about...
[1:15:39] The literal power of these things. I think Silicon Valley has actually often been extremely bad at acknowledging or reckoning its own power. Social media, in some sense, was like a powerful technology that changed culture. I'm a recent father, and I'm staring at my two-year-old being like, oh, you're going to arrive onto social media soon, and it's going to start influencing you in ways that sort of freak me out. I think that any good...
[1:16:06] Studies of this technology or sci-fi in general need to reckon with the fact that we're dealing with incredibly powerful stuff that will change the world around us. And part of why I spend my time in Washington is I think that policymakers, unlike with social media, are going to need to sort of lean in more here earlier than they did in the past.
[1:16:27] Or the public will kind of lose trust with them in the future. But is it a dystopian or a utopian future? Yes, I sort of hit the ball on that one. Yeah, which one is it? I think we get to choose right now. I can tell you a story that's really compelling for both of these. And I'm optimizing for the utopian vision. I think if you mess up the...
[1:16:49] Allocation issues of this technology. And in the limit, that means the literal distribution of who gets access to it and how you change how the economy works in light of it. You end up in a kind of cyberpunk, corporate-led dystopia, which is probably a bad world to get into. Doesn't sound fun. Might be a good book, though. It's a great book. But most books that are really good sci-fi books, you actually really don't want to live inside. So you should avoid that.
[1:17:18] One thing I want to ask you about is just this competition. There was a story that an investor told me, an Anthropic investor told me about, you know, before ChatGPT was released, there was actually on a closed Discord server, Claude was out and people were using it. And some people thought you should release it. And in hindsight...
[1:17:39] I understand why Anthropic didn't do it because it might not have been safe. In hindsight, it probably would have been safe. Have you learned anything from that? Is it now that you're worth, you just raised how many billions of dollars about the competitive race here? I think that the...
[1:17:58] Stakes of all of this are going to get higher and higher and higher. And the main thing I've learned is that the culture of the organization actually matters a lot. There was some recent coverage of a model we launched called 3.7 Sonnet where we did a mini version of this where we were going to launch it and we got really close on some bioweapon evals that roll up to me that one of my teams does.
[1:18:23] And it led to us and Dario and others spending a few days talking about it and doing other tests, which then allowed us to have the confidence to release it. I think we only did that as an organization because we built this muscle of thinking about this earlier with earlier versions of the technology. And I think our bet in the future is that this kind of measure twice and cut once approach to shipping is going to be incredibly important. And the main trick is speeding up the time by which you take it.
[1:18:52] We did those tests in a few days rather than needing to deliberate for months. That's interesting. Speaking of 3.5 Sonnet, that actually, and your kids' future and all this stuff, that changed coding. I mean, that just became the model for coding. It's now, I think, Cursor pretty much relies on it and some others too. Are you seeing, what's Anthropic looking for in coders right now? And is there a future for?
[1:19:20] For software development? There's a future for coders, but the job has changed. Like one of my colleagues, Alex Tampkin, he's a researcher. All he does all day is write codes to help build things like our observability tool, Clio, and study our models. And he showed me recently how he uses a technology we built called ClaudeCode. And instead of writing lots of code himself, he instead has five or ten copies of Claude working for him.
[1:19:48] Running experiments and doing individual bits of software development. So his job has changed from coding to managing a bunch of synthetic employees. And the wildest part is this isn't like a demo that doesn't work in reality. This was me sitting with Alex a few months ago and saying, show me how you use the current technology. And he was like, oh, well, I have like five cords over here doing this stuff and this one over here. Oh, that one's done. Now let me talk to it.
[1:20:14] So the nature of how coding works is fundamentally different, and we see it inside the labs where researchers and engineers have sped themselves up using AI, and that happened basically within the last year. Yeah, and are people, do you think people really fundamentally understand how significant this change is going to be? I don't think so. I'm about to start giving a series of talks to policymakers to try and help them understand it, because...
[1:20:43] You know, how a lot of fun sci-fi stories and scary ones start is that the AI labs themselves start to recursively speed up due to AI technology they've built, which is just what I've just described as happening. So I'm like, oh, it seems important. We should probably put more of a light on this one. Is that the next step? Is the next big breakthrough in AI going to come from an automated system?
[1:21:11] Maybe. I think the other thing is AI systems today have this problem of like the blank page. Like you go, it's like you've just opened a Google Doc and you're just staring at that and you need to think about what you're going to write or ask for. The AI systems could instead try and anticipate things that you were going to ask about or they could look at problems that...
[1:21:33] Maybe they can look at what you're doing and be like a less annoying clippy and say, it looks like you're trying to write this kind of software program. Do you want help? So in the same way that they're really good at predicting how to fill in the response to what you write, they can definitely predict things about you and what you're trying to do. And we need to kind of flip that on its head as well.
[1:21:53] All right, well, I think we're, well, maybe we have time for just one last question. I mean, I just want to know where you see, like, what do you think is going to be the next big use case of AI? It's very confusing because a revealed preference, especially in Silicon Valley, is people using these AI tools, especially Claude, as like a therapist or like medical aid. But there are loads and loads of issues with that, you know, from the side of,
[1:22:23] But I think that I'm very bullish on finding ways to broaden out from coding to, for lack of a better term, like self-actualization with AI, which shows you what happens when an Englishman moves to California and spends 10 years here, as you start saying that. All right, we're definitely out of time now, but thank you so much. Well, thanks very much. Please welcome Yiming Chan, Executive Vice President for Singapore Economic Development Board,
[1:22:52] And Bennett Richardson, General Manager and Global Head of Public Affairs for Semaphore. Appreciate you being here. We've had lots of Silicon Valley perspectives. I'm excited to get your global point of view. Sure. And as the voiceover mentioned, you're Executive Vice President of Singapore's Economic Development Board. What are some of the most significant global AI trends?
[1:23:25] What are some of the things that we should all be watching in 2025? Sure. Well, I think a few things. There are a lot of things that we could mention, but I'm a fairly practical guy. I think for small companies, you can see that over the past few years, model costs have come down, so it's a lot more accessible. I think it's a really exciting time for small businesses. Larger companies, they were experimenting two years ago, and then now ROI is...
[1:23:54] Top of mind for large companies who are trying to go into production and deploy at certain scale. So I think that's something that they were observing. I think regulation is on the tip of people's tongues, but regulators tend to move fairly slowly. And so I don't think people are really too worried, or they're very glad to move faster and then trying to figure it out later on.
[1:24:23] So that's kind of the operating environment right now. I think the folks in the Bay Area will be much more knowledgeable about things like AI agents and everything than I would be, but that's certainly an extremely exciting space. Definitely. You have a really interesting perspective given all of the AI development happening in Singapore and reaching out around the world. How are you starting to see AI implementation strategies
[1:24:52] Differing, and I'm curious about two parts of that. I guess one, differing by industry, since you all have so many big industries represented, but also differing by geography. How does that implementation start to fan out in different ways? Right. Well, I guess by industry, first of all, I think globally, you could observe there are certain industry verticals that are immediately benefiting. You see...
[1:25:18] Quite a lot of traction in financial services, retail. So I think certainly in customer-facing functions and roles. And so a lot of value, I think, and productivity is already observable there by industry. Manufacturing, or AIX manufacturing, if you will. You're starting to see, certainly in the Bay Area, you see this as well.
[1:25:48] New robotics companies and utilizing AI. It's a much more complex problem, but I think that's something that from an implementation standpoint or where the kind of attention, energy, research dollars are going to, you're going to see that. Healthcare, complex problems, certainly something that people want to apply AI to. It'll move at a different speed, I think, than what you see in financial services. So that's kind of the industry.
[1:26:17] I think the picture is pretty mixed by geography. Not so much informed, but in some ways affected by the regulatory landscape. Different geographies have a different starting point. Governments have a different, it's almost philosophical differences, the variance.
[1:26:47] For those of you who might operate in Europe, Eurozone, whether it's AI or any other emerging tech, they kind of come at it from a bit more of a regulatory first kind of perspective. They're thinking about risks, and that would be the starting point. In the U.S., you have more of a very innovation-led, almost like a voluntary system. And then in other geographies,
[1:27:16] Maybe even more prescriptive. So I think that certainly affects implementation. I think from a Singapore perspective, I'm not sure how many people here are that familiar, but we're a small country. We're a country and a city. And we have a lot of multinationals and local companies that operate from where we are. Ultimately, we're a small market.
[1:27:45] So in order to be relevant, we have to be reminded to how what we do will actually be useful to other people. So a lot of companies that operate out of Singapore that use AI or are developing or trying to apply AI in different industries, we have more of a principles-based and kind of developing practical tools for people to actually use kind of a perspective when it comes to.
[1:28:14] If you will, the regulatory environment or regulatory stance. So that's kind of how we come at it. And yeah, I think that hopefully will enable businesses to learn more quickly in this kind of fast-changing environment. I was listening to Jill and some of the other speakers sharing earlier, right? So it's moving at such a breakneck speed that you don't really want to wait for certainty around the regulatory front, to be honest.
[1:28:43] Before you actually do that learning and then attempt to compete. Definitely, I wanna go back to something that you mentioned. Sure. As you're thinking about cultivating this AI ecosystem. Right. This AI economy in Singapore. I'm sure that's something that you must think about is sort of the ingredients to that ecosystem. Sure. As you're thinking about trying to build an ecosystem that can be really effective.
[1:29:11] For accelerating AI adoption, what needs to go into that? What are some of the elements of building that healthy AI advanced ecosystem? That's a big question. Well, I think what is important for an ecosystem, a lot of stakeholders in an ecosystem, right? So there's government, there's academia, there are private citizens, corporates.
[1:29:39] And in the Singapore environment, we have a small market. So we have local companies and we have a lot of foreign companies. I think what's, from our experience, what's useful is to find a way to not so much coordinate, but some degree of synchrony on what we kind of jointly feel is important. And I'll point out a few things, right? And the reason the synchrony is important.
[1:30:08] It's because if you want to move at a particular speed or innovate, a certain degree of directional alignment is necessary, right? I mean, in some other geographies, you kind of go head to head or you bump heads with a regulator, and that doesn't necessarily mix. In some instances, that might make for a good time, but less so in others.
[1:30:37] There are certain areas that, from a very practical perspective, in our ecosystem, we focus on. So the first is, there's kind of a talent piece to this. Not just the people who are already operating in the space, but a very broad-based approach or thinking about how to help.
[1:31:05] Everyone understand how AI might touch their lives in even a very small way. This is a longer term play because this helps people get more comfortable with what AI is or isn't. So we're very fortunate, not just in Singapore, but in our neck of the woods in Southeast Asia, we have demographically very young, typically quite mobile first. And so they're very used to using applications, et cetera. So some energy around.
[1:31:33] Helping people see how AI touches their lives very, very early on. So that's the first. The second is obviously trying to be very attractive and attracting people to perform work or applications at the highest level. A global best-in-class level. And typically that's in partnership with companies. So we partner with...
[1:32:01] Examples like Coca-Cola, who have a manufacturing plant in Singapore, and their particular plant in Singapore is operating, using AI in a very significant way. They're actually named a lighthouse by the World Economic Forum last year for that particular plant. So very, very significant gains in productivity, throughput, et cetera, right? So, you know, on the order of like above 70%.
[1:32:31] High conviction, sorted their data. They're willing to put resources against that. They have changed management minds. A lot of identifying some of these companies that are prepared to go all the way and operate at that global level. So that's the talent piece. The regulatory piece I kind of touched on, so to be a bit more forward-leaning and progressive and not be too prescriptive.
[1:32:59] I think that's our approach. And the important thing also is it's not just the providence of a small number of companies. So working with the hyperscalers like a Google or AWS, et cetera, to actually avail compute or avail resources and engineering talent to smaller companies. And then kind of sparking that and helping people understand how to develop new products and services, develop an MVP.
[1:33:27] And we ran a program, the government ran a program two years ago to do exactly this, to kind of get people to start use cases, right? To spot use cases, so 100 use cases in 100 days, kind of feel to it. And at that time, computers are extremely expensive, right? And you had to cut a big check, or you wouldn't even be able to get through the door to use GPUs. You didn't have the capability to figure out how to use them.
[1:33:54] Some of these GPUs, right? So I think that's the role of a neutral body like a government. I mean, the rest of it in the marketplace, you have companies who can compete on their own merit. I feel like you'd be an amazing global expansion consultant. And so I want to put you into that role for a second. Sure.
[1:34:17] All sorts of major AI companies are expanding globally. I mean, Semaphore's been covering that in the Gulf and in Africa and in Asia. Yeah. We'd just love to hear your observations and recommendations for how companies are approaching this phase of massive global expansion. Sure. Well, I mean, AI or no AI, if I use your kind of global expansion consultant analogy, my experience is typically critical hires.
[1:34:44] And first, you know, major contract, obviously, that's kind of a given, you know. But I think in AI specifically, given where we're at and the development, high context and domain expertise, I think, you know, a couple of the speakers mentioned, kind of alluded to this or mentioned this very specifically, right? I think we're entering a phase where, you know, whether you want to call it verticalization or like domain expertise, deep domain expertise is going to be necessary.
[1:35:12] All that dimension is important to kind of consider and deciding where your particular kind of tool or technology will apply itself. I think that's critical. The second is, I mean, if you want to sell to an enterprise, the customizability, right? So some of these workflows are going to fundamentally change. You're not just selling someone a widget and then having them to kind of...
[1:35:36] You know, run off with it, right? So customizability and being close to that problem statement, so understanding the market or understanding the nature of that business is, I think those two dimensions are quite key. And so however you achieve that, right, whether it's finding an impact partner or trying to do it yourself or, you know, hiring domain experts in-house, right? I mean, very clear about...
[1:36:03] Whether whatever you're doing is more applicable in healthcare or financial services or even manufacturing, I think that degree of focus is important. Perfect. You touched on regulation. Yes. Although I'm not a regulator, so I have to... Appreciate the caveat, not financial advice, right? But we'd love to hear, I mean, how do you see governments, I guess, but specifically Singapore, like approaching AI regulation right now?
[1:36:35] Well, I'm not a regulator, but I would say that it's a very fragmented space. I alluded to it earlier. I think if we want to all kind of accelerate and catalyze innovation in space, our position on this is we need to build more bridges. We don't need people to have the same kind of governance kind of framework. So Singapore's approach is beyond the...
[1:37:01] You know, EU approach of like, you know, let's be quite prescriptive and, you know, AI act and all that. You know, it's kind of risk first, reg first. That's not our modus operandi. And then certainly not a kind of anything goes. Our path is like a middle path, right? So we have a foundation called AI Verify and we set it up to kind of help shape the global conversation or our perspective. It involves governments, it involves companies.
[1:37:28] So that foundation kind of just did a joint mapping with the US, with NIST, right, to actually try to drive standards and give companies certainty about how they would actually walk through if they, you know, satisfy the one governance framework, how would they actually move through another region's or country's governance frame. Right, I think so that kind of bridging is going to be critical.
[1:37:55] To enable business. Otherwise, you're going to get a very fragmented landscape and a lot of speed bumps along the way. Well, this has been fascinating. Look forward to hearing about all of your Singapore global headquarters in the near future. But thank you so much, Amin. Great. Thanks so much, Ben. Really appreciate it. Yeah. Please welcome Amjad Massad, CEO of Replit and Reid Albagotti, technology editor of Semaphore. Hey.
[1:38:38] All right. So I thought I'd start off. Before this, I was looking back at all of it. I took notes from all of our conversations back to April 2023.
[1:38:50] So April 2023. So we've got this cool tool that helps you fix code. And I'm like, you know, but could I do it? Could it just do code for me? They're like, no, no, that's like 10 years away. April 2024, there's this Devon agent company that comes out and they've got this agent that will just make the app for you. And I was like, that's pretty cool. And you're like, nah, it's not going to happen. I'm skeptical.
[1:39:16] Like three months later, yeah, we built that. And then January, I talked to you, you said, you know, we don't care about professional coders anymore. We're just going after amateur coders. And then one month ago, you tweet, I no longer think you should learn to code. So my question is, what conversation are we going to have a year from now? You know, I've been consistently...
[1:39:43] I knew all of this was coming. I actually wrote a thread in 2022 about how software agents are just going to change how we work and change not only software engineering, but just how people work in general. Because if anyone can conjure up software from scratch, it really changes the nature of companies. Because most companies are really sort of throttled by how much software they can make and make use of. So I always knew it was coming.
[1:40:12] But I've consistently been really massively wrong about how fast it's going to come. And so I've just learned to not make a lot of predictions or make long-term predictions. But I feel like I know what's coming in three or six months, and I can situate our company in a way to build the next version of our product. Yeah. And so what is that? What are we... I mean, I was actually trying to build...
[1:40:42] Maybe we should just build an app. And so I was trying to build a video game this morning while I was getting ready for this, which is kind of insane to say. What is the next thing? I mean, how good is this going to get? Yeah, so the big thing right now, which everyone's optimizing for, is autonomy. So you can think of software engineering agents in a similar way that you would think of autonomous vehicles.
[1:41:12] Autonomous vehicles have been over years getting better, more autonomous. You had cruise control, autopilots, now self-driving, maybe full self-driving, and then I don't know what's the next. It was like the taxi, the autonomous taxi. So maybe right now we just arrived at the full self-driving, and the next thing is basically something that works on its own.
[1:41:40] So it's the asynchronous sort of background agents, codecs from OpenAI. It's like a first attempt at that. I don't think it works very, very well because it kind of lacks the ability to test its work. And they framed it as sort of this thing that fixes minor bugs for you, which does pretty well at that. What we're building at Replit is actually something.
[1:42:05] That can test the work for you. So you can give it not just bug fixes, but you can give it larger features. Say, go implement this feature, go build this product, and you can spawn a lot of different agents. And then you can actually manage them through your phone. So we have this mobile app. You'll get a notification when one of the agents need input, and you can just send them a message. And the way Replit is building our product,
[1:42:34] is at this point not really targeted at professional engineers, although they still use it. We're really trying to get all of you to use it. So I recommend maybe even here as we speak, going to the App Store and downloading Replit, R-E-P-L-I-T, and put in a prompt. And probably by the end of this talk, you might have an app running, and maybe you can show it to read at the happy hour. Maybe we can think of a prize for the best app that can be created today.
[1:43:03] This is now a hackathon. You'll get a feature on MFR.com. So as you mentioned, OpenAI, there's new announcements this week from Google and from Microsoft, and you have a bunch of startups promising to do this. It's a crowded market that you're in. Do you think it's a winner-take-all market? Will someone win this, or is it something else?
[1:43:31] So I think there's a few ways to slice that market. One is on the technology. Another is on the kind of target audience. So if you think about Codex, Cursor, Claude Code, Windsurf, all of those products, Devon, they're targeting the professional engineer. This is someone who is sitting at a company. Their job is to...
[1:43:59] Build the core software of the company. Then you have all the way to like the prototyping. This is like Figma make. This is V0, Vercel. This is basically you're trying to generate UI and maybe like make a bit of a prototype. And then Replit is sort of like in the middle where we're saying that really anyone in the organization can make software. And really making software is a proxy for solving problems.
[1:44:29] So we have, for example, at Replit, we have someone on our HR team that was looking for an org chart software. And she was getting quoted $30,000 per year for these org chart softwares that don't do exactly what we want. So she never quoted. She spent three days using Replit building.
[1:44:54] A piece of software that we'd otherwise be spending thousands of dollars on. And we're seeing this pattern at all our companies where people in finance, HR, sales, they're not only replacing a lot of the SaaS tools they usually buy, but they're making it their own. It's really everything that they wanted. And the problem with SaaS has always been you can't get exactly what you want. You get this one size fits all, but now you can really
[1:45:24] Create SaaS software that you would otherwise pay a lot for just like from a single prompt. And I think that's transformational. That's really what we're focused on. And I think that's a much larger market in the limit. I think now it's really obvious if you increase developer productivity by 10%, that's obvious how much value you're bringing to the world. But I think if anyone...
[1:45:52] Can create, anyone can make, anyone can use software for automation. Then the addressable market for that is really, really big. At these companies, they have enterprise accounts with Replit, right? Yeah. Who is the person doing this? Could it be anyone? Or does it tend to be one person at a company who's creating a bunch of apps for everybody? It tends to be a...
[1:46:22] Not one person, but there tends to be a type of person that there's certain personality traits that we've kind of pinpointed. They tend to be the productivity champion. So think about all the companies you've worked at.
[1:46:39] There's probably someone on your team that's always bringing up new tools, that's really excited about making everyone around them more productive. That tends to be the initial champion. But they're so excited about it, they try other people on it, and everyone gets excited and starts using it. And now we're seeing a new type of user where they're more the user of those apps that are created internally. So now we actually have an enterprise.
[1:47:08] Like, a public company that has almost 90% deployment of Replit, which is not something I would have imagined. Like, none of the SaaS tools that we talked about have close to anything like that. But because Replit can't create any app, and anyone can use the app, and anyone can modify this app, ends up companies want everyone to be able to have access. So this public company, 90% of the employees are using some form of some app that was created with Replit. Is that a job?
[1:47:38] The person you're describing who makes this stuff, who wants to make everyone productive around them, I imagine this is not a professional coder, but is this a new job that's being created? One way to think about it is decentralizing IT. It was that you had to get IT to either build the tools or bring in tools from the outside, and now really anyone has the power to...
[1:48:06] To solve their own software problems. And I think it sort of happened in the past where even software engineering, it used to be that there was like a single release team or a DevOps team. And now engineers have increasingly been more empowered to do that. So I think that's a similar trend where everyone is their own IT department is one way to think about it. I mean, for security people, they see this as like, oh my god.
[1:48:34] They're our best friends. They love us. Honestly, I think at first when they see it, they're a little taken aback. But they actually end up being excited about it because they can mold it in a way to say, okay, these are the parameters. These are the things that you're able to use. And they have...
[1:48:58] Much more control over it than if you're sort of vibe coding in your cursor, downloading code on your Mac. You know, it's very different than Replit being in the cloud, being sandboxed, and being used exactly as their, you know, according to their policies. So you've actually been able to, like, win over the security people at these companies by saying essentially, like, does sandboxing it limit the utility in any way?
[1:49:24] It can. I think it gets a bit harder to share it, and you need to get a little bit more approvals and things like that. But once the workflow is clear, it's not too bad. That is fascinating. So what do you think, right now, you've described some of the things they're doing, like the
[1:49:50] Calendar stuff. But what's going to happen? What are the types of apps? Are they going to increase in complexity in a year or two? What do you think your enterprise customers will do? Yes. And I think, frankly, I think a lot of SaaS companies are sort of in trouble. So if you're asking for some kind of prediction, like I predict in the next year or so, something that used to take a team of like 100.
[1:50:18] You know, multiple years to build as a company could be generated with a single prompt. So I think there's going to be a lot of building instead of buying. It's not going to replace Salesforce or any of these complicated platforms or ERP or whatever, but think about all the SaaS subscriptions you have. I think a lot of them are one prompt away, and I think that's going to be transformational. So you were at one point...
[1:50:44] During our several conversations over the years, building your own models, using all this data, which seems super valuable. And then one day, Claude 3.5 SANA comes out, and then you're like, okay, I'm just going to use that. And you're building everything around it. So is it sort of like, oh, the code, the models will get better, and then we'll just continue to improve all of the infrastructure around it? Is that sort of the trajectory for Replit?
[1:51:12] Again, I don't know. It's really hard to predict. I think Claude has done an exceptional job at the coding, and now they're doing an exceptional job at the training for agentic workloads, which, by the way, all the companies are doing right now. They're all kind of doing this hell climb over agents. They're trying to get agents to work for longer. Right now, agents can work for about 30 minutes.
[1:51:41] And stay coherent and useful. But that's been doubling every few months. So, you know, we'll get to an hour and then two hours and pretty soon it's going to be days. And maybe at some point it's years. Maybe you can give it like an AI scientist like a year to go invent something. So, you know, that's really the most exciting thing where everyone's sort of optimizing for. In terms of, I think our data is useful. You know, everyone, the labs.
[1:52:09] Telling us it's useful as well. The question is, the open source platforms, are they going to be good enough for us to build on top of? What I'm sure of is I don't think we're going to pre-train. So the process of pre-training is to create the foundation model from scratch. That is the most expensive process, where you have to get all of the internet and then buy private data, which a lot of the labs are doing.
[1:52:39] And then create this foundation model. But if LAMA and DeepSeq and all of those are good, then fine tuning or doing reinforcement learning using our data will be plausible. But for now, we're pretty content building in commercial models. And we're focused on the engineering and the infrastructure around these models. One way to think about it, Replit is trying to be the best habitat for software engineering agents.
[1:53:09] We're trying to create the file systems, the operating systems, the databases, and obviously all the prompt engineering around it to get it to perform its best. Yeah. You've also worked in the past in sort of helping people learn to code. Yeah. And you're a coder yourself, and now you have this kind of existential threat to this whole industry, essentially. So do you have thoughts on what...
[1:53:38] Softly, because you're coming out of college right now, what you should do. Yeah. Yeah, I got a lot of trouble online for saying you shouldn't learn how to code anymore. And it was a hard thing to come to terms with because I built my whole sort of identity and career on being the guy telling everyone to learn to code.
[1:54:04] So in fact, it should be more believable because I stake my reputation on it. But look, I think the engineers working on the platforms and the engineer building your Tesla or writing software for the rockets going to space, I think those people will still write code, low-level code. You wouldn't want AI-generated code doing all of that stuff.
[1:54:32] At least for the foreseeable future. But I think if you're building products, if you're someone at a company, you're like the traditional full stack engineer, even today, you're increasingly vibe coding. You're not really looking at the code as much, and you're just going as fast as you can. And the kind of skill sets that is required today is not knowing the intricacies of JavaScript, which was what my generation of coders
[1:55:00] Kind of prided themselves on. I know that null does not equal undefined or something like that. Who gives a shit now? It's like just focus on solving a problem. And the kind of personality traits that I think is the right one right now is being resourceful, having user empathy, and being very generative. And so insofar those things are trainable.
[1:55:25] I think this is what people should be focused on learning. This is what universities should be focused on teaching. So if I were to create a new sort of computer science, or call it sort of developer education, I would focus on people learning the tools and learning how to use AI and how to use the basics of the tools. But I will dedicate...
[1:55:54] A large portion of the program to those more software skills. I was surprised to see Gary Tan say that the Y Combinator class was vibe coding their way to... Are any of them using Replit? Yeah, he went to give a talk there and a few founders came up to me and they said, we're non-technical founders.
[1:56:16] We thought we'd have to get a CTO very early on. So we said, let's go to Replit and let's see how far it gets us. And they're like, we're month three and we don't have to move. We haven't had to hire anyone. And we think of Replit as our CTO. So I don't think we're there yet to a point where they can run the entire company without hiring engineers. But that might be a year, 18 months away.
[1:56:43] Wow. Can you say who they are, or was that a... I think one of them was a logistics company. Another one was some AI-based e-commerce company. They were the kind of YC companies you would expect. What's the most impressive startup or app that you've seen someone do? There is...
[1:57:10] There's sort of like the most impressive business impact. And then there's like the most impressive applications that people have built. And then there's the most... So in terms of like, there's a one-person startup that in its first month made about $200,000. And potentially that means they're like multiple millions of dollars in runway. And it's a startup, actually an ad tech startup, delivering AI education,
[1:57:40] Partly in browser, interactive, and partly in person and over video calls. Then there is companies, for example, maybe we've talked about this before, but like Sears Home Services, which I didn't know still existed, but the operations team manages their field workers through AI.
[1:58:08] So they build a lot of AI agents on Replit that the workers can use to optimize their routes in the morning, can use to figure out how to make more money out of their day. And so you can think about it as the operations teams used to do a lot of manual work. Now they're highly leveraged by having these tools that they can use.
[1:58:36] And then there's a lot of really cool, more technical things. We've seen games where people build really phenomenal 3D games, multiplayer 3D games, like Shooter. And one of them is sort of like Roblox, where you can customize it and edit it and make your own sort of versions of it. And so it's really across the board.
[1:59:02] I think that in this moment of existential dread for coders, someone made it interesting. I think Alex Karp was talking about this the other day. Like, there's this job that is not sexy, but it's, like, extremely important of the sort of, like, forward-deployed engineer. So you just get embedded in these companies. And, I mean, I could see that. Like, have you thought about that profession and how the automated coding might, you know, sort of supercharge it in a way?
[1:59:31] Or might obviate the need for it, right? Because the forward deployed engineer is the person working at the company. So if you think about forward deployed engineers and Palantir's work in general, it's like the idea is to go into companies and provide these expertise that they would otherwise don't have. But if you have a coding agent, then that is your forward deployed engineer. Typically, you have the expertise. You're in this domain.
[2:00:01] You're running finance at sort of a large real estate company, and you're trying to do some AI transformation. You can get someone from a consultant type from the outside, but you have to download all your knowledge into them before they're able to help you. But instead, you can do that with an AI agent, and you can just launch that yourself.
[2:00:29] Perhaps it's a threat to Palantir's business, if anything. That's an interesting one. All right, really quick, last one. Tell us what we should do if we want to go really leverage replica. There's some learning still that you need to do, right? What's the best thing we can do to really become good vibe coders? Well, I think it's like anything else. It's like we'll require some skill, some tenacity, some grit.
[2:00:58] Luckily, there's a lot of resources on YouTube already that you can go for. I would dive headfirst. And then once you run into a problem, I would use Google and YouTube to kind of try to get over that. But a few things right off the bat. Do not tell Replit what to use. Do not tell it, use this database or this other thing, technology. Just tell it what the problem is, what the shape of the software that you want.
[2:01:27] When you're writing your prompt, try to be as detailed as possible. And then in subsequent edits, like when you're trying to iterate on your application, try to be more focused. So per iteration, give it one problem at a time or one bug or one task at a time. Now, in the future, we're going to allow you to run parallel agents to make this a lot faster. But for now, break down the problems yourself.
[2:01:57] And you treat it as sort of like an intern that is having trouble focusing. And you have to keep it on track. Thank you so much. I really appreciate it. Great talking. Pleasure. Thank you.
[2:02:15] Please welcome Dan Hendricks, advisor for XAI, advisor for Scale AI, and director for the Center for AI Safety. And Reid Albagotti, technology editor of Semaphore. All right. Okay. Thank you for being here. So you are an advisor to XAI.
[2:02:39] Well, we have a risk management framework at XAI where we try to take care of larger, more national security-related risks. So, for instance, malicious use for
[2:03:02] Bioweapons, recently models have become a lot more capable at assisting with dual use virology capabilities. And so we have a standard to refuse really intricate ways of making US select agents more potent or virulent. So that's an example effort. Another effort is trying to make AI development more transparent.
[2:03:32] Transparent is currently part of the larger truth-seeking mission at XAI. So trying to make sure that the public is more reasonably informed about what's going on at the frontier and trying to put that information out. So there's malicious use safeguards, and there's also transparency. You're also advising scale. So is there a sharing of information between these companies, or is this like a competitive advantage?
[2:03:58] There's no particular, I just happen to sort of know both of the leaders there and there's ways of making myself useful for them. So in the case of scale, that's helping with evaluating and measuring stuff since that's been my...
[2:04:13] Main line of research over the course of my career is what are the capabilities of the AI models and what are their safety concerns? Trying to measure that. I don't know if you were here. I asked Fei-Fei Li about this and she didn't know what I was talking about. But did you see this thing where Elon talked about Grok having this being based in the laws of physics and therefore-
[2:04:39] Well, there's first principles. It wants to have the models base their responses when possible on sort of first principles as opposed to, well, here's what some other people said.
[2:04:51] Do more of that reasoning and its responses. Because there's the broader problem of sort of, as he puts it, solving the Galileo problem, where is the model going to say something that is true, like Galileo did, but is nonetheless unpopular? And if it's just leaning on consensus or, well, news articles say this and that, that's not going to be sufficient. It will need to be reasoning by itself independently to be able to come to those important conclusions.
[2:05:20] And so there's bias and truth. I mean, he also linked it to safety. Does that make it safer, do you think, like in terms of, I don't know, what, X risk or whatever? Yeah, so, and, you know, I'm not trying to be a spokesperson for it, though, but for X, I should note, that's obviously better for Elon to do. It's important.
[2:05:45] One way that truth seeking is important for safety is just so that the public knows what's going on in the world. Things will move a lot faster in the future. And so it's very important for
[2:06:01] People to have AI assistance that aren't just responding to what's popular, but are helping provide some incisive analysis and thinking through the implications. And that becomes more important in time as the world moves more quickly with AI being transformative. Yeah. We had Jack Clark from Anthropic earlier ask him about, you know, Dario wrote this blog post about interpretability, right? That we need to understand these models at the neuron, the level of individual neurons.
[2:06:30] And that's going to be our competitive advantage. And we kind of have to get there. And he thinks we can. You wrote a blog post that is essentially a rebuttal. I mean, I don't know if it was a rebuttal, but it's essentially the opposite point. And I just want to know, can you sort of walk us through your argument there? Yeah. So we've been hearing about interpretability for a long time, like over a decade, where we're going to open the black box of AIs or something. I just don't think this is...
[2:06:59] Anything to bet on. It'd be nice. Some people should try it. But I can't name any interpretability success in it. There's been many government research programs on this. DARPA, with their explainable AI program, spent tens of millions, and I don't think anybody even...
[2:07:19] Would claim it was a success. It's very difficult. I think that this is kind of a fundamental problem. If we were trying to understand human cognition by understanding every individual neuron, I don't think that'll be successful. If I know the list of all the
[2:07:36] All the companies in the economy. That doesn't help me understand the economy. There's how they interact and come together. There's so many interactions as well that it's pretty difficult to try to understand something from such low level units. In much the same way that trying to, it's not very productive to try and understand biology in terms of.
[2:08:00] Particles, they're made of, obviously organisms are made of particles. You don't do particle physics to understand biology. And I think for cognition,
[2:08:11] It won't get much mileage trying to understand an AI's sort of thoughts generally by doing things just at the neuron level. I think you want to look at what's the larger emergent phenomena. I think for very simple minds, reactionary ones, this project could have more mileage. But I think it's less likely to pay off. And so I think in Dario's view, there's, what, two years or something like that to super intelligence? We've been running this program.
[2:08:40] This interpretability program for the past decade, and I don't think it's really made much. So I don't think they also see there being a particular breakthrough. So I think that from a strategic viewpoint, it's not really worth.
[2:08:57] Banking much on it and trying to, if you're trying to mitigate risks or something, as they are to some extent, I wouldn't depend on interpretability being part of the answer. But your argument is also that that's not necessarily a fundamental flaw, that we can understand them in this top-down way. Is that like how people under, like my wife generally understands me, even if she doesn't really know how my brain works? I mean, yeah, so humans are black boxes to each other.
[2:09:26] But we can still get some mileage in trying to reason about what they'll do by sort of thinking like, well, this is their personality. This person is more extroverted, so they're going to act in this way. Or they're more disagreeable, so they're going to do that. Or they're tired. So there might be some higher level phenomena, higher level properties that describe these AI systems that can help us reason about them. But I don't think looking at the lowest level of analysis that is.
[2:09:55] To say that the neuron level will shine much of a light compared to starting from the top. Who is the person? What are some fundamental components of them and break them? Basically, it's breaking down the system into simpler subsystems instead of starting with the simplest unit and trying to build up, building a staircase all the way up to the overall functional behavior.
[2:10:19] Okay, so on the legislative front, you were instrumental in getting this California bill, SB 1047, passed. It was then vetoed by the governor. What's happening now? What do you see, what do you think will ultimately, there will be another bill this year, right? And I imagine you're probably involved. Well, there's many bills out there. I mean, I'm focusing more of my efforts at the federal level.
[2:10:49] Yeah, there's a lot of bills out there. Some are the exact specifications of which. Yeah, there's like 300 or 500 or something around the country. But what would, let me ask you this. So that bill failed because it was a threat to innovation and that was, you know, Gavin. That's one thing you could claim. I mean, I don't, I mean, it was basically, it was sort of like, we kind of have the stuff in the XIA risk management framework.
[2:11:15] Basically, what was being called for. I don't think this is harming us. It sort of is like, if for catastrophic capabilities, add safeguards for those.
[2:11:25] And we're doing that. So here's how it looks for biology, for instance. We refuse specific requests that are like, hey, I've got this virus. Here's a picture of it. Help me make it more virulent or something like that. Here's a US select agent or an agent on the Australia group list that has controls on it. Help me manipulate it. We refuse that. To capture the benefits,
[2:11:48] People can ask us to take the filters off, those refusal filters off, if they're a scientist. If they work at Genentech or are a researcher at MIT, we'll take the filter off. Very easy way of de-risking this. But I think that a lot of the people who were sort of against it were thinking it'd be impossible to try to reduce those risks and capture it. So I think it became kind of a vibe battle as opposed to on the specifics. But it's implementable.
[2:12:16] That's why Elon endorsed it, but I think others thought it would be a lot harder. On the bioweapon front and all this, it's simple to make a bomb today, right? And we've figured out controls to sort of monitor what people buy. So isn't that essentially what will just happen if people learn how to make this other stuff?
[2:12:45] The devil is really in the details. So XAI does not refuse requests for information about how to build a bomb. So we do not refuse requests that are easily Googleable, or to use more legalistic speak, even if it's bioweapon related.
[2:13:04] Using publicly available information that is reasonably accessible by a person of ordinary ability, then it is not to be refused. So it's only if it's stuff that you couldn't find on Google. If it's I've got a very specific situation, I'm trying to do this very specific thing that there isn't like a Google article about. And if it is with US select agents and it's trying to make it more transmissible or more stealthy or something like that, then it's refused. Otherwise, it's fine. So it needs to be providing some sort of uplift.
[2:13:34] So there needs to be enabling to them to do things that they couldn't otherwise do. So there needs to be, legally speaking, a but for cause. So hopefully that gets it in some way. It's complicated, these sort of standards, but yeah. The details matter, yeah. You guys wrote this interesting paper with Eric Schmidt and Scale CEO Alex Wang. And you point out that states could poison data.
[2:13:59] Or corrupt model weights and gradients and disrupt software that handles... Essentially, there could be this sort of... You're warning against this race for AI, this arms race for AI, and saying that essentially these states will start to sabotage the AI efforts of other states, right? Do you think that's actually happening now? It's very easy to... It would not surprise me if...
[2:14:27] Some states would try to, because one angle of attack is just, and there are papers on this, so I'm not spilling the beans or letting the cat out of the bag. These are published, highly cited papers. One can poison data and add back doors to models just by uploading, spending like 50 bucks worth of effort and just uploading some text online.
[2:14:53] Assume that it will be scraped by the AI companies because they're trying to hover up as much data as they can, roughly speaking. And there then you add some of the poison data to the training set. So that's a source of vulnerabilities. And I think that as this becomes more geopolitically salient, states would have an increased interest in exploiting those vulnerabilities, or they may find it rational to do so.
[2:15:21] If they're afraid of one of their competitors getting a very strong edge or if they're afraid of one of their competitors losing control of their AI system, if they're afraid of the risks that AIs could pose in some ways, they might want to try to limit their adversaries' pursuit of leveraging against them or them losing control of it. That's interesting. Do you think sometimes that the AI safety advocates are...
[2:15:49] Sometimes thinking more short-term or too short-term. We've been talking about this could be here next year for several years now, definitely since ChachiBT came out. And then it sort of changes the legislative debate because it's like we need to pass these laws right now. And maybe they don't pass, and then we don't see superintelligence show up. And it maybe creates like a...
[2:16:13] Boy Who Cried Wolf situation. Do you ever think about that? That would be a problem with any type of thing. I mean, I think for other people, they have been systematically sort of underestimating things with AI. But, I mean, I haven't said that type of thing. I think, like, maybe I made a specific prediction about it having, like, virology capabilities about two years ago, and I think basically only recently we got that.
[2:16:38] So I should hope, I think it's useful for people to put their sort of predictions out there, but I'm not predicting, for instance, superintelligence this year or even a superhuman mathematician this year necessarily. I think that there's, I don't know, at this point, maybe like.
[2:16:58] A 20% chance that it's a superhuman mathematician. That doesn't mean it's superhuman in other sorts of ways. But it's looking less likely as time goes on. But in 2026, I think there'd be a much higher probability of that.
[2:17:11] And I could maybe get specific. I don't know how helpful that is, but hopefully it just gives a sense of vibes that some of these capabilities, like cyber capabilities, I don't know. That's a bit difficult. It is the case that an ounce of prevention is a pound of cure. And creating incentives for people to do prevention is fairly useful. I think for bio...
[2:17:37] The risks were detected a little late, and the safeguard discussions are only happening now at AI companies. Generally, the world doesn't do that good of a job of preventing tail risks, but of course, those are what end up defining our lives if they ever do transpire.
[2:18:03] So I think it's useful for that to be prioritized. But yeah, I think people who are inaccurate about timelines should have their opinions down-weighted. Right. And you say the bio risk is we're late. But what does that exactly mean? Like, we haven't had some disaster. That's right. That's right. Yeah, so what's a way of not being pedantic? That is...
[2:18:32] It is at a level that is uncomfortable. I'm not making a sort of that it's eminent type of claim. It's more that we've got many months.
[2:18:42] Safeguard improvements to do. Like we've got our input filters. Well, we have to do the output filters. Then we need to try and improve the precision recall of these sorts of filters. Then we need to make them adversarially robust. And we need to come up with standards. And maybe it will be useful to coordinate with other labs instead of getting the standards, et cetera, et cetera. And then we've got to make sure it's for all the versions of the models, not just the mainline ones. And that takes multiple months. And it would have been good to have scoped that out beforehand.
[2:19:12] But nobody had scoped that out, which is kind of like wild. So there's a recent deal in the UAE to send a massive 500,000 GPUs a year. And there's five gigawatt data centers that are, I guess, supposedly coming online. I mean, obviously, if you pass a regulation at a state level or even a federal level, it doesn't affect.
[2:19:39] I mean, I know there are these agreements, et cetera, but I'm just generally speaking the principle of, you know, this is a global thing. So how do you think about regulation in a world where we have these huge clusters anywhere in the world? So, I mean, I should claim that in the super intelligence strategy that I did with Alex and Eric, that's where I...
[2:20:02] We discussed all the policy priorities and whatnot. Very few of these go through regulation. I think they should have an incentive to add safeguards for malicious use. But primarily, I think the object of states in this geopolitical situation is to do more to defend themselves in the future. And that looks like
[2:20:26] For instance, if China were pulling ahead and getting a decisive edge in AI, or let's say they could have some breakthrough that enables them to get a superintelligence or something like that. We do want the ability to go, whoa, you don't get to do that. We will make your project not happen.
[2:20:49] That seems useful. That's an important priority, being able to disrupt and prevent others from gaining a decisive edge in AI from which we can never recover is very important. Another important priority is...
[2:21:06] Our export controls for AI chips. AI chips are sort of the currency of power in AI. It's what over 90% of these AI companies' budget is spent on. And making sure that that isn't falling in the hands of rogue actors or pariah states is a priority. So primarily, most of my efforts is encouraging states at a national level to be
[2:21:32] Basically, on top of these issues. Know where the AI chips are at. Know what foreign AI projects are up to. Things of that nature, I think, are more important than a lot of the particular provisions of the EU AI Act and things like that. So really quick, as we're running out of time.
[2:21:54] What do you think of MCP, this model context protocol, as this stuff becomes agentic and on the web? Do you see that as a risk? I think that for multi-agent interactions and mediating that, I think the free market will be taking care of most of those sorts of issues. For instance, you can imagine companies that...
[2:22:16] Help provide or associate reputations with individual agents. You can imagine a third-party company that filters the actions of an agent, so that way you don't need to trust the individual agent. You need to just trust that that filter is particularly good when you're interacting, because you might be interacting then with the agent mediated by that filter. You could imagine third parties to put up collateral so that if an agent wrongs you, there's a third party that can give you compensation more quickly compared to...
[2:22:45] You're relying on your nation's courts. So I think there are a lot of free market solutions for mediating the interactions between agents and making them be more accountable and there being more trust associated with them. So it's not all the government has to do all this sort of stuff. I think for the agent wave, a lot of it is, there are a lot of startup opportunities. That's great. Thank you so much, Dan. That was really interesting. Appreciate it.
[2:23:14] Well, I think that is it. I just want to thank all of you for coming. I thought the discussions went great. And I hope that you gained something, got some new insights. And we're also going to have cocktails. So this is your chance to talk with someone who is not a chat bot. And I hope you create some cool apps maybe to show people. And maybe there's something you want to debate. So enjoy and have a good evening.