[0:00:07] Welcome. Today's session's called AI and Leviathan because AI implicates the government as well as technology and society. So that's why I'm happy to have Jack Clark here, co-founder of Anthropic, who also heads their policy team. So I guess my first question for you is, what is something that SF techies do not understand about DC?
[0:00:31] I think that the DC ecosystem is like a lot more awake to AI than people think and in some parts a lot more sophisticated. So years ago, I think people working in AI really wanted to sort of wake up the government and now it's a bit like you wish on the monkey's paw and like one of the fingers curl and then four other fingers do the same thing. The government is now extremely awake and kind of tooled up in interesting ways.
[0:00:58] We should expect more opinionated kind of technical rulemaking to come sooner than people expect. And I think it's very fashionable to assume that the government's asleep on a lot of this stuff. But there are pockets that are paying very, very close attention now, which means the stakes of what we do and other technologists working in policy has gone way up because the possibility for random things to happen due to what you say has increased a lot as well. So they should go back to sleep.
[0:01:28] It's interesting you say that. I agree there's definitely pockets, but a different company, OpenAI, when Sam Altman testified famously in the Senate, he said, more or less, we're building super intelligence that could be here this decade. And Senator Marshall Blackburn said, what does that mean for music royalties? So I think there's still a seriously, but not literally gap here. So how does that get communicated? Yeah, I think this is also a challenge with...
[0:01:56] you know press and policy makers is that people don't take these AI companies like OpenAI or Anthropic or DeepMind literally when they say they're building AGI and they're like oh cool you're building like advanced technology you know I'm building technology that I think will like change the contours of the state and the security environment and then they say what does that mean for jobs you're like well lots of things it means lots of things for jobs as well as everything else so the question is do you want to like
[0:02:26] Wake up the government to something like AGI when I think most technologists don't even have a shared understanding or agreement on what that is. And would that be a good idea? So what is AGI? I think AGI, the simplest way I've thought about it is it's kind of like a country in a box.
[0:02:49] If you were to be able to hand a magic piece of paper to a country that was like, carry out a research project in this area or do some analytical work of this form, the country would bring to bear resources and people and processes to deliver you an answer that was really, really, really good. And I think AGI looks a lot like that, like synthetic, miniaturized kind of countries that anyone can access. And if you...
[0:03:16] Sit with that concept for a while. It's actually completely crazy. I mean, companies are not meant to build like miniature synthetic countries. Governments would be extremely interested if I was going to DC and saying, I am building a technology that just creates lots and lots of countries. How's that? And I think that that like illustrates the challenge of this technology is wrapping your head around that concept is, I think, quite counterintuitive. But if we end up being right.
[0:03:45] That's where this ends up in most people's, I think, conception of getting the technology right and succeeding. That should at least wake up some of the anti-immigration folks, perhaps. I think maybe the message would be more received if when you go to the Senate or wherever and you say, you know, we are building the ultimate cyber weapon and along the way we're going to improve your search experience.
[0:04:16] A commonly good business tactic is to say stuff like that. And I think this illustrates the challenges. You're trying to build technology that just like delivers lots of value today, is helpful to people today. In the back of your head, you keep on thinking, if we keep succeeding, I'm going to need to have the, oh, it's also a crazy cyber weapon talk. But if you lead with that, I think you risk creating an environment that's probably just bad for industry and bad for the sector.
[0:04:44] But if you ignore that completely, then when the thing starts like popping out cyber weapons, people will be somewhat surprised at what's happening. And I find myself just trying to be frank and also confused in public with these people. And then they say, that's a really confusing issue. What are you going to do about it? And I'm like, it is very confusing. Please join the club. Okay, so what are we going to do about it? So I don't know, maybe you can speak a little bit to your own internal timelines.
[0:05:15] But suppose you had AGI next year. Are we ready for that? Paint a vivid picture. What does that mean for the world? What does that mean for governments? Are we ready? I'll break this into two. I'll do the next year scenario and then how it might go better. So if we had AGI next year, you'd have something which was able to do arbitrary amounts of scientific discovery. It was able to do arbitrary work in domains like cyber defense or offense.
[0:05:45] And I think you'd be starting to think about how you, almost like you become the Federal Reserve and you're setting the interest rate of how much intelligence to go into the world. You'd need to figure out some release process because if you just turned it on, loads of chaotic and confusing things would happen. I think if it happened next year, we don't really have the institutions or conceptual frameworks to make this go well at all. And so this is one area where I'm pleased that we may not hit that.
[0:06:15] Target next year. I don't think any of the companies will. But how you'd make this go better is you kind of need to build deliberative capacity in government to think about this stuff. And a few years ago, that seemed like a completely crazy possibility. But then a year ago, we had the formation of the first AI safety institute in the UK. And now there's a bunch of, you know, machine learning PhDs and engineers employed by the British government who do testing for
[0:06:43] Cyber defense, cyber weapons, cyber offense, bio, chem, all kinds of things. And they're actually very, very sophisticated and good. So we've ended up on a Pareto frontier improvement in the ability for the government to know about this stuff. And if we build more of those institutions, we'll have the kind of system that we need to make this stuff go better. The question is if we can do that in time. Right. And it also is sort of premised on a degree of centralization.
[0:07:13] To be able to turn those interest rate knobs. And at least in the recent history, we've seen now dozens, maybe hundreds soon open weight models, alternative models at GPT-4 levels. And that's maybe a two-year lag. And so is it sort of like once AGI is achieved internally, there's a sort of a stop clock that goes on where this becomes democratized? Because obviously the set of issues when you have something
[0:07:43] Behind a closed API versus accessible to anybody are very different and the set of solutions are also different. Yeah, I think the thing we've learned at Anthropic and before Anthropic I worked at OpenAI from 2016 is whenever you invent anything, an invisible clock starts counting down to zero and that thing gets published on archive or released by someone, usually on the order of six months to two years after you've figured it out. So the challenge here is you can't just...
[0:08:08] You know, pause this stuff. You can't stop this stuff. All you can really do is figure out what information you kind of rapidly start emitting and how you make that useful. So I think the way I think of it is we're like a new generation of companies building something akin to nuclear power. We have to build like new reactors, learn how to make them safe and deploy them and do that relatively rapidly because really, really small, cheaper reactors are definitely coming right behind us.
[0:08:37] And I think the onus is on us to learn a lot through deployment and share what we learn, because that's going to increase the chance of none of the smaller ones going bang after us. And I think if you had something like an AI-driven disaster, it's the kind of thing that nerfs the whole industry. And that's, to me, one of the most important things to avoid in the next few years, because then you end up with...
[0:09:01] Extremely bad policy hangover that means you never get the benefits of this technology at the scale you'd want, like the nuclear power sector, which is only just recovering here today. So we need to walk this tightrope really carefully. Right, so that brings up regulation. SP 1047 has passed. What's that? You tell me. So originally Anthropic had a support of amendment.
[0:09:30] Many of those amendments were adopted. Some of them weren't. Some of them were adopted in maybe a different way than you have ideally had. Where does Anthropic now fall on this? And imagine you had a line item veto or maybe an enhanced line item veto where you can sort of add things here and there and not change the whole thing but make a better version. So the position we took with the latter was we thought it had a core which was
[0:09:58] Basically reasonable. It's saying if you're trying to train a model that costs $100 million, you should document your security procedures so it doesn't get stolen and some of the safety testing you're doing so you don't have a proverbial kind of reactor meltdown. There was a huge amount of other stuff attached to the bill that didn't seem reasonable at all and in fact seemed like either a side quest or an example of kind of overreach of the government. They went through some amendments and then the...
[0:10:27] The need for companies to kind of document what they're doing remains, but there are still some additional things which, as we said in our letter following the amendments, are really annoying and confusing. They killed something called the frontier model division, and instead they embedded a load of those functions in something called government operations in the Sacramento kind of bureaucracy.
[0:10:48] And I ended up looking like Charlie from Always Sunny in Philadelphia, where I'd graphed out the different responsibilities of GovOps and auditors and the companies and had this crazy diagram. You could just get rid of that, because I think whenever you see legislation that has wildly complicated things that you need to go into a sort of fugue state to understand, you probably don't have something especially pragmatic.
[0:11:13] However, Anthropic spent the time to kind of give her opinion on this because we think these safety issues may eventually become real and the onus is on AI companies to not just blanket reject this stuff, but try and give an earnest kind of intellectual account of what they actually think about the bill. And it hasn't won us a ton of friends, but I think we've been very consistent with what we ultimately believe is going to be important. What do you think is the main crux?
[0:11:42] In the debate over SP1047 and just AI safety regulation in general, my perception is that it's sort of billed as a values debate. Some people want openness and some people are maybe high on neuroticism or something like that versus maybe the real crux is just where is this technology heading? How soon? Is scaling hitting a ceiling? And obviously the labs have...
[0:12:08] Much better insight into that because you have the biggest models. You're running all kinds of experiments. It's very non-transparent from the outside. We're having to read the Sam Altman cryptic tweets to figure out where things actually stand. Yeah, I think my spicy take on this is that if you're against a lot of the ideas behind 1047, you're basically a technological pessimist. You think that this AI stuff will just kind of...
[0:12:37] Hover around near where it is today, in which case a lot of the purpose of that regulation is completely like obviated. There's probably not any need for any of it if the technology stays roughly where it is today or gets just a tiny bit better. But if you believe that the technology is going to continue to scale, that the scaling laws will kind of hold for another few orders of magnitude, which I think is the institutional belief of the AI companies.
[0:13:02] Then you have to reckon with the fact that these systems may truly become like extremely powerful and capable in ways where you'd be glad that you had that. But I think ideological beliefs about technology in the future are probably not well suited to doing policy in the present. So a lot of the anger about it is it's clearly like a belief driven piece of legislation. And I think people are rightly going to say, well, let's wait until the evidence is in.
[0:13:33] Is this something where you have to be early because you can't afford to be late? Or do you think we have morning shots? Well, a known thing about the government is that it moves extremely quickly and very effectively at all times. So I think one of the best arguments for trying to do some of this stuff now is that it takes a while. Let's say the AI safety institutes, we did a pre-deployment test of our latest model with them. I don't think...
[0:14:01] We or them were expecting to find some crazy buyer risk or chemical warfare risk or cyber offense risk from these models. But you kind of want to get reps in so that if those risks are out there, you have an institution that could actually credibly identify them. So in some sense, you're trying to build these institutions ahead of when they're needed, because if you start them from the ground up at the time they're needed, they'll almost certainly like instantly fail. It seems to me another crux is
[0:14:31] Disagreement over what AI is. Is it just a tool? Is it a new form of life? Is it sentient? All these heavy philosophical questions. Sometimes folks want to have it both ways. I know from the Andreessen Techno-Optimist Manifesto, he sort of at one point says it's the do-everything machine. We're making sand think. And then other times it's just linear algebra. So how do you see this? Do you see...
[0:15:00] Anthropic is sort of birthing a new life form, or is that just hype? Is that just marketing? I mean, I had a baby recently, so I was really sitting with that and thinking of the terror that instills for a moment. But I think we think of ourselves as trying to figure out exactly how powerful this technology can become and get it to do useful things in the present. I think it's very hard to know what form it takes because lots of that is kind of choice-based. Like today...
[0:15:26] These AI systems are kind of like, almost like brains in jars that you hand some notes to and they think of it and then they hand a note back to you. Sometimes people are connecting that brain in a jar to a load of APIs so that it can take more actions. And some people have these ideas that you'll kind of give it, you know, arms and legs. You'll give it the ability to spin up its own servers, to carry out independent actions. All of these are choices. And I think we don't know what the correct choice is yet to get all of the benefits or not.
[0:15:56] And I think that life forms are kind of contingent on their action space. So I think the companies are just trying to build the extremely capable brain in a jar and then are doing a lot of experimentation of what you actually give that thing access to as that's where a lot of the potential really crazy things can happen. Have you personally had sort of that metaphysical moment working with Claude? I had actually used the right term.
[0:16:25] Since I had a baby recently, I was keeping some journals about having a baby and my relationship with my wife and my new child. I like to write. And I handed it to Claude and I said, what is the author of this not writing? And Claude said, oh, it's interesting. The author, though they allude to working in AI, it appears they are not reckoning with the metaphysical shock they may be experiencing.
[0:16:51] And I got really mad and I went on a five-hour walk being like, am I not reckoning with the metaphysical shock implied by what I'm doing? Which is very strange. I went on a walk because an AI system said something to me that annoyed me. And it was reminiscent of, since I'm English, I moved to California and after 10 years, I did a year with a therapist because it took that long to break through my English resolve. But, you know, sometimes the therapist would like catch me dead on on something and I'd leave the session kind of fuming, being like,
[0:17:21] How dare this person like nail me on something? And it felt eerily similar. And that to me was interesting. Like I now do this with actually, and you might want to do this with your own writing, where if you've got some long essay that you've written, ask an AI to predict things about the author of the essay. And sometimes it'll get aspects of your character that feels extremely eerie. And to me is one of the best bits of evidence that this stuff is doing something a little more complicated than next token prediction.
[0:17:50] And it makes sense, you know, John Searle's like Chinese room argument where you're feeding bits of paper into some room, something is translating the paper from, you know, Chinese into English, it's coming out the other side, does the thing in the room, you know, is it really a thinking creature? Well, if you wanted to get really good at dealing with paper that comes into your kind of windowless cell, you'd start to try and predict what the person or thing was that was handing you the paper.
[0:18:18] And your ability to accurately predict that or make really, really good predictions is contingent on your ability to model something outside of your observable space. And I think without any magic, the way that we train this technology in a really basic way pushes you towards that. So I think that there's some level of spooky modeling that these systems are starting to do, which is hand in hand with how they've become capable at a variety of tasks. And that's vaguely metaphysical, I think. You felt seen.
[0:18:47] It wasn't even multimodal. Claude saw me. Yeah, very strange. So, you know, one way I think of Anthropic differentiating from DeepMind, OpenAI is, you know, one, a big focus on safety, sort of trying to engineer a race to the top on safety, but then also very, you know, high conviction in scaling. And, you know, that metaphysical shock you felt that...
[0:19:15] In some ways, there's a byproduct of scale. I'm sure there's lots of other bags of tricks that go into it, but scale is probably the biggest thing. And now I feel like we're sort of in this liminal space between the current generation of models and the newest generation of models that have yet to come out, where a lot of people are going to be updating one way or the other. I think people won't update at all. And I'll tell you why. I think that these models have got sufficient good that they're increasingly hard to judge if you're a regular person.
[0:19:43] I don't have an advanced degree in physics and a bunch of my colleagues do. So they give the new models like very advanced physics problems and then sometimes they say like, wow, look at what it's done. What do you think, Jack? And then I have to read like a graduate level physics prompt. I'm like, well, gee, I guess you seem excited by this.
[0:20:04] And I think that it's going to look a lot like that, where these AI systems are going to start advancing, but your ability to evaluate where they're advancing is kind of contingent on your own expertise. So maybe I want to register a prediction that the next generation of models will be materially better, but people will assume that scaling has ended because most of the ways you can evaluate how much better they are are extremely difficult and not available to people. But I think scaling will continue for a while.
[0:20:33] So then is it incumbent on the labs to sort of have their own, to broadcast their own internal timelines to have better ways of conveying the ways the models are improving that may be sort of esoteric or opaque to non-experts? I think this is the one case where you actually want government in a non-regulatory sense. You know, with the safety institutes, you want to arrive at some set of measures which everyone would look at and agree are like legitimate measures for capability or risk.
[0:21:03] And I think if we as a lab would make any claims about our model being very capable or perhaps having some risk, you wouldn't naturally believe us and that would be appropriate. And you'd want these safety institutes to kind of validate that. I think for us all to understand this domain in general, we need third parties because the companies have terrible incentives here. They have incentives to market or to sandbag or just to outright lie as these systems become more powerful. But if you have third parties, there's some chance of getting to...
[0:21:32] Kind of truth and having truth available to us. And also, if we succeed and these systems become truly powerful, for all of its drawbacks, I would want someone outside of the companies in government to have that be legible to them. Who's best to be on the receiving end of that within the federal government? I mean, the AI Safety Institute sort of has an evaluation role, but, you know...
[0:22:03] This is sort of like when policy goes to the FTC versus the FCC or sometimes even the FAA. The Secretary of Defense. That's bad, by the way. I'm being very sarcastic. I think that would lead to extreme pain for everyone. But I think that in some sense, we need to work out what the other kind of institutions are going to be. Some of this will relate to trade. Some of it will relate to kind of security. Some of it will relate to, you know, aspects like defense and intelligence. But it's not going to go to like one person.
[0:22:32] I think ultimately you want this to just go extremely broad. I think the safe world is that you have a load of things that are almost equivalent to warrant canaries that can start to disappear or appear to indicate certain things. And you want certain types of information to just be fully public. So it's interesting. So I think there's at least a perception, maybe a misperception, that Anthropic is more on the EA side of the spectrum and believes in like...
[0:23:00] Only we are responsible enough to control this model when you're suggesting that actually increasing the number of stakeholders, not letting it become the Manhattan Project of the Department of Defense. I think I could have a fun discussion with my colleagues about this. So I don't represent the total in-house view. I almost represent an internal skeptic view.
[0:23:22] I think the more time you spend with technology, the more you end up thinking, oh, well, we should make all of the decisions about this. And this is just like a natural consequence of working on it. And history mostly says that that's a terrible idea, I think, up to beyond a certain point. So I believe that the control of this technology beyond a certain level will need kind of institutions that are yet to be built, and it won't be done solely by companies.
[0:23:49] I'm not saying that about today's technologies. I think that would be like an asinine, stupid thing to do and would break chunks of the industry. I'm sort of saying if we end up in this world where we succeed at building stuff at this scale and capability, it can't just be a company in San Francisco calling the shots on exactly what to do. So does that lead to some kind of nationalization? This is something we heard last night. And it's not the 1940s anymore. The U.S. doesn't...
[0:24:19] Straight up nationalized companies, but we did do Operation Warp Speed, which was advanced market commitments, public-private partnerships. Where do you see this heading vis-a-vis the security apparatus? Do you want DMV super intelligence? I think it's a really good prompt. I also think you should think about the worst version of it you can imagine. And if anyone works at the DMV, I'm sorry, but you need to...
[0:24:46] You have substandard customer service and infrastructure. Like a world where the DMV steps in and is like, right, we're going to build a large-scale data center. We're going to acquire the power. We'll do a training data set. I imagine you get your first tokens out of the model in 15 years or something, after which you've been overtaken by events.
[0:25:12] So most forms of nationalization, I think, break for these reasons. And I think forms of securitization run this huge risk of causing the thing they're most afraid of. Like part of why you do national projects is to sort of win some geopolitical rival. But if in doing the project, you just get the rival to take the thing really, really, really seriously, and then you end up in a much closer fight than if you'd been in otherwise, how much good did you do? So I feel like...
[0:25:42] There are some lessons here from things like commercial crew where NASA is relying on outside contractors like SpaceX for getting mass to orbit. There's some stuff to be done where government may end up kind of using the private sector in a quasi-national way for projects, and that's likely how it works. But I think we need to be extremely careful of things that look like kind of defense-oriented militarization or securitization because it probably gets us a load of like extremely...
[0:26:12] It pushes us into the dangerous world, probably more than the safe world under most ways that you can think it through, I think. And I say that as someone who became an American citizen a couple of years ago and sort of believe strongly in America as a country, but America also has a history of having unintended military consequences around the world. So we need to be mindful of that. Right, so earlier we heard about Little Tech and the case for Little Tech.
[0:26:42] There's always the wrapper on GPT meme. Bigger picture, beyond militarization, I also think about the fact that Microsoft is the biggest IT vendor in the US government. If there comes a day where we need AI capabilities in government, it would be a kind of takeover if the entire US government was to be running on one company's rails. On the other hand, Anthropic last year had a pitch deck that
[0:27:11] suggested that the company that builds the best model in the 2025-2026 cycle could have a lead that no company catches up to. So what is the room for little tech and AI? Are all the app companies going to get steamrolled? Is this going to naturally lead to a winner-take-all? I think the AI companies don't understand much about expert or specific domains, and so lots of things will end up looking like people with...
[0:27:41] Very specific domain knowledge or access to like an idea on how to build very specific and good proprietary data sets are going to build their own businesses or they're going to build things that solve problems which are specific in the world. And that will be immensely successful. I think the AI companies don't naturally find that stuff. I think market concentration is going to be an issue and you're going to end up needing to think about this stuff like a...
[0:28:08] How do you conceptualize having a maximally competitive market here? And I think that one failure mode is ending up with certain forms of procurement that funnel you towards a less competitive market. But I think the government has a history of both making good decisions here and also bad decisions. So that's kind of up for debate as to where we end up. And to your question about, could some company develop a model which generates some lead?
[0:28:36] I think this kind of relates to whether a model can generate synthetic data to train a successor model. And I don't think synthetic data is going to be the like all-in silver bullet. I think it'll be important and new models will be able to do that to varying degrees. But I don't see some kind of like run away with the whole thing scenario for the next few generations. For the next few generations? For the next few generations.
[0:29:05] That's the technological pessimist in me talking in a sense. And is a generation roughly two years? Yeah, I think it's a year and a half to two years. And I think it's kind of like Intel had this concept called TikTok where you build a new chip on kind of a new architecture and then you refine that architecture, then a new architecture comes along. And I think the models have sort of been doing that. Then you have a defect and lose half your market value.
[0:29:33] In the Intel case. Probably not the best example given the struggles. So over these next two generations, what will be, if this does not come to pass, what will be the best counterfactual reason why? Yeah, I think it could be that data is incredibly valuable and very hard to create, and there's some kind of data wall in front of us that the companies run into. That could be one thing. The other could be...
[0:30:01] You can't build a broad enough distribution of environments to do reinforcement learning on top of. You know, lots of the reason coding stuff has got better is you can verify that code is correct. Same with math for approvers out there. You can use that to generate like good data that you can train off of. People probably assume it's easy to generate a whole bunch of environments here and it may end up being extremely hard.
[0:30:24] You know, in a famous Google paper from Noam Chazir, who subsequently went on to do character AI, he wrote about some technical thing, and he said something like, only God knows why this works, which is, I think, his way of being epistemically humble about how some of this stuff is just mysterious, so I think it could stop working, and we would say the gods have denied us this technology or something.
[0:30:49] But you might just genuinely not know. It's a confusing field and sometimes confusing stuff happens that will take a while to figure out how to fix. Yeah, on that point, is it true that the reason 3.5 seemed to be degrading quality was because it was August and that's when Europeans go on vacation? The French condition, yeah. I don't think so, but I also...
[0:31:19] I would not outright deny this. And I think it touches on an interesting thing. How do you know if your model gets lazy in August? What is the evaluation criteria for working out if your model has become slightly work shy and lazy over a large distribution of tasks? It's actually quite hard. So we're building an HR function to assess how well Claude does and build the measures to figure it out. But it points to something.
[0:31:47] I think that is genuinely confusing about these systems, which is that they contain kind of multitudes, and when they go slightly off-piste, it may end up being because the model thinks it's European, and we may discover that. So you agree that Claude is more European? As an American immigrant, I'm more patriotic than standard Americans, so there's no way.
[0:32:16] But I think that you've seen this in the kind of like Western models where, you know, ChatGPT had some kind of laziness accusations, I think, around Christmas as well. So similarly, it was like, well, it's December, so we'll see. And I think that just reflects to some extent, for all of the advanced stuff we've talked about, these models are also doing...
[0:32:43] Like data modeling of a big text distribution. And there's like lots of human patterns baked into that. Like these weird behaviors that happen around European summers or most many, many people in Christmas being slightly lazier. And I think they have inherited some of that in ways that are kind of hard to test for. They're a mirror on a reality, a kind of a shadow. Yeah, yeah. I mean, these things are reflecting us back at...
[0:33:11] Back at ourselves, it's kind of like a funhouse mirror on all of human culture that we've bottled up and now work with. So you have a somewhat unique background. You sort of started as a tech reporter, first neural nets reporter, right? Yeah, I called myself the world's first neural net reporter because it was very easy to do that in 2013. And you also...
[0:33:37] Wonderful substack. Everyone should subscribe to Import AI. I've been writing that since 2016 or so. You often end with what you call tech tales, sort of speculative science fiction, sort of telling some story. How did that begin? And how do you also sort of think about the fact that you're building a technology that is now, you know, commoditizing creative writing?
[0:33:59] So I'll start with how it began and I'll say the unflattering truth about myself, which is I am still a little European in that I'm chronically lazy and I stopped being a journalist and I had two months off before I started at OpenAI. I was like, oh, I need like a forcing function to make sure I keep working. What if I use my English fear of shame and embarrassment in public to get me to write a newsletter where I have to read loads of research papers every week?
[0:34:26] And it continued because I think, especially if you work in policy, the amount of technical research you're going to interact with necessarily decreases over time. And I wanted something that forces me every week to read a load of the latest papers and try and understand them. I think that's so important. And otherwise you end up saying completely banana stuff. And in terms of the, you know, writing.
[0:34:50] I think writing is a way to kind of view the world sideways or view the world from a different angle. And it's also creating an interesting data set that I can use to ask Claude to model the author of these stories about to further plumb the kind of metaphysical depths. On how this stuff influences creative writing, creative writing as an industry has been on like a very weird journey for the last 20 years or so. And I think it mostly this stuff.
[0:35:21] Doesn't seem to have like much of an effect unless you were maybe writing 99 cent like Kindle fiction, in which case you're now competing with other people writing 99 cents Kindle fiction who are trying to do it more effectively than you. But that's not most of creative writing. Most of creative writing is passion projects by people trying to understand themselves and the world. And I think that's been barely changed by this stuff other than maybe making it slightly cheaper to get an editor sometimes.
[0:35:50] So Nick Land, the ARCH accelerationist, he sort of talks about hyperstition, we're assembling the AI from the future and the present, the kind of role of sci-fi driving self-fulfilling prophecy. How do you think about that? What is the role of sci-fi in AI beyond just being an inspiration? There's been a really funny thing recently of people in the long-term AI safety community stopped
[0:36:19] are saying, stop writing detailed scenarios or stories about how AI takes over. You're like, you're giving it the plans, which is, I think, maybe a little paranoid. But I do have this belief that, you know, and I became obsessed with this maybe in 2005 or 2006, where I was like, hang on, we have companies like Google that have a business model that forces them to cover the world in data centers.
[0:36:47] And then you're following some of the really early machine learning research for ad targeting. And you're like, hang on. Google's business model is incentivizing it to train increasingly smart intelligences, which it sells, that loop back resources into themselves and get Google to stamp out more data centers. This feels like quite a self-reinforcing business process. And I do think that some of this AI stuff has a kind of inevitability around it. It creates demand for
[0:37:15] The infrastructure that you use to train the next generations of it. And it constantly makes itself cheaper and cheaper to train. You know, some of my colleagues now just run local models on their laptop to do like local, completely private work. You know, because why wouldn't you? It's incredibly useful. I thought of this when I was on a plane dealing with United Wi-Fi recently. And Claude can't fit on my MacBook, but I wished I had like a llama on there to talk to. And it says to me that all we can really choose to do
[0:37:45] In some sense is bring the future like into the present kind of intentionally and study it really, really, really carefully while there's still some delta between those two things. And mostly I think what we're going to learn is just how useful and profoundly strange this stuff is. But I think some of my hope is if we learn that there is any genuine risk out there, you have some period of time you can then play with to deal with or mitigate that risk before it.
[0:38:13] Before it comes down the escalator of Moore's law and just arrives as a fait accompli all around you. Yeah, that's what I worry about most. That's my optimism. I mean, there's an optimism. You know, I often think about, you know, if we were having internet safety discussions in 2000, we would talk about, you know, cyber crime or online bullying. We wouldn't talk about information tsunami delegitimizing Western democracies and causing an Arab Spring.
[0:38:41] And now imagine if all those particles of data were agentic and highly intelligent. I tend to think that it's great to have some ability to adapt some warning shots, some delta, but as this trickles down, it's going to require different kinds of solutions, right? And do we have the regulatory tools, period, let alone...
[0:39:12] Yay or nay regulation? No. But we're an inventive species and we're doing our best to come up with it. I think that all we can really do is build the institutions that get really good at kind of measurement and assessment and rapid, continuous kind of analysis of the space. And that's one of the best tools you have. The social media companies did this in a very ad hoc and piecemeal way, but they were the ones able to...
[0:39:40] On that note, help me thank Jack Clark for being here.
[0:40:10] Introduce the next group.