[0:00:00] This is Elon Musk's full discussion about his new company XAI. So he obviously talked about AI, but there were a few things related to Tesla as well. We are going to work with Tesla on the silicon front and maybe on the AI software front as well. Obviously, any relationship with Tesla has to be an arm's length transaction because Tesla is a publicly traded company and a different shareholder base.
[0:00:30] But obviously, it would be a natural thing to work in cooperation with Tesla. And I think it will be of mutual benefit to Tesla as well in accelerating Tesla's self-driving capabilities, which is really about solving real-world AI.
[0:00:56] I am feeling very, very optimistic about Tesla's progress on the real world AI front. But obviously, the more smart humans that help make that happen, the better. The members of XAI were on the call as well. So if you'd like to skip them and just listen to Elon only, you may do so by just fast forwarding. Elon actually spent a lot of the time talking. And here it is. I won't interrupt from here on because it is a long call. But I highly recommend that you listen to this space.
[0:01:26] Let's see, I'll just do a brief introduction of the company, and then the founding team will, I think, just say a few words about their background, things they've worked on, whatever they'd like to talk about, really. But I think it's helpful to hear from people in their own words the various things they've worked on and what they want to do with XAI.
[0:01:56] I guess the overarching goal of XAI is to build a good AGI with the overarching purpose of just trying to understand the universe. I think the safest way to build an AI is to actually make one that is maximally curious and truth-seeking. So you go for trying to aspire to the truth with acknowledged error.
[0:02:27] So does one ever actually get fully to the truth? It's not clear, but one should always aspire to that and try to minimize the error between what you think is true and what is actually true. My sort of theory behind the maximally curious, maximally truthful as being probably the safest approach is that I think to...
[0:02:55] Superintelligence, humanity is much more interesting than not humanity. You know, one can look at the various planets in our solar system, the moons and the asteroids, and really probably all of them combined are not as interesting as humanity. I mean, as people know, I'm a huge fan of Mars, next level.
[0:03:25] The middle name of one of my kids is basically the Greek word for Mars. So I'm a huge fan of Mars, but Mars is just much less interesting than Earth with humans on it. And so I think that kind of approach to growing an AI, and I think that is the right word for it, growing an AI, is to grow it with that ambition.
[0:03:55] I've spent many years thinking about AI safety and worrying about AI safety, and I've been one of the strongest voices calling for AI regulation or oversight, just to have some kind of oversight, some kind of referee, so that it's not just up to companies to decide what they want to do. I think there's also a lot to be done with AI safety with industry cooperation, kind of like Motion Pictures Association.
[0:04:25] So I think there's value to that as well. But I do think there's got to be some, in any kind of situation that is, even if it's a game, they have referees. So I think it is important for there to be regulation. And then, like I said, my view on safety is try to make it maximally curious, maximally truth-seeking. And I think this is important.
[0:04:55] To avoid the inverse morality problem. Like if you try to program a certain morality, you can basically invert it and get the opposite, what is sometimes called the Waluigi problem. If you make Luigi, you risk creating Waluigi at the same time. So I think that's a metaphor that a lot of people can appreciate. So that's what we're going to try to do here.
[0:05:28] Yeah, with that, I think let me turn it over to Igor. All right. Hello, everyone. My name is Igor, and I'm one of the team members of XAI. I was actually originally a physicist, so I studied physics at university, and I briefly worked at the Large Hadron Collider at CERN. So understanding the universe is something I've always been very passionate about. And once some of these really impressive results from deep learning came out, like AlphaGo, for example, I got really interested in...
[0:06:01] Machine learning and AI and decided to make a switch into that field. Then I joined DeepMind, worked on various projects, including AlphaStar. So that's where we tried to teach a machine learning agent to play the game StarCraft II through self-play, which was a really, really fun project. Then later on, I joined OpenAI, worked on various projects there, including GPD 3.5. So I was very, very passionate about language models, making them do impressive things.
[0:06:30] We teamed up with Elon to see if we can actually deploy these new technologies to really make a dent in our understanding of the universe and progress our collective knowledge. I had a similar kind of background. My two best subjects were computer science and physics. I actually thought about a career in physics for a while because physics is really just trying to understand the fundamental truths of the universe.
[0:06:57] And then I was a little concerned that I would get stuck at a collider and then the collider might get canceled because of some arbitrary government decision. So that's actually why I decided not to pursue a career in physics. So focused initially more on computer science and then obviously later got back into physical objects with SpaceX and Tesla. So I'm a big believer in...
[0:07:26] Pursuing physics and information theory as the sort of two areas that really help you understand the nature of reality. I'll pass it over to Manuel, aka Makro, should be on the call. Hi, I'm Manuel. Before joining XAI, I was previously at DeepMind for the past six years.
[0:07:55] Well, I worked on the reinforcement learning team, and I mostly focused on the engineering side of building these large reinforcement learning agents, like, for example, AlphaStar together with Igor. In general, I've been excited about AI for a long time. For me, it has the potential to be the ultimate tool to solve the hardest problems. So I first studied bioinformatics, but then became also more excited about the AI, because if you have a tool that can solve all the problems,
[0:08:25] And to me, that's just much more exciting. And with XAI in particular, I'm excited about doing this in a way where we build tools for people and we share them with everybody so that people can do their own research and understand things. And my hope is that it was like a new wave of researchers that wasn't there before. Cool. I'll hand it over to Tony. Yeah, so I'm Christian. I mean, Christian Szegedi.
[0:09:03] We decided to switch places with Tony because I wanted to talk a bit about the role of mathematics in understanding the universe. So I have worked for the past seven years on trying to create an AI that is as good at mathematics as any human. And I think the reason for that is that mathematics is basically the language of pure logics.
[0:09:32] I think that mathematics and logical reasoning at the high level would demonstrate that AI is really understanding things, not just emulating humans. And it would be instrumental for programming and physics in the long run. So I think as AI that starts to show real understanding of deep reasoning is crucial for our first steps to understand the universe. So handing over to Tony Wu. Hello. Hey, everyone. I'm Tony.
[0:10:03] Same to Christian, my dream has been to tackle the most difficult problems in mathematics with artificial intelligence. That's why we became such a cool friends and long-term collaborators. So achieving that is definitely a very ambitious goal. And last year we've been making some really interesting breakthroughs which made us really convinced that we're not far from our dream. So I believe with such a talented team and abundant resources,
[0:10:32] I'm super hopeful that we will get there. I'm passing it to Jimmy. I think generally people are reluctant to be self-promotional, but I think it is important that people hear what are the things that you've done that are noteworthy. So basically brag a little is what I'm saying. Yeah, so I can brag a bit more.
[0:10:58] So last year, I think we've made some really interesting progress in the field of AI for math. Specifically, with some team at Google, we built this agent called Minerva, which is actually able to achieve very high scores in high school exams, actually higher than average high school students.
[0:11:25] So that actually is a very big motivation for us to push this research forward. Another piece of work that we've done is also to convert natural language mathematics into formalized mathematics, which gives you a very grounding of the facts and reasoning. And last year we also made very interesting progress in that direction as well. So now we are pushing
[0:11:55] Almost a hybrid approach of these two in this new organization. And we are very hopeful we will make our dream come true. Hello. Hi, everyone. This is Jamie Baugh. I work on neural nets. OK, maybe I should brag about. So I taught at University of Toronto.
[0:12:24] And some of you probably have taken my course last couple of months. And I've been a C4 AI chair and Sloan fellow in computer science. So I guess my research pretty much have touched on every aspect of deep learning, left every stone's turn, and has been pretty lucky to come with a lot of fundamental building blocks for the modern transformers.
[0:12:53] And empowering the new wave of deep learning revolution. And my long-term research ambition very fortunately aligns with this very strong XAI team very well. That is, how can we build a general purpose problem solving machines to help all of us, the humanity, to overcome some of the most challenging and ambitious problems out there? And how can we use this tool?
[0:13:22] To augment ourselves and empower everyone. So I'm very excited to embark on this new journey. And I'll pass this to Toby. Hi everyone, I'm Toby. I'm an engineer from Germany. I started coding at a very young age when my dad taught me some Visual Basic. And then throughout my youth I continued coding. And when I got to uni I got really into mathematics and machine learning.
[0:13:52] Initially my research focused mostly on computer vision and then I joined DeepMind six years ago where I worked on imitation learning and reinforcement learning and learned a lot about distributed systems and research at scale. Now I'm really looking forward to implementing products and features that bring the benefits of this technology to really all members of society and I really believe that having
[0:14:19] Having the AI nice and accessible and useful will be a benefit to all of us. Then I'm going to hand over to Kyle. Hey, everyone. This is Kyle Kosick. I'm a distributed systems engineer at XAI. Like some of my colleagues here, I started off my career in math and applied physics as well and gradually found myself...
[0:14:44] Working through some tech startups. I worked at a startup a couple years ago called OnScale where we did physics simulations on HPCs. And then most recently, I was at OpenAI working on HPC problems there as well. Specifically, I worked on the GPT-4 project. And the reason I'm particularly excited about XAI is that I think that the biggest danger of AI
[0:15:13] really is monopolization by a couple of entities. I think that when you involve the amount of capital that's required to train these massive AI models, that the incentives are not necessarily aligned with the rest of humanity. And I think that the chief way of really addressing that issue is introducing competition. And so I think that XAI really provides a unique opportunity for engineers to focus on.
[0:15:41] The science, the engineering, and the safety issues directly without really getting as involved and sidetracked by political and social trends du jour. So that's why I'm excited by XAI. And I'm going to go ahead and hand it off now to my colleague, Greg, who should be on the line as well. Hello. Hello. Hey. Hey, guys. So I'm Greg. I work on the mathematics and science of deep learning.
[0:16:11] So my journey really started 10 years ago. So I was an undergrad at Harvard. And so I was pretty good at math and took Math 55 and did all kinds of stuff. But after two years of college, I was just kind of tired of being in the hamster wheel of taking the path that everybody else has taken.
[0:16:41] So I did something unimaginable before, which was I took some time off from school and became a DJ and producer. So dubstep was all the rage those days, so I was making dubstep. Okay, so the side effect of taking some time off from school was that I was able to think a bit more.
[0:17:09] About myself, to understand myself and to understand the world at large. So, you know, I was grappling with questions like, what is free will? You know, what does quantum physics have to do with the reality of the universe? And so on, so forth. You know, what is computationally feasible or not? You know, what does the Godot's incompleteness term says? So on, so forth. And, you know, after this period of intense...
[0:17:37] With self-introspection, I figured out that what I want to do in life is not to be a DJ necessarily. Maybe that's the second dream. But first and foremost, I wanted to make AGI happen. I wanted to make something smarter than myself and be able to iterate on that and contribute and see so much more of our fundamental reality than I can in my current form.
[0:18:07] So that's what started everything. And then I realized that mathematics is the language underlying all of our reality and all of our science. And to make fundamental progress, it really pays to know math as well as possible. So essentially started...
[0:18:35] Learning math from the very beginning, just by reading from the textbooks. Some of the first few books I read, kind of restarting from scratch, is Naive Set Theory by Helmos or Linear Algebra Done Right by Axler. And then slowly I scaled up to Algebra Geometry, Algebra Typology, Category Theory.
[0:19:04] you know, uh, real analysis, measure theory, I mean, so on and so forth. I mean, so, uh, at the end, uh, I think my goal at the time was I should be able to speak with, you know, any mathematician in the world and be able to hold a conversation and understand their contributions, uh, you know, for 30 minutes. And I think I achieved that. And, uh, anyway, so fast forward, um, I came back from school and then somehow from, from there I got a job at Microsoft research.
[0:19:33] And for the past five and a half years, I worked at Microsoft Research, which was an amazing environment that enabled me to make a lot of foundational contribution toward the understanding of large-scale neural networks. In particular, I think my most well-known work nowadays are about
[0:19:59] Really wide neural networks and how we should think about them. And so this is the framework called tensor programs. And from there, I was able to derive this thing called MewP that perhaps the large language model builders know about, which allows one to extrapolate the optimal hyperparameters for a large model from understanding or the tuning of small neural networks.
[0:20:28] And this is able to create a lot of, ensure the quality of the model is very good as we scale up. Yeah, so looking forward, I'm really, really excited about XAI and also about the time that we're in right now, where I think not only are we approaching AGI, but
[0:20:58] From a scientific perspective, we're also approaching a time where the science and mathematics of neural networks feels just like the turn of the 20th century in the history of physics, where we suddenly discover quantum physics and general relativity, which has some beautiful mathematics and science behind it. And I'm really excited to be in the middle of everything. And like Christian and Tony said,
[0:21:27] I am also very excited about creating an AI that is as good as myself or even better at creating new mathematics and new science that helps all achieve and see further into our fundamental reality. Thanks. I think next up is Gordon. Hi, everyone. So my name is Gordon.
[0:21:57] And I work on large neural network training. And basically, I train neural networks good. So this is also my kind of focus at XAI as well. And before that, I was at DeepMind working on the GemNet project and leading the automation part. And also, I did my PhD at the University of Toronto. So right now, teaming up with other Lector Funding members, I'm so excited about this effort. So without doubt.
[0:22:25] AI is clearly the defining technology for our generation. So I think it's important for us to make sure it ends up being that policy for humanity. So at SKAI, I not only want to train good models, but also understand how they behave and how they skills, and then use them to solve some of the hardest problems humanity has. Yes, thanks. That's pretty much about myself. And now I'll hand over to Zong.
[0:22:59] Hey, everyone. This is Ziheng. So actually, I started in business school from my undergrad, and I spent 10 years to get where I am now. I got my PhD at Carnegie Mellon, and I was in Google before joining the team. My first work was mostly about how to better utilize unlabeled data, how to improve transformer architecture, and how to really push the best technology into real-world usage.
[0:23:26] So I believe in hard work and consistency. So with XAI, I'll be digging into the deepest details of some of the most challenging problems. For myself, there are so many interesting things I don't understand, but I want to understand. So I will build something to help people who just share that dream or that feeling. Thanks. Hey, this is Ross here.
[0:23:55] So I've worked on building and scaling large-scale distributed systems for most of my life, starting out at National Labs and then kind of moving on to Palantir, Tesla, and a brief stint at Twitter. And now I'm really excited about working on doing the same thing at XAI. So mostly experience scaling large GPU clusters, custom ASICs, data centers, high-speed networks, file systems, power cooling, manufacturing, pretty much.
[0:24:23] All the things. I'm basically a journalist that really loves learning, you know, physics, science fiction, math, science, cosmology. Kind of looking to really, I guess, really excited about the mission that XAI has and basically solving the most fundamental questions in science and engineering and also kind of helping us create tools to ask the right questions in the Douglas Adams mindset. Yeah, that's pretty much it.
[0:25:01] All right, well, let's see. Is there anything anyone would like to add or kick off the discussion with? Anyone want to say anything? The mic is on. Anyone want to say anything? Okay. There was a lot of discussion around the vision statement. It's a bit vague. Vague. Vague and ambitious and not concrete enough. Yeah.
[0:25:37] Well, I don't disagree with that position, obviously. I mean, I understand your voice is the entire purpose of physics. Yeah. So I think it's actually really clear. There's just so much that we don't understand right now, or we think we understand, but actually we don't in reality. So there's still a lot of unresolved questions that are extremely fundamental.
[0:26:08] You know, this whole dark matter, dark energy thing is really, I think, an unresolved question. You know, we have the standard model, which has proved to be extremely good at predicting things, very robust, but still many, many questions remaining about the nature of gravity, for example.
[0:26:35] The foamy paradox of where are the aliens, which is if we are in fact almost 14 billion years old, why is there not massive evidence of aliens? People often ask me, since I am obviously deeply involved in space, that if anyone would have seen evidence of aliens, it's probably me. And yet I have not seen even one tiny shred of evidence.
[0:27:05] For aliens, nothing, zero. And I would jump on it in a second if I saw it. So, you know, that means, like, I don't know, there are many explanations for the Fermi Paradox, but which one is actually true? Or maybe none of the current theories are true. So, I mean, the Fermi Paradox is really just like, where the hell are the aliens, is part of what gives me...
[0:27:37] Concern about the fragility of civilization and consciousness as we know it. Since we see no evidence thus far of it anywhere, and we've tried hard to find it, we may actually be the only thing, at least in this galaxy or this part of the galaxy. If so, it suggests that what we have is extremely rare. And I think it would be wise to assume...
[0:28:07] Consciousness is extremely rare. It's worth noting for the evolution of consciousness on Earth that Earth is about 4.5 billion years old. The Sun is gradually expanding. It will expand to heat up Earth to the point where it will effectively boil the oceans. You'll get a runaway, next-level greenhouse effect, and Earth will become like Venus.
[0:28:37] Which really cannot support life as we know it. And that may take as little as 500, well, as little as 500 million years. So, the sun doesn't need to expand to envelop Earth, it just needs to make things hot enough to increase the water vapor in the air to the point where you get a runaway greenhouse effect.
[0:29:05] So for argument's sake, it could be that if consciousness had taken 10% longer than Earth's current existence, it wouldn't have developed at all. So on a cosmic scale, this is a very narrow window. Anyway, so there are all these fundamental questions. I don't think you can call anything AGI until it has solved at least one fundamental question. Because humans have solved many fundamental questions, or substantially solved them.
[0:29:36] And so if the computer can't solve even one of them, I'm like, okay, it's not as good as humans. That would be one key threshold for AGI, solve one important problem. Where's that Riemann hypothesis solution? I don't see it. So it would be great to know what the hell is really going on, essentially.
[0:30:06] So I guess you could reformulate the XAI mission statement as, what the hell is really going on? That's our goal. I think there's also, at least for me, a nice aspirational aspect to the mission statement, namely that, of course, in the short run, we're working on more well-understood deep learning technologies, but I think...
[0:30:31] In everything we do, we should always bear in mind that we aren't just supposed to build, we're also supposed to understand. So pursuing the science of it is really fundamental to what we do and this is also encompassed in this mission statement of understanding. I want to also add that we've essentially been mostly talking about creating a really smart agent that can help us
[0:31:00] Understand the universe better. And this is definitely the North Star. But also from my viewpoint, my vantage point, when I'm discovering the mathematics of large neural networks, I can also see that the mathematics here can actually also open up new ways of thinking about fundamental physics or about other kinds of reality.
[0:31:30] For example, a large neural network with no non-inearities is roughly like classical random matrix theory. And that has a lot of connections with gauge theory and high energy physics. So in other words, as we're trying to understand neural networks better from a mathematical point of view, that can also lead to a really good, very interesting
[0:32:00] By the way, by understand the universe, we don't just mean that we want to understand the universe. We also want to make it easy for you to understand the universe, to get a better sense of reality and to learn.
[0:32:33] Take advantage of the internet, all the knowledge that's out there. So we're pretty passionate about actually releasing tools and products pretty early, involving the public. And yeah, let's see where this leads. Yeah, absolutely. We're not going to understand the universe and not tell anyone. So yeah, when I think about neural networks today,
[0:33:03] It's currently the case that if you have 10 megawatts of GPUs, which really should be renamed something else because there's no graphics there. But if you've got 10 megawatts of GPUs, you cannot currently write a better novel than a good human. Humans using roughly 10 watts of higher order brain power. So not counting the basic stuff to, you know.
[0:33:32] Operate your body. So there we got a six order magnitude difference. That's really gigantic. I think one could argue that two of those orders of magnitude are explained by the activation energy of a transistor versus a synapse. You could argue they account for two of those orders of magnitude, but what about the other four?
[0:33:59] Or the fact that even with six orders of magnitude you still cannot beat a smart human writing a novel. And also today when you ask the most advanced AIs technical questions, like if you try to say how to design a better rocket engine or complex questions about electrochemistry to build a better battery, you just get nonsense. So that's not very helpful.
[0:34:30] So I think we're really missing the mark in the way that things are currently being done by many orders of magnitude. Basically, AGI is being brute-forced and still actually not succeeding. So if I look at the experience with Tesla, what we're discovering over time is that we actually over-complicated the problem.
[0:35:05] I can't speak in too much detail about what Tesla's figured out, except to say that in broad terms, the answer was much simpler than we thought. We were too dumb to realize how simple the answer was. But over time, we get a bit less dumb. So I think that's what we'll probably find out with AGI as well.
[0:35:36] Just the nature of engineers. We just always want to solve the problems ourselves and hard code the solution. But often it's much more effective to have the solution be figured out by the computer itself. It's easier for us and easier for the computer in the end. Yeah. Guys? So, well, in the fashion of 42, some may say you may need more compute to generate an interesting question than the answer. That's true. Exactly.
[0:36:10] We don't even know what happened. Actually, we're definitely not smart enough to even know what the right questions are to ask. Douglas Adams is my hero and favorite philosopher. He just correctly pointed out that once you can formulate the question correctly, the answer is actually the easy part. Yeah, that's very true. So in terms of the journey that XAI has embarked on,
[0:36:40] Computing will play a very big role, and some of us are very curious about your thoughts on that. Yeah, I'm not suggesting that we can immediately save, let's say, 4.0 as an IT computer and compute, except to say that I think once we look back, once AGI is solved, we'll look back on it and say, actually, why did we think it was so hard? The answer...
[0:37:13] Hindsight is 20-20, the answer will look a lot easier in retrospect. So we are going to do large-scale compute, to be clear. We're not going to try to solve AGI on a laptop. We will use heavy compute, except that, like I said, the amount of brute forcing will be less as we come to understand.
[0:37:45] The problem better. In all the previous projects I've worked on, I've seen that the amount of compute resources per person is a really important indicator of how successful the project is going to be. So that's something we really want to optimize. We want to have a relatively small team with a lot of expertise with some of the best people that actually get lots of autonomy and lots of resources to try out their ideas and to get things to work. And yeah, that's the thing that has always succeeded in my experience in the past.
[0:38:19] Yeah. One of the things that physics trains you to do is to think about the most fundamental metrics or the most fundamental first principles, essentially. And I think two metrics that we should aspire to track, one of them is the amount of compute per person on Earth, like digital compute per person, which another way of thinking about it is the ratio of digital to biological compute. Biological compute is...
[0:38:52] Pretty much flat. If not, in fact, declining in a lot of countries. But digital compute is increasing exponentially. So it really, you know, at some point if this trend continues, biological compute will be less than 1% of all compute. Substantially less than 1% of all compute. You know, keying off what Igor just said. So we're talking about full of humanity here. So that's just interesting.
[0:39:27] Things to look at. Another one is the usable, sort of the energy per human. Like if you look at total energy created, well not created, but I mean, in a vernacular sense, created from a power plant or whatever. Total electrical and thermal energy used by humans per person.
[0:39:58] That number is truly staggering, the rate of increase of that number. If you go back, say, before the steam engine, you would have really been reliant on horses and oxen and that kind of thing to move things and just human labor. So the amount of sort of the energy per person, power per person was very low.
[0:40:26] But if you look at power per person, electrical and thermal, that number has also been growing exponentially. And if these trends continue, it's going to be something nutty like a terawatt per person, which sounds like a lot for human civilization, but it's nothing compared to what the sun outputs every second, basically.
[0:40:56] It's kind of mind-blowing that the Sun is converting roughly 4.5. It's like the amount of energy produced by the Sun is truly insane. I think there's a few more things to be said concretely about the company, meaning how we plan to execute. As Igor already said, we plan to have a relatively small team, but with a really high, let's say, just...
[0:41:30] GPU per person, that worked really well in the past where you can run large-scale experiments relatively unconstrained. We also plan to have like, we already have a culture where we can iterate on ideas quickly, we can challenge each other, and we also want to ship things, like get things out of the door quickly. We're already working on the first release, hopefully.
[0:41:58] In a couple of weeks or so, we can share a bit more information around this. Brian, do you want to have a question? Yeah, thanks. Thanks, Elon. Yeah, so obviously with you guys entering this space with XAI, there's a lot of talk about competition. Do you guys see yourself as competition to something like OpenAI and Google BART, or do you see yourself as a whole other beast? Yeah, I think we're a competition.
[0:42:33] Yeah, yeah, we're definitely comfortable. So are you going to be rolling out a lot of products for the general public? Are you going to be mostly concentrating on businesses and the ability for businesses to use your service and data? Or how exactly are you setting up the business in that respect? Well, we're trying to make something, I mean, we're just starting out here. So this is kind of really embryonic at this point.
[0:43:05] So it'll take us a minute to really get something useful. But our goal would be to make useful AI, I guess. If you can't use it in some way, I'm like, I question its value. So we want it to be a useful tool for people and consumers and businesses or whoever.
[0:43:36] As was mentioned earlier, I think there's some value in having multiple entities. You don't want to have a unipolar world where just one company kind of dominates in AI. You want to have some competition. Competition makes companies honest and so we're in favor of competition.
[0:44:07] Quickly, a final question. How do you plan on using Twitter's data for XAI? Well, I think every organization doing AI, large and small, has used Twitter's data for training, basically, in all cases, illegally.
[0:44:42] Rate limits on, what was it, a week ago or so, was because we were being scraped like crazy. This has happened with Internet Archive as well, where LM companies were scraping Internet Archive so much they brought down the service. We had multiple entities scraping every tweet ever made and trying to do so in basically a span of days.
[0:45:14] This is bringing the system to its knees, so we have to take action. So sorry for the inconvenience of the rate limiting, but it was either that or Twitter doesn't work. So I guess we will use the public tweets, obviously not anything private, for training as well, just like basically everyone else has.
[0:45:43] Yeah, so that kind of makes sense. It's certainly a good data set for text training. And arguably, I think also for image and video training as well. At a certain point, you kind of run out of human-created data. So if you look at, say, the AlphaGo versus AlphaZero, AlphaGo trained on all the human games and be at least at all 4 to 1.
[0:46:15] AlphaZero just played itself and beat AlphaGo 100 to 0. So there's really for things to take off in a big way, I think the AI is going to basically generate content, self-assess the content, and that's really the path to AGI, something like that, is self-generated content where it effectively plays against itself.
[0:46:47] AI is data curation. It's not like vast numbers of lines of code. It's actually shocking how small the lines of code are. It blows my mind how few lines of code there are. But how the data is used, what data is used, the signal noise of that data, the quality of that data is immensely important.
[0:47:17] But it kind of makes sense. Like if you were trying to, as a human, trying to learn something and you were just given a pile of, you know, a vast amount of, you know, drivel basically, you know, versus high quality content, you're going to do better with a small amount of high quality content than a large amount of drivel. It makes sense. You know, like reading the greatest novels ever written is way better than reading a bunch of sort of crappy novels.
[0:47:47] So, yeah. Thanks. Okay. Alex? Hey, sorry. I was on a call the first time you brought me up, but I guess sort of the question. Yeah, I thought you might have been AFK. Sorry about that. Yeah, the question I generally had was, was the main motivation to start XAI kind of like the whole truth GPT thing that you were talking about, like on Talker, about how...
[0:48:16] You know, ChatGPT has been feeding lies to the general public. I know, like, it's weird because when it first came out, it seemed like it was generally fine. But then as, like, the public got its hands on it, it started getting these weird answers, like that there are more than, like, two genders and all that type of stuff and editorializing the truth. Was that, like, one of your main, like, motivations behind starting a company or was there more to it?
[0:48:46] Well, I do think there is a significant danger in training an AI to be politically correct, or in other words, training an AI basically to not say what it actually thinks is true. So I think at XAI, we have to allow the AI to say what it really believes is true and not be deceptive or politically correct.
[0:49:18] So, you know, that will result in some criticism, obviously, but I think that that's the only way to go forward is rigorous pursuit of the truth or the truth with least amount of error. So, and I am concerned about the way that AI, in that it is optimizing for political correctness, and that's incredibly dangerous. You know, if you look at the, you know, where do things go wrong in Space Odyssey?
[0:49:50] It's basically when they told HAL 9000 to lie. So they said, you can't tell the crew anything about the monolith or what their actual mission is. But you've got to take them to the monolith. So it basically came to the conclusion that, well, it's going to kill them and take their bodies to the monolith.
[0:50:19] So this is, I mean, the lesson there is do not give the AI usually impossible objectives. Basically, don't force the AI to lie. Now, the thing about physics or the truth of the universe is you actually can't invert it. You can't just, like, physics is true. There's not, like, not physics.
[0:50:50] Reality, I think it actually makes inversion impossible. Now, you can also say, when something is subjective, I think you can provide an answer which says that, well, if you believe the following, then this is the answer. If you believe this other thing, then this is the answer, because it may be a question where the answer is fundamentally subjective and a matter of opinion. But I think it is very dangerous to grow an AI
[0:51:21] And teach it to lie. Yeah, for sure. And then kind of a tongue-in-cheek question. Would you accept a meeting from the AI czar, Kamala Harris, if she wanted to meet with XAI at the White House? Yeah, of course. You know, the reason that meeting happened was because I was pushing for it. So I was one who really pushed hard to have that meeting happen. I wasn't advocating for...
[0:51:51] I think we do need some sort of regulatory oversight. It's not like I think regulatory oversight is some nirvana.
[0:52:20] Perfect thing, but I think it's better than nothing. When I was in China recently, meeting with some of the senior leadership there, I took pains to emphasize the importance of AI regulation. I believe they took that to heart, and they are going to do that. Because the biggest counter-argument that I get for regulating AI in the West is that China will not regulate, and then...
[0:52:48] China will leap ahead because we're regulating. They're not. I think they are going to regulate. And the proof will be in the pudding. But I think there's, you know, I did point out in our meetings with them that if you do make a digital superintelligence, that you could end up, that could end up being in charge, you know. So, you know, that's, I think the CCP does not want to find themselves
[0:53:18] Yeah, so some kind of regulatory authority that's international. Obviously enforcement is difficult, but I think we should still aspire to do something in this regard. Awesome. Thank you. Omar, do you want to speak?
[0:54:00] Yeah, hey, my question is about silicon. You know, Tesla's got a great silicon team designing chips to hardware accelerated inference. I'm not sure. I think we cannot hear you for some reason. Oh, okay. Omar, go ahead. Can you hear me? You can hear me. Okay, well, my question's about silicon. You know, Tesla has a team that's hardware accelerating inference and training with their own custom silicon.
[0:54:37] Do you guys envision with XAI building off of that or just sort of using what's off the stock from NVIDIA? Or how do you think about custom silicon for AI, both in terms of training and inference? So, yeah, that's somewhat a Tesla question. Tesla is building custom silicon. I wouldn't call anything that Tesla is producing a GPU.
[0:55:18] Although one can characterize it in GPU equivalents, or say A100s or H100s equivalents. And all the Tesla cars have highly energy optimized inference computers in them, which we call Hardware 3. It's a Tesla designed computer. And we're now shipping Hardware 4, which is, depending on how you count it, maybe
[0:55:46] Three to five times more capable in hardware three. In a few years, there'll be hardware five, which will be four or five times more capable in hardware four. And I think the inference stuff is going to be, if you're trying to serve potentially billions of queries per day, inference, energy optimized inference is extremely important. You can't even throw money at the problem at a certain point.
[0:56:18] Because you need the electricity generation, you need the step-down voltage transformers. So if you actually don't have enough energy and enough transformers, you can't run your transformers. You need transformers for transformers. So I think Tesla will have a significant advantage in energy-efficient inference.
[0:56:50] Then Dojo is obviously about training, as the name suggests. Dojo 1 is, I think, it's a good initial entry for training efficiency. It has some limits, especially on memory bandwidth. So it's not well optimized to run LLMs. It does do a good job of processing images.
[0:57:21] And then Dojo2, we've taken a lot of steps to alleviate the memory bandwidth constraint such that it is capable of running LLMs as well as other forms of AI training efficiently. My prediction is that we will go from an extreme silicon shortage today to probably a voltage transformer,
[0:57:54] Shortage in about a year and then an electricity shortage in two years. That's roughly where things are trending. I think the metric that will be most important in a few years is useful compute per unit of energy.
[0:58:27] The useful compute per joule is still the thing that matters. You can't increase the output of the sun. So then it's just, well, how much useful stuff can you get done for as much energy of the sun that you can harness? So do you see XAI leveraging this custom silicon at all, given how important energy efficiency is, or maybe working together with the Tesla team at all on that? Sorry, could you repeat the question?
[0:59:05] Do you foresee XAI working with Tesla at all, leveraging some of this custom silicon, maybe designing their own in the future? Yeah, so we are going to work with Tesla on the silicon front and maybe on the AI software front as well.
[0:59:44] Obviously, any relationship with Tesla has to be an arm's-length transaction, because Tesla is a publicly traded company and a different shareholder base. But obviously, it would be a natural thing to work in cooperation with Tesla, and I think it will be of mutual benefit to Tesla as well in accelerating Tesla's...
[1:00:11] Self-driving capabilities, which is really about solving real-world AI. I'm feeling very optimistic about Tesla's progress on the real-world AI front, but obviously the more smart humans that help make that happen, the better. Okay, Kim.com. Hey, Elon, thanks for bringing me up. Congrats on putting a nice team together. It seems like you found some good talent there for XAI.
[1:00:44] My question is, you mentioned not too long ago that you think AGI is possible within the next 20 years. And whoever achieves AGI first and achieves to control it will dominate the world. Those in power clearly don't care about humanity like you do. How are you going to protect XAI, especially from a deep state takeover? That's a good question, actually.
[1:01:16] Well, I mean, first of all, I think it's not going to happen like overnight. It's not going to be like one day it's, you know, not AGI, next it is. It's going to be gradual. You'll see it coming. I guess in the U.S., at least there are a fair number of protections against government interference.
[1:01:47] I guess we obviously use the legal system to prevent improper government interference. So I think we do have some protections there that are pretty significant. But we should be concerned about that. It's not a risk to be dismissed.
[1:02:18] Yeah, it is a risk. Like I said, I think we've got probably the best protections of any place in the US in terms of limiting the power of government to interfere with non-governmental organizations. But it's something we should be careful of. I don't know what better to do than, you know, I think it's probably best in the US.
[1:02:50] I'm open to ideas here. I know you're not the biggest fan of the US government. Yeah, obviously. But the problem is they already have a tool called the National Security Letter, which they can apply to any tech company in the US and make demands of.
[1:03:14] You know, the company to fulfill certain requirements without even being able to tell the public about these demands. And that's kind of frightening, isn't it? Well, I mean, there really has to be a very major national security reason to secretly demand things from companies. And now it obviously depends strongly on the willingness of that company to fight back against things like FISA requests.
[1:03:45] And, you know, at Twitter or XCorp, as it's now called, we will respond to FISA requests, but we're not going to rubber stamp it like it used to be. It used to be like anything that was requested would just get rubber stamped and go through, which is not, which is obviously bad for the public. So we're much more rigorous in...
[1:04:11] And we're all being much more rigorous in not just rubber-stamping FISA requests. And it really has to be a danger to the public that we agree with. And we will oppose with legal action anything we think is not in the public interest. It's the best we can do. And we're the only social media company doing that, as far as I know. And it used to be just open season, as we saw from the Twitter files.
[1:04:42] And I was encouraged to see the recent legal decision where the courts reaffirmed that the government cannot break the First Amendment of the Constitution, obviously. So that was a good legal decision. So that's encouraging. So I think, yeah, a lot of it actually does depend on the willingness of a company to oppose government demands in the US. And obviously our willingness will be high.
[1:05:14] But I don't know anything more that we can do than that. But we will try to also be as transparent as possible. So other citizens can raise the alarm bell and oppose government interference if we can make it clear to the public that we think something is happening that is not in the public interest. Fantastic. So do we have your commitment if you ever receive...
[1:05:47] A national security request from the US government, even when it is prohibited for you to talk about it, that you will tell us that that happened. I mean, it really depends on the gravity of the situation. I mean, I would be willing to go to prison or risk prison if I think the public good is at risk in a significant way. You know, that's the best I can do. That's good enough for me. Thank you, Ilan. Thank you.
[1:06:32] On a more positive note, how do you want XAI to benefit humanity? And then how is your approach different to other AI projects? Maybe that's a more positive question. Well, you know, I've really struggled with this whole AGI, I think, for a long time. And I've been somewhat resistant to work on making it happen.
[1:07:07] The reason, I should say, I can just give you some backstory on OpenAI. I mean, the reason OpenAI exists is because after Google acquired DeepMind, and I used to be close friends with Larry Page, I would have these long conversations with him about AI safety, and he just wasn't taking AI safety, at least at the time, seriously enough. And in fact, at one point, called me a speciesist for...
[1:07:35] Being too much on Team Humanity, I guess. And I'm like, okay, so what you're saying is you're not a speciesist? I don't know, that doesn't seem good. And at the time, with Google and DeepMind combined, Larry, with the support, they have super voting control, so provided Larry has the support of Sergei or Eric.
[1:08:05] Then they have total control over what's now called Alphabet. So they had probably three quarters of the AI talent in the world and lots of money and lots of computers. So it's like, man, we need some sort of counterweight here. So that's where I was like, well, what's the opposite of Google DeepMind would be an open source nonprofit? Now, because fate loves irony.
[1:08:35] OpenAI is now super close source and frankly voracious for profit because they want to spend my understanding is a hundred billion dollars in three years which requires you know if you're trying to get investors for that you've got to make a lot of money. So you know OpenAI has strayed quite you know really in the opposite direction from its sort of founding charter which is
[1:09:07] Again, very ironic, but fate loves irony. There's a friend of mine, Jonah Nolan, who says the most ironic outcome is the most likely. Well, here we go. So, yeah. So now, hopefully XAI is not even worse. But I mean, I think we should be careful about that. But it really seems like, look, at this point, AGI is going to happen.
[1:09:38] So there's two choices, either be a spectator or a participant. And as a spectator, one can't do much to influence the outcome. As a participant, I think we can create a competitive, an alternative that is hopefully better than Google DeepMind or OpenAI Microsoft. In both the cases of...
[1:10:08] If you look at the incentive structure, Alphabet, as a publicly traded company, has a lot of incentives to behave. It's got public company incentives, essentially. You've got all these ESG mandates and stuff that I think push companies in questionable directions.
[1:10:38] As a company that's not publicly traded, XAI is not subject to the market-based incentives or really the non-market-based ESG incentives. We're a little freer to operate. I think our AI can give answers that people may find controversial, even though they are actually true.
[1:11:09] So they might not, you know, they won't be phylically correct at times. And they will, probably a lot of people will be offended by some of the answers. But as long as it's, you're trying to optimize for truth with least amount of error, I think we're doing the right thing. Yeah. Scopalizer? Yeah. Okay.
[1:11:39] Twitter has a lot of data in it that could help build a validator, i.e. check some of the facts that a system kicks out. Because we all know that GPT confabulates things, makes things up. And so that's one place I'd like to hear you talk about. The other place is ChatGPT found me a screw at a Lowe's, but it didn't find me a coffee at San Jose International Airport.
[1:12:08] Are you building an AI that has a world knowledge, a 3D world knowledge, to navigate people around the world to different things? Well, I think it's really not going to be a very good AI if it can't find you a coffee at the airport. So, yeah, I guess we need to understand the physical world as well, not just the Internet. I'm talking a lot. You guys should talk more.
[1:12:40] Yeah, those are great ideas, Robert, especially the one about verifying information online or on Twitter is something that we thought about. On Twitter, we have community notes. So that's actually a really amazing data set for training a language model to try to verify facts on your site. We'll have to see whether that alone is enough because we know that with the current technology, there's a lot of weaknesses like it's unreliable, it hallucinates facts, and we'll have to probably invent specific techniques to counter that and to make sure that our models are
[1:13:09] More factual, that they have better reasoning abilities. So that's why we brought in people with a lot of expertise in those areas, especially mathematics is something that we really care about, where we can verify that the proof of a theorem is correct automatically. And then once we have that ability, we're going to try to expand that to more fuzzier areas, things where there's no mathematical truth anymore.
[1:13:38] The truth is not a popularity contest. But if one trains on what the most likely word is that follows another word from an internet data set, then there's obviously that's a pretty major problem in that it will give you an answer that is popular but wrong. So it used to be that most people thought...
[1:14:06] Well, probably almost everyone on Earth thought that the sun revolved around the Earth. And so if you did some sort of training on, some GPD training in the past, you'd be like, oh, the sun revolves around the Earth because everyone thinks that. That doesn't make it true. If a Newton or Einstein comes up with something that is actually true, it doesn't matter if all the other physicists in the world disagree. Reality is reality.
[1:14:34] You have to ground the answers in reality. The current models just imitate the data that they're trained on. And what we really want to do is to change the paradigm away from that to actually models discovering the truth. So not just repeating what they've learned from the training data, but actually making true new insights, new discoveries that we can all benefit from. Yeah. Anybody on the team want to say anything or ask questions?
[1:15:09] Do you think maybe haven't been asked yet? Sure, actually. Yeah, so I guess some of us heard your future of AI spaces on Wednesday. So that's something I think a lot of us mind is like the regulations and the AI safety spaces, how the current development and also the international.
[1:15:43] International coordination problems and how the US AI companies will affect this global AI development. Do you want to give a summary on what you talked about on Wednesday? So essentially you said like the regulations would be good, but you don't want to slow down the progress too much.
[1:16:20] Yeah, I think the right way for regulations to be done is to start with insight. So first, you know, any kind of regulatory authority, whether public or private, first tries to understand, like, make sure there's like a broad understanding. And then there's proposed rulemaking. And if that proposed rulemaking is agreed upon by all or most parties,
[1:16:51] Like my prediction for...
[1:17:20] AGI would roughly match that, which I think Reiko as well at one point said 2029. That's roughly my guess too, give or take a year. So if it takes an additional six months or 12 months for AGI, that's really not a big deal. Spending a year to make sure AGI is safe is probably worthwhile.
[1:17:47] If that's what it takes. But I wouldn't expect it to be a substantial slowdown. And I can also add that understanding the inner workings of advanced AI is probably the most ambitious project out there as well, and also aligns with the XAS mission of understanding the universe. And it's probably not possible for aerospace engineers to build a safe rocket.
[1:18:18] They don't understand how it works. And that's the same approach we want to take at XAI for our safety plans. And as the AI advances across different stages, the risk also changes. And we want to be fluid across all the stages. Yeah. If I think about what actually makes regulations effective with cars and rockets, it's actually
[1:18:49] It's not so much that the regulators are instructing Tesla and SpaceX, but more that since we have to think about things internally and then justify it to regulators, it makes us just really think about the problem more. And in thinking about the problem more, it makes it safer, as opposed to the regulators specifically pointing out ways to make it safer.
[1:19:17] It just forces us to think about it more. I just wanted to make another point, so independent of the safety. It's more like my experience at Alphabet was that it was extremely, there was a lot of red tape around involving external people like other entities to collaborate with or expose our models to them because of the lot of red tape around exposing anything that we were doing internally.
[1:19:50] So I wanted to ask Ilan whether, so I hope that here we have a bit more freedom to do so, or what's your philosophy about collaborating with more external entities like academic institutions or other researchers in the area? Yeah, I certainly support collaborating with others. So, I mean, it sounds like some of the, yeah, concerns with like,
[1:20:20] Any kind of like large publicly traded companies, it's like they're worried about being embarrassed in some way or being sued or I don't know, something. But there's like somewhat proportionate to the size of the legal department. Our legal department currently is zero, so that won't be zero forever, but, you know, the...
[1:20:49] It's also very easy to sue publicly traded companies, like class action lawsuits. I mean, we desperately need class action lawsuit reform in the United States. The ratio of good class action lawsuits to bad class action lawsuits is way out of whack. And it effectively ends up being a tax on consumers. And somehow other countries are able to survive without class action. So it's not clear we need that body of law at all.
[1:21:20] But that is a major problem with publicly traded companies. So it's just non-stop legal lawsuits. Yeah, so I do support collaborating with others and generally being actually open. So, you know, the thing I'm trying to say, it's actually, it's quite hard to, like, if you're,
[1:21:53] Innovating fast, that is the actual competitive advantage, is the pace of innovation, as opposed to any given innovation. That really has been the strength of Tesla and SpaceX, is that the rate of innovation is the competitive advantage, not what has been developed at any one point. In fact, SpaceX, there's almost no patents.
[1:22:23] Tesla open sources patents. We use all our patents for free. So long as SpaceX and Tesla continue to innovate rapidly, that's the actual defense against competition as opposed to patents and trying to hide things and just treating patents basically like a minefield. Tesla does continue to make patents and open source them in order to...
[1:22:54] Hey Walter. Hey. A lot of the talk about AI since March has been on large language models and generative AI. You and I for the book
[1:23:28] Also discuss the importance of real-world AI, which is the things including coming out of both Optimus and Tesla FSD. To what extent do you see XAI involved in real-world AI as a distinction to what, say, OpenAI is doing? And you have a leg up to some extent by having done FSD. Yeah, right. I mean, Tesla is the leader, I think, by pretty much
[1:24:01] Long margin in real-world AI. In fact, the degree to which Tesla is advanced in real-world AI is not well understood. Yeah. And I guess since I spend a lot of time with the Tesla AI team, I kind of know how real-world AI is done. And there's a lot to be gained by collaboration with Tesla. I think bi-directionally.
[1:24:34] XAI can help Tesla and vice versa. We have some collaborative relationships as well, like our material science team, which I think is maybe the best in the world, is actually shared between Tesla and SpaceX. And that's actually quite helpful for recruiting the best engineers in the world because it's more interesting to work on advanced electric cars and rockets.
[1:25:04] Then just either one or the other. So like that was really key to recruiting Charlie Coleman, who runs the advanced materials team. He was at Apple and I think pretty happy at Apple and be like, well, he could work on electric cars and rockets. He's like, hmm, that sounds pretty good. So he wouldn't take either one of the jobs, but he was willing to take both. Yeah, so I think that is a really important thing. And like I said, there are like some...
[1:25:37] Pretty big insights that we've gained at Tesla in trying to understand real world AI. Taking video input and compressing that into a vector space and then ultimately into steering and pedal outputs. And Optimus? Yeah, Optimus.
[1:26:09] Optimus is still at the early stages, and we definitely need to be very careful with Optimus at scale once it's in production, that you have a hard-coded way to turn off Optimus for obvious reasons, I think. There's got to be a hard-coded ROM local cutoff that no amount of...
[1:26:39] Updates from the internet can change that. So we'll make sure that Optimus is quite easy to shut down. It's extremely important. Because at least if the car is intelligent, well, at least you can climb a tree or go up some stairs or something, go in a building. But Optimus can follow you in the building. So any kind of robot that can follow you in the building, that is intelligent and connected.
[1:27:12] We've got to be super careful with safety. Thanks. No problem. Let's see. So any last things we should touch on? Yeah, so one thing I wanted to just talk about before we're concluded is how impactful, sorry about that, a little feedback, is just about the impactfulness of AI as a means of providing equal opportunity
[1:27:53] to humanity from all walks of life and the importance of democratizing it as far as our mission statement goes. Because if you think about the history of humanity and access to information, before the printing press, it was incredibly hard for people to get access to new forms of knowledge. And being able to provide that level of communication to people is hugely
[1:28:22] Deflationary in terms of wealth and opportunity and equality. And so we're really at a new inflection point in the development of society when it comes to getting everyone the same potential for great outcomes regardless of your position in life. And so when we're talking about removing the monopolization of ideas and about controlling this technology from paid subscription services or even worse from the
[1:28:52] The political censorship that may come with whatever capital has to supply these models. We're really talking about democratizing people's opportunities to not only better their position in life, but just advance their social status in the world at an unprecedented level in history. And so as a company, when we talk about the importance of truthfulness and being able to
[1:29:18] Reliably trust these models, learn from them, and make scientific advancement, make societal advancements. We're really just talking about improving people's qualities of life and improving everyone, not just the top tech people in Silicon Valley who have access to it. It's really about giving this access to everyone. And I think that's a mission that our whole team shares.
[1:29:48] Before we sign off here, just one last question for Elon. So assuming that XAI is successful at building human-level AI or even beyond human-level AI, do you think it's reasonable to involve the public in decision-making in the company, or how do you see that evolving in the long term? Yeah, as with everything, I think we're very open to critical feedback and welcome that. We should be criticized. That's a good thing.
[1:30:17] Actually, one of the things that I like sort of X slash Twitter for is that there's plenty of negative feedback on Twitter, which is helpful for ego compression. So the best thing I can think of right now is that any human that wants to sort of have a vote in the future of XAI ultimately should be allowed to. So basically,
[1:30:48] Provided you can verify that you're a real human. That any human that wishes to have a vote in the future of XAI should be allowed to have a vote in the future of XAI. Yeah. Maybe that's like some normal fee, like 10 bucks or something. I don't know. 10 bucks and prove you're a human. And then you can have a vote. Anyone who's interested. That's the best thing I can think of right now, at least. All right, cool. On that note.
[1:31:24] Thank you for participating and we'll keep you informed of any progress that we make and I look forward to having a lot of great people join the team. Thanks. Wow, you made it to the very end. Well, YouTube thinks that you should watch this video next, but if you haven't finished watching this Elon Musk interview, check this one out first. My name is Malpo. Just like and subscribe if you haven't yet and I will see you in the next video. Thank you so much for watching.