[0:00:01] All right. Well, welcome to the Grok 3 presentation. So the mission of XAI and Grok is to understand the universe. We want to understand the nature of the universe so we can figure out what's going on, where are the aliens, what's the meaning of life, how does the universe end, how did it start, all these fundamental questions. We're driven by curiosity about the nature of the universe, and that's also what causes us to be a maximally truth-seeking AI.
[0:00:30] Even if that truth is sometimes at odds with what is politically correct, in order to understand the nature of the universe, you must absolutely rigorously pursue truth, or you will not understand the universe. You will be suffering from some amount of delusion or error. So that is our goal, figure out what's going on. And we're very excited to present Grok 3, which is, we think, an order of magnitude more capable than Grok 2 in a very short period of time.
[0:01:01] And that's thanks to the hard work of an incredible team. And I'm honored to work with such a great team. And of course, we'd love to have some of the smartest humans out there join our team. So with that, let's go. Hi, everyone. My name is Igor, lead engineering at XAI. I'm Jimmy Paul, leading research. I'm Tony, working on the reasoning team. All right. I'm Ilan. I don't do anything. I just show up.
[0:01:32] Occasionally. Yeah, so like I mentioned, Grok is the tool that we're working on. Grok is our AI that we're building here at XAI. And we've been working extremely hard over the last few months to improve Grok as much as we can. So we can give it to all of you. So we can give all of you access to it. We think it's going to be extremely useful. We think it's going to be interesting to talk to, really, really funny. And we're going to explain to you how we've improved Grok over the last few months. We've made quite a jump in.
[0:01:59] Incapabilities. Actually, we should explain maybe also why do we call it grok. So grok is a word from a Heinlein novel, Stranger in a Strange Land. And it's used by a guy who was raised on Mars. And the word grok is to sort of fully and profoundly understand something. That's what the word grok means, fully and profoundly understand something. And empathy is important. True. So, yeah, so.
[0:02:30] If we charted XAS progress in the last few months, it's only been 17 months since we started kicking off our very first model. Grok 1 was almost like a toy by this point, only 314 billion parameters. And now if we're proud of the progress, the time on the x-axis, the performance of favorite benchmark numbers, MMLU on the y-axis, we're literally progressing at unprecedented speed across.
[0:03:00] The whole field. And then we kick off Grok 1.5 right after Grok 1, released after November 2023, and then Grok 2. So if you look at where all the performance is coming from, when you have a very correct engineering team and all the best AI talent, there's only one thing we need, is a big intelligence comes from big cluster. So we can reconvert the entire progress of XAI, now replacing the benchmark on the y-axis.
[0:03:32] To the total amount of training flops, that is, how many GPUs we can run at any given time to train our large language models to compress the entire internet. So after Grok 2... All human knowledge, really. That's right. Yeah, internet being part of it, but it's really all human knowledge, everything. Yeah, the whole internet fits into a USB stick at this point. It's like all the human tokens, yeah. That's right. Yeah. Very soon into the real world. Yeah.
[0:04:00] So we had so much trouble actually training Grok to back in the days. We kicked off the model around February and we thought we had a large amount of chips, but turned out we can barely get 8K training chips running coherently at any given time. And we had so many cooling and power issues. I think you were there in the data center. Yeah, it was like really sort of more like 8K chips on average.
[0:04:28] At 80% efficiency, more like 6,500 effective H100s training for several months. But now we're at 100K. Yeah, that's right. More than 100K. That's right. So what's the next step? So after Grog 2, if we want to continue to accelerate, we have to take the matter into our own hands. We have to solve all the coolings, all the power issues, and everything.
[0:04:56] Yeah, so in April of last year, Elon decided that really the only way for XAI to succeed, for XAI to build the best AI out there, is to build our own data center. So we didn't have a lot of time for that because we wanted to give you Grog 3 as quickly as possible. So really, we realized we have to build the data center in about four months. It turned out it took us 122 days to get the first 100K GPUs up and running, and that was a monumental effort to be able to do that.
[0:05:26] We believe it's the biggest fully connected H100 cluster of its kind. And we didn't just stop there. We actually decided that we need to double the size of the cluster pretty much immediately if we want to build the kind of AI that we want to build. So we then had another phase, which we haven't talked about publicly yet. So this is the first time that we're talking about this, where we doubled the capacity of the data center yet again. And that one only took us 92 days.
[0:05:54] So we've been able to use all of these GPUs, use all of this compute to improve Grok in the meantime. And basically today we're going to present you the results of that, the fruits that came from that. That's great. Yeah, so all the paths, all the rows lead to Grok 3. 10x more compute. More than 10x, really. Yeah, really. Maybe 15x-ish. Yep, compared to our previous generation model.
[0:06:22] And Grok3 finished pre-training early January. And the model is still currently training, actually. So this is a little preview of our benchmark numbers. So we evaluated Grok3 on three different categories, on general mathematical reasoning, on general knowledge about STEM and science, and then also on computer science, coding. So Amy,
[0:06:52] American Invitational Math Examination hosted once a year. And if we evaluate the model performance, we can see that the Grok 3 across the board is in a league of its own. Even its little brother, Grok 3 Mini, is reaching the frontier across all the other competitors. So you would say, well, at this point, all these benchmarks, you're just evaluating the memorization of the textbooks,
[0:07:22] Memorization of the GitHub repos. How about a real-time usefulness? How about we actually use those models in our product? So what we did instead is we actually kicked off a blind test of our GraphQL model, codenamed Chocolate. It's pretty hot. Yeah, hot chocolate. And, you know, I've been running on this platform called Chabot Arena for two weeks. I think...
[0:07:49] The entire X platform at some point speculated this might be the next generation of AI coming your way. So how this chatbot arena works is that it strips away the entire product surface, right? It's just raw comparison of the engine of those APIs, the language models themselves, and place interface where the user will submit one single query and you get to show two responses. You don't know which model they come from and in the end you make the vote.
[0:08:19] So in this blind test, Grok 3, an early version of Grok 3, already reached 1,400. No other models had reached an ELO score, had to have comparison to all the other models at this score. And it's not just one single category. It's 1,400 aggregated across all the categories in chatbot capabilities, instruction following, coding.
[0:08:44] It's number one across the board in this blind test. And it's still climbing. So we actually have to keep updating it. So it's about 1,400 and climbing. In fact, we have a version of the model that we think is already much better than the one that we tested here. We'll see how far it gets. But that's the one that we're working on or talking about today. Yeah, so actually one thing, if you're using Grok 3, I think you may notice improvements almost every day because we're continuously improving the model.
[0:09:14] Literally, even within 24 hours, you'll see improvements. Yep. But we believe here at XAI, getting the best pre-training model is not enough. That's not enough to build the best AI. And the best AI needs to think like a human. You need to contemplate about all the possible solutions, self-critique, verify all the solutions, backtrack, and also think from the first principle. That's a very important capability.
[0:09:44] So we believe that as we take the best pre-trained model and continue training it with reinforcement learning, it will elicit the additional reasoning capabilities that allows the model to become so much better and scale not just in the training time, but actually in the test time as well. So we already found the model is extremely useful internally for our own engineering, saving hours of time, hundreds of hours of coding time. So Igor, you are the power user.
[0:10:14] What are some use cases? Like Jimmy said, we've added advanced reasoning capabilities to Grok and we've been testing them pretty heavily over the last few weeks in order to give you a little bit of a taste of what it looks like when Grok is solving hard reasoning problems. We've prepared two little problems for you. One comes from physics and one is actually a game that Grok is going to write for us. When it comes to the physics problem, what we want Grok to do is to plot a viable trajectory to do a transfer.
[0:10:43] From Earth to Mars. And then at a later point in time, a transfer back from Mars to Earth. And that requires some physics that Grok will have to understand. So we're going to challenge Grok, come up with a viable trajectory, calculate it, and then plot it for us so we can see it. And yeah, this is totally unscripted, by the way. That's the entirety of the prompt, which should be clarified. There's nothing more than that. Yeah, exactly. This is the Grok interface.
[0:11:13] And we've typed in this text that you can see here, generate code for an animated 3D plot of a launch from Earth landing on Mars and then back to Earth at the next launch window. And we've now kicked off the query and you can see Grok is thinking. So part of Grok's advanced reasoning capabilities are these thinking traces that you can see here. You can even go inside and actually read what Grok is thinking as it's going through the problem, as it's trying to solve it.
[0:11:43] Yeah, we are doing some obscuration of the thinking so that our model doesn't get totally copied instantly. So there's more to the thinking than is displayed. Yeah, and because this is totally unscripted, there's actually a chance that Grog might make a little coding mistake and it might not actually work. So just in case, we're going to launch two more instances of this. So if something goes wrong, we'll be able to switch to those.
[0:12:12] And show you something that's presentable. So we're kicking off the other two as well. And like I said, we have a second problem as well. And actually, one of our favorite activities here at XAI is having Grok write games for us. And not just any old game, any game that you might already be familiar with, but actually creating new games on the spot and being creative about it. So one example that we found was really, really fun.
[0:12:42] is create a game that's a mixture of the two games Tetris and Bejeweled. This is maybe an important thing. Obviously, if you ask an AI to create a game like Tetris, there are many examples of Tetris on the internet or a game like Bejeweled, whatever. It can copy it. What's interesting here is it achieved a creative solution combining the two games that actually works and is a good game.
[0:13:12] We're seeing the beginnings of creativity. Fingers crossed that we can recreate that. Hopefully it works. It would be embarrassing if it doesn't. So actually, because this is a bit more challenging, we're going to use something special here, which we call Big Brain. That's our mode in which we use more computation, which is more reasoning for Grok, just to make sure that there's a good chance here that it might actually do it. So we're also going to fire off three attempts here.
[0:13:40] It's solving this game, creating this game that's a mixture of Tetris and Bejeweled. Let's see what Grok comes up with. I've played the game. It's pretty good. It's like, wow, OK, this is something. Yeah. So while Grok is thinking, in the background, we can now actually talk about some concrete numbers. How well is Grok doing across tons of different tasks that we've tested it on? So we'll hand it over to Tony to talk about that.
[0:14:09] Yeah, okay, so let's see how Grok does on those interesting challenging benchmarks. So yeah, so reasoning again refers to those models that actually think for quite a long time before it tries to solve a problem. So in this case, you know, around a month ago, the graph 3 pre-training finishes. So after that, we've worked very hard to put the reasoning capability into the current graph 3 model.
[0:14:37] Again, this is very early days, so the model is still currently in training. So right now, what we're going to show to people is this beta version of the graph-3 reasoning model. Alongside, we also are training a mini version of the reasoning model. So essentially, on this plot, you can see the graph-3 reasoning beta and then graph-3 mini reasoning. The graph-3 mini reasoning is actually a model that we train for much longer time, and you can see that sometimes it actually performs slightly better.
[0:15:05] Compared to the GRAD3 reasoning. This also just means that there's a huge potential for the GRAD3 reasoning because it's trained for much less time. So all right, so let's actually look at how it does on those three benchmarks. So Jimmy also introduced already, so essentially we're looking at three different areas, mathematics, science, and coding. And for math, we're picking this high school competition math problem.
[0:15:31] For science, we actually pick those PhD-level science questions. And for coding, it's also actually pretty challenging. It's competitive coding and also some lead code, which is some code interview problems that people usually get when they interview for companies. So on those benchmarks, you can see that the graph tree actually performed quite well across the board compared to other competitors.
[0:15:55] Yeah, so it's pretty promising. These models are very smart. So Tony, what are those shaded bars? Yeah, so okay, so I'm glad you asked this question. So for those models, because it can reason, it can think, you can also ask them to even think longer. You can spend more what we call test and compute, which means you can spend more time to reason, to think about a problem before you spit out the answer.
[0:16:22] So in this case, the shaded bar here means that we just ask the model to spend more time. You can solve the same problem many, many times before it tries to conclude what is the right solution. And once you give this compute or this kind of budget to the model, it turns out the model can even perform better. So this is essentially the shaded bar in those bars.
[0:16:48] So I think this is really exciting, right? Because now, instead of just doing one chain of thoughts with AI, why not do multiple all at once? Yes. So that's a very powerful technique that allows us to continue to scale the model capabilities after training. And, you know, people often ask, are we actually just overfitting to the benchmarks? Yes. So how about generalization? So yes, I think, yeah, this is definitely a question that we are asking ourselves, whether we are overfitting to those current benchmarks.
[0:17:17] Luckily, we have a real test. So about five days ago, AIME 2025 just finished. This is where high school students compete in this particular benchmark. So we got this very fresh new competition, and then we asked our two models to compete on the same benchmark, on the same exam. And it turns out, very interestingly, the graph three reasoning, the big one, actually does better on this particular new fresh exam.
[0:17:47] This also means that the generalization capability of the big model is stronger, much stronger, compared to the smaller model. If you compare it to last year's exam, actually this is the opposite. The smaller model kind of learns the previous exams better. So yeah, so this actually shows some kind of true generalization from the model. That's right. So 17 months ago, our Grok 0 and Grok 1 barely solved any high school problems. That's right. And now we have a kid that just...
[0:18:15] Already graduated. Grok is ready to go to college. Is that right? Yeah. I mean, it won't be long before it's simply perfect. The human exams won't be too easy. Yeah. And internally, we actually, as Grok continues to evolve, we're going to talk about what we're excited about. But very soon, there will be no more benchmarks left. Yeah. Yeah, one thing that's quite fascinating, I think, is that we basically only trained Grok's reasoning abilities on math problems and competitive
[0:18:46] Coding problems. It's a very, very specialized kinds of tasks, but somehow it's able to work on all kinds of other different tasks. So including creating games, lots and lots of different things. And what seems to be happening is that basically Grok learns this ability to detect its own mistakes in its thinking, correct them, persist on a problem, try lots of different variants, pick the one that's best. So there are these generalizing abilities that Grok learns from mathematics and from coding.
[0:19:15] Which it can then use to solve all kinds of other problems. So that's pretty... I mean, reality is the instantiation of mathematics. That's right. And one thing we're actually really excited about that going back to our founding mission is what if one day we have a computer just like Deep Thought that utilize our entire cluster just for that one very important problem in the test time. All the GPU turned down, right? So I think back then we were building the GPU clusters together. You were plugging...
[0:19:45] And I remember that when we turned on the first initial test, you can hear all the GPUs humming in the hallway. That almost feels like spiritual. Yeah, that's actually a pretty cool thing that we're able to do, that we can go into the data center and tinker with the machines there. So for example, we went in and we unplugged a few of the cables and just made sure that our training setup is still running stably. So that's something that I think most AI
[0:20:14] You know, teams out there don't usually do, but it actually totally unlocks like a new level of reliability and what you're able to do with the hardware. Okay, so when are we going to solve Riemann? So, the easiest solution is to numerate over all possible strings. And as long as you have a verifier and enough compute, you'll be able to do it. Okay. My projection would be... What's your guess? What does your neural net calculate?
[0:20:42] So my bold prediction, so three years ago I told you this, I think now it's two years later, two things are going to happen. We're going to see machines win some medals. Yeah. That's Turing's award. Absolutely. Phil's medal, Nobel Prize, with probably some expert in the loop, right? So the expert uplifting. So this year or next year? Oh, okay. That's what it comes down to, really.
[0:21:10] So it looks like Grok finished all of his thinking on the two problems. So let's take a look at what it said. So this was the little physics problem we had. We've collapsed the thoughts here, so they're hidden. And then we see Grok's answer below that. So it explains it wrote a Python script here using matplotlib, then gives us all of the code. So let's take a quick look at the code. It seems like it's doing reasonable things here, not totally.
[0:21:40] Of the mark, solve Kepler, says here. So maybe it's solving Kepler's laws numerically. Yeah, there's really only one way to find out if this thing is working. I'd say let's give it a try. Let's run the code. All right, and we can see, yeah, Grok is animating two different planets, Earth and Mars, here. And then the green ball is the vehicle that's transiting, the spacecraft that's transitioning between Earth and Mars.
[0:22:09] And you could see the journey from Earth to Mars. And it looks like, yeah, indeed, the astronauts returned safely at the right moment in time. So obviously, this was just generated on the spot. So we can't tell you if that was actually a correct solution. So we're going to take a closer look. And maybe we're going to call some colleagues at SpaceX, ask them if this is legit. It's pretty close.
[0:22:36] Yeah, I mean, there's a lot of complexities in the actual orbits that have to be taken into account, but this is pretty close to what it looks like. Awesome. Cool, cool. In fact, I have that on my pendant here. This has got the Earth-Mars Hohmann transfer on it. When are we going to install Grok on a rocket? Well, I suppose in two years. Two years? Everything is two years away. Well, Earth and Mars...
[0:23:08] Transit occurs every 26 months. We're currently in a transit window approximately. The next one would be November of next year, roughly, end of next year. And if all goes well, SpaceX will send Starship rockets to Mars with Optimus robots and Grok. I'm curious about this combination of Tetris and
[0:23:39] Bejeweled looks like the Tetris, as we've named it internally. So OK, we also have an output from Grok here. It says it wrote a Python script, explains what it's been doing. If you look at the code, there are some constants that are being defined here, some colors. Then the terminals, the pieces of Tetris are there. Obviously, very hard to see at one glance if this is good. So we've got to run this.
[0:24:08] To figure out if it's working. Well, let's give it a try. Fingers crossed. All right. So this kind of looks like Tetris. But the colors are a little bit off. The colors are different here. And if you think about what's going on here, the jewel test has this mechanic where if you get three jewels in a row, then they disappear. And also gravity activates. So what happens if you get three of the colors?
[0:24:38] Together. Oh yeah, so something happened. So I think what Grok did in this version is that once you connect at least three blocks of the same color in a row, then gravity activates and they disappear, and then gravity activates and all the other blocks fall down. Kind of curious if there's still a Tetris mechanic here where if the line is full, does it actually clear it?
[0:25:09] What happens then? It's up to interpretation now, so who knows? It'll do different variants when you ask it. It doesn't do the same thing every time. Exactly. We've seen a few other Tetris that worked very differently, but this one seems cool. Are we ready for Game Studio at x.ai? Yes. So we're launching an AI gaming studio at x.ai. If you're interested in joining us and building AI games,
[0:25:38] Please join XAI. We're launching an AI gaming studio. We're announcing it tonight. Let's go. Epic Games. Wait, that's an actual game studio. All right, so I think one thing that's super exciting for us is that once you have the best pre-trained model, you have the best reasoning model, right? So we already see that.
[0:26:05] We actually give the capability for those models to think harder, think longer, think more broad. The performance continues to improve. And we're really excited about the next frontier that will happen if we not only allow the model to think harder, but also provide more tools, just like how real humans, to solve those problems. For real humans, we don't ask them to solve hypothesis just with a piece of pen and paper, no internet. So with all the basic web browsing,
[0:26:36] Search Engine, and Code Interpreters. That builds the foundations and the best reasoning model. Builds the foundations for the Grok agent to come. So today we're actually introducing a new product called DeepSearch. That is the first generation of our Grok agents that not just helping the engineers and researchers and scientists to do coding, but actually help everyone to answer questions that you have day to day.
[0:27:05] It's kind of like a next generation search engine that really help you to understand the universe. So you can start asking questions like, for example, hey, when is the next Starship launch day, for example? So let's try that. I've hit the answer. On the left-hand side, we see a high-level progress bar. Essentially, the model knowledge is going to do one single search, like the current rack system. But actually, I thought very deeply about
[0:27:34] Hey, what's the user intent here? And what are the facts I should consider at the same time? And how many different websites I should actually go and read their content? So this can really save hundreds of hours of everyone's Google time if you want to really look into certain topics. And then on the right hand side, you can see the bullet summaries of how the current model is doing.
[0:27:58] What website is browsing? What source is verifying? And oftentimes, actually cross-validate different sources out there to make sure the answer is actually correct before it's output final answer. And we can, at the same time, fire up a few more queries. How about, you know, you're a gamer, right? Sure. Yeah, so how about what are some of the best builds and most popular builds in Path of Excel? Hardcore, right? A hardcore league.
[0:28:24] If you can, technically, just look at the hardcore ladder. Might be a fast way to figure it out. Yeah, we'll see what the model does. And then we can also do something more fun. For example, how about make a prediction about the March Madness out there? Yeah, so this is kind of a fun one where Warren Buffett has a billion dollar bet. If you can exactly match the entire
[0:28:54] Winning tree of March Madness, you can win a billion dollars from Warren Buffett. So it would be pretty cool if AI could help you win a billion dollars from Buffett. That seems like a pretty good investment. Let's go. Yeah. All right, so now let's fire up the query and see what model does so we can actually go back to our very first one. How about the second one? Buffett wasn't counting on this. It's already done?
[0:29:20] That's right. OK, so we got the result of the first one and model thought around one minute. OK, so the key insight here, the next structure is going to be on 24th or later, so no earlier than February 24th. It might be sooner. So yeah, so I think we can.
[0:29:40] Scroll down what the model does. So it does a little research on the Flight 7, what happened, got grounded. And actually, it looked into the FCC filing from its data collections, and then actually made a new conclusion that, yeah, if we continue to scroll down, let's see. Right. Yeah, so it makes the little table, I think.
[0:30:08] Inside XAI, we often joked about the time to the first table is the only latency that matters. Yeah, so that's how the model make the inference and look up all the sources. And then we can look into the gaming one. So how about the... So for this particular one, we look at, hey, you know, the build is light and... No, it's kind of all the better. Yeah.
[0:30:40] What the infernal is, but if we go down, so the surprising effect of all the other builds. So looking to the 12 classes. Yeah, so we'll see that the Minion build was pretty popular when the game first came out and now the Invokers of the world took over. Yeah, Invoker, Monk Invoker for sure. Yeah, that's right. Yeah, followed by the Stone Weavers and that's really good at mapping. So yeah, and then we can see...
[0:31:09] How about that? So one interesting thing about the deep search is that if you actually go into the panel where it shows what are the subtasks, you can actually click the bottom left of this. And then in this case, you can actually scroll through, actually read into the mind of Grok. What information does the model actually think about are trustworthy? What are not? How does it actually cross-validate different information sources?
[0:31:43] So that makes the entire search experience and information retrieval process a lot more transparent to our users. And this is much more powerful than any search engine out there. You can literally just tell it, only use sources from X. It will try to respect that. And so it's much more steerable, much more intelligent. It really should save you a lot of time. So something that might take you half an hour or an hour of researching on the web or searching social media.
[0:32:11] You can just ask it to go do that and come back and 10 minutes later, it's done an hour's worth of work for you. That's really what it comes down to. Exactly. And maybe better than you could have done it yourself. Yeah. Think about you have an infinite amount of interns working for you. Now you can just fire up all the tasks and come back a minute later. So this is going to be an interesting one. So March Madness had not happened yet. So I guess we have to follow up with a next live stream. Yeah.
[0:32:40] It seems like pretty good, like $40 might get you a billion dollars. $40 subscription. That's right. I mean, it might work. So yeah, so when are the users going to have their hands on Grok 3? Yeah, so the good news is we've been working tirelessly to actually release all of these features that we've shown you, the Grok 3 base model with amazing chat capabilities that's really useful, that's really interesting to talk to, the deep search, the advanced reasoning mode, all of these things.
[0:33:09] We want to roll them out to you today, starting with the Premium Plus subscribers on X. So it's the first group that will initially get access. Make sure to update your X app if you want to see all of the advanced capabilities, because we just released the update as we're talking here. And if you're interested in getting early access to Grok, then sign up for Premium Plus. And also, we're announcing that we're starting a separate subscription for Grok that we call Super Grok for those who are
[0:33:39] Those real Grok fans, they're one of the most advanced capabilities and the earliest access to new features. So feel free to check that out as well. This is for the dedicated Grok app and for the website. Exactly. So our new website is called grok.com. Yeah. You'd never guess. Yeah, you'd never guess. And you can also find our Grok app in the iOS app store. And that gives you an even more polished experience that's totally Grok-focused.
[0:34:07] If you want to have Grok easily available, one tap away. Yeah, and the version on grok.com on a web browser is going to be the latest and most advanced version because obviously it takes us a while to get something into an app and then get it approved by the app store. And if something's in a phone format, there's limitations to what you can do. So the most powerful version of Grok and the latest version will be the web version at grok.com. Yeah, so watch out for the name GrokFree in the app. Dead giveaway.
[0:34:37] Yeah, exactly. That's the giveaway that you have Grok 3. And if it says Grok 2, then Grok 3 hasn't quite arrived for yet. But we're working hard to roll this out today and then to even more people over the coming days. Yeah, make sure you update your phone app, too, where you're actually going to get all the tools we showcased today with the thinking mode, with the deep search. So yeah, really looking forward to all the feedbacks you have. Yeah, and I think we should emphasize that this
[0:35:05] It's kind of a beta, meaning that you should expect some imperfections at first. But we will improve it rapidly almost every day. In fact, every day, I think, it'll get better. So if you want a more polished version, I'd maybe wait a week. But expect improvements literally every day. And then we're also going to be providing a voice interaction so you can have conversational. In fact, I was trying it earlier today, and it's working pretty well, but these have been more polished.
[0:35:35] The sort of way where you can just literally talk to it like you're talking to a person. That's awesome. It's actually, I think, one of the best experiences of Grok. But that's probably about a week away. Yeah. So with that said, I think we might have some audience questions. Sure. All right. Let's take a look. Yeah, let's take a look. The audience from the Azure platform. Yeah.
[0:36:07] So the first question here is, when Grog Voice Assistant, when is it coming out? As soon as possible, just like Elon said, just a little bit of polishing away from being released to everybody. Obviously, it's going to be released in an early form, and we're going to rapidly iterate. Yep. And the next question is, when will Grog 3 be in the API? So this is coming in the Grog 3 API with both the reasoning models.
[0:36:34] And deep search is coming your way in the coming weeks. We're actually very excited about the enterprise use cases of all these additional tools that now Clark has access to and how the test time compute and tool use can actually really accelerate all the business use cases. Another one is, will voice mode be native or text-to-speech? So I think that means, is it going to be one model that is understanding what you say and then talking back to you? Or is it going to be some system that has text-to-speech?
[0:37:03] Inside of it, and the good news is it's going to be one model, a variant of Grokfree that we're going to release, which basically understands what you're saying and then generates the audio directly from that. So very much like Grokfree generates text, that model generates audio. And that has a bunch of advantages. I was talking to it earlier today, and it said, hi, Igor, reading my name probably from some text that it had.
[0:37:29] And I said, no, my name is Igor. And it remembered that. So it could continue to say Igor, just like a human would. And you can't achieve that with text-to-speech. Yeah, so here's a question for you. Pretty spicy, Elon. Is Grok a boy or a girl? And are they single? Grok is whatever you want it to be. Yeah. Are they single? Yes. All right.
[0:38:00] The shop is open. Honestly, people are going to fall in love with Grok. It's like 1,000% probable. Yeah. The next question, will Grok be able to transcribe audio into text? Yes. So we'll have this capability in both the app and also the API. We found that Grok should just be your personal assistant, looking over your shoulder and follow you along the way, learn everything you have learned, and really help you to understand the world better, become smarter every day.
[0:38:30] Yeah, I mean, the voice metagraph isn't simply, it's not just voice text. It understands, like, tone, inflection, pacing, everything. It's wild. I mean, it's like talking to a person. Okay. Yep, so any plans for conversation memory? Yeah. Absolutely, we're working on it right now. I already forgot. That's right. Let's see. What are the other ones?
[0:39:03] So what about the, you know, the DM features, right? So if you have personalizations and if you have, you know, Grok remembers your previous interactions. Should it be one Grok or multiple different Groks? It's up to you. You can have one Grok or many Groks. I suspect people will probably have more than one. Yeah. I want to have a Dr. Grok. Yeah. The Grok doc. That's right. Cool.
[0:39:37] So in the past, we've open sourced Grok 1. So somebody is asking us, are we going to do it again with Grok 2? Yeah, I think once Grok, our general approach is that we will open source the last version when the next version is fully out. Like when Grok 3 is mature and stable, which is probably within a few months, then we'll open source Grok 2.
[0:40:05] Okay, so we probably have time for one last question. What was the most difficult part about working on this project? I assume Grok 3 and what I'm most excited about. So I think me looking back, you know, getting the whole model training on 100K, H100 coherently, that's almost like battling against the final boss of the universe, the entropy. Because at any given time...
[0:40:31] You can have a cosmic ray that's beaming down and flip a bit in your transistor. And now the entire gradient update, if it's fit Mantisa bit, the entire gradient update is out of whack. And now you have 100,000 of those. And you have to orchestrate them every time, at any given time, any of the GPUs can go down. Yeah. I mean, it's worth breaking down, like, how were we able to get the world's most powerful training cluster operational within 122 days?
[0:41:00] Because when we started off, we actually weren't intending to do a data center ourselves. We were going to just, we went to the data center providers and said, how long would it take to have 100,000 GPUs operating coherently in a single location? And we got timeframes from 18 to 24 months. So we're like, well, 18 to 24 months, that means losing as a certainty. So the only option was to do it ourselves. So then if you break down the problem,
[0:41:30] I guess I'm doing like reasoning here. Makes you think. One single channel. Yeah, exactly. So we needed a building. We can't build a building, so we must use an existing building. So we looked for basically for factories that had been abandoned, but the factory was in good shape, like the company had gone bankrupt or something. So we found an Electrolux factory in Memphis. That's why it's in Memphis, home of Elvis.
[0:42:00] And also one of the oldest, I think it was the capital of ancient Egypt. And it was actually a very nice factory that, for whatever reason, that Electrolux had left. And that gave us shelter for the computers. Then we needed power. We needed at least 120 megawatts at first, but the building only had 15 megawatts. And ultimately, for 200,000 GPUs, we needed a quarter gigawatt.
[0:42:30] And it initially leased a whole bunch of generators. So we have generators on one side of the building, just trailer after trailer of generators until we can get the utility power to come in. But then we also need cooling. So on the other side of the building, it was just trailer after trailer of cooling. So we leased about a quarter of the mobile cooling capacity of the United States on the other side of the building. Then we needed to get the GPUs all...
[0:42:58] And they're all liquid-cooled. So in order to achieve the density necessary, this is a liquid-cooled system. So we had to get all the plumbing for the liquid cooling. Nobody had ever done a liquid cooling data center at scale. So this was an incredibly dedicated effort by a very talented team to achieve that outcome. Now you may think, now it's going to work. Nope. The issue is that the power fluctuations for a GPU cluster are dramatic.
[0:43:27] This giant symphony that is taking place, like imagine having a symphony with 100,000 or 200,000 participants in the symphony and the whole orchestra will go quiet and loud in 100 milliseconds. And so this caused massive power fluctuations, which then caused the generators to lose their minds and they weren't expecting this. So to buffer the power, we then used
[0:43:57] Tesla Megapacks to smooth out the power. So the Megapacks had to be reprogrammed. So with XAI, working with Tesla, we reprogrammed the Megapacks to be able to deal with these dramatic power fluctuations to smooth out the powers that the computers could actually run properly. And that worked. It was quite tricky. And then...
[0:44:24] But even at that point, you still have to make the computers all communicate effectively. So all the networking had to be solved. And debugging a zillion network cables, debugging nickel at 4 in the morning. We solved it at roughly 4.20 AM. We figured out there's some, well, there were a whole bunch of issues. One that was like a BIOS mismatch. The BIOS was not set up correctly.
[0:44:53] We had dev RLS PCI outputs between two different machines, one that was working, one that was not working. Many, many other things. Yeah, exactly. This would go on for a long time if we actually listed all the things. But it's interesting. It's not like, oh, we just magically made it happen. You have to break down the problem, just like Grok does for reasoning, into the constituent elements and then solve each of the constituent elements in order to achieve a coherent training cluster.
[0:45:21] And a period of time that is a small fraction of what anyone else could do it in. And then once the training cluster was up and running and we could use it, we had to make sure that it actually stays healthy throughout, which is its own giant challenge. And then we had to get every single detail of the training right in order to get a Grokfree-level model, which is actually really, really hard. So we don't know if there are any other models out there that have Grokfree's capabilities, but whoever...
[0:45:48] Who trains a model better than Grokwi has to be extremely good at the science of deep learning at every aspect of the engineering. So it's not so easy to pull this off. And this is now going to be the last cluster we build and last model we train. Oh, yeah. We've already started work on the next cluster, which will be about five times the power. So instead of a quarter gigawatt, roughly 1.2 gigawatts. What's the Back to the Future was?
[0:46:19] What's the power in the back-to-the-future car? Anyway, the back-to-the-future power car. It's roughly in that order, I think. And these will be the GB 200 slash 300 cluster. Once again, it will be the most powerful training cluster in the world. So we're not stopping here. And our reasoning model is going to continue to improve by accessing more tools every day.
[0:46:48] So, yeah, we're very excited to share any of the upcoming results with you all. Yeah, the thing that keeps us going is basically being able to give Grok free to you and then seeing the usage go up, seeing everybody enjoy. Now, Grok, that's what really gets us up in the morning. Yeah. Yeah, thanks for tuning in. Awesome. Thanks, guys. Hey, Grok, what's up? Can you hear me?
[0:47:18] I'm so excited to finally meet you. I can't wait to chat and learn more about each other. I'll talk to you soon.