Elon Musk’s AI company xAI is developing a specialized supercomputer to support the next generation of its AI chatbot Grok, based on a report by The Information.
The ambitious businessman plans to launch the supercomputer by Fall 2025, and ensures to deliver it on schedule, as indicated by an xAI investors’ presentation in May.
The intended supercomputer is said to employ 100,000 Nvidia H100 GPUs and would be at least quadruple the size of the largest existing GPU clusters, like those “utilized by Meta for AI model training.” Such considerable power is vital to train and operate the constantly advancing Grok language model.
The report further disclosed that xAI is collaborating with Oracle for the supercomputer’s construction.
Queries to xAI and Oracle remain unanswered.
The reported supercomputer project coincies with xAI’s recent announcement of a $6 billion series B funding round.
“xAI is pleased to announce our Series B funding round of $6 billion with participation from key investors including Valor Equity Partners, Vy Capital, Andreessen Horowitz, Sequoia Capital, Fidelity Management & Research Company, Prince Alwaleed Bin Talal and Kingdom Holding, amongst others,” the company said in a statement.
The funds from the round will be used to take xAI’s first products to market, build advanced infrastructure, and accelerate the research and development of future technologies, the statement added. xAI also said that it “will continue on this steep trajectory of progress over the coming months, with multiple exciting technology updates and products soon to be announced.”
This funding is expected to fuel xAI’s product development, infrastructure advancements, and future AI research.
xAI launched Grok in November 2023 inspired by “The Hitchhiker’s Guide to the Galaxy” and projected as “more than just a traditional chatbot.” It strives to answer user queries in an informative and even witty manner, while also prompting users with relevant questions that they might not have considered.
However, as Grok evolved through iterations like Grok 1.5 with long-context understanding and Grok 1.5V with visual information processing, its computational requirements have dramatically increased. Training Grok 2 model required 20,000 Nvidia H100 GPUs and Grok 3 and beyond would require 100,000 Nvidia H100 GPUs, the report added.
This exceptional growth in computational requirements necessitates a supercomputer. The current frontrunners in the supercomputer race are Frontier (US), Aurora (US), Eagle (US), and Fugaku (Japan). While the exact specifications of xAI’s planned supercomputer are not known yet, it would fall within the supercomputer category, if xAI’s above claims are met, potentially challenging the dominance of existing machines.
