Say what you want about Elon Musk, but once this technological innovator sets his sights on something, he fights to win.
His newest artificial intelligence startup, xAI, which was only founded last July, just brought online a new supercomputer called “Colossus” over Labor Day weekend, which is designed to train a large-scale language model (LLM) called Grok, a rival to Open AI’s better-known GPT-4.
Grok is limited to paid subscribers of Musk’s social media platform X, but many Tesla experts speculate that it will eventually form the artificial intelligence that powers the electric vehicle maker’s humanoid robot Optimus.
Musk estimates that the strategic lighthouse project could eventually earn Tesla $1 trillion in annual profits.
New xAI data center in Tennessee will house 100,000 servers Nvidia has benchmarked its Hopper H100 processor more than any other AI computing cluster..
“From start to finish, it was done in 122 days,” Musk wrote, calling Colossus “the world’s most powerful AI training system.”
This weekend, the @xAI team brought the Colossus 100k H100 training cluster online, taking 122 days from start to finish.
Colossus is the most powerful AI training system in the world, and will double in size to 200k (50k H200s) within the next few months.
wonderful…
— Elon Musk (@elonmusk) September 2, 2024
The xAI effort isn’t stopping there: Musk estimates he could double Colossus’ computing power within months if he can procure 50,000 chips from Nvidia’s newer, more advanced H200 series, which are roughly twice as powerful.
Musk and xAI did not respond to Fortune’s requests for comment.
Built to train Grok-3, which could become the next leader in AI models
Considering xAI only selected its Memphis location in June, the pace at which Colossus is being established has been ferocious.
Additionally, several major tech companies, including Microsoft, Google and Amazon, are joining Musk in the current AI gold rush to vie for Nvidia’s popular Hopper series AI chips.
But the AI entrepreneur is a valued Nvidia customer, pledging to spend $3 billion to $4 billion on CEO Jensen Huang’s hardware this year just for Tesla.
Moreover, xAI has had a head start by leveraging supplies of AI chips already delivered to EV maker Tesla.
The Memphis cluster will train Musk’s third generation Grok robots.
“We’re hoping to release Grok-3 by December, at which point it will be the most powerful AI in the world,” he told conservative podcaster Jordan Peterson in July.
An early beta version of Grok-2 was made available to users last month.
The model was trained on just about 15,000 Nvidia H100 graphics processors, but by some measures it is already one of the most capable AI large-scale language models, according to a leaderboard of competing chatbots.
Increasing the number of GPUs by nearly seven-fold suggests Musk isn’t ready to cede the race to develop artificial general intelligence to OpenAI, which he helped co-found in late 2015 out of concern that Google had a monopoly on the technology.
Musk has since clashed with CEO Sam Altman and is suing the company for a second time.
To overcome this disadvantage, xAI In May, the company raised $6 billion in a Series B funding round, led by venture capitalists such as Andreessen Horowitz and Sequoia Capital. Fidelity and Saudi Arabian Prince Alwaleed bin Talal’s Kingdom Holdings.
Tesla could be the next company to invest in Musk’s xAI
Musk also said he intends to propose to Tesla’s board of directors a vote on whether to invest $5 billion in xAI, a move welcomed by many shareholders.
Of the roughly $10 billion in AI spending that I said Tesla will do this year, about half is internal, primarily in our Tesla-designed AI inference computers and sensors in all of our cars, and in Dojo.
NVidia hardware is a good choice for building AI training superclusters.
— Elon Musk (@elonmusk) June 4, 2024
But the xAI supercomputer cluster is causing unease in Memphis because city officials rushed to agree to the project, which would bring economic activity back to a part of the city that was last home to an Electrolux white goods factory.
One of the main concerns is It would put a strain on city resources. Officials at the city’s utility company, MLGW, said the Colossus Up to one million gallons of water would be required per day to cool the servers, consuming as much as 150 megawatts of electricity.
But Musk is a big thinker, and anything worth doing is worth doing fast, or risk falling behind the competition.
Speaking with podcaster Lex Friedman after he toured xAI’s fast-growing operations, Musk said speed is a key element of the company’s five-step management process.
“You can speed up anything. No matter how fast you think you can do it, you can do it faster,” he said last month.
The new special issue features a Wall Street legend getting a dramatic makeover, tales of cryptocurrency shenanigans and badly behaved poultry royalty, and more.
Read the story.
