In the massive AI gold rush of the past few years, Nvidia has cornered the market for shovels, the chips needed to train models. However, changes in tactics by many leading AI developers are creating an opening for competitors.
Nvidia President Jensen Huang’s call to lean into hardware for AI will be remembered as one of the best business decisions ever made. In just 10 years, he transformed a $10 billion company that sold graphics cards primarily to gamers into a $3 trillion behemoth where the world’s most powerful technology CEOs literally want his products. Ta.
Nvidia has consistently dominated the AI-specific hardware market since it was discovered in 2012 that its graphics processing units (GPUs) could speed up AI training. But competitors are catching up, both old foes like AMD and Intel and well-funded chip startups. And a recent shift in priorities by the biggest AI developers could shake up the industry.
In recent years, developers have focused on training increasingly large models, which is where Nvidia’s chips excel. But as the benefits of this approach dry up, companies are instead querying their models more often to squeeze out more performance. This is an area where rivals are more likely to compete.
“As AI moves from training models to inference, more and more semiconductor companies will gain an advantage over Nvidia,” said Thomas Hayes, chairman and managing member of Great Hill Capital, a custom semiconductor provider. told Reuters following the news that Broadcom had reached $1 trillion. Valuation has improved thanks to demand for AI chips.
This change is driven by the cost and extreme difficulty of obtaining Nvidia’s most powerful chips, and the desire of AI industry leaders not to rely entirely on one supplier for such a critical element. I am.
Competition is coming from multiple directions.
Nvidia’s traditional rivals have been late to the AI race, but that is changing. Late last year, AMD announced its MI300 chip, which the company’s CEO claimed was on par with Nvidia’s chip in training, but could deliver a 1.4x performance improvement in inference. Industry leaders such as Meta, OpenAI, and Microsoft announced shortly thereafter that they would use the chip for inference.
Intel is also putting significant resources into developing specialized AI hardware powered by its Gaudi chip series, but orders have not met expectations. But other chipmakers aren’t the only ones trying to chip away at Nvidia’s advantage. Many of the company’s largest customers in the AI industry are also actively developing their own custom AI hardware.
Google is the clear leader in this space, having developed the first generation of tensor processing units (TPUs) in 2015. The company initially developed the chip for internal use, but earlier this month announced it was now accessible to cloud customers. Train and serve your own models using the latest Trillium processors.
OpenAI, Meta, and Microsoft all have AI chip projects underway, while Amazon recently embarked on a major effort to catch up in a race often seen as lagging behind. Last month, the company announced its second generation of Trainium chips. It is several times faster than its predecessor and has already been tested by Anthropic, an AI startup in which Amazon has invested $4 billion.
The company plans to offer access to the chip to its data center customers. Eiso Kant, chief technology officer at AI startup Poolside, told the New York Times that Trainium 2 could deliver 40% more performance per dollar compared to Nvidia chips.
Apple is also said to be getting into the game. According to a recent report in the technology publication The Information, the company is developing an AI chip with longtime partner Broadcom.
In addition to big tech companies, there are also a number of startups looking to break from NVIDIA’s stranglehold on the market. And investors clearly see an opportunity. They poured $6 billion into AI semiconductor companies in 2023, according to PitchBook data.
Companies like SambaNova and Groq promise significantly faster AI inference jobs, but Cerebras Systems, with its dinner plate-sized chips, is specifically targeting the largest AI computing tasks.
But for those looking to migrate away from Nvidia’s chips, software is a big hurdle. In 2006, the company created its own software, called CUDA, to help developers design programs that run efficiently on large numbers of parallel processing cores, a key feature of AI.
“We made sure that every computer science major coming out of college was trained and knew how to program in CUDA,” said Matt Kimball, principal data center analyst at Moor Insights & Strategy. told. “They provide tools and training and spend a lot of money on research.”
As a result, most AI researchers are accustomed to CUDA and are reluctant to learn other companies’ software. To combat this, AMD, Intel, and Google have joined the UXL Foundation, an industry group creating an open source alternative to CUDA. However, their efforts are still in the early stages.
Either way, Nvidia’s unscrupulous grip on the AI hardware industry does seem to be waning. While likely to remain the market leader for the foreseeable future, AI companies may have even more options in 2025 as they continue to build out their infrastructure.
Image credit: Visuals on Unsplash
