The AI computing market is likely to change in 2025, opening up opportunities for small and medium-sized businesses. AI computing is dominated by Nvidia. Evolving workloads can benefit competitors. Companies like Groq, Positron, and SambaNova are focusing on inference to counter Nvidia’s market dominance.
In 2025, the tide may turn for companies looking to compete with the $3 trillion gorilla in the AI computing space.
Nvidia holds an estimated 90% of the market share in AI computing. Still, workloads are expected to change as the use of AI increases, and this evolution could present opportunities for companies with competitive hardware.
In 2024, the majority of AI computing spending will shift to inference, Thomas Sommers, CEO of chip startup Positron AI, told BI. This “will continue to grow exponentially,” he added.
Inference in AI is the computation required to generate a response to a user’s query or request. The computing required to teach the model the knowledge it needs to respond is called “training.” For example, the creation of OpenAI’s image generation platform Sora represents training. Each user who directs the creation of an image represents an inference workload.
OpenAI’s other models have Somers and others excited about the increased computing needs in 2025.
OpenAI’s o1 and o3, Google’s Gemini 2.0 Flash Thinking, and several other AI models use more computationally intensive strategies to improve post-training results. These strategies are often referred to as reasoning time computing, thought chains, reasoning chains, or reasoning models.
Simply put, the more the model thinks before answering, the better the response will be. That idea costs time and money.
Startups vying for some of Nvidia’s market share are trying to optimize one or both.
Nvidia is already benefiting from these innovations, CEO Jensen Huang said during the company’s November earnings call. Companies hoping to compete with Huang are betting that new post-training strategies for AI will benefit all inference chip providers in 2025.
Business Insider spoke to three challengers about their hopes and expectations for 2025. Their New Year’s resolutions are:
What’s the one thing in your control that could make 2025 a big year for alternative chips?
Mark Heaps is chief technology evangelist at Nvidia challenger Groq. Grok
Mark Heaps, Chief Technology Evangelist at Groq, said:
“Run, run, run. Right now, everyone at Groq has decided not to take a vacation this year. We’re all running, we’re building systems. We’re all going to make sure we realize the opportunities we have. It is under our control.
Let me tell you, our funnel is bubbling with carbon dioxide right now. Incredible customer interest. We have to build more systems and ramp them up to meet demand. We want to serve all of our customers. We want to increase rate limits for everyone. ”
Rodrigo Lien, CEO of SambaNova Systems:
“For SambaNova, the most important element is executing the transition from training to inference. The industry is rapidly moving towards real-time applications, and inference workloads account for the largest share of AI demand. Our focus is on ensuring that our technology allows businesses to scale efficiently and sustainably. ”
Related articles
Positron CEO Thomas Somers:
“My thinking is that if we can actually deploy enough compute, which I think we can, thankfully from a supply chain perspective, by deploying more inference-only compute, It would be possible to increase the adoption rate of “chains” and other inferential additional calculations. ”
What are you looking forward to in 2025 that is out of your control?
Rodrigo Liang, CEO and Co-Founder of SambaNova Systems. Samba Nova Systems
Heaping amount:
“It’s about customers recognizing that there are new advancements to existing technology. ‘We like what you have, but to paraphrase the old adage, no one wants to There are a lot of people who tell us, “Nobody got fired for making a purchase.” Insert current employee.
But we know it’s starting to boil over. People are starting to realize that it’s hard to get chips from incumbents and they don’t perform as well as Groq. So my hope is that people take advantage of that opportunity and really look at these new technologies. ”
Liang:
“If I had a magic wand, I would address the power challenges of AI deployments. Today, most of the market is powered by power-hungry hardware that is not designed for large-scale inference. The result is an approach that is both economically and environmentally unsustainable.
At SambaNova, we’ve proven there’s a better way. Our architecture consumes 10x less power, enabling businesses to deploy AI systems that meet their power budgets and carbon targets without exceeding them. I would like to see the market move more quickly towards adopting technologies that prioritize efficiency and sustainability. By doing so, we can enable AI to scale globally without burdening the infrastructure that supports it. ”
Somers:
“I want people to really adopt these thought-chain capabilities as quickly as possible. I think from a capability standpoint, this is a big change. There are 8 billion parameter models, , the number of parameter models is over 70 billion, so I will do everything in my power to make that happen.”
What are your New Year’s resolutions?
Positron AI executive. Left to right: Edward Kmet, Thomas Somers, Adam Hewson, Greg Davis. Positron AI
Heaping amount:
“Over the past six months, I’ve been to a lot of hackathons and met developers, and I’m so inspired. So my New Year’s resolution is to learn about the good things people are doing with AI. It’s about trying to amplify the signal.”
Liang:
“I make time for music. Playing the guitar is something I’ve always loved and I want to get back to it. Music has a great way of clearing the mind and stimulating creativity. I find this invaluable as I work to bring SambaNova’s AI to new corners of the globe. ”
Somers:
I want to do everything I can to encourage the use of these new tools to help my mother. One of the reasons I’m interested in technology is that these tools allow people to do more with their time and learn whatever they want, no matter what job they’re doing. Because I wanted to see it. These things will enable the spread.
Also, personally, I’ve been using the o1 Pro model like crazy for the past few weeks and it’s been great for my personal work, so I’d love to see more of these things outside of just a work context. I would like to try using it. But when I gave my mom access, it was very interesting to see what she did with it. The kind of daily human work for these things is really an assistant. ”
