AWS has launched a new AI chip that competes with Nvidia’s GPUs. AWS says its goal is not to unseat Nvidia in the AI chip market, but to offer customers more choice. Gadi Hutt, senior director at AWS, works with Intel, Anthropic, and AMD.
Amazon Web Services this week launched an upgraded line of AI chips, putting the company in direct competition with Nvidia.
However, AWS doesn’t think that way.
AWS’ new AI chips won’t follow Nvidia’s launch, said Gadi Hutt, senior director of customer and product engineering at the company’s chip design subsidiary Annapurna Labs. The market is large enough for multiple vendors, so the goal is to offer customers lower-cost options, Hutt told Business Insider in an interview at AWS’s re:Invent conference.
“The goal is not to displace Nvidia,” Hutt said, adding, “What’s really important is to give customers choice.”
AWS has spent tens of billions of dollars on generative AI. The company this week announced a cutting-edge AI chip called Trainium 2, which costs about 40% less than Nvidia’s GPUs, and a new supercomputer cluster called Project Rainier that uses the chip. Previous versions of AWS’ AI chips had mixed results.
Hutt insists that this is not a competition, but a collaborative effort to increase the size of the overall market. They also have different customer profiles and target AI workloads. He added that Nvidia’s GPUs will continue to dominate for the foreseeable future.
In the interview, Hutto talked about AWS’ partnership with Anthropic, which will be Project Rainer’s first customer. The two companies have been working closely together over the past year, with Amazon recently investing an additional $4 billion in the AI startup.
He also shared his thoughts on the partnership between Intel and AWS, whose CEO Pat Gelsinger has just stepped down. He said AWS will continue to work with the struggling semiconductor giant as customer demand for Intel’s server chips remains high.
AWS announced last year that it was considering selling AMD’s new AI chips. But Hutt said these chips aren’t yet available on AWS because they haven’t shown strong customer demand.
This Q&A has been edited for clarity and length.
There have been a lot of headlines about Amazon looking to buy Nvidia for new AI chips. Can you talk about that?
I always laugh a little when I see these headlines. Because this isn’t about unseating Nvidia. Nvidia is a very important partner for us. It’s really about giving customers a choice.
We are putting a lot of work into making these chips available to more customers on an ongoing basis. And Nvidia isn’t going anywhere. They have great solutions and solid roadmaps. There is also continued investment in the Nvidia product line, as we just announced P6 instances, AWS servers powered by Nvidia’s latest Blackwell GPUs. It’s really about giving customers a choice. Nothing more.
Nvidia is a great supplier of AWS, and our customers love them. I don’t mean to disrespect NVIDIA in any way, shape, or form.
Want to see more use cases for Nvidia on AWS?
If customers believe that’s the way they should go, they’ll do it. Of course, if it’s good for our customers, it’s also good for us.
The market is so large that there is room for multiple vendors to enter. We are not forcing anyone to use these chips, but we are working hard to ensure that our main tenets of high performance and low cost are realized and benefit our customers.
Does that mean it’s okay for AWS to be in second place?
It’s not a competition. There is no machine learning awards ceremony every year.
For customers like Anthropic, there is clear scientific evidence that larger computing infrastructure allows them to build larger models using more data. This will improve accuracy and performance.
Our ability to scale capacity to hundreds of thousands of Trainium 2 chips gives us the opportunity to innovate in ways we couldn’t before. Increase your productivity by 5x.
Is it important to be No.1?
The market is big enough. 2nd place is a very good position.
By the way, I’m not saying I’m second or first. But that’s not really what I think. We’re still in the early stages of machine learning in general, the industry in general, and chips in particular, serving customers like Anthropic, Apple, and all the other companies.
We also haven’t done any competitive analysis with Nvidia. I’m not benchmarking against Nvidia. There’s no need for that.
For example, there is MLPerf, an industry performance benchmark. Companies that participate in MLPerf have performance engineers who work solely to improve MLPerf numbers.
It’s a total distraction for us. We don’t want to waste time on benchmarks that aren’t customer-centric, so we don’t participate in them.
On the surface, it seems like helping companies grow on AWS isn’t necessarily beneficial for AWS’s own products. Because we are competing with AWS.
We’re the same company that’s the best that runs Netflix, and we also run Prime Video. It’s part of our culture.
I would say there are a lot of customers who are still using GPUs. Many customers love their GPUs and don’t plan on moving to Trainium anytime soon. That’s fine. Because, again, we give them a choice and they decide what they want to do.
Do you think these AI tools will become more commoditized in the future?
I hope so.
When we started this in 2016, the problem was that there was no operating system for machine learning. So we had to invent all the tools to utilize these chips to make them work as seamlessly as possible for our customers.
The more machine learning becomes commoditized on the software and hardware sides, the better for everyone. This means that using these solutions is easy. But doing machine learning meaningfully is still an art.
What are the different types of workloads that customers want to run on GPUs and Trainium?
GPUs are more of a general-purpose processor for machine learning. Every researcher and data scientist in the world knows how to use Nvidia. If you invent something new, run it on a GPU and it will work.
If you invent something new on a specialized chip, you need to be sure that the compiler technology understands what you’ve built, or you need to create your own compute kernel for that workload. We primarily focus on use cases where our customers have said, “I need this.” Typically, the customers we acquire are those who are concerned about increased costs and are looking for alternatives.
So are the most advanced workloads typically reserved for Nvidia chips?
The usual. If data scientists need to run experiments continuously, they’ll likely do it on a GPU cluster. Once you know what you want to do, you have more options. This is where Trainium truly shines, offering high performance at low cost.
AWS CEO Matt Garman has previously said that the majority of workloads will continue to reside on Nvidia.
That makes sense. We provide value to customers who are spending a lot of money and are looking for ways to manage their costs a little better. When Matt says “majority of workloads,” he means medical imaging, speech recognition, weather forecasting, and all sorts of other workloads, but we have larger customers who ask us to do bigger jobs. Because of that, I’m not focusing on it much right now. Therefore, that statement is 100% correct.
In short, we want to continue to be the place of choice for GPUs and, of course, for Trainium when our customers need it.
What has Anthropic done to help AWS in the AI space?
They have very strong opinions about what they need, and they come back to us and say, “Hey, we need something. “Can we add feature A to future chips?” It’s a conversation. Some of the ideas they came up with were impossible to even implement on a piece of silicon. We actually implemented some ideas and came up with better solutions for others.
They’re very expert at building the underlying models, so this really helps them build chips that are really good at what they do.
We just announced Project Rainier together. This is someone who wants to spend a lot of chips as quickly as possible. This isn’t an idea, it’s actually being built.
Can you talk about Intel? AWS’ Graviton chips are replacing many Intel chips in AWS data centers.
I would like to make a correction here. Graviton is not a replacement for x86. We’re not removing x86 and putting in Graviton. But again, at customer request, more than 50% of our recent CPUs were Graviton.
This means that customer demand for Graviton is increasing. But we still sell a lot of x86 cores to customers, and we think we’re the best place to do that. We are not in competition with these companies, but we treat them as great suppliers and we have a lot of business to work with.
How important is Intel going forward?
They will continue to be a great AWS partner. There are many use cases that work very well with Intel cores. It is still being implemented. I have no intention of quitting. It really responds to customer needs.
Is AWS still considering selling AMD’s AI chips?
AMD is a great partner for AWS. We sell a large number of AMD CPUs to our customers as instances.
Machine learning product lines are constantly being considered. If the customer insists on needing it, there’s no reason not to implement it.
And we haven’t seen it yet with AMD’s AI chips?
still.
How cooperative are Amazon CEO Andy Jassy and Garman of the AI chip business?
they are very cooperative. We meet with them regularly. There is a focus across the company’s leadership on ensuring that customers who need ML solutions get them.
Internally, we also have a lot of collaboration with the science and services teams that are building solutions on top of these chips. Other teams within Amazon, such as Rufus, the AI assistant available to all Amazon customers, are running entirely on Inferentia and Trainium chips.
Do you work at Amazon? Any tips?
Contact reporter Eugene Kim via the encrypted messaging app Signal or Telegram at +1-650-942-3061 or by email at ekim@businessinsider.com. Communicate using non-work devices. For more tips on sharing information safely, check out Business Insider’s source guide.
