No one expected Nvidia to release something like GB10. After all, why did a technology company that transformed itself into the most valuable company ever by selling parts that cost hundreds of thousands of dollars suddenly decide to sell entire systems for a fraction of the price? Is it?
I believe that Nvidia wants to revolutionize computing in the same way that IBM did with the original IBM PC some 45 years ago.
It may be time to introduce Jensen’s Law to complement Moore’s Law. With comparable AI performance, it would take about 100 months to reduce the price per FLOP by 25.
D. Ahau
As a reminder, Project DIGITS is a fully formed, off-the-shelf supercomputer built into something the size of a mini-PC. This is essentially a smaller version of the DGX-1, which was the first of its kind launched in April 2016, about 10 years ago. At the time, it had a 16-core Intel Xeon CPU and eight P100 GPGPU cards and sold for $129,000. Digit costs $3,000.
Nvidia has confirmed that it has 1,000 teraflops of AI performance at FP4 precision (dense/sparse?). Although a direct comparison is not possible, we can estimate that this little supercomputer has about half the processing power of a fully loaded 8-card Pascal-based DGX-1.
At the heart of Digits is a GB10 SoC with 20 Arm cores (10 Arm Cortex-X925 and 10 Cortex-A725). Other than the confirmed presence of a Blackwell GPU (a lite version of the B100), we can only speculate on the power consumption (100W) and bandwidth (825GB/s according to The Register).
You should be able to connect two (but more) of these devices via Nvidia’s proprietary ConnectX technology to tackle larger LLMs, such as Meta’s Llama 3.1 405B. Squeezing these tiny mini PCs into a 42U rack seems nearly impossible at the moment, as it would compromise Nvidia’s far more profitable DGX GB200 system.
Everything about the moat
Why did Nvidia launch Project DIGITS?I think the point is to strengthen the company’s outer moat. Making their products so sticky that it’s nearly impossible to transfer to competitors has worked very well for other companies, such as Microsoft and Windows, Google and Gmail, and Apple and the iPhone.
The same thing happened with Nvidia and CUDA. With Nvidia in the driver’s seat, it was able to shuffle the goalposts and cheat its competitors.
By moving inference to FP4, Nvidia has delivered impressive results, including “Blackwell achieved 2.5x performance over previous generation on FP8 for per-chip training and 5x performance on FP4 for inference.” We were able to achieve a benchmark. Of course, AMD doesn’t offer FP4 compute on the MI300X/325X series, and we’ll have to wait until later this year for it to roll out on the Instinct MI350X/355X.
NVIDIA is therefore laying the ground for future incursions, for lack of a better word or analogy, from existing and future competitors, including its own customers (think Microsoft and Google). Masu. Nvidia CEO Jensen Huang’s ambitions are clear. He wants to expand his control of the company beyond the hyperscaler realm.
“AI will become mainstream in every application in every industry. Project DIGITS will bring the Grace Blackwell Superchip to millions of developers and put an AI supercomputer on the desk of every data scientist, AI researcher, and student. will be in place, allowing them to build and shape the era of AI,” Huang recently commented.
This is a sign that, apart from renaming Nvidia to Nvid-ai, Huang is pursuing his own ambitions to make his company’s name synonymous with AI, much like Tarmac and Hoover before him (albeit in a more niche field). It’s as close as I’ve come to admitting.
Like many people, I was confused by the relationship with Mediatek. The rationale for this partnership is stated in Mediatek’s press release. The Taiwanese company says it has “bred its Arm-based SoC performance and power efficiency design expertise into (a) breakthrough device for AI researchers and developers.”
I believe this partnership will benefit Mediatek more than Nvidia, and I think Nvidia will quietly go it alone in the short term. According to Reuters, Huang dismissed the idea that NVIDIA would go after AMD and Intel, saying, “Now they (Mediatek) can give it to us, and they can keep it themselves and bring it to the market. Deaf,” he said. And it was a great win-win. ”
However, this doesn’t mean Nvidia won’t offer more mainstream products. It’s just that these products are aimed at businesses and professionals, not consumers, where fierce competition has made things more difficult (and the margins are much thinner).
The Reuters article quotes Huang as saying. “We’re going to make this a mainstream product, do everything we can to support professional, high-quality software, and PC[manufacturers]will make it available to everyone.” End users . ”
Staring into my crystal ball
One theory I came across while researching this feature is that more data scientists are adopting Apple’s Mac platform because it offers a balanced approach. Thanks to its unified memory architecture, it delivers decent performance at an “affordable” price. Mac Studio with 128GB unified memory and 4TB SSD currently retails for $5,799.
So where does Nvidia go from there? The obvious move is to integrate memory into the SoC, similar to what Apple did with its M-series SoCs (and AMD’s HBM-based Epyc). This not only saves you money, but also improves performance. This is something its sibling GB200 already achieves.
It depends on whether Nvidia wants to offer more products for the same price, or the same performance at a lower price point (or both). Nvidia could take Intel’s path and use GB10 as a prototype to encourage other major partners (PNY, Gigabyte, Asus) to launch similar projects (Intel is calling the Next Unit of Computing ( NUC).
I’m also particularly interested in finding out what happens to the Jetson-Olin family. The NX 16GB version was upgraded just a few weeks ago and now delivers 157 TOPS with INT8 performance. Although this platform is aimed at fulfilling more DIY/edge use cases rather than pure training/inference tasks, I can’t help but think of “what if” scenarios.
Nvidia is clearly disrupting itself before other companies try to disrupt it. The question is how far it will go.
