China is trying to establish a domestic GPU market The idea is to move away from dependence on licensed Nvidia GPUs, but compatibility and cost pose significant problems
U.S. export controls have targeted China’s access to advanced chips over concerns that the technology could strengthen China’s military.
These sanctions have forced China to step up efforts to develop its own GPU technology, and Chinese startups have already made significant progress in developing GPU hardware and software.
However, the transition from globally recognized Nvidia chips to homegrown alternatives will require extensive engineering, slowing the rate of AI progress. Despite China’s progress in this area, challenges posed by incompatible systems and technology gaps remain significant.
High cost and complexity
As a result, a government-backed think tank in Beijing suggested that Chinese data centers continue to use Nvidia chips because of the high cost and complexity involved in moving to domestic alternatives.
Nvidia’s A100 and H100 GPUs, widely used for training AI models, were banned from export to China in August 2022, prompting the company to create modified versions like the A800 and H800 . However, these chips were also banned by the US government in October 2023, limiting China’s access to the advanced hardware it had relied on.
Despite the rapid development of Chinese GPU startups, the think tank noted that it remains difficult to transfer AI models from Nvidia hardware to domestic solutions due to differences in hardware and software. The extensive engineering required for such a migration imposes significant costs on data centers, making Nvidia chips more attractive despite availability constraints.
Despite US sanctions, China’s AI computing capabilities continue to grow at a rapid pace. As of 2023, China’s computing power, including both central processing units (CPUs) and GPUs, increased 27% year over year to reach 230 Eflops.
GPU-based computing power, which is essential for training and inferring AI models, has grown even faster, increasing by 70% over the same period. Additionally, China’s AI hardware landscape is expanding significantly, with more than 250 Internet Data Centers (IDCs) completed or under construction by mid-2023.
The centers are part of a larger push for “new infrastructure” with support from local authorities, national telecommunications operators and private investors. However, this rapid ramp-up has also raised concerns about overcapacity and underutilization.
“If conditions permit, (data centers) can choose (Nvidia’s) A100 and H100 high-performance computing units. If computing power needs are limited, choose the H20 or an alternative domestic solution. ,” the China Academy of Information and Communication Technology (CAICT) said in a report on China’s computing capacity development published on Sunday.
“The trend of fragmentation of computing power is becoming increasingly severe, with average GPU utilization below 40%…There are significant differences in IDC hardware, including GPUs, AI accelerators, and network structures, which require management and “Hardware resources are becoming scarce to support the differential computing needs of AI tasks that are becoming harder to dispatch, further hampering utilization,” the report adds.
Via SCMP
