Stay up to date with free updates
Simply sign up to the Artificial Intelligence myFT Digest and have it delivered straight to your inbox.
China’s efforts to match U.S. computing power in artificial intelligence have been hampered by buggy software, with customers of leading AI chipmaker Huawei complaining about performance issues and difficulty switching from Nvidia products.
Chinese tech giants are leading the race to develop a homegrown alternative to industry leader Nvidia after the U.S. government further tightened export controls on high-performance silicon last October.
The company’s Ascend series has become an increasingly popular choice for Chinese AI groups to perform inference, the process used by applications such as OpenAI’s ChatGPT to generate responses to queries.
But several industry sources, including AI engineers at partner companies, said the chip still lags far behind Nvidia’s when it comes to initial training of models, blaming stability issues, slow connections between the chips and poor Huawei-developed software called Cann.
Nvidia’s software platform, Cuda, is known as the company’s “secret sauce” because it’s developer-friendly and can significantly speed up data processing. Huawei is one of many companies trying to break Nvidia’s stranglehold on AI chips by creating alternative software.
Huawei employees themselves have complained about Kang: One researcher, speaking on condition of anonymity, said Kang made Ascend’s products “difficult to use and unstable,” hindering testing efforts.
“When random errors occur, it’s very hard to figure out the cause because of poor documentation. To find out what’s wrong requires a talented developer to read the source code, which slows everything down. The coding is incomplete,” they said.
Another Chinese engineer who was briefed on Baidu’s use of Huawei processors said the chips crashed frequently, complicating AI development work.
Huawei researchers said the crashes occurred because the hardware was difficult to use. “People don’t know much about the hardware itself, which makes it easier for bad outcomes to occur,” the researchers said.
To address the issue, Huawei is sending engineers to help customers on-site migrate training code previously written in CUDA to Cann, people familiar with the matter said. Tech companies that have received engineering teams include Baidu, iFlytek and Tencent, the people said.
Huawei declined to comment. Baidu, iFlytek and Tencent did not respond to requests for comment.
“Huawei has great customer service, so it naturally sends engineers to its big customers to help them use its chips,” said a former Baidu employee.
Recommendation
Huawei can accelerate the transition by leveraging its vast workforce: The company says more than 50% of its 207,000 employees are in research and development, including engineers deployed to deploy technology for customers.
“The advantage Huawei has over Nvidia is that it can work closely with its customers,” said Tilly Zhang, a technology analyst at consulting firm Gavekal. “Unlike Nvidia, Huawei has a large team of engineers who can solve customers’ problems and help them migrate to its hardware.”
Huawei has also set up an online portal where developers can provide feedback on how to improve the software.
After the United States tightened export controls in October, Huawei raised the price of its training chip, the Ascend 910B, by 20 to 30 percent, according to people familiar with the matter.
Huawei customers have also expressed concern about supply constraints for Ascend chips due to manufacturing difficulties as the Chinese company cannot buy cutting-edge chip-making machinery from Dutch company ASML.
Huawei is seeing strong demand for its AI chips. The company on Thursday reported a 34% increase in first-half revenue but did not break out sales by business unit.
More than 50 foundational models have been “trained and iterated” on the Ascend chip, Huawei executive Zhang Ping’an said at the World Artificial Intelligence Conference in Shanghai in July.
iFlytek said its large-scale language models have been trained exclusively on Huawei chips since Huawei sent a group of engineers to its headquarters in Hefei, eastern China, last year to integrate the technology.
