Preparing for the upcoming AI chip shortage

Semiconductor supply and demand is a delicate balance that can quickly shift, as the industry and its customers have known all too well over the past few years. The pandemic-induced chip shortage has passed, but executives are starting to prepare for the next crisis, which will be caused by (you guessed it) artificial intelligence.

As AI adoption accelerates across industries, the insatiable demand for computing resources to train and run large language models (LLMs) will collide with supply chain constraints, putting pressure on the supply of Graphics Processing Units (GPUs) for data centers. Additionally, the upcoming proliferation of AI-enabled devices is expected to drive a surge in purchases of new personal computers (PCs) and smartphones, with significant implications for the entire semiconductor supply chain.

The semiconductor supply chain is so complex that any increase in demand of more than about 20% would likely tip the balance and create chip shortages. An explosion of AI at the confluence of large end markets could easily push past that threshold, creating vulnerable bottlenecks throughout the supply chain (see Figure 1).

Balancing semiconductor supply and demand has always been difficult because of the industry’s rapid technology advances, large capital requirements, and long lead times to ramp up production capacity. But chip suppliers and buyers will need to act fast to get ahead of the next big crisis. Let’s look at how things could play out with potential demand and supply shocks.

Data Center Demand

The breakthrough of generative AI in the second half of 2022 has been a boon for the semiconductor industry so far. Chipmakers, from big GPU distributors like Nvidia to vendors supplying other chips to data centers, such as Broadcom (switches) and SK Hynix (high-bandwidth memory), have seen huge increases in revenue and valuations. Spending on data centers and the specialized chips that run them shows no signs of slowing down. Large cloud service providers are expected to increase capital expenditures 36% year-over-year in 2024, driven heavily by investments in AI and accelerated computing. Demand for GPUs will continue to grow as LLMs expand their ability to process multiple data types simultaneously (text, images, voice) and venture capitalists pour more money into AI startups.

If data center demand for current generation GPUs doubles by 2026 (a reasonable assumption given the current trajectory), based on Bain’s forecasting model that accounts for the complexities of the multi-level semiconductor supply chain, key component suppliers will need to increase production by more than 30% in some cases (see Figure 1 above). This pull-through demand will be concentrated in advanced packaging and memory. In the above scenario, manufacturers of chip-on-wafer-on-substrate (CoWoS) packaging components will need to nearly triple their production capacity by 2026.

Enabling the growth of AI requires integrating complex supply chain elements, from building data centers and wafer fabs to ensuring access to advanced packaging and sufficient power. Obtaining many of these critical elements requires long lead times that may not be able to keep up with demand (see Figure 2).

Importantly, many of these supply chain elements are shared with other parts of the technology ecosystem, and all are exposed to capital, geopolitical, and timing risks. A shortage of one chip can disrupt the entire system, as happened during the last supply shortage when a critical chip was missing and many new cars went unsold.

Demand for PCs and smartphones

Personal device manufacturers are already rapidly building AI capabilities directly into their products: Our benchmarks show that the silicon surface area of ​​the average notebook core processing unit (CPU) and smartphone processor has increased by roughly 5% and 16%, respectively, to accommodate neural processing engines for on-device AI.

More importantly, as AI applications become more useful, buyers looking to upgrade may accelerate their purchases of new devices, sparking increased demand in the same way that the pandemic caused a short-term spike in PC demand (see Figure 3).

Given the long list of components associated with smartphone and PC devices compared to GPUs, the demand for AI will have a wide-reaching impact on the semiconductor supply chain for these devices. The weakest link in the supply chain for these devices is the state-of-the-art fabs that manufacture cutting-edge chips. In a rapid AI adoption scenario in which PC sales grow 31% and smartphones 15% between 2023 and 2026, state-of-the-art fabs will need to increase production by an estimated 25% to 35%. This would require building four or five additional state-of-the-art fabs, costing an estimated $40 billion to $75 billion, which would help justify the many fabs already being built by major foundries.

Don’t forget about supply risks

Extreme weather, natural disasters, geopolitical conflicts, pandemics and other major disruptions over the past decade have made it abundantly clear that supply shocks can severely limit the industry’s ability to meet demand. Much of the pressure on GPU supply over the past 18 months was caused by disruptions to less visible elements of the supply chain, such as CoWoS’s advanced packaging capabilities.

Geopolitical tensions, trade restrictions, and multinational technology companies decoupling their supply chains from China continue to pose serious risks to semiconductor supplies. Factory construction delays, material shortages, and other unpredictable factors could also create a pinch. Barring these uncertainties, we expect growing demand for high-bandwidth memory components, advanced packaging fab and tool builds, and substrate fab builds to pose the biggest supply risks.

Lessons for managers

For semiconductor buyers across industries, navigating these supply chain complexities starts with a deep understanding of the components they procure. Effective leaders will pay special attention to AI data center and related components, such as switches, transceivers, and power management integrated circuits. They will also closely monitor PC and smartphone refresh cycles and associated peripherals, such as Wi-Fi routers and networking equipment. Spikes in these areas will have cascading effects throughout the supply chain, so each must be tracked closely.

Leading companies will take lessons learned from the recent semiconductor shortage and safely balance their inventory between shortages and surpluses. They will enter into long-term purchasing agreements to ensure access to chips and manufacturing capacity based on expected future needs (and share this visibility with suppliers). The “just-in-time” inventory strategies that dominated the past few decades will continue to be replaced by costlier but more resilient “just-in-case” approaches. More companies will design their products to use industry-standard semiconductors whenever possible, rather than application-specific chips. They will also continue to invest in the resilience of their supply chains to geopolitical uncertainties such as tariffs and regulations. Finally, they will closely monitor their supply of silicon advanced packaging and substrates, as well as their front-end semiconductor manufacturing capabilities.

Executives may still be feeling fatigued from the semiconductor supply disruptions caused by the pandemic, but there’s no time to rest as the next big supply shock looms. But this time the signs are clear and the industry has an opportunity to prepare. The road ahead will require vigilance, strategic foresight, and swift action to strengthen supply chains. By taking proactive measures, business leaders can ensure resilience and success in an increasingly AI-enabled world.

Continued in the report