I spoke last spring with a procurement manager at a medium-sized IT services company. The company was in the process of expanding a data center, and the manager had created a spreadsheet that showed lead times, assigned vendors for certain models and SKUs of graphics cards, and then the current spot market price for each of those parts. The goal of the spreadsheet was to come up with the best possible schedule for bringing in the hardware needed for the data center expansion. She told me that the supply chain had “gone to hostage” and she was just trying to get the best schedule.
So here we are. The problems that the huge number of companies are going through are not problems of the hyperscalers. They do have very large teams of procurement people but they also have preferred suppliers with agreed terms and conditions. For the rest of us in the IT world (mid-market IT projects, on moderate budgets, trying to run a moderately sized project on a reasonable time scale and hit the end of the project on time), we are all being crushed by factors that are outside of our control.
The AI boom didn’t come with a warning label
The cloud providers have been growing their data centers to accommodate demand. However, the cloud providers did not anticipate the massive demand for GPU clusters to train frontier models of large language models, image generation, and other inference workloads. Training a single large model requires thousands of thousands of high-end GPUs running 24/7 for weeks. The sudden appearance of dozens of deep-pocketed companies to ship similar products suddenly creates a massive demand for GPUs that far exceeds the supply of chips that the foundries can make. The lead times for new semiconductor capacity are years, not quarters. The supply chain for the most advanced semiconductors is an old-growth forest, not a widget factory. TSMC, Samsung and a few other companies control the most advanced nodes and are not inclined to suddenly expand their capacity to meet the demands of a market that went nearly vertical in 2022 and has not come back down since.
The root of all of this is that there simply are not enough GPUs in supply to meet the needs of the industry. There are many factors that play into this but in short, data centers need GPU’s to be used as servers and currently, there are not enough in the market to go around. Larger data centers that are operated by the hyperscalers will be able to get the GPUs that they need for their facilities, likely via preferred relationships with a select group of suppliers. But for the mid-market data center looking to grow in order to better serve their customers, things are bleak. In short, the market for GPU’s is like a hostage situation and although it may be polite and include a lot of smiles between the various players, that does not change the basic dynamics at play. And as has been discussed above, alternative SKUs (such as arm-based processors) are not suitable for use in place of GPUs in most cases.
Leading edge semiconductors at advanced nodes require years of investment to design process and equipment, and then many months to bring up high volume production. High end foundries such as TSMC, Samsung, and a few others have developed the deep expertise and knowledge of specialized equipment and materials required to support production at these leading edge nodes. They are expanding capacity but their roadmaps will not dramatically bend to address the massive increase in demand for GPUs that occurred in 2022.
Cloud infrastructure is making it worse, not better
You might assume that as more cloud based infrastructure is being created by the large cloud providers, that the demand for hardware to be installed in Data Centers to run cloud services would decrease and that this would help ease the pressure on other organizations to source their required hardware. Unfortunately this is not the case, the large cloud providers are buying massive quantities of the same type of hardware that other organizations are trying to purchase and this is putting a huge squeeze on the available supply of new hardware for other organizations to purchase.
However, that has now become a classic case of the hyperscalers mopping up all of the supply of all other chip buyers. And not just buying at the front of the line, but renting the entire line in advance to keep others out. This is terrible for other chip buyers. They are left trying to purchase in the spot market, which is charged at higher rates, or purchase alternative GPUs that do not have the same level of performance, but are plentiful in the market.
More info on projected gpu shortages for 2026 here. The shortages and consequent inflation of prices on the semiconductor market will vary, but for most companies and organizations it’s going to mean extended time to complete needed procurements.
On the Ground
However for the IT teams and the procurement teams within the organizations the pain of the shortage will be felt in many different ways.
- Lead times that were once four to six weeks are now stretching to six months or longer for certain high-demand SKUs
- Spot market pricing that bears almost no resemblance to MSRP — sometimes comically, sometimes devastatingly divorced from it
- Vendor allocations that favor the biggest buyers, leaving mid-market companies picking through whatever’s left
- Project timelines that slip and slip again because a data center buildout is gated on hardware that simply hasn’t arrived
It’s not pretty. But the IT organizations managing best are those treating GPU procurement as a strategic sourcing issue and are doing their best to optimize it over the long haul. They are very patient and have established long-term relationships with good reseller partners to aid in their purchase of the limited supply of GPUs currently available in the market.
Rethinking how you plan for hardware procurement
What is interesting is that the companies managing to procure GPUs best are not necessarily the ones with the deepest pockets. Rather, they are treating GPU procurement as a strategic sourcing problem – something to be planned for, and sourced, over a long period of time. And they are rewarding their procurement teams for taking a long view, for building good relationships with resellers, and for being patient in the face of rapidly shifting market conditions.
(There’s also something quietly humbling about watching a procurement manager outmaneuver a much larger competitor simply because she called her reseller rep in January instead of April.)
- Extend your procurement timeline to at least 12 months out for high-demand hardware
- Cultivate relationships with multiple authorized resellers, not just one preferred vendor who may not have allocation to spare
- Model infrastructure plans around realistic availability, not ideal availability. The gap between those two things is where projects go to die
- Seriously explore refurbished or previous-generation hardware for workloads that don’t demand cutting-edge specs
The deeper issue nobody wants to say out loud
All semiconductor manufacturing capacity today takes years to design, qualify, fabricate and ship. This creates a huge void between the front end supply of chips and the massive increase in demand for frontier class chips created by the AI boom. Water always finds every crack in a foundation and in this case the void between supply and demand is creating massive amounts of pressure that will continue to find every weakness in the entire semiconductor supply chain until the two surfaces match up.
Until the industry has brought on line sufficient fabrication capacity to meet demand, the current gap will remain. This is true even though there are announcements of new fab construction. Yes, these are real, but their timelines are long, and the initial yields on the leading-edge nodes will be uncertain. Demand will continue to rise.
| Factor | Before 2022 | Current situation |
| Average GPU lead time | 4 to 8 weeks | 6 to 12+ months for flagship SKUs |
| Primary demand driver | Gaming, professional visualization | AI training, inference, data centers |
| Competitive buying environment | Manageable, largely predictable | Hyperscalers crowding out mid-market buyers |
| Spot market premium over MSRP | Minimal | Significant, sometimes extreme |
| Procurement planning horizon | Quarterly | Annual or longer to be safe |
The procurement manager with the spreadsheet-within-a-spreadsheet eventually got her hardware. Four months late. The project adapted, barely. But she told me she’d never again treat hardware ordering as the easy part of a data center project. Because for a lot of companies right now, it isn’t the easy part. It’s the whole problem.
