Markups
As I get deeper into the AI Infrastructure industry, I realise how complex the supply chain is. The current supply chain for trading firms is Hedgefund -> Value Added Reseller + Data Centre -> OEM -> ODM. At each stage, there is roughly a 30% markup. "VAR" buys servers and manages the entire project. Their value lies in knowing the perfect hardware to fully utilise the client's software model. Recently, I came across the VAR's latest AI server list, published in late 2024. I was curious why they were only offering older models like the R760xa from 2021, along with several servers dating back to 2019. None is liquid-cooled, and there were no signs of the latest XE models from Dell. My hypothesis is:
1. OEMs like Dell no longer prioritise VAR, since hyperscalers have created huge AI server backlogs.
2. The VAR lacks the networking and integration expertise needed for the newest servers.
3. With little competition, VAR can get away with charging high margins despite weaker offerings.
4. Many clients care more about cost than cutting-edge hardware, and competitive pressure hasn’t yet squeezed trading profits.
I started to think - can I source from ODMs directly and deliver the same quality of product? So I decided to reach out to ODMs in Taiwan.
They replied initially and stopped - they only entertain large customer orders.
Risks and Realities of AI Clusters
I spoke with a leader managing 3,000 Nvidia GPUs. They were trying to build a data centre, the first-of-its-kind, locally. The current process is inefficient - the entire contract is awarded to a local telecom company, which must meet strict criteria or face penalties. The telecom then acts as a project manager, coordinating with about 20 different vendors. Very little is handled in-house. I suspect this lack of expertise translates into operational inefficiency. I’m especially sceptical about their ability to network all these clusters effectively.
They then sell these GPU clusters to clients, mainly university research departments. They keep spare nodes but still face daily customer complaints. Building perfectly reliable hardware is very hard, yet clients tolerate it because there’s no better alternative. Meanwhile, China continues to struggle with procuring H800s needed for training.
Their biggest concern is cybersecurity. As she put it, “It only takes one disaster to ruin trust.” The data centre’s location is top secret and is part of the government’s critical infrastructure law.