As organizations advance their AI programs, many IT leaders are shifting AI workloads from public clouds to private or on-premises environments. This move is primarily driven by the need for better cost control and data privacy. However, experts warn that transitioning to an AI-ready infrastructure is not as simple as just adding GPUs; it often involves significant retrofitting of legacy data centers which can come with substantial costs.
These updates can potentially reach tens of millions of dollars, especially for companies anticipating high-intensity AI workloads. Initial expenses for preparing co-location centers or on-premises settings can start at several hundred thousand dollars, escalating depending on the scale of required upgrades.
Despite the daunting costs, many CIOs find that on-premises infrastructure can yield more predictable pricing models compared to public cloud services reliant on per-use pricing. This is particularly appealing as they clarify their AI workload requirements.
Transforming a traditional data center into an AI-capable one involves a variety of considerations. While some companies may undertake small-scale upgrades for specific AI applications, most are looking at integrating AI more comprehensively, which requires substantial capacity and resources. According to industry experts, retrofitting for AI workloads necessitates substantial enhancements to cooling, power, and structural systems.
Reports indicate that the costs for constructing a new, AI-ready data center can range from $11 to $15 million per megawatt, exclusive of computing power. As AI technology evolves rapidly, CIOs must also evaluate how these advancements affect their data center investments.
The density of racks is a crucial factor as traditional models were based on kilowatt usage that vastly differs from modern AI training demands. CIOs now face the challenge of outdated facilities that cannot support the electricity and cooling requirements for contemporary AI applications, leading them to either retrofit existing structures, build new facilities, or outsource their needs.
Implementing efficient cooling systems, such as switching to liquid-cooled infrastructure, can help alleviate energy burdens and free up electrical capacity for production use, ultimately generating more value per watt.
The costs associated with retrofitting, generally between $4 million and $8 million per megawatt, do not include hardware. Experts recommend conducting thorough audits of structural integrity and power distribution before proceeding with upgrades.
Understanding AI workloads is critical, as training and inference require different infrastructures. A clear strategy around data location, movement speed, and model scaling is essential for determining suitable upgrade paths, whether that involves on-premises renovation or partnership with co-location providers.
The trend towards decentralized and federated architectures is reshaping traditional centralized data models, which will necessitate flexibility, scalability, and designed-in governance from the outset to meet the challenges of modern AI infrastructure.
