Data centre storage games: Winning the AI race

The UAE is known for meeting ambitious development goals, and this certainly applies to the country’s digital transformation and its goal of becoming an AI leader.

The region, which appointed the world’s first minister of AI in 2017 and established the world’s first university dedicated to AI research in 2019, has attracted international interest, with initiatives such as the UAE-US AI Campus or the MGX Fund to mobilise up to $100 billion investment in AI infrastructure.

When building these AI ecosystems, the focus is on generative AI models (GenAI), graphics processing units (GPUs), central processing units (CPUs), fast data storage, and highly skilled workers. However, fundamental IT infrastructures such as hard disk drive-based storage are often undervalued, even though they store huge unstructured data sets cost-effectively.

To tap into the full potential of AI and keep the region at the forefront of technological advancements, regional CIOs and IT decision-makers need to look beyond processing power and rethink their storage approach to AI data workflows.

The importance of HDDs for AI

AI revolves around unstructured data — from ingestion and data preparation (including vectorisation, labelling) to training, interference, content generation and monitoring – and every step needs and creates more data. Given the global AI boom, the amount of information is exploding. According to IDC, the annual volume of data generated is expected to more than double to 527.5 zettabytes in 2029.

To illustrate: if this data were to be stored on an HDD with 32 terabytes, 16.48 billion drives would be required. Stacked, they would build a 438.5km tall tower, stretching nearly eleven times around the Earth. To store this unimaginable amount, there is one technology that performs this task cost-effectively and energy efficiently while remaining powerful at scale.

Maintaining a share of 80 per cent in the cloud by 2028, according to IDC, high-capacity HDDs with up to 32TB are becoming indispensable in AI data centres just like GPUs and CPUs.

To fully enable effective storage systems for these data-hungry pipelines, IT decision makers and CIOs should consider:

Highly Dense Architectures

HDDs with innovations such as energy-assisted magnetic recording (EAMR), shingled magnetic recording (SMR), and helium-sealing are fuelling capacity increases without expanding the physical footprint of data centres. This saves on server racks or storage enclosures and can decrease cooling and power consumption costs. In fact, HDDs are approximately six times more cost-efficient than flash solutions.

Moving from 26TB to 32TB drives to deploy one exabyte of storage, for example, can lead to 18.7 per cent fewer racks, 18.8 per cent fewer drives, and 18.8 per cent lower total power consumption, including a boost in cooling and power usage effectiveness.

While results vary depending on workload requirements and setup, such efficiencies can reduce the total cost of ownership, helping IT teams meet performance demands without building more or bigger facilities and increasing costs.

Energy efficiency and sustainability

With AI systems scaling up, so does the environmental cost of working with unstructured data. Modern technologies consume enormous amounts of energy when processing and storing the constantly growing information volumes.

According to current estimates, the electricity demand of data centres worldwide will more than double to around 945 terawatt hours (TWh) by 2030 – roughly equivalent to the total electricity consumption of the UAE between 2017 and 2023 (952 TWh).

While this may sound bleak, it can be an opportunity to lead the way in sustainability and help meet regional net-zero targets. Again, HDDs can play a pivotal role in this transformation.

To help reduce power consumption and emissions, IT decision makers must improve hardware efficiency. As high-capacity HDDs are the foundation of today’s cloud environments, any effort to consolidate more capacity in the same 3.5in footprint can go a long way.

Over the past years, HDD manufacturers have constantly innovated the technology to enable even lower energy use per TB (kW/TB). Drives from Western Digital with up to 32TB, for example, leverage the latest generation of HelioSeal technology. This innovation helps to reduce the total power consumption (including energy and cooling costs) of these HDDs by up to 28 per cent compared to standard air-filled drives. Ultimately, this can contribute to greater sustainability in the data centre and extend the hardware lifespan.

Disaggregated storage

In addition to HDD-based storage, disaggregated storage based on Non-Volatile Memory Express over Fabric (NVMe-oFTM ) is a transformative approach to modern data centres. The concept is no longer confined to hyperscalers; it’s rapidly gaining traction to help support AI and accelerate workloads.

Disaggregated infrastructures enable right-sized, independent scaling of compute, storage, and networking, eliminating the need for costly overprovisioning.

Unlike traditional storage architectures that require adding entire servers when only one element needs expansion, disaggregated storage separates the GPU, CPU, and storage servers to better meet the different power, cooling, and space requirements. This gives customers better flexibility, unlocks further efficiencies, and lowers the overall TCO compared to tightly coupled HCI environments.

CIOs Need to Prioritise HDD Storage Now

The future of the UAE IT infrastructure is about building and using resources smartly. CIOs who embrace innovative HDD storage solutions and disaggregated storage architectures today will be best positioned to scale effectively without waste, unlocking the full potential of their AI investments for the future.