How UCloud Cuts Archive Storage Costs by 80% with SMR Disks and Smart IO Scheduling

UCloud officially launched a new generation of archive storage product based on US3 in August 2020. The product uses UCloud’s self‑developed storage architecture, achieving nearly 80% lower storage cost compared to standard storage and about 30% cheaper than comparable market offerings.

IDC predicts that global newly generated data will reach 175 ZB by 2025, but only about 15 ZB will be actually stored, meaning a loss rate over 91%. In today’s enterprise data iceberg, 80% of data is cold. UCloud believes there is still huge potential to improve capacity‑type storage in the public cloud through technology.

Maximizing the use of the latest high‑capacity hardware to further reduce storage cost and ensuring data security in long‑term archive scenarios require substantial optimization of the US3 I/O path and hardware adaptation, while keeping the product easy to use.

Increasing Storage Density with SMR Disks and JBOD

Cost reduction at the hardware level is mainly achieved by increasing storage density. After evaluating various media such as Blu‑ray, tape, and hard drives—and referencing Microsoft’s Pelican system—UCloud chose high‑density HDDs for archive cloud storage, discarding Blu‑ray and tape because the target is to enable data activation within minutes in emergencies and within hours under normal conditions.

Traditional hard drives are vertical magnetic recording (PMR) devices that store data on parallel, non‑overlapping tracks; capacity can only be increased by adding more tracks.

Shingled Magnetic Recording (SMR) drives increase storage density by overlapping new tracks over previously written ones, making earlier tracks narrower. This “shingling” allows higher track density, similar to overlapping roof tiles.

SMR drives face write challenges because overlapping tracks prevent random overwrites. Consequently, SMR disks are divided into zones; most zones allow only append‑only writes, while about 1% are CMR zones that support random read/write.

To hide SMR limitations, UCloud adopts a host‑managed approach, managing SMR reads and writes at the block layer without relying on a file system.

We embed a small amount of metadata with each I/O, including a CRC to detect silent data corruption, enhancing reliability for massive cold‑storage workloads.

If metadata becomes unusable due to catastrophic hardware/software failures, it can be used to reconstruct the overall structure, albeit at high cost.

Embedding metadata reduces write amplification, avoiding double I/O writes that would hurt HDD performance.

We select several CMR zones at the disk head to store and redundantly replicate metadata. Because CMR zones already occupy about 1% of the disk, we abstract both CMR and SMR zones into append‑only Data Zones to maximize usable space.

These optimizations increase single‑disk storage space by 150% and, by using JBOD instead of traditional 36‑bay high‑density chassis, raise rack‑level capacity by 5.375× and disk count by 59%.

Additionally, a dual‑head hardware architecture ensures that all JBOD disks are visible to two hosts, so if one host fails the system can instantly failover, maintaining service availability.

Optimizing IO Scheduling to Reduce Operating Costs

Higher density reduces CAPEX, but long‑term OPEX (especially electricity) remains significant for archive storage. UCloud adds a spin‑up/spin‑down aware scheduling algorithm to the I/O layer, cutting power waste from idle disks in dense configurations.

Disks are grouped by failure domain within JBOD, allowing EC striping and fault tolerance at both disk and JBOD levels. Spin‑up/down operations are performed per group, balancing user‑required rapid reads with limits on spin cycles to extend disk lifespan.

Write performance is ensured by matching the bandwidth of each disk group to the network capacity, so sequential writes and compaction on SMR drives fully utilize available bandwidth without waste.

Final Thoughts

By increasing disk storage density and lowering operating costs through intelligent IO scheduling, the US3 archive storage engine achieves dramatically lower costs while guaranteeing high reliability for long‑term cold data. Future work will focus on further improving user experience, automating data cooling, exploring smarter cost‑reduction techniques, and investigating tape and other media for deep‑archive scenarios.