What’s Driving the Next Wave of HPC: Cloud, Big Data, and Deep Learning?

Interview Overview

At the 2015 High‑Performance Computing User Forum (HPCUF2015), three experts—Prof. Qian Depei from Beihang University, Liu Jun, General Manager of High‑Performance Computing at Inspur, and Earl C. Joseph II, Vice President of IDC and Executive Director of the IDC HPC User Forum—were interviewed by CSDN reporters to analyze HPC trends in the era of cloud and big data.

Key Perspectives on HPC Development

Qian Depei emphasized that the Top‑500 ranking fluctuations in China are less important than the overall development level of HPC. He argued that HPC should not be confined to a narrow niche; a sufficient scale is needed to stimulate diverse applications.

Earl C. Joseph II highlighted that the rise of big data has turned HPDA (HPC‑based data analysis) into the next growth point, with 67% of HPC resources now dedicated to HPDA. Machine learning and deep learning are typical HPDA workloads, and scaling out with many cores is a common solution. He warned that cloud‑based HPC should avoid heavy virtualization because it can degrade performance.

Liu Jun noted that cloud, big data, and deep learning are reshaping HPC hardware requirements. He advocated for a unified computing architecture that supports heterogeneous and hybrid workloads, leading to the concept of “software‑defined HPC” and the “rack‑as‑HPC” hardware model.

HPDA and Real‑World Use Cases

HPDA blends traditional HPC performance with big‑data analytics. IDC predicts HPDA server revenue could reach $1.1 billion in 2015. Notable HPDA applications include:

Supercomputing‑driven analytics transforming the medical industry.

USPS’s Total Revenue Protection program, which processes 4 billion transactions in six hours using a memory‑centric database.

PayPal’s fraud‑detection system, saving $700 million and supporting massive daily login, transaction, insert, and select volumes.

IDC also points out that storage is the fastest‑growing segment of the HPDA market, while interconnect technologies and data migration remain major challenges.

Cloud, Big Data, and the Emergence of “Big Computing”

Liu Jun described a reciprocal relationship: HPC benefits cloud by providing shared resources and ease of use, while big data relies on HPC for performance. This synergy leads to “Big Computing,” a fusion of traditional HPC, cloud, and big‑data capabilities.

Application Diversity and Future Trends

Qian Depei stressed that the distinction between capacity‑oriented and performance‑oriented workloads is blurring. Modern supercomputers must support both, as many applications evolve from capacity‑type to performance‑type as they scale.

According to Earl C. Joseph II, the following trends will shape HPC:

Scalability across many cores, not just a few.

Integration of cloud, big data, and HPC while avoiding performance‑penalizing virtualization.

Reliability in cloud environments, ensuring continued operation despite hardware failures.

Longer application lifespans (5‑6 years) requiring hardware‑agnostic designs.

Industry‑specific customization, especially in CFD and automotive design.

Deep Learning as a Prime HPDA Application

Deep learning, driven by large datasets and massive compute, is a flagship HPDA use case. Inspur reports that deep learning and industrial applications have fueled nearly 100% growth in its HPC business over the past year.

While deep learning is important, Qian Depei cautioned against treating it as the sole focus; HPC must remain flexible to accommodate a wide range of emerging workloads.

Software‑Defined HPC

Liu Jun outlined Inspur’s progress toward software‑defined HPC, including plans for 64‑node systems with up to 48 TB memory and a “SmartRack”‑style integrated infrastructure that merges compute and storage.

Inspurs’ roadmap includes a “rack‑as‑HPC” product that delivers compute‑storage convergence and initial software‑defined networking capabilities.

China vs. International HPC Landscape

Earl C. Joseph II observed that Chinese HPC is newer and built on more standardized hardware, whereas Western HPC has a 35‑year legacy with many legacy applications tied to outdated architectures. This gives China an opportunity to develop modern, commercial‑ready HPC software.

He praised Chinese vendors such as Inspur, Sugon, Huawei, and Lenovo, noting strong growth for Inspur, HP, and Cray, while IBM, Hitachi, and Dell have seen declines.

Strategic Advice for Chinese HPC Vendors

Target open markets in Europe (e.g., Germany, UK, Italy) where demand is high.

Focus on the U.S. industrial sector and universities, which receive strong government support.

Establish local partnerships to navigate regional regulations and market dynamics.