Intel Announces Aurora genAI: A Trillion-Parameter Generative AI Model Powered by the Aurora Supercomputer

As ChatGPT continues to blaze for months, with tech giants like Microsoft, Google, and Meta joining in, Intel has finally announced its entry into the competition.

Last weekend at the International Supercomputing Conference (ISC) in Hamburg, Germany, Intel showcased its leading performance in HPC and AI workloads and announced an unexpected plan: in partnership with Argonne National Laboratory, it will develop a generative AI model, Aurora genAI, on the Aurora supercomputer with up to one trillion parameters.

For reference, ChatGPT has only 175 billion parameters, so Aurora genAI will be at least five times larger.

(Image source: Intel website)

AI model will be powered by Aurora supercomputer

According to Intel, the Aurora genAI model will be built on two frameworks: NVIDIA's Megatron and Microsoft's DeepSpeed.

Megatron: an architecture for distributed training of large language models, optimized for Transformers, supporting both data parallelism and model parallelism.

DeepSpeed: focuses on optimizing training of large deep learning models, enhancing scale, speed, cost, and usability, enabling training of 100‑billion‑parameter models and greatly facilitating large‑model training.

In addition to these frameworks, the Aurora genAI model will draw compute power from the Aurora supercomputer—a system Intel designed for Argonne, which after many delays has finally been realized.

Public data shows the Aurora supercomputer is powered by Intel Xeon CPU Max and Xeon GPU Max chips, comprising 10,624 nodes, 63,744 Ponte Vecchio GPUs, 21,248 Sapphire Rapids Xeon CPUs, 1,024 DAOS storage nodes, and 10.9 PB of DDR5 persistent memory.

Intel also disclosed early performance results: Aurora delivers leading performance on scientific and engineering workloads, twice the performance of AMD MI250 GPUs, 20% faster than H100 on the QMCPACK quantum mechanics application, and scales near‑linearly across hundreds of nodes.

Notably, compared with the original 1 Exaflop goal, the Aurora supercomputer is expected to deliver over 2 Exaflops of double‑precision performance at launch, surpassing the current Top500 leader Frontier (1.194 Exaflop/s).

Science‑focused generative AI model

With the powerful Aurora supercomputer as its foundation, Aurora genAI will be large; Intel notes that Argonne National Lab is leading international collaborations for the model.

Argonne deputy lab director Rick Stevens said: “The project aims to harness the full potential of the Aurora supercomputer to produce a resource for downstream science at DOE labs and to collaborate with other institutions.”

Overall, Aurora genAI is a science‑focused generative AI model that will be trained on general text, code, scientific literature, and data from biology, chemistry, materials science, physics, and medicine.

The resulting AI model will have up to one trillion parameters, encompassing knowledge from molecular and material design to millions of data sources, applicable to scientific domains such as systems biology, cancer research, climate science, cosmology, polymer chemistry, and materials, and also to fields like financial modeling, NLP, machine translation, image and speech recognition.

Planned completion in 2024

Beyond this, Intel has not disclosed further details, but media reports indicate Intel plans to develop and complete Aurora genAI in 2024, so the model may arrive soon if progress stays on track.

The announcement has attracted attention, and Intel's entry into large AI models with a trillion‑parameter start raises expectations for the future of GPT‑4 and competing models.

“A trillion parameters is a special limit, though one could be skeptical and view it as a huge number to watch. Undoubtedly, if this model resembles GPT‑4, it adds another data point. With companies constantly announcing new models, I wonder if we’ll peak by June.”

“People are building new systems with H100, and better AI GPUs are already emerging; if this continues, NVIDIA may need to announce new cards quickly to stay ahead.”

“I guess GPT‑4 will soon stop being SOTA on many benchmarks, and perhaps it will be trained on the world’s fastest supercomputer. For reference, OpenAI’s supercomputer has about 10,000 GPUs, while Aurora has 63,744 GPUs.”

Reference links:

https://www.intel.com/content/www/us/en/newsroom/news/intel-delivers-ai-accelerated-hpc-performance.html#gs.yhhdis

https://www.business2community.com/tech-news/intel-announces-1-trillion-parameter-chatgpt-competitor-aurora-genai-02695750#

https://www.reddit.com/r/singularity/comments/13ozabi/intel_announces_aurora_genai_generative_ai_model/