Ray: The Distributed Framework Powering the Next Generation of Generative AI

Ray is an open‑source distributed computing framework launched by the RiseLab lab at UC Berkeley. Since 2018, Ant Group has been a major contributor, adding core features and architectural improvements, and by August 2022 the company had eight committers, occupying twelve of the top‑50 contributor seats and contributing 26.3% of the Ray core code, second only to Anyscale.

Ant Group treats Ray as a high‑performance computing platform that forms the backbone of its internal compute infrastructure, supporting privacy‑preserving computation, federated learning, graph processing, video processing, and online training‑inference workloads. Leveraging Ray, Ant has built a real‑time training‑inference engine and scientific computing engine for AI scenarios, which have been open‑sourced to the community.

The engine powers Alipay’s app home page, wealth management, and digital finance online learning services, integrating a variety of deep‑learning frameworks such as TensorFlow, PyTorch, EasyDL, Alps, and Triton. It enables large‑scale models up to 1.2 trillion parameters (approximately 5 billion parameters per model) for real‑time inference.

The article, translated from Business Insider, highlights that Ray is the underlying framework used by OpenAI to train ChatGPT and other large language models, and that OpenAI’s leadership has publicly praised Ray for its flexibility, scalability, and ability to manage heterogeneous hardware across clouds like Google Cloud and AWS.

Ray is positioned alongside other emerging machine‑learning platforms such as Google’s JAX and the Dask framework from Coiled, all of which are being described as the “internal combustion engines” for the next generation of large language models developed by companies like Meta, Hugging Face, OpenAI, and Google.