JD Donates Oxygen xLLM Inference Engine to OpenAtom Foundation to Accelerate Domestic AI Infra

Open‑source donation

On 25 June 2026 JD donated the Oxygen xLLM large‑model inference engine—including source code, patents, trademarks and related rights—to the OpenAtom Open Source Foundation under the Apache 2.0 license. The project is hosted at https://github.com/jd-opensource/xllm.

Engineering Intelligence (EI) vision

EI aims to make the AI infrastructure stack itself intelligent: the scheduler automatically senses workload characteristics and re‑optimises resource allocation; the inference engine generates optimal execution plans based on model topology and hardware capabilities; the entire pipeline gains self‑awareness, self‑decision and self‑optimisation.

Service‑Engine decoupled architecture

Service layer (xLLM‑Service) provides unified elastic scheduling for online and offline tasks, supports dynamic PD (parameter‑distribution) separation to absorb traffic spikes, and maintains a global KV cache with fast fault‑recovery to guarantee large‑scale availability.

Engine layer (xLLM‑Engine) implements multi‑stage pipelines that fully overlap computation and communication, an adaptive graph mode with efficient memory management to handle dynamic inputs and GPU memory allocation, and specialised optimisations for Mixture‑of‑Experts (MoE), speculative decoding and generative‑recommendation scenarios, thereby maximising hardware utilisation.

Hardware and model support

The framework exposes a unified AI Gateway and an OpenAI‑compatible SDK. It natively supports GPU, NPU and MLU accelerators and abstracts heterogeneous chips so that LLM, VLM, DiT, text‑to‑image/video and generative‑recommendation models can run on mixed domestic hardware without code changes.

Core technical highlights

Architectural innovation – first‑ever service‑engine decoupling enables independent evolution of scheduling and compute while preserving collaborative efficiency.

Performance breakthrough – multi‑stage pipelines, adaptive graph mode and dynamic PD separation improve throughput and resource utilisation under strict SLO constraints, surpassing existing state‑of‑the‑art inference frameworks.

Heterogeneous unification – a unified inference abstraction layer masks differences among chips and models, supporting a wide range of model families on mixed domestic accelerators.

High‑availability guarantees – global KV cache management, distributed fast fault recovery, health monitoring and automatic inspection ensure stable production at scale.

Domestic adaptation – comprehensive support for GPU, NPU and MLU fills the gap of “heterogeneous‑chip unified inference” and lowers deployment barriers for Chinese AI hardware.

Industrial validation

E‑commerce – in JD’s customer‑service large‑model service, cluster utilisation increased by >35 % and P99 latency decreased by 28 % during peak traffic.

Power‑grid inspection – inspection efficiency improved ~3×, outage‑incident rate fell by 30 %, and emergency‑repair efficiency rose by 20 %.

Public‑safety – partner Lianhui’s edge deployment achieved a 227 % increase in inspection efficiency, a 127 % boost in concurrent handling, and a 50 % reduction in time‑to‑first‑trace.

Community status

Since open‑sourcing, the repository has earned over 1.4 k GitHub stars, 235 forks and contributions from major domestic chip and model vendors.

Roadmap

Planned milestones include full multimodal model support (text‑to‑image, video, Omni), comprehensive adaptation to mainstream domestic chips and a commercial enterprise edition by 2026, with the contributor base targeted to exceed 200 developers. From 2027 onward the project aims to become the de‑facto standard for large‑model inference on domestic chips.