JD Retail Technology

Aug 21, 2026 · Artificial Intelligence

Janus: Dual‑Timescale Scheduling for Production‑Scale Multi‑LLM Serving

Janus, a Service‑Engine co‑design system built on Oxygen xLLM, uses dual‑timescale scheduling, performance‑oracle‑driven placement, and a three‑state model lifecycle to handle bursty traffic, power‑law application hotness, and heterogeneous resource demands, achieving 0.97–1.0 SLO rates while cutting device usage by 27%.