Ling-3.0-flash: Open-Source LLM Designed for Real-World Deployment

Model Overview

Ling-3.0-flash is a native mixture-of-experts (MoE) model with 124 B total parameters and 5.1 B active parameters per token, preserving the knowledge capacity of trillion-parameter models while reducing per-token compute.

Four precision variants are released: FP8, FP4, MXFP4 and INT4, allowing selection based on accuracy, hardware, inference cost and data‑security requirements.

Deployment Options

API access – cloud endpoint for rapid integration without building inference infrastructure.

Single-machine private deployment – MXFP4 and INT4 versions run end-to-end on a single NVIDIA DGX Spark (128 GB LPDDR5x), keeping data on-premise.

High-performance deployment – on a specified GPU configuration the model achieves >1100 tokens/s average output rate for latency-sensitive services.

Performance Benchmarks

Artificial Analysis leaderboard: 353 tokens/s output speed.

AA Intelligence Index: weighted average task cost $0.04, average decoding time 1.4 minutes.

Local MXFP4 inference: ~80 tokens/s; prefill speed 2500–3500 tokens/s; supports 3–4 concurrent users on a DGX Spark.

High-performance batch‑1 test (TP4, input 8192, output 1024): average 1100+ tokens/s, token-per-output-time ≈0.8 ms.

Optimization Techniques for High-Performance Deployment

Inference-engine engineering : CPU/GPU overlap, operator-level overlap, KDA operator tuning and operator fusion to reduce synchronization stalls.

Speculative decoding : DSpark Draft Model generates candidate token blocks; block size can be tuned for general workloads or refined with domain data for vertical use cases.

Local Deployment Case Study – Medical Research Data

On a single NVIDIA DGX Spark the model processes sensitive medical research data in a closed-loop pipeline:

Sensitive information handling : de-identifies personal identifiers and assigns dynamic IDs to samples.

Data structuring : extracts vital signs, biochemical indicators and imaging reports, converting unstructured records into structured analysis‑ready tables.

Workflow assistance : groups samples according to research rules; flags ambiguous or insufficient cases as “Unclassified” for expert review.

This demonstrates three benefits: data remains local, external dependencies are eliminated, and small teams can integrate the model into internal systems.

Decision Guidance

Use the API for quick validation and elastic usage. Choose MXFP4/INT4 for on-premise workloads with limited users or data‑privacy constraints. Adopt high-performance hardware and DSpark optimizations when ultra‑low latency is required for real-time agents, continuous tool calls or interactive code completion.

Community Integration

The model has been tested with agents and developer tools such as OpenCode, Kilo Code and nanobot, and integrated with inference services including Novita AI, Fireworks AI, DeepInfra and the open-source SGLang framework.

Model Weights and Code

Hugging Face repository: https://huggingface.co/inclusionAI/Ling-3.0-flash

FP8 variant: https://huggingface.co/inclusionAI/Ling-3.0-flash-fp8

FP4 variant: https://huggingface.co/inclusionAI/Ling-3.0-flash-fp4

INT4 variant: https://huggingface.co/inclusionAI/Ling-3.0-flash-int4

ModelScope repository: https://modelscope.cn/models/inclusionAI/Ling-3.0-flash

FP8 on ModelScope: https://modelscope.cn/models/inclusionAI/Ling-3.0-flash-fp8

FP4 on ModelScope: https://modelscope.cn/models/inclusionAI/Ling-3.0-flash-fp4

INT4 on ModelScope: https://modelscope.cn/models/inclusionAI/Ling-3.0-flash-int4

Illustrative Images