Meta's 816B Muse Spark Open Offers Full Agent Training Stack, But Chinese Models Lead

Model Release and Specifications

On October 15, Meta released Muse Spark Open, an open-weight multimodal model with 816 billion total parameters (approximately 42 billion active parameters per token), a 1 million token context window, and support for text, image, video, and audio inputs with text output. The model shares its base with Muse Spark 1.3 and adds safety post-training covering multi-file refactoring, debugging, and long-running development tasks. Its architecture uses a mixed sliding-window and global attention mechanism to manage long-context cache overhead.

Integrated Post-Training Ecosystem

Meta simultaneously provided a full training stack: TorchTitan adds support for multi-turn supervised fine-tuning, LoRA, and TitanRL reinforcement learning paths; SGLang and vLLM handle deployment; Miles and verl provide distributed RL training routes. For teams not wanting to manage clusters, hosted post-training services are also available. This allows developers to continue training the model on proprietary data and tasks.

Full-Parameter GRPO Experiments

SGLang and Miles teams ran GRPO training on the full language model using the DAPO-Math-17K dataset. Experiments used 64 H200 GPUs and 32 B300 GPUs. Miles contributed distributed training support with optimizer state offloading to CPU memory or NVMe to alleviate resource pressure. A critical multi-turn agent issue was fixed: chat template re-application during training must preserve the exact token prefixes generated during sampling to keep trajectories consistent. SGLang and Miles patched the template behavior accordingly.

On Geometry3K with frozen visual encoder, 30 updates lifted test accuracy from 88.2% to 94.9%.

LoRA RL with NVFP4 Quantization

vLLM and verl offered an alternative path: NVFP4 quantized weights with LoRA reinforcement learning. vLLM handles sampling, verl FSDP2 trains adapters, and adapters are synced back to the inference engine without full-model synchronization each round. On 16 GB300 GPUs, 150 GRPO steps raised AIME 2024 mean@8 accuracy from 0.56 to 0.85.

Benchmark Comparison with Chinese Models

Meta labels Muse Spark Open as the "strongest US open-weight model," but its own evaluation tables show several Chinese models scoring higher on coding benchmarks:

Terminal-Bench 2.1: Muse Spark Open 76.4 vs Qwen 82.0, GLM 83.9, Kimi K3 85.0

DeepSWE 1.1: Muse Spark Open 60 vs GLM 67, Kimi 68

These results indicate Chinese models already lead on agent coding tasks.

Chinese Vendors' Comparable Infrastructure

Chinese companies have already deployed similar end-to-end agent training infrastructures:

Kimi K3 ships with Kimi Code CLI, supports vLLM and SGLang deployment, and integrates quantization-aware training from the SFT stage to improve deployment efficiency.

DeepSeek operates the DSec sandbox platform co-designed with its RL framework, supporting over 380,000 concurrent sandboxes and creating more than 5,000 sandboxes per second in production for code execution, tool use, and trajectory generation.

Zhipu (GLM) open-sourced the slime RL framework used to train GLM, and the official GLM repository provides continued training paths via slime and ms-swift for both RL and SFT.

Conclusion

Meta's release of Muse Spark Open with a ready-to-use post-training pipeline lowers the barrier for building custom agents, but the technical landscape is already competitive. Chinese models not only outperform on key coding benchmarks but also offer mature, production-grade training and deployment stacks. The next practical questions for developers are which training path to choose, the associated cost, and integration feasibility into existing systems.