How Meta’s Open‑Source 30B Muse Glimmer Fits Into 24 GB VRAM for Always‑On Local Agents

Meta Superintelligence Labs released Muse Glimmer, an open‑source 30B dense causal transformer (≈29.6 B parameters) with a 1.8 B‑parameter visual perception encoder, supporting text and image inputs and a context length of over 131 k tokens, explicitly designed for “always‑on local agent workflows” and released under the Apache 2.0 license.

The training pipeline consists of three stages: (1) Pre‑Training uses logit distillation from Muse Spark; (2) Mid‑Training raises the share of long‑context, agent data, and reasoning traces; (3) Post‑Training combines supervised fine‑tuning, on‑policy distillation, and reinforcement learning to cover general, reasoning, coding, and agentic tasks, embedding tool use, multi‑step reasoning, multimodal understanding, and failure recovery directly into the model.

The “always‑on” focus stems from the need for agents that continuously read files, interpret screenshots, call tools, manage schedules, execute code, and retry after failures; cloud‑based APIs introduce latency and cost that are unacceptable for such high‑frequency, stateful operations, making offline residency and fast repeated inference essential.

Quantization reduces the model’s memory footprint: full‑precision requires >55 GB, while 4‑bit quantization compresses weights below 20 GB, freeing space for KV cache, the visual encoder, and a DFlash speculative decoding drafter. Two quantization targets are offered: K‑Quant‑Dynamic for 32 GB VRAM (≈0.2 % average benchmark accuracy loss) and K‑Quant‑17GB for 24 GB VRAM (≈1.0 % loss), with Meta confirming that task performance remains usable after compression.

Benchmark comparisons on several agentic tasks show Muse Glimmer leading many scores: MCP Atlas 75.5 (vs. Gemma‑4‑31B 54.2, Qwen‑3.6‑27B 62.5), DeepSearch QA 74.6 (vs. 61.7 and 71.1), and SWE‑Bench Pro 51.2 (vs. 36.9 and 50.2). It trails on some tests such as SWE‑Bench Verified and OSWorld‑Verified, highlighting its design trade‑off between model size, deployment cost, and agentic capability.

Speed measurements reveal baseline decoding rates of 74.9 tok/s on an RTX 5090, 26.6 tok/s on a MacBook M5 Max, and 23.7 tok/s on an M4 Max. With DFlash speculative decoding (predicting a 16‑token block and parallel verification), average speeds rise to 233.4 tok/s (RTX 5090, ~3.1×), 50.2 tok/s (M5 Max, ~1.8×), and 37.8 tok/s (M4 Max, ~1.5×), enabling near‑real‑time interaction for continuous agent inference.

These results shift the deployment boundary for agentic AI: a 30B multimodal model can reside in consumer‑grade memory (24‑32 GB) and run at interactive speeds, supporting a hybrid architecture where the local model handles frequent, high‑throughput tasks while a larger cloud model tackles complex reasoning, turning agents from one‑off API calls into persistent, on‑device runtimes.