How PrismML’s Bonsai Compresses a 27B Model to 3.9 GB for Browser and Phone Deployment

PrismML released two edge‑focused variants of the Qwen3.6 27B model, called Bonsai. The Ternary version occupies 5.9 GB (1.71 effective bits per weight) for notebook‑class inference, while the 1‑bit version is only 3.9 GB (1.125 effective bits per weight) and can run on an iPhone 17 Pro.

Fitting a 27B model onto a mobile device is challenging: at 16‑bit precision the model needs 54 GB, and even 4‑bit quantization still requires 18 GB. Mobile apps typically have only ~6 GB of usable memory after accounting for KV cache and activations, so traditional quantization cannot meet the constraint. The 1‑bit Bonsai therefore represents the first model that breaks this barrier.

On the Math‑500 benchmark (introduced by OpenAI’s “Let’s verify step by step” paper) the 1‑bit Bonsai scores 99.2 %, placing it ahead of DeepSeek‑R1, Kimi K2 and GLM‑4.5. PrismML published results on fifteen tasks; the full model scores 85.0 overall, the Ternary variant 80.5 and the 1‑bit variant 76.1. Detailed scores include:

Math: 95.3 (full), 93.4 (Ternary), 91.7 (1‑bit)

Programming: 88.7, 86.0, 81.9

Agent/Tool calling: 80.0, 74.0, 66.0

Instruction following: 78.4, 71.8, 65.8

Knowledge/STEM: 83.1, 77.0, 73.4

Vision: 72.6, 65.2, 59.6

The Ternary variant retains about 95 % of the full‑precision capability, while the 1‑bit version keeps roughly 90 %, with minimal loss in mathematics and programming performance—crucial for agent scenarios that require many repeated calls.

Tim Carambat, after receiving early access, compared the Bonsai 1‑bit model with the OpenRouter Qwen3.6 27B and found them 100 % comparable. On an M4 Pro laptop the model runs at ~40 tokens / s using a fork of llama.cpp. An AMD RX 7800 XT can run the model with a 90k context in ~13 GB VRAM at the same throughput.

PrismML introduces the notion of "intelligent density": the 1‑bit Bonsai delivers 0.53 units of intelligence per GB, more than ten times the baseline full‑precision model. This makes local inference a turning point for agent applications, eliminating per‑token cloud costs and preserving user privacy.

Because the model fits on‑device, agents can execute hundreds of steps without network latency, enabling offline assistants, local document reasoning, and other privacy‑sensitive use cases that were previously infeasible due to model size.

The project is backed by a Caltech research team with support from Khosla Ventures, Cerberus, Google, and Samsung. The model is released under Apache 2.0, supports 256 K context, multimodal input, and the following resources are provided:

HuggingFace model collection: https://huggingface.co/collections/prism-ml/bonsai-27b

llama.cpp fork: https://github.com/PrismML-Eng/llama.cpp

MLX branch: https://github.com/PrismML-Eng/mlx

Browser demo (WebGPU kernels): https://huggingface.co/spaces/webml-community/bonsai-webgpu-kernels

Technical report: https://paperswithcode.co/paper/103979