Mac Studio + RTX 5090 eGPU Runs Qwen3.8-Flash-Next at 210 tok/s, 2.5x M4 Max

How It Works

The breakthrough relies on the TinyGPU driver from the tinygrad team, which Apple has approved. This driver lets Apple Silicon Macs use NVIDIA (Ampere and newer) or AMD (RDNA3+) GPUs via Thunderbolt/USB4 eGPU docks for AI inference — compute only, no display output. Joseph Savage ( @joseinvestments) built Apple NVIDIA Sidecar on top of TinyGPU, using the Strata engine to split the workload between the Mac and the RTX 5090. The project was open-sourced on November 8 (repository: joesinvestments/apple-nvidia-sidecar, weights and drivers not included).

Speed Ladder: 210 tok/s in Context

The author benchmarked the same Qwen3.8-Flash-Next model with NVIDIA's official FP4 weights across several machines (methods differ, so treat as order-of-magnitude):

Apple NVIDIA Sidecar (M5 Max 64GB + RTX 5090 eGPU): 210 tok/s

RTX PRO 5000 72GB: 159 tok/s

Single RTX 5090 PC: 115 tok/s

Pure M4 Max: 83 tok/s

Sidecar tops the chart, and it's not merely "borrowing the 5090's power" — the next section explains why it beats a bare 5090 PC.

Why It's So Fast: MoE Name-Real Separation

Qwen3.8-Flash-Next is a 125B total / 6B active Mixture-of-Experts model, plus a 51B n-gram embedding and a 4B MTP head. NVIDIA's official FP4 weights are about 96 GiB , which exceeds the 64GB unified memory of the Mac — a pure Mac can only run a quantized 3.3-bit "reduced" version.

Sidecar's solution is a "split the loot" approach: the RTX 5090 holds the most frequently used experts, while the Mac holds the remaining experts and uses its own GPU to compute. Each layer passes work back and forth over the Thunderbolt link. This combines the Mac's 614 GB/s memory bandwidth with the 5090's 1.8 TB/s bandwidth, pushing single-stream decode to 210 tok/s.

Four Hard Boundaries

The author lists limitations explicitly:

Single session only: Cold start with 1–16 concurrent streams drops total throughput to ~81 tok/s — unsuitable for multi-user or batch workloads.

Driver not upstreamed: The driver modifications are local and not merged into the repository; you must compile the driver yourself to reproduce.

Slow prefill: Prefill performance remains a bottleneck.

DIY complexity: Requires an M5 Max/Studio, a spare RTX 5090, a Thunderbolt eGPU enclosure, and willingness to compile kernels.

Should You Do This? Decision Guide

Already have M5 Max/Studio + idle RTX 5090 + love tinkering: Go for it. 210 tok/s on a 125B model is 2.5× the pure M4 Max (83 tok/s) and ~1.7× the author's own M5 Max single-machine mlx-serve (121 tok/s). Real-world coding and long-document reading feel noticeably faster.

Pure Mac user, no NVIDIA GPU: Wait. The author's own comparison shows a pure Mac (M5 Max running mlx-serve 3.3-bit) hits 89–121 tok/s, plenty for daily use. Revisit when Sidecar fixes driver and concurrency.

Planning to buy hardware specifically for large models: Buy a 5090 PC or wait for official solutions. Sidecar is currently a "geek toy," not a product — don't spend extra money for it.

Need stable production / multi-concurrency: Pass for now. Single-session + DIY driver is far from production-ready.

Data source: Facts from Joseph Savage ( @joseinvestments ) Nov 8 post and community mirrors; speed numbers are author-reported / per-machine measurements, not a unified benchmark. Qwen3.8-Flash-Next specs (125B total / 6B active / NVFP4 ~96 GiB) from NVIDIA Hugging Face model card and multiple hardware reviews. Chinese translation by Wai ( @avi_wuing ) was one clue source.