Local LLM Compatibility Tool: 201 Models, 126 Hardware Configs, 12K Speed Benchmarks
The author built a Chinese-language tool covering 201 models and 126 hardware devices with 12,609 estimated speed data points, letting users select their device and memory to instantly see which quantized LLMs fit, their expected token generation speeds, and Ollama install commands, while revealing insights like multi-GPU diminishing returns and Mac speed being chip-dependent not memory-dependent.
Why Build Another Site
Running LLMs locally requires piecing together three fragmented data sources: model quantized sizes, your machine's usable memory, and real-world speed benchmarks. These are scattered across different sites with inconsistent quantization labels (Q4 vs Q8) and hardware bases (A100 vs M-series). Critically, fitting weights into memory is only the entry ticket; generation speed depends on memory bandwidth. The same 27B Q4 model runs twice as fast on an RTX 5090 as on an older RTX 3090.
The site turns "which model can my machine run" from a research task into a three-click query: pick device, pick memory, get results.
Four Modules in One Page
The site packs four modules into a single page:
Selection Guide — Choose device class (Mac, MacBook, Mac Studio, NVIDIA, AMD) then specific model or memory tier (16GB, 32GB, 64GB). Returns a list of runnable models with Ollama install commands.
Speed Ladder — Top 8 real-device uploads, then a full table of 72 devices ranked by estimated speed on a fixed reference model (qwen3.8-27b-q4). Switching to qwen2.5-7b-q4 expands coverage to 125 devices. Horizontal comparison only makes sense when the model is fixed.
Memory Comparison — 14 tiers from 8GB to 512GB. Each tier shows dense/MoE model size limits, recommended models, and how many models fit.
Model Library + Statistics — 201 models, 15 per page, searchable, filterable by family, sortable by column. Statistics split into 14 memory-tier details and 17 key facts, all paginated on the same page.
Speed ladder: fixed reference model (qwen3.8-27b-q4) across 72 devices, paginated. Site's four modules: selection guide, speed ladder, memory comparison, model library + statistics. Device selection area with eight device-class icons. Memory comparison: 14 tiers from 8GB to 512GB showing dense/MoE limits and model counts. Model library: search, family filter, run-mode filter, sortable columns, paginated. Full statistics: 14 memory-tier details and 17 key facts, paginated. Four Counter-Intuitive Insights from the Speed Ladder Four insights derived from the speed ladder data. Multi-GPU is not 1+1=2. 2×RTX 5090 yields 86.7 tok/s, slightly lower than a single RTX 5090 at 89 tok/s. Parallelism overhead means stacking cards does not linearly stack speed — a cost-saving reminder for multi-GPU workstation buyers. Mac speed is determined by the chip; memory only decides capacity. M5 Ultra runs at 55.4 tok/s whether configured with 96GB, 256GB, or 512GB. More memory lets you load larger models, but does not increase generation speed. Top GPUs hit bandwidth saturation. Both RTX 5090 and RTX PRO 6000 Blackwell score 89 tok/s on 27B Q4, indicating the model size already saturates single-card memory bandwidth; adding VRAM does not raise throughput. These speeds are estimates, not measured runs. The ladder uses a bandwidth-based formula, not your actual hardware benchmark. Head-to-Head: RTX 5090 vs Mac Studio on 27B Q4 Key model speed comparison: RTX 5090, Mac Studio M5 Ultra, and other configurations on 27B Q4. Direct conclusion: NVIDIA single GPU is faster; Mac Studio wins on maximum model size. RTX 5090 leads at 89 tok/s. Mac Studio M5 Ultra delivers 55.4 tok/s, identical across 96GB and 512GB — confirming memory only affects capacity. Cross-generation Mac Ultra speeds: M5 Ultra 55.4 > M3 Ultra 37.1 > M2 Ultra 34 > M1 Ultra 30.6. Chip generation is the speed divider. The 512GB M5 Ultra buys you the ability to load 1T-scale models, not faster tokens. Data Provenance and Trustworthiness Data scope and reliability: sources, methodology, and error metrics. One-sentence summary: The dataset is open and verifiable, but ladder speeds are bandwidth-derived estimates for horizontal comparison — not a guarantee of your machine's actual throughput. The original site collects real-device uploads (CLI prompts before upload, no PII or file contents). The Chinese mirror places those 8 real runs at the top. The remaining 12,609 speed cells come directly from the official ModelFit open dataset API — cell-by-cell diff is zero. The "estimation" is ModelFit's own methodology: a physics-based formula using memory bandwidth and model size, fitted on 4,831 real runs across 104 hardware types with leave-one-out validation, yielding a median absolute error of 10.7%. A collapsible "Data Sources & Methodology" block at the bottom of the page exposes these numbers and the API link for independent verification. Data sourced from ModelFit's CC BY 4.0 dataset, updated to 2026-11-16. The Chinese site is independent, unaffiliated with ModelFit, and retains attribution in the footer. How to Use Open the site via the link at the bottom of the original article (WeChat converts links to plain text, so copy the URL): https://workbuddy.link/p/UNhCUIKUELAnBDKFw1x235?source=2 Top search box accepts model or hardware names; pressing / also triggers search. Background switches automatically between light (day) and dark (night) per Beijing time, with manual override. Final question for readers: What's your current machine's memory, which model are you running, and what real tok/s do you measure? Post your config and numbers in the comments — large deviations will be used to refine the estimation model.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
