Skip to content
Tagged articles

contract extraction

2 articles · Page 1 of 1
DataFunSummit
DataFunSummit
Nov 15, 2026 · Artificial Intelligence

284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512

The llama-modchip project runs a 284B-parameter DeepSeek-V4-Flash model on a 64GB RAM CPU-only machine via aggressive quantization to 1.71 bits/weight (56.56 GiB), achieving 9–10 tokens/sec, but a 96GB Xeon server fails due to custom GGUF tensor types and missing AVX-512, highlighting hardware-software compatibility gaps.

AVX-512CPU inferenceDeepSeek-V4-Flash
0 likes · 13 min read
284B DeepSeek on 64GB CPU: Quantization Works, 96GB Xeon Fails on AVX-512
DataFunTalk
DataFunTalk
Nov 8, 2026 · Artificial Intelligence

DeepSeek-V4-Flash on 64GB CPU: Why 96GB Xeon Can't Load It

The article explains how llama-modchip runs 284B DeepSeek-V4-Flash on a 64GB RAM CPU-only machine via aggressive quantization (1.71 bits/weight), while a 96GB Xeon server fails due to tensor format mismatch and missing AVX-512. It breaks down inference latency for long contracts (~10 minutes for 20k tokens) and proposes three evaluation metrics: time to first token, key-field extraction accuracy, and total processing time.

AVX-512CPU inferenceDeepSeek-V4-Flash
0 likes · 12 min read
DeepSeek-V4-Flash on 64GB CPU: Why 96GB Xeon Can't Load It