DeepHub IMBA

Oct 22, 2026 · Artificial Intelligence

Gufo's XDNA2 NPU Offload for LLM Prefill: 5-10% Speedup, Numerical Accuracy Issues

Gufo's experimental branch enables GPU-NPU cooperative inference on AMD Ryzen AI MAX+ 395, offloading select Prefill GEMM operations to the XDNA2 NPU for 5-10% throughput gains, but introduces numerical errors reducing Top-1 token consistency to ~94% due to 8-bit block floating-point accumulation with repeated BF16 rounding.