Go SIMD Deep Dive: Hardware, Compiler Intrinsics & 9x Dot Product Speedup

This article is a translation of Jesús Espino's deep-dive "SIMD and Go," covering Go's experimental SIMD support introduced in Go 1.26 (simd/archsimd) and Go 1.27 (simd). All examples target amd64 with GOEXPERIMENT=simd enabled.

What SIMD Means

SIMD (Single Instruction, Multiple Data) lets one CPU instruction process multiple data elements simultaneously. A scalar loop adds pairs one by one; a SIMD instruction loads eight float32 values into a 256-bit YMM register and adds them in parallel. The constraint: all lanes must perform the identical operation. Vector registers (128/256/512-bit) are divided into lanes (e.g., eight float32 lanes in 256-bit). The register itself is just bits; each instruction decides how to interpret lanes.

SIMD Lifecycle: Load, Operate, Store

Almost all SIMD code follows three steps:

Load: One instruction reads eight float32 values from a slice into a register's lanes (e.g., a[0] → lane 0). Remaining elements (tail) wait for the next load.

Operate: Vertical operations — each lane computes independently with its counterpart in another register (e.g., lane 3 of a + lane 3 of b → lane 3 of result). No cross-lane data flow.

Store: Write all lanes back to memory in one instruction. Alternatively, extract a single lane to a scalar variable, but repeated extraction kills performance.

Go's Two SIMD Packages

simd/archsimd: Fixed-Size, Architecture-Bound

Types map 1:1 to hardware registers: Float32x8 (256-bit, eight lanes), Float32x4 (128-bit), Float32x16 (512-bit). Methods like LoadFloat32x8, Add, Store, GetElem mirror the lifecycle. Example adding two slices in chunks of eight:

func add(a, b, out []float32) {
	for i := 0; i+8 <= len(a); i += 8 {
		x := archsimd.LoadFloat32x8(a[i:])
		y := archsimd.LoadFloat32x8(b[i:])
		x.Add(y).Store(out[i:])
	}
}

simd: Portable, Dynamically Sized

Types have no size suffix: Float32s, Int8s, Mask32s. Lane count is decided at program startup via v.Len() (4, 8, or 16). Same method names ( Add, MulAdd, Store). The loop uses dynamic stride:

func add(a, b, out []float32) {
	var v simd.Float32s
	n := v.Len()
	for i := 0; i+n <= len(a); i += n {
		x := simd.LoadFloat32s(a[i:])
		y := simd.LoadFloat32s(b[i:])
		x.Add(y).Store(out[i:])
	}
}

Core SIMD Operations

Broadcast: Fill All Lanes with One Scalar

Needed for mixing vectors with constants. BroadcastFloat32x8(5) → [5 5 5 5 5 5 5 5]. Cheap; usually done once before the loop.

Fused Multiply-Add (FMA): Two Steps in One

Computes x*y + z in a single instruction, same latency as multiply. Essential for dot products: each lane maintains a running sum. Example:

x1 := archsimd.LoadFloat32x8([]float32{1,2,3,4,5,6,7,8})
x2 := archsimd.LoadFloat32x8([]float32{8,7,6,5,4,3,2,1})
var acc archsimd.Float32x8
acc = x1.MulAdd(archsimd.BroadcastFloat32x8(2), acc) // [2 4 6 8 10 12 14 16]

Comparisons Produce Masks

SIMD cannot branch per lane. a.Greater(broadcast(5)) returns a Mask32x8 where each lane is all-ones (true) or all-zeros (false). mask.ToBits() yields a bitmask (e.g., 0b10101010).

Blend / Masked: SIMD's If-Else

All lanes compute both outcomes; mask selects per lane. x.IfElse(mask, y) picks from x where mask true, else y. x.Masked(mask) blends with zero vector (filter).

Permute: Cross-Lane Reordering

An index vector dictates lane destinations. Example reversing a vector:

x := archsimd.LoadFloat32x8([]float32{1,2,3,4,5,6,7,8})
idx := archsimd.LoadUint32x8([]uint32{7,6,5,4,3,2,1,0})
r := x.Permute(idx) // [8 7 6 5 4 3 2 1]

Horizontal Operations: Cross-Lane Reduction

ConcatAddPairs

adds adjacent lanes across two registers: x.ConcatAddPairs(y) → [x0+x1, x2+x3, y0+y1, y2+y3]. Used for folding partial sums.

Partial Loads: Safe Tail Handling

For slices not a multiple of lane count, LoadFloat32x8Part loads only valid elements, zero-pads the rest, and returns the count. In Go 1.27.0 amd64 it still reads 32 bytes then zeros extra lanes (fixed upstream).

s := []float32{1,2,3,4,5,6,7,8,9,10}
x, n := archsimd.LoadFloat32x8Part(s[8:]) // x=[9 10 0 0 0 0 0 0], n=2

Compiler Magic: How Go Methods Become CPU Instructions

archsimd: Compiler Intrinsics

Methods like Add have no body — they are compiler intrinsics. The compiler matches x.Add(y) in an intrinsic map ( simdAMD64intrinsics.go), emits an SSA op, rewrites to VADDPS ( simdAMD64.rules), and allocates a physical YMM register. The type Float32x8 carries a marker telling the compiler to keep it in a vector register for its entire lifetime.

simd: Multi-Version Compilation (Midway)

Since lane count is only known at runtime, the compiler generates multiple copies of each function: dot@simd128, dot@simd256, dot@simd512, plus a pure-Go fallback dot@simd0. A dispatcher (in rewrite.go) picks the best version at startup based on CPU features (e.g., AVX2 → simd256). Override with GODEBUG=simd=256 or =0. Stack traces show the chosen variant (e.g., dot@simd256).

End-to-End Example: Dot Product

Complete archsimd implementation for []float32 dot product:

func dotSIMD(a, b []float32) float32 {
	var acc archsimd.Float32x8
	i := 0
	for ; i+8 <= len(a); i += 8 {
		x := archsimd.LoadFloat32x8(a[i:])
		y := archsimd.LoadFloat32x8(b[i:])
		acc = x.MulAdd(y, acc)
	}
	x, _ := archsimd.LoadFloat32x8Part(a[i:])
	y, _ := archsimd.LoadFloat32x8Part(b[i:])
	acc = x.MulAdd(y, acc)
	s := acc.GetLo().Add(acc.GetHi())   // 8 → 4 lanes
	s = s.ConcatAddPairs(s)             // 4 → 2
	s = s.ConcatAddPairs(s)             // 2 → 1 (replicated)
	return s.GetElem(0)
}

Step-by-step for 10-element slices:

Full block (i=0): Load a[0:8], b[0:8]; MulAdd → acc = [10,18,24,28,30,30,28,24].

Tail (i=8): Partial load of last two elements (9,10 and 2,1), zero-padded; MulAdd adds [18,10,0,0,0,0,0,0] → acc = [28,28,24,28,30,30,28,24].

Fold: GetLo() ( [28,28,24,28]) + GetHi() ( [30,30,28,24]) = [58,58,52,52]. ConcatAddPairs → [116,104,116,104]. Again → [220,220,220,220].

Extract: GetElem(0) → 220.

Generated assembly for the hot loop is three instructions:

VMOVQQU 0(DX), Y2
VMOVQQU 0(R11), Y3
VFMADD213PS Y1, Y3, Y2

Fold uses VEXTRACTF128, VADDPS, VHADDPS, VPEXTRD. Zero function-call overhead, no heap allocation, data stays in registers.

Benchmark: scalar loop ~3.9 µs, SIMD ~0.41 µs — nearly 9× faster. Caveats: higher maintenance cost, CPU feature checks required (AVX2+FMA), benefits only for long contiguous slices; branching, sparse access, or tiny inputs may see no gain or regression. Always benchmark.

Summary

SIMD = one instruction, multiple lanes in wide vector registers (128/256/512-bit).

Pattern: load → vertical ops → store/extract. Extended with broadcast, FMA, masks, blend, permute, horizontal adds, partial loads. simd/archsimd = direct hardware mapping, fixed sizes, intrinsics, manual CPU checks. simd = portable, dynamic sizing, multi-version compilation + runtime dispatch.

Dot product demo shows 9× speedup via FMA, horizontal folding, and zero-overhead assembly.

Experimental APIs may change; underlying concepts are stable. Measure before optimizing.