Unlock CPU Performance: How MegPeak Measures Instruction Throughput and Latency

Background and Motivation

With the rapid growth of AI compute demand, extracting maximum performance from existing hardware requires algorithmic and implementation optimizations tailored to specific processors.

Optimization Directions

Reduce memory traffic and compute workload while preserving accuracy.

Implement algorithms to fully exploit processor capabilities.

MegPeak Overview

MegPeak is an open‑source processor debugging tool (GitHub: https://github.com/MegEngine/MegPeak) that helps developers evaluate performance and obtain optimization guidance.

Key Features

Peak instruction bandwidth

Instruction latency

Memory peak bandwidth

Combined peak bandwidth for arbitrary instruction mixes

Why Use MegPeak

Chip datasheets often lack detailed performance numbers for specific instruction combinations. MegPeak provides direct, accurate measurements that are difficult to obtain otherwise.

Usage

Build and usage instructions are in the repository README ( https://github.com/MegEngine/MegPeak#build). An example measures the peak bandwidth and latency of the ARMv8 fmla instruction.

Example Output

Measurements on an ARMv8 processor show the computed peak bandwidth and latency for the fmla instruction.

Interpretation

The measured values can be used to plot Roofline models, assess the optimization potential of a program, and explore theoretical peaks of instruction combinations. By comparing actual performance to the Roofline, developers can identify whether memory or compute is the bottleneck.

Underlying Principles

MegPeak measures instruction throughput by eliminating data, structural, and control hazards. Inline assembly is used to control hazards and prevent compiler optimizations that could skew results.

Measuring Instruction Throughput

To obtain the peak compute rate of an instruction, the tool repeats the instruction in a loop without any data dependencies, using multiple registers to avoid RAW/WAW/WRA hazards. Twenty registers are chosen because they are sufficient to break dependencies while staying within the 32‑register limit of ARM64.

static int fmla_throughput() {
    asm volatile(
        "eor v0.16b, v0.16b, v0.16b
"
        "eor v1.16b, v1.16b, v1.16b
"
        "..."
        "eor v19.16b, v19.16b, v19.16b
"
        "mov x0, #0
"
        "1:
"
        "fmla v0.4s, v0.4s, v0.4s
"
        "fmla v1.4s, v1.4s, v1.4s
"
        "..."
        "fmla v19.4s, v19.4s, v19.4s
"
        "add x0, x0, #1
"
        "cmp x0, %x[RUNS]
"
        "blt 1b
"
        : : [RUNS] "r"(megpeak::RUNS) : "cc", "v0", "v1", "v2", "v3", "v4", "v5", "v6", "v7", "v8", "v9", "v10", "v11", "v12", "v13", "v14", "v15", "v16", "v17", "v18", "v19", "x0");
    return megpeak::RUNS * 20;
}

Measuring Instruction Latency

Latency is measured by creating a strict RAW dependency chain, ensuring each iteration depends on the result of the previous one.

static int fmla_latency() {
    asm volatile(
        "eor v0.16b, v0.16b, v0.16b
"
        "mov x0, #0
"
        "1:
"
        "fmla v0.4s, v0.4s, v0.4s
"
        "// repeat 20 times
"
        "...
"
        "add x0, x0, #1
"
        "cmp x0, %x[RUNS]
"
        "blt 1b
"
        : : [RUNS] "r"(megpeak::RUNS) : "cc", "v0", "x0");
    return megpeak::RUNS * 20;
}

Applications

Draw Roofline models to guide performance tuning.

Evaluate the optimization space of existing code.

Explore theoretical peak performance of instruction mixes.

Future Directions

Support additional processor metrics such as L1/L2 cache sizes and automatic exploration of dual‑issue instruction combinations.

Extend mobile OpenCL support with details like warp size and local memory size.