How to Solve Matrix and Z‑Buffer Precision Loss in WebGL Maps

Matrix Precision Loss Problem and Solutions

Most GPUs only support 32‑bit floating‑point numbers, so using 32‑bit floats in GLSL can lead to precision loss when handling large values.

In map scenes, road and terrain coordinates are small (0‑256) and do not suffer from precision loss, but elements such as text and labels use pixel coordinates relative to the whole map; as the map zooms, these values become large and cause jitter due to precision loss in the shader.

Solution 1: Pre‑compute Screen‑Centered Coordinates in JavaScript

This approach avoids precision loss but adds CPU overhead because coordinates must be recomputed each frame for all elements, which hurts performance as the number of labels grows.

Solution 2: Sub‑Coordinate Systems

Divide the large world coordinate system into smaller sub‑systems; when the zoom level exceeds a threshold, use the appropriate sub‑system so that coordinate values stay small, reducing matrix translation magnitude and preventing precision loss. This avoids per‑frame CPU calculations but requires managing multiple sub‑systems and may need further subdivision at higher zoom levels.

Solution 3: Relative‑to‑Eye (RTE) Coordinates in the Shader

Convert element coordinates to a view‑relative system and perform the conversion on the GPU, using two 32‑bit floats to represent a 64‑bit value.

function doubleToTwoFloats(value) {
    var high;
    var low;
    if (value >= 0) {
        tempHigh = Math.floor(value / 65536) * 65536;
        high = tempHigh;
        low = value - tempHigh;
    } else {
        tempHigh = Math.floor(-value / 65536) * 65536;
        high = -tempHigh;
        low = value + tempHigh;
    }
    return [high, low];
}

The shader stores the high and low parts of the eye position in two uniforms, and each vertex provides high and low parts for its coordinates, keeping the original values unchanged.

Z‑Buffer Precision Loss Problem and Solutions

Z‑fighting occurs when two surfaces compete for the same depth value. In map rendering, the depth buffer is typically 24‑bit, concentrating precision near the near plane, which can cause fighting for distant geometry.

To mitigate this, elements are grouped by type and depth ranges are assigned using gl.depthRange (e.g., background 0‑0.2, roads 0.2‑0.4). This creates separate depth intervals for each group.

If multiple layers need to be drawn together, an additional per‑vertex layer index can be used to offset the clip‑space Z value:

gl_Position = u_proj_matrix * u_mv_matrix * vec4(a_pos, 0, 1);
gl_Position.z -= a_layer_index;

Higher layer indices render on top, and the Z offset can be tuned to avoid precision loss.

JavaScript Precision Loss Problem and Solutions

When performing coordinate transformations in JavaScript, many matrix libraries use Float32Array, which also suffers from precision loss. Switching to Float64Array for matrix calculations resolves this issue.

Summary

For a small number of elements, converting to screen‑relative coordinates works but adds CPU cost.

Moving calculations to the GPU by representing a 64‑bit value with two 32‑bit floats mitigates precision loss.

Use gl.depthRange to limit Z‑buffer ranges for different element groups.

Adjust the clip‑space Z value in the shader to control layering and avoid Z‑fighting.

In JavaScript, employ Float64Array for matrix operations to preserve precision.