Databases 11 min read

Why InfluxDB’s max‑value‑per‑tag Error Occurs and How to Resolve It

This article explains the cause of InfluxDB’s max‑value‑per‑tag error when Prometheus remote‑writes high‑cardinality tags, analyzes why the built‑in memory index triggers OOM protection, and presents three practical solutions—including moving indexes to disk, storing high‑cardinality tags as fields, and filtering them before write—to ensure stable monitoring data persistence.

dbaplus Community
dbaplus Community
dbaplus Community
Why InfluxDB’s max‑value‑per‑tag Error Occurs and How to Resolve It

Problem Scenario

InfluxDB is widely used as a time‑series database for persisting monitoring data. During write operations, the max‑value‑per‑tag error can appear, which is a protective measure to prevent out‑of‑memory (OOM) crashes when the number of distinct tag values grows too large.

The article examines this error from the perspective of Prometheus remote‑write to InfluxDB, illustrating the error with screenshots and describing the impact on continuous monitoring.

Root Cause Analysis

When Prometheus writes metrics to InfluxDB, tags with high cardinality—such as a globally unique trace ID or a transaction timestamp—are included. These tags cause the following chain of events:

Exporter exposes metrics that contain high‑cardinality tag elements.

Prometheus continuously streams these metrics to InfluxDB, using Snappy compression and protobuf for transport.

InfluxDB parses the incoming stream, treating the measurement name as the table name, tags as index fields, and field values as regular columns.

All tag values are stored in memory by default to speed up index lookups. To avoid excessive memory consumption, InfluxDB enforces the max‑values‑per‑tag limit (default 100 000). When a tag exceeds this threshold, write operations fail with the protective error. Consequently, after a period of continuous writes, the number of distinct high‑cardinality tag values reaches the limit, causing subsequent writes to be rejected.

Solutions

Solution 1 – Move Indexes to Disk

InfluxDB allows configuring the index storage engine to persist indexes on disk instead of keeping them entirely in memory. By setting index-version=tsi1 in the configuration file, the database creates an index directory under /var/lib/influxdb/data . This eliminates the OOM protection trigger but increases disk usage. Testing under a simulated 100 TPS workload showed that after 12 hours the system remained stable, while disk consumption grew by roughly 300 MB per hour (index ≈ 50 MB, TSM ≈ 120 MB, _series ≈ 130 MB).

Solution 2 – Store High‑Cardinality Tags as Fields

Since high‑cardinality tags are mainly used for alert context rather than analytical dimensions, they can be demoted to regular fields. This requires customizing InfluxDB’s write path: when a remote‑write request arrives, retain only the measurement name as the index and convert the high‑cardinality tags into fields. After applying this change and running the same 12‑hour test, the max‑value‑per‑tag warning disappeared and disk growth was limited to a few tens of megabytes. However, because multiple metrics share the same batch timestamp, InfluxDB treated them as updates to the same series, resulting in data loss for the monitoring use‑case.

Solution 3 – Filter High‑Cardinality Tags Before Write

The final approach is to prevent high‑cardinality tags from reaching InfluxDB at all. By editing Prometheus’s remote_write configuration, tags identified as high‑cardinality are stripped from the payload before it is sent. Running the same 12‑hour workload confirmed that InfluxDB operated normally, without the max‑value‑per‑tag warning, and resource consumption remained moderate.

Conclusion

The root cause of the max‑value‑per‑tag error is the presence of high‑cardinality tags in monitoring metrics, which InfluxDB is not designed to index efficiently. For stable operation, such tags should either be stored on disk (accepting higher storage costs), demoted to fields (risking update‑collision issues), or filtered out before ingestion. In practice, filtering high‑cardinality tags offers the most balanced solution for monitoring data persistence.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

monitoringInfluxDBTime SeriesDatabase Configurationhigh-cardinality tagsmax-value-per-tag
dbaplus Community
Written by

dbaplus Community

Enterprise-level professional community for Database, BigData, and AIOps. Daily original articles, weekly online tech talks, monthly offline salons, and quarterly XCOPS&DAMS conferences—delivered by industry experts.

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.