What’s New in Apache IoTDB? Exploring the Latest Features for Industrial IoT

Apache IoTDB Overview

Apache IoTDB is an open‑source time‑series database targeting industrial IoT. It stores timestamped measurements, supports high‑write workloads, complex metadata, and edge‑cloud collaboration.

Release History and Key Improvements

First open‑source release 2017; Apache incubation 2018; first Apache release 2019.

2019 – out‑of‑order data handling.

2020 – major query‑performance boost and memory control.

2021 – 0.12 series (0.12.0‑0.12.4) with incremental patches.

Subsequent releases (0.13…) continue to add UDF, trigger, distributed time‑partitioning.

Core Architectural Concepts

Entity : physical device (e.g., wind turbine, vehicle).

Storage Group : logical isolation of a set of entities on disk.

Measurement : a physical quantity; can be single‑variate or multi‑variate.

Time Series : combination of entity + measurement.

Template : reusable set of measurements for many similar entities, reducing metadata overhead.

Hierarchical Storage Groups replace flat tables, allowing heterogeneous sampling rates and avoiding frequent schema changes.

Encoding and Compression Options

IoTDB provides several encodings that can be selected per series: PLAIN – for high‑variance data. RLE – for repeated values. TS_2DIFF – for stable numeric series. GORILLA – for floating‑point series with small variations. DICTIONARY – for low‑cardinality TEXT.

Choosing the appropriate encoding reduces storage size, especially for multi‑variate series where timestamps are shared.

Edge‑Defined Schema (Feature 1)

Data models can be defined on edge devices, enabling dynamic addition or removal of measurement columns without central schema changes. This matches industrial scenarios where equipment upgrades frequently alter the set of sensed points.

Hierarchical Load Management (Feature 2)

Instead of a single wide table, IoTDB uses a tree‑structured storage group. Each leaf node stores series for a specific measurement, allowing independent sampling rates and simplifying write paths.

In‑Database Processing (Feature 3)

IoTDB supports three processing stages:

On‑ingest processing – sliding‑window or point‑wise calculations applied as data arrives.

Pre‑analysis indexing – materialized indexes created after ingestion to accelerate later queries.

Query‑time computation – user‑defined functions (UDF) executed during query execution.

UDF types:

UDTF (User‑Defined Table Function): input multiple series, output a single series.

UDAF (User‑Defined Aggregate Function): input multiple series, output a single aggregated point.

Quality Assurance and Trigger Mechanisms

The IoTDB‑Quality library demonstrates integration of data‑quality algorithms via UDFs (e.g., custom down‑sampling, anomaly detection). Real‑time alarm filtering can be built as a pipeline: jump‑change removal → threshold filtering → true‑alarm matching → anomaly scoring → manual confirmation.

Integration with the Apache Ecosystem

IoTDB can be combined with ingestion tools (EDGENT, PLC4X), big‑data processing frameworks (Spark, Hive), and visualization platforms (Grafana, Calcite, Karaf) to realize a full‑lifecycle solution for time‑series data.

Future Directions

Ongoing work includes expanding UDF and trigger capabilities, improving time partitioning, adding virtual storage groups for distributed deployments, and enhancing performance for higher data utilization.