Why OpenTSDB Is the Ultimate Time‑Series Monitoring Solution for Scalable Operations

Background

The Chinese space agency used a next‑generation numerical weather forecasting system to ensure the successful docking of Tiangong‑1 and Shenzhou‑9, highlighting the importance of precise, high‑resolution monitoring for critical missions. Similarly, operations teams need comparable monitoring systems to quickly detect instability and meet SLA requirements.

Problems with Traditional Monitoring

Legacy monitoring platforms often suffer from:

Centralized data storage leading to single points of failure.

Limited storage capacity.

Stale data due to time delays.

Difficulty customizing visualizations.

Inability to scale to billions of data points.

Metrics cannot scale to the K‑level.

No support for sub‑second data.

OpenTSDB Overview

OpenTSDB is an open‑source monitoring system that addresses the above issues by using HBase to store all time‑series data without sampling, creating a distributed, scalable time‑series database.

It stores metrics at second‑level granularity, supports permanent storage, and integrates easily with existing alerting systems.

It can ingest metrics from large clusters—including network devices, OS, and applications—and present them in web‑based, graphical dashboards.

Architecture Overview

OpenTSDB uses HBase as its storage backend, allowing collection of billions of data points with second‑level monitoring. The TSD daemon communicates with HBase without a master/slave distinction, eliminating single points of failure. Access is provided via Telnet, HTTP, or RPC, and each server that needs metrics runs a Collector script.

Storing Time Series in HBase

OpenTSDB leverages async HBase, a fully asynchronous, non‑blocking, thread‑safe API that reduces thread, lock, and memory usage while delivering high throughput for massive write workloads. Table designs such as tsdb-uid and tsdb are critical for performance.

Example Use Case

A company experiencing rapid traffic growth and increasing complexity used OpenTSDB to monitor engine performance, including full‑ and incremental growth trends, latency, and log metrics. Visualizations of these metrics helped identify bottlenecks and guide capacity planning.

Data Point Format

Each data point in OpenTSDB consists of:

A metric name.

A UNIX timestamp.

A value (64‑bit integer or double).

A set of tags (key‑value pairs) identifying the source.

Example data points (derived from the company’s dashboard):

index.full_count 1341069600 156866750 domain=domain_E area=1 app=jqb cluster=epid partition=partition_16384_32767
index.full_count 1341069600 155819640 domain=domain_E area=1 app=jqb cluster=epid partition=partition_32768_49151
index.full_size 1341069000 18561 domain=domain_D area=1 app=jqb cluster=b2c partition=partition_0_16383
index.full_size 1341069000 18554 domain=domain_D area=1 app=jqb cluster=b2c partition=partition_16384_32767
index.full_count 1341069200 11421051 domain=domain_G area=1 app=jqb cluster=b2c partition=partition_16384_32767

tcollector Integration

Building on the open‑source tcollector, the company added custom scripts to collect all required metrics, manage connections to TSD, schedule periodic script execution, deduplicate data, and support multiple data exchange protocols, providing extensible data collection.

Deploying the customized etao‑tcollector across all machines enables remote start/stop control, sending timestamped metric data to TSD, which can then be visualized via OpenTSDB’s web UI for real‑time monitoring.

Conclusion

OpenTSDB, backed by HBase, offers high scalability, flexible metric addition, lossless storage, and powerful analysis and visualization capabilities, making it an excellent choice for operations teams seeking robust, customizable monitoring solutions.