Understanding OpenStack Ceilometer: Architecture, Components, and Data Collection Methods

OpenStack is a widely used open‑source cloud platform whose common services include Nova, Swift, Glance, Keystone, Neutron, Cinder, Horizon, Ceilometer, Heat, Trove, TripleO, and Sahara.

Nova: Provides the Compute service, managing the full lifecycle of virtual machine instances.

Glance: Offers an Image Service for discovering, registering, and retrieving VM images (it does not store the image files itself).

Swift: Supplies Object Storage for unstructured data; image files are often stored here.

Ceilometer is the telemetry service that collects almost all internal OpenStack events for billing, monitoring, and other services.

Its main responsibilities are:

Metering – gathering and storing metrics related to billing.

Monitoring – collecting and storing non‑billing metrics and status information.

Alarming – generating performance‑related alerts and fault notifications via plugins.

Ceilometer’s architecture centers on a database and message queue, with the following components:

ceilometer-api: Exposes aggregated data to users.

ceilometer-polling: Uses polling plug‑ins to fetch data.

ceilometer-agent-central: Calls various component APIs to monitor resource existence (deprecated after the Liberty release).

ceilometer-agent-compute: Monitors hypervisors or libvirt to obtain VM performance data and publishes it via the message queue (deprecated after Liberty).

ceilometer-agent-ipmi: Retrieves physical host information from IPMI sensors (deprecated after Liberty).

ceilometer-agent-notification: Receives messages from other components via the message queue.

ceilometer-collector: Consumes messages from the queue, distributes data to appropriate databases.

ceilometer-alarm‑X: Handles alarm‑related functionality.

Except for the compute‑ and IPMI‑agents, all components are deployed on one or more controller nodes and heavily rely on a message‑queue service for inter‑component communication.

Ceilometer collects data via two methods:

Push: Components emit event messages (e.g., VM creation, image upload) to the message queue, which are then processed by the ceilometer-collector ’s notification‑agent, transformed by pipelines, and stored in MongoDB.

Poll: Pollsters run on controller or compute nodes, periodically invoking component APIs or libvirt to gather statistics and real‑time data, which follow the same pipeline and storage path as push data.

The processing pipeline assembles zero or more Transformers into a chain that modifies the samples before a Publisher sends them back to the message queue for the collector.

The collector continuously listens to the queue, persisting the final metrics in MongoDB .

OpenStack has become the de‑facto standard for private cloud deployments across many industries, and the author has compiled the knowledge into the ebook OpenStack Technology and Practice for further study.