Mastering Memcached: Features, Use Cases, and Prometheus Monitoring

Overview

Memcached is a free, open‑source, high‑performance distributed memory object cache. It stores arbitrary data as key‑value pairs in RAM, providing sub‑millisecond response times and reducing load on persistent databases.

Key Characteristics

In‑memory storage : All data resides in RAM, enabling billions of operations per second.

Distributed design : Multi‑threaded architecture allows horizontal scaling by adding nodes; each node can use multiple CPU cores.

Key‑value store : Supports any data type via simple commands (set, get, delete, inc, dec, cas) for fast lookups.

Client libraries : Available for Java, C/C++, Go, Python, Ruby and many other languages.

Typical Use Cases

Cache static assets : HTML, CSS, images, session data, etc., to accelerate page loads.

Database front‑end : Acts as an in‑memory layer between applications and persistent databases, reducing read/write traffic.

High‑frequency read/write workloads : Stores hot data (e.g., recent comments) to avoid repeated disk access.

Unsuitable Scenarios

Objects larger than 1 MiB – Memcached is not designed for large media blobs.

Workloads requiring data traversal – only a limited command set is supported.

Strong durability or failover requirements – data is volatile and not persisted.

Core Concepts

Memory Management (Slab Allocator)

Memcached divides its memory pool into slabs . Each slab class manages chunks of a fixed size; a slab consists of one or more pages (default 1 MiB, configurable with -I). When an item is stored, Memcached selects the smallest slab class that can accommodate the item size. The next slab class’s maximum size is 1.25× the previous one by default, but this growth factor can be changed with the -f flag. This design eliminates external fragmentation and keeps allocation/deallocation O(1).

LRU Eviction

Memcached uses a modified Least‑Recently‑Used (LRU) algorithm with four queues:

HOT : Frequently accessed items stay here; active items are never evicted directly from HOT.

WARM : Buffer for items that have been accessed a few times.

COLD : Holds the least‑recently used items; eviction occurs from the tail of this queue when memory is full.

TEMP : Holds short‑lived items with very low TTL; they are not part of the normal LRU flow.

Items move between queues based on access patterns, allowing efficient reclamation of cold data while preserving hot data.

Version Highlights

1.6.0 – Added external flash storage support, more protocols, and network optimizations.

1.5.0 – Improved LRU implementation and storage efficiency.

1.4.0 – Introduced binary protocol and client libraries for C, Java, and others.

Recommendation: use the latest stable release.

Prometheus Monitoring

Memcached can be monitored with the memcached_exporter, which exposes a wide range of metrics. Below are the most important metric groups and example metric names.

System Metrics

memcached_process_system_cpu_seconds_total

– CPU time spent in kernel mode. memcached_process_user_cpu_seconds_total – CPU time spent in user mode. memcached_limit_bytes – Configured memory limit. memcached_up – 1 if the instance is reachable, 0 otherwise. memcached_uptime_seconds – Seconds since the process started. memcached_version – Version string.

Read/Write Metrics

memcached_read_bytes_total

– Total bytes read from the network. memcached_written_bytes_total – Total bytes written to the network. memcached_commands_total – Total number of commands processed. memcached_slab_commands_total – Commands broken down by slab class.

Storage Metrics

memcached_items_total

– Total number of items stored. memcached_items_evicted_total – Items evicted due to memory pressure. memcached_items_reclaimed_total – Expired items reclaimed. memcached_current_items – Current live item count. memcached_malloced_bytes – Total bytes allocated for slabs. memcached_slab_chunk_size_bytes – Size of a chunk in bytes. memcached_slab_chunks_free / memcached_slab_chunks_used – Free vs used chunks per slab class.

LRU Metrics

memcached_lru_crawler_enabled

– Whether the LRU crawler is active. memcached_lru_crawler_hot_percent, memcached_lru_crawler_warm_percent – Percentage of slab memory allocated to HOT/WARM. memcached_lru_crawler_moves_to_cold_total, memcached_lru_crawler_moves_to_warm_total – Number of items moved between LRU queues.

Connection Metrics

memcached_connections_total

– Total connections accepted. memcached_current_connections – Currently open connections. memcached_connections_rejected_total – Connections rejected due to the -c limit. memcached_connections_yielded_total – Connections throttled by the -R limit.

Typical Alerting Thresholds

While alert rules depend on specific service‑level objectives, common thresholds include:

Instance health : memcached_up == 0 for 5 minutes.

Memory usage : Warn if memcached_limit_bytes * 0.80 < memcached_malloced_bytes; critical if > 90 %.

OOM events : Alert on any increase of memcached_slab_items_outofmemory_total.

Connection limits : Alert when memcached_current_connections approaches the configured max or when memcached_connections_rejected_total > 0.

References

Memcached official site: https://memcached.org/about

Modern LRU design article: https://memcached.org/blog/modern-lru/

AWS Memcached guide: https://www.scaler.com/topics/aws-memcached/