Understanding SHOW ENGINE ROCKSDB STATUS: Statistics and Background Thread Implementation in MyRocks

In Facebook's MyRocks (a MySQL variant using RocksDB), the optional RocksDB storage engine offers significant disk‑space savings compared to InnoDB, making it attractive for large‑scale internet applications. To safely adopt a new engine, internal metrics—exposed via SHOW ENGINE ROCKSDB STATUS and INFORMATION_SCHEMA tables—are essential for performance tuning and debugging.

STATISTICS

Enabling STATISTICS adds a modest 5%‑10% overhead but records cumulative counts and timings for every RocksDB operation across all threads. The implementation relies on instrumentation points such as RecordTick and MeasureTime embedded in core functions like GetEntryFromCache .

Cache::Handle* GetEntryFromCache(Cache* block_cache, const Slice& key,
                                 Tickers block_cache_miss_ticker,
                                 Tickers block_cache_hit_ticker,
                                 Statistics* statistics) {
  auto cache_handle = block_cache->Lookup(key, statistics);
  if (cache_handle != nullptr) {
    PERF_COUNTER_ADD(block_cache_hit_count, 1);
    RecordTick(statistics, BLOCK_CACHE_HIT);
    RecordTick(statistics, BLOCK_CACHE_BYTES_READ,
               block_cache->GetUsage(cache_handle));
    RecordTick(statistics, block_cache_hit_ticker);
  } else {
    RecordTick(statistics, BLOCK_CACHE_MISS);
    RecordTick(statistics, block_cache_miss_ticker);
  }
  return cache_handle;
}

The public API resides in include/rocksdb/statistics.h and monitoring/statistics.h . Users create a Statistics object via rocksdb::CreateDBStatistics() and attach it to Options.statistics . Available ticker levels include kExceptDetailedTimers , kExceptTimeForMutex , and kAll . Statistics data can be retrieved through methods such as Statistics::getTickerCount , Statistics::histogramData , and Statistics::ToString .

STATISTICS Implementation

RocksDB implements StatisticsImpl , which stores per‑thread TickerInfo and HistogramInfo structures. Thread‑local counters are merged into global sums only when a thread exits, minimizing cache‑coherency traffic and keeping the overhead low.

MyRocks Usage

MyRocks initializes its own statistics object and assigns it to the DB options:

rocksdb_stats = rocksdb::CreateDBStatistics();
rocksdb_db_options->statistics = rocksdb_stats;

It also creates a custom histogram for commit latency:

commit_latency_stats = new rocksdb::HistogramImpl();
rocksdb::StopWatchNano timer(rocksdb::Env::Default(), true);
... 
commit_latency_stats->Add(timer.ElapsedNanos() / 1000);

When SHOW ENGINE ROCKSDB STATUS is invoked, MyRocks calls rocksdb_stats->ToString() to render ticker data and outputs histogram percentiles (50%, 95%, 99%, 100%) for commit latency.

Background Threads (BG_THREADS)

The command also reports BG_THREAD information, e.g., a compaction thread’s type, operation stage, elapsed time, bytes read/written, and job identifiers. Sample output shows a thread in CompactionJob::ProcessKeyValueCompaction with detailed metrics.

Thread‑Status Interface and Implementation

RocksDB provides thread‑status facilities via headers such as include/rocksdb/thread_status.h and monitoring/thread_status_updater.h . The key classes are ThreadStatusUpdater (holds per‑thread status) and ThreadStatusUtil (static helpers for registration and updates).

Threads register themselves with ThreadStatusUtil::RegisterThread and unregister with ThreadStatusUtil::UnregisterThread . The environment’s GetThreadList() returns a vector of all background thread statuses, which rocksdb_show_status iterates over to produce the BG_THREADS section.

Specific enums define properties for compaction ( CompactionPropertyType ) and flush ( FlushPropertyType ) operations, enabling fine‑grained monitoring of job IDs, input/output levels, byte counts, and other metrics.

Conclusion

The article covered how SHOW ENGINE ROCKSDB STATUS exposes STATISTICS and BG_THREAD data in MyRocks, the underlying RocksDB APIs and data structures, and how developers can extend these mechanisms for deeper performance insight.