Why Pika Is the Cost‑Effective, Large‑Scale Alternative to Redis

Pika Overview

Pika is a high‑capacity, Redis‑compatible storage system jointly developed by 360’s Web Platform DBA and infrastructure teams. It is not intended to replace Redis but to complement it in scenarios where Redis’s memory‑centric design becomes costly or limited.

Why Pika Was Created

Redis delivers excellent performance because it stores data entirely in memory, but memory prices have risen sharply, effectively doubling the cost of a stable Redis deployment without any change in data volume. This price surge forces developers to compress data, limit memory usage, or endure expensive hardware procurement.

The rising memory cost creates a heavy burden for both procurement and development teams, prompting the need for a cheaper, high‑capacity alternative.

Key Benefits of Pika

Full Redis protocol compatibility – existing Redis drivers work without code changes.

Persistent disk storage – data is written to SSD/HDD, eliminating the need for costly memory.

Built‑in compression (default Snappy) reduces data size to 1/3–1/8 of the original.

Multi‑threaded architecture – configurable worker threads mitigate the single‑thread bottleneck of Redis.

Simple operations – slaveof, cascade clustering, and other Redis‑like commands work out of the box.

Fast backup and recovery – non‑blocking snapshot backups complete in milliseconds for large datasets.

Rich monitoring – QPS, connections, memory/disk usage, binlog, and monitor commands are available.

Cost Comparison

SSD prices are far lower than memory (≈2.5 CNY/GB vs. ≈60 CNY/GB). Combined with Pika’s compression (at least 3×), the overall storage cost can be as low as 1/72 of an equivalent Redis deployment.

Architecture

Pika consists of three core components:

Network module pink – a framework supporting Redis, HTTP, and protobuf protocols with both single‑thread and multi‑thread worker models.

Storage engine nemo – built on RocksDB, it implements Redis data structures (hash, list, set, zset) by mapping them to key‑value pairs.

Binlog module – provides binary log based replication and disaster‑recovery capabilities.

Data Structure Implementation

For a hash, nemo concatenates the key and field to form a new key stored in RocksDB, and also stores a meta‑key that records the number of field‑value pairs. Similar techniques are used for lists, sets, and sorted sets.

Deployment Guide

After compiling and installing, start a single node with ./pika -c pika.conf . For production, create a master‑slave cluster by adding a slave node and executing slaveof <master‑ip> <master‑port> . The slave clones data and continues incremental sync automatically.

Operational Practices

Deploy on SSDs for optimal performance.

Configure a large open_files_limit for RocksDB when storing massive data.

Monitor QPS, connections, memory/disk usage, and binlog status via the info command.

Use bgsave for non‑blocking snapshot backups; restore by renaming the snapshot directory to db and restarting.

Performance Insights

Pika can sustain around 80 k QPS in real deployments. While its KV performance is strong, operations on extremely large multi‑data‑structure keys (e.g., a hash with billions of elements) can be up to ten times slower than Redis.

Future Plans

The next step is a “pika‑cluster” that overcomes the local‑disk capacity limit (currently around several hundred GB per node) by integrating with Codis and enabling distributed storage beyond a single SSD.

Frequently Asked Questions

Typical use cases : high‑cost memory scenarios, massive data volumes that exceed a single Redis server’s capacity, and workloads with occasional high‑complexity commands that would block Redis.

Cost‑control measures : application forms require users to justify large Redis allocations, effectively reducing over‑provisioning.

Migration tools : redis_to_pika (full + incremental AOF migration), ssdb_to_pika , and pika_to_redis (for moving data back to Redis).

Disaster recovery : binlog‑based incremental sync allows automatic recovery of slaves without full data reload; for full restores, use slaveof with the force flag.

Community adoption : over 600 instances serve more than 10 companies (e.g., Sina Weibo, Meituan, Xiaomi), handling 5 TB of data and 500 billion requests per day.