How to Pick the Best Storage Engine for High‑Throughput Browsing Records: Redis, MySQL or Tair?

Background

The e‑commerce system records user browsing behavior, which can be 10‑100 times the order volume. Users may browse many stores, products, or videos before placing an order, resulting in a write traffic of over 100K TPS during peak hours, while read traffic stays between 1K‑10K QPS. The new requirement adds support for multiple content types but still limits visible records to the last 30 days.

Storage Model Selection

The author evaluates four middleware candidates—pure Redis, Redis + MySQL, pure MySQL, and Tair—against criteria of data reliability, cost, read/write performance, and implementation difficulty.

Data size estimate (20 bytes per record, 10 records per user per day, 100 M active users, 30 days): 20 * 10 * 30 * 100000000/(1024**3) = 558 G Thus, 30‑day data exceeds 500 GB.

Comparison summary (derived from the original table):

Redis : high cost, unreliable data, extremely high read/write performance, low implementation difficulty.

Redis + MySQL : moderate cost, reliable, extremely high read/write performance, high implementation difficulty.

MySQL : moderate cost, reliable, poor read/write performance, high implementation difficulty.

Tair : low cost, reliable, high read/write performance, low implementation difficulty.

Pure Redis is discarded because memory cost for >500 GB is prohibitive and it cannot guarantee data durability. Pure MySQL is discarded due to inability to sustain >10K QPS reads/writes and the heavy archiving burden.

Redis + MySQL is also rejected: writes must hit both cache and DB, stressing the DB with >100K TPS, and daily archiving adds extra load.

Consequently, Tair emerges as the most balanced choice.

Storage Structure Design

The author first explains a Redis‑centric design and then contrasts it with Tair.

List Structure

Using a Redis List, new records are LPUSH ed to the head, guaranteeing the newest record is first. Pagination is done with LRANGE using an offset and count.

LPUSH KEY_NAME VALUE1
# Add a record: LPUSH ${userId} {${productId}, ${viewTime}}

# Paginated query: LRANGE ${userId} ${offset} ${offset}+${count}

Tair’s List behaves similarly but only supports tail insertion, making it impossible to paginate from the newest side.

Sorted‑Set Structure

Sorted Sets store a score (timestamp) and a value (product ID). Pagination uses ZREVRANGE to retrieve records in reverse chronological order.

ZADD KEY_NAME SCORE1 VALUE1
# Add a browsing record: ZADD ${userId} ${viewTime} ${productId}

ZREVRANGE key start stop WITHSCORES  # Paginated retrieval

Redis cannot expire individual Sorted‑Set elements, so a background archiving job is required for data older than 30 days. Tair also lacks this structure, so the Sorted‑Set approach is abandoned.

Hash Structure

Hashes are more complex but can store per‑user, per‑day data. Three variants are discussed:

Scheme 1: ${userId} as key, product ID as field, timestamp as value.

Scheme 2: ${userId} as key, day as field, JSON value containing product IDs and timestamps.

Scheme 3: ${userId} as key, day as field, JSON value (same as Scheme 2) but with a separate key for daily totals ( ${userId}_dayinfo).

Scheme 1 requires pulling all 30 days of data for pagination, leading to high memory and GC pressure. Schemes 2 and 3 store only daily counts, enabling pagination by first fetching the per‑day counts, then retrieving the needed days’ records.

Performance comparison:

Scheme 2: concurrent writes cause whole JSON updates, higher network cost.

Scheme 3: concurrent writes only update the day’s field, reducing write overhead.

Both schemes need to read all 30 days of count fields, but the data volume is tiny (≈150 bytes total), so the read cost is negligible.

Edge case: a single day with an extremely high number of records could still cause a slow query; the author suggests capping daily records (e.g., 30) to keep queries fast.

Concurrency Control

Three options are examined:

Distributed lock per userId : feasible for 100K TPS.

No control : tolerate occasional data loss, assuming duplicate browsing events are rare.

Kafka sharding by userId : route all events of a user to a single partition to guarantee ordered processing.

The author concludes that, given the non‑critical nature of browsing logs, the risk of occasional loss is acceptable, and no strict concurrency control is needed.

Summary

The scenario presents a classic high‑write, low‑read workload with strict 30‑day retention. After weighing cost, reliability, performance, and implementation difficulty, Tair is selected as the storage engine. The final design adopts a Hash‑based structure (Scheme 3) with per‑day fields, leverages Tair’s persistence and high read/write throughput, and relies on simple concurrency handling (either a lightweight lock or Kafka‑based ordering) because absolute consistency is not required.