Why Our E‑commerce Home Page Slowed to 20 seconds and How We Fixed It

Preface

Our online store suddenly became extremely slow: the home page took about 20 seconds to return data, causing user complaints.

1. Incident Scene

During a regular release, a new feature that recommended products by region and category dramatically increased the amount of data stored in Redis, exhausting the 1 GB memory allocated on Alibaba Cloud.

When Redis failed, the application fell back to the database, which slowed the response time even more.

2. Quick Fix

The immediate solution was to increase Redis memory from 1 GB to 4 GB, which restored the home page speed.

However, this only addressed the symptom, not the root cause.

3. Postmortem

We identified two main problems:

The new recommendation feature stored excessive and unnecessary fields in Redis, leading to an unreasonable data structure.

The fallback logic directly queried the database when Redis was down, which could overload the database under higher traffic.

4. Optimization

We explored several options:

4.1 Page Staticization

Static pages would be ideal but required too many front‑end and back‑end changes for the current traffic level.

4.2 Local Cache

Adding a local cache could protect the home page from Redis failures, but the application’s memory might not be sufficient to hold all recommendation data.

4.3 Switch to MongoDB

MongoDB stores data on disk with memory‑mapped access, making it more suitable for large structured datasets than Redis, though it might be slower for hot reads.

4.4 Local Cache + MongoDB

Combine a local cache for hot data (updated every 5 minutes) with MongoDB as the primary store.

Requests first check the local cache; if missing, they fall back to MongoDB.

5. Fallback Strategies

To handle MongoDB failures, we considered:

5.1 Apollo Configuration

Return default recommendation data configured in Apollo.

5.2 Database Access

Query the relational database directly if MongoDB is unavailable.

5.3 Redis Access

Retrieve hot product data from Redis as a secondary source.

5.4 Additional Local Cache

Store default data in a local cache for 24 hours after the first database fetch.

6. Final Solution

After discussion, we adopted a layered approach: local cache + MongoDB + default local cache + database.

The chosen solution balances performance, data volume, and reliability, acknowledging that no single technology is perfect; the best fit depends on business needs, resources, and team capabilities.