How to Scale High‑Concurrency Write Operations with Asynchronous Patterns

1. Business Scenario

In a teaching platform, teachers log in, view a list of courses, and watch course videos. The front end fetches video metadata from a Redis cache, while the back end records each watch event in a MySQL table. As concurrent users increase, write requests grow exponentially, causing thread blockage and slow responses.

2. Thread‑Pool Mode

Inspired by a previous experience with a high‑traffic red‑packet system, the write operation was off‑loaded to an independent thread pool. The controller returns immediately, while the pool asynchronously executes the activation logic, stabilizing the service without timeouts.

Key considerations for this mode:

Do not set the thread count too high to avoid exhausting database connections.

Size the thread‑pool queue appropriately to prevent memory overflow.

3. Local Memory + Scheduled Task

The open‑source community’s view‑count solution stores incoming events in an in‑memory LinkedBlockingQueue. A background thread wakes every minute, batches the queued items into a List, and executes Jdbc.batchUpdate to write them to the database.

This approach requires no changes to the existing business logic, offers high throughput, and keeps latency low, but it introduces the risk of memory overflow if the queue grows unchecked.

4. MQ Mode

Message queues provide asynchronous processing and decoupling. The workflow is:

Controller receives the write request and converts the watch record into a message.

The teaching service publishes the message to the MQ.

A consumer service pulls messages from the MQ and performs batch database writes.

Advantages include high availability, efficient asynchronous delivery, and built‑in persistence. The downside is the added operational complexity of managing an extra component.

5. Agent Service + MQ Mode

An independent Agent process runs on the teaching server. After the service receives a write request, it writes the request payload (e.g., JSON) to a local file and immediately returns success to the front end.

The Agent watches the file, streams new entries to the MQ, and a consumer later persists them to MySQL. This architecture keeps the business service free of MQ dependencies while still gaining asynchronous durability.

6. Summary

Asynchronous processing is valuable when write‑heavy operations consume excessive resources and can tolerate slight delays. Four practical patterns—thread‑pool, in‑memory queue with scheduled batch, pure MQ, and Agent‑plus‑MQ—share the core idea of queuing write commands, responding instantly to the client, and letting a background worker handle persistence. Choose the pattern that balances performance, complexity, and reliability for your specific load.