How to Call Third‑Party APIs Asynchronously Without Losing DB Writes

Background

In many real‑world projects, business scenarios require asynchronous processing, especially when a database operation must be followed by a call to a third‑party service.

Question

A colleague asked how to implement a solution where, after inserting, updating, or deleting data in the database, a third‑party API is called, but a failure of that call should not affect the database operation. They also wondered which async mechanism to use (Redis, NSQ, message queue, etc.), how to guarantee ordering when re‑trying, and how to handle data if the async call fails.

Answer

The key is to keep the database transaction separate from the third‑party call. Perform the CRUD operation inside a transaction, commit it, and then invoke the external service asynchronously if a delay is acceptable.

Common Asynchronous Implementations

Start a new 线程.

Use a thread pool ( 线程池).

Use a job scheduler.

Use a message queue (MQ).

Use Redis’s queue feature.

Listen to MySQL binlog with Canal.

When the system restarts, a plain 线程 or 线程池 may lose data, so the data should be persisted first.

Job‑based Asynchronous Solution

Add a 任务表 whose insert is part of the same transaction as the business data. Then use elastic-job with sharding to execute the job every 1, 3, 5, or 7 seconds. If the remote call fails, retry immediately up to three times.

Message‑Queue Solution

After the database operation, publish a message to an MQ server. A consumer reads the message and calls the third‑party API with automatic retries. This approach may struggle with message ordering and duplicate consumption.

Redis Queue Solution

After the DB operation, push the API request parameters into a Redis queue. An asynchronous worker reads from the queue and retries on failure. The main challenge is ensuring consistency between the database and Redis data.

Canal Binlog Listener Solution

Use Canal to monitor MySQL binlog changes. When new data appears, invoke the third‑party API with up to three automatic retries. If all retries fail, additional handling logic is required.

Recommendation

Among the options, the job‑based approach generally introduces fewer problems and is recommended.