Why ‘Insert‑Then‑Publish’ Isn’t Enough: Ensuring Reliable Message Delivery in Microservices

Problem with Simple Insert‑Then‑Publish

In a microservice architecture, a common pattern is to write data to the database first and then publish a message to a message bus. The article shows a typical flow where a user service registers a user, inserts the record, and then emits a registration event for an email service.

Although this approach looks straightforward, it hides several consistency problems.

Failure Scenarios

Scenario 1: Database insert succeeds, message publish fails

If the message bus does not receive the event, the database contains the user record while downstream services never see the event, leading to data inconsistency.

Scenario 2: Database insert fails

Even with ACID guarantees, two sub‑cases exist: the transaction rolls back (consistent) or the transaction commits but the response to the caller fails (the database has the record while the caller assumes failure).

Distributed Transaction Perspective

Both cases illustrate a classic distributed‑transaction problem: the database and the message bus are separate resources that must be updated atomically. Full strong consistency is often unnecessary; eventual consistency (weak consistency) is sufficient for many business scenarios.

Local Message Table Solution

The article proposes using a local message table that stores the outbound message inside the same database transaction as the business data. The pseudocode is:

var content = processRequest(httpRequest);</code>
<code>var message = prepareMessage(httpRequest);</code>
<code>DB.begin();</code>
<code>DB.insert(content);</code>
<code>DB.insert(message);</code>
<code>DB.commit();</code>
<code>Message.publish(message);</code>
<code>DB.delete(message);</code>
<code>// Compensation task (periodic)
Executor.execute(new Task() {
    public void run() {
        while (true) {
            var msg = DB.selectMessage();
            Message.publish(msg);
            DB.delete(msg);
        }
    }
});

This leverages the atomicity of the local DB transaction to guarantee that either both the business record and the outbound message are persisted or neither is. A background task later retries failed publishes.

Handling Duplicate Messages

If a message is published but the subsequent delete fails, the compensation task may resend it, causing duplicate delivery. The article recommends making downstream consumers idempotent. Modern message brokers (e.g., Kafka 0.11+) also support deduplication based on message IDs, reducing the impact of duplicates.

When to Apply This Pattern

For high‑traffic, large‑scale distributed systems, the local‑message‑table approach improves reliability and maintainability. Smaller services with low traffic may accept the risk of occasional inconsistency to avoid the added complexity.

Understanding the underlying distributed‑transaction issue allows engineers to choose the appropriate strategy rather than blindly following the "insert‑then‑publish" rule.