How to Ensure Consistent State in Event‑Driven Microservices: 3 Proven Patterns

Scenario Example

In an e‑commerce system, placing an order requires updating the order table and the rewards table to record earned points.

In a monolithic architecture this is simple: start a transaction, update both tables, and commit; any error rolls back everything.

In a distributed microservice architecture, order history and rewards are handled by different services, making consistency more complex. This is known as a distributed transaction: multiple services must process a single event, and if any service fails, all others must roll back.

If an error occurs right after writing the order table—e.g., server crash, timeout, or the rewards service cannot process the event—the system becomes inconsistent: the order exists but the reward points are missing.

An event‑driven architecture can ensure the rewards service processes the event at least once after the event is placed on a queue. After inserting the order, the order service publishes an event that the rewards service eventually consumes to update the rewards table.

We have achieved eventual consistency, but we still have not solved the problem of a server crash before the event is written to the queue. The following patterns address this issue.

Note: When using a message broker, duplicate messages may appear. Design services to be idempotent regardless of the pattern chosen.

Pattern 1: Outbox Pattern

The outbox pattern stores events in an outbox table that resides in the same database as the order table. Both the order and the outbox event can be inserted within a single transaction, so if either operation fails, both are rolled back.

The rewards service can use Change Data Capture (CDC) to monitor changes in the outbox table and process them. For example,

Amazon DynamoDB

provides

DynamoDB Stream

to capture table changes.

Remember to clean up processed events from the outbox table to prevent it from growing indefinitely.

Pattern 2: Original Event Handling

Instead of a separate outbox table, this pattern uses CDC directly on the order history table. Whenever a new order is added, CDC captures the change and forwards it as an event to the rewards service.

Pattern 3: Self‑Read Pattern

Both previous examples rely on a single point to emit the event within one transaction. In this pattern, the order‑history service publishes the event, and both the order‑history service and the rewards service consume it.

If step (2) successfully publishes the message, both the order and rewards services will eventually process the event, keeping the system state consistent.