Transaction Management Patterns in Microservices: Blocking Retry, Async Queue, TCC, and Local Message Table

In modern distributed systems and microservice architectures, service-to-service call failures are common, making exception handling and data consistency critical.

Blocking Retry

Blocking retry is a typical approach where a request is retried a limited number of times before giving up. The following pseudocode demonstrates the idea:

m := db.Insert(sql)

err := request("B-Service", m)

func request(url string, body interface{}) {
    for i := 0; i < 3; i++ {
        result, err = request.POST(url, body)
        if err == nil {
            break
        } else {
            log.Print()
        }
    }
}

This method can cause duplicate data when the downstream service succeeds but the caller times out, generate dirty data if the request fails after a DB insert, and increase latency, putting extra pressure on downstream services.

Asynchronous Queue

Introducing a message queue decouples processing and improves reliability. After writing to the database, a message is published to the queue for an independent consumer to handle the business logic:

m := db.Insert(sql)

err := mq.Publish("B-Service-topic", m)

Although queues are more stable than direct service calls, publishing can still fail (e.g., network issues), leading to the same inconsistency problem where the DB write succeeds but the message is not delivered.

TCC Compensation Transaction

TCC (Try‑Confirm‑Cancel) splits each service call into three phases to achieve atomicity across services:

Try: check resources and reserve them (e.g., inventory lock).

Confirm: commit the reservation.

Cancel: release the reservation if Try fails.

Example pseudocode for a shopping scenario that calls inventory, amount, and points services:

m := db.Insert(sql)
aResult, aErr := A.Try(m)
bResult, bErr := B.Try(m)
cResult, cErr := C.Try(m)
if cErr != nil {
    A.Cancel()
    B.Cancel()
    C.Cancel()
} else {
    A.Confirm()
    B.Confirm()
    C.Confirm()
}

Key issues include empty releases (when a failed Try actually succeeded), ordering problems (Cancel arriving before Try), and failures during Confirm/Cancel, which may require additional retry or manual intervention.

Local Message Table

The local message table, originally proposed by eBay, stores a message record in the same database transaction as the business data, guaranteeing that either both succeed or both fail. After the main transaction commits, the message is sent to a queue; if the send fails, the message remains with a try status and is retried asynchronously.

messageTx := tc.NewTransaction("order")
messageTxSql := tx.TryPlan("content")

m, err := db.InsertTx(sql, messageTxSql)
if err != nil { return err }

aErr := mq.Publish("B-Service-topic", m)
if aErr != nil { // push failed
    messageTx.Confirm() // set status to confirm
} else {
    messageTx.Cancel() // delete the message
}

Consumers listen for confirm or try messages and retry until successful, ensuring eventual consistency without blocking the original request.

Independent Message Service and MQ Transaction

Separating the message table into an independent message service provides the same reliability guarantees but cannot be wrapped in the same local transaction, so a “prepare” state may appear if the message is created but the subsequent business operation fails.

err := request.POST("Message-Service", body)
if err != nil { return err }

aErr := request.POST("B-Service", body)
if aErr != nil { return aErr }

Some MQ implementations (e.g., RocketMQ) support transactional messages that follow the same Try‑Confirm‑Cancel pattern, adding a prepare state that the consumer must confirm before committing.

Summary

Ensuring data consistency in distributed systems inevitably requires extra mechanisms. TCC offers a flexible, service‑level solution but demands three APIs per service and careful failure handling. Local message tables are simple and work well with service calls and MQs, while MQ‑based transactions and independent message services provide a decoupled approach at the cost of added latency and limited broker support.