Transaction Consistency Strategies in Microservices: Blocking Retry, Async Queues, TCC, and Local Message Tables
The article examines common microservice transaction consistency techniques—including blocking retries, asynchronous queues, TCC compensation transactions, local message tables, and MQ transactions—explaining their mechanisms, advantages, drawbacks, and practical code examples for ensuring data integrity across distributed services.
In modern distributed and microservice architectures, inter‑service call failures are common, making data consistency a critical challenge. This article reviews several approaches to handle such failures, focusing on blocking retries, asynchronous queues, TCC compensation transactions, local message tables, and MQ transactions.
Blocking Retry
Blocking retry is a straightforward method where a request is attempted multiple times before giving up. The following pseudo‑code demonstrates a three‑attempt retry loop:
m := db.Insert(sql) err := request(B-Service,m) func request(url string, body interface{}) { for i := 0; i < 3; i++ { result, err = request.POST(url, body) if err == nil { break } else { log.Print() } } }This approach can cause duplicate data, dirty records, and increased latency, especially when the downstream service is unavailable.
Asynchronous Queue
Introducing a message queue decouples the service call from the business logic. After persisting data, a message is published to the queue for asynchronous processing:
m := db.Insert(sql) err := mq.Publish("B-Service-topic", m)While queues improve reliability, publishing to the queue can still fail, leading to the same consistency issues as blocking retries.
TCC Compensation Transaction
TCC splits each service call into three phases—Try, Confirm, and Cancel—to achieve distributed consistency without tightly coupling services.
Try: validate and reserve resources.
Confirm: commit the reservation.
Cancel: release the reservation on failure.
Example pseudo‑code for a shopping scenario involving inventory (A), payment (B), and points (C) services:
m := db.Insert(sql) aResult, aErr := A.Try(m) bResult, bErr := B.Try(m) cResult, cErr := C.Try(m) if cErr != nil { A.Cancel() B.Cancel() C.Cancel() } else { A.Confirm() B.Confirm() C.Confirm() }TCC solves cross‑service consistency but introduces challenges such as empty releases, ordering issues, and handling failures of Confirm/Cancel operations.
Local Message Table
Originating from eBay, a local message table stores messages in the same database transaction as business data, enabling reliable asynchronous retries.
Typical workflow: insert business data and a corresponding message; if subsequent processing succeeds, delete the message; otherwise, keep it for retry.
Example pseudo‑code:
messageTx := tc.NewTransaction("order") messageTxSql := tx.TryPlan("content") m, err := db.InsertTx(sql, messageTxSql) if err != nil { return err } aErr := mq.Publish("B-Service-topic", m) if aErr != nil { messageTx.Confirm() // keep message for retry } else { messageTx.Cancel() // delete message }The underlying SQL for the message table insertion is:
insert into `tcc_async_task` (`uid`,`name`,`value`,`status`) values ('?','?','?','?')MQ Transaction
Some MQ implementations (e.g., RocketMQ) support transactional messages, which follow the same Confirm/Cancel pattern as TCC, adding a prepare state that must be resolved by the consumer.
Summary
Ensuring data consistency in distributed systems inevitably requires additional mechanisms. TCC offers flexibility without tying to a specific database but demands extensive API implementation. Local message tables provide a simple, service‑agnostic solution, while MQ transactions abstract the pattern into the messaging layer. Selecting the appropriate strategy depends on consistency requirements, operational complexity, and available infrastructure.
IT Architects Alliance
Discussion and exchange on system, internet, large‑scale distributed, high‑availability, and high‑performance architectures, as well as big data, machine learning, AI, and architecture adjustments with internet technologies. Includes real‑world large‑scale architecture case studies. Open to architects who have ideas and enjoy sharing.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.