seadt: Golang Distributed Transaction Solution – Design, Integration, and Code Implementation

seadt is a distributed transaction solution built with Golang for Shopee Financial Products, providing a TCC (Try‑Confirm‑Cancel) mode that aims to make distributed transactions feel like local transactions.

Design goals :

Transaction localization – the transaction initiator operates distributed transactions as if they were local DB operations.

Service atomicity – participants expose simple atomic interfaces without worrying about concurrency, idempotency, empty commit/rollback, or transaction hanging.

The article explains the business integration of seadt, covering initiator (TM) and participant (RM) code, the underlying TM/RM/TC components, and detailed exception handling.

1. Business Integration

1.1 Application Scenarios

Two main types of scenarios are described:

Type 1 – invoking multiple downstream transactional interfaces (e.g., double‑write during system upgrades, loan transfers).

Type 2 – invoking a single downstream transactional interface (e.g., freezing quota).

Both fit the TCC model and can be used in loan‑process workflows.

1.2 Initiator (TM) Integration

The initiator starts a global transaction using tcc.WithGlobalTransaction() . A simplified code example shows how to wrap business logic inside the global transaction:

func DoSubmit(...) {
    // start distributed transaction
    tcc.WithGlobalTransaction(ctx, func(ctx context.Context) {
        // call Try of participant A
        ***.TryFrozenQuotaForLoan(...)
        // call Try of participant B
        ***.TryFrozenVoucherForLoan(...)
        // local DB operation
        ***.Insert(ctx, record)
    }, &tcc_model.GlobalTransactionExtraInfo{TimeOutSecond: int64(constant.DEFAULT_ASYNC_PROCESS_TIMEOUT), TransactionName: "quota_frozen"})
}

This makes the four steps behave like a single local transaction.

1.3 Participant (RM) Integration

Participants implement the ITccResourceService interface with Try, Confirm, and Cancel methods. Example implementations:

// Try implementation
func (t *QuotaFrozenTccImpl) Try(ctx context.Context, payload interface{}) (bool, error) {
    if req, ok := payload.(*api.ClFrozenQuotaReq); ok {
        ***.QuotaAvailableToFrozen(ctx, req)
        return true, nil
    }
    return false, nil
}

// Confirm implementation (no operation in this example)
func (t *QuotaFrozenTccImpl) Confirm(ctx context.Context, payload interface{}) bool {
    log.Info(***, "call confirm")
    return true
}

// Cancel implementation
func (t *QuotaFrozenTccImpl) Cancel(ctx context.Context, payload interface{}) bool {
    if req, ok := payload.(*api.ClFrozenQuotaReq); ok {
        ***.QuotaFrozenToAvailable(ctx, req)
        return true
    }
    return false
}

The SDK generates a proxy that registers the participant with the RM, handling the TCC lifecycle automatically via AOP.

2. Core Components

2.1 TM (Transaction Manager)

TM provides tcc.WithGlobalTransaction() which internally calls transaction.WithTransaction() to start a global transaction, registers callbacks, and persists the transaction record.

func WithGlobalTransaction(ctx context.Context, process func(ctx context.Context), extraInfo *tcc_model.GlobalTransactionExtraInfo) {
    transaction.WithTransaction(ctx, func(ctx context.Context) {
        rootContext := tm.RefTransactionManager().StartGlobalTransaction(ctx, extraInfo)
        process(rootContext)
    })
}

TM also registers before‑commit, before‑completion, and after‑completion callbacks to handle timeout, status reporting, and interaction with TC.

2.2 RM (Resource Manager)

RM abstracts the participant side. It registers the participant’s Try method, creates branch transactions, and reports status to TC. The proxy generated by the SDK contains reflective method descriptors for Try/Confirm/Cancel.

type ResourceServiceDescriptor struct {
    Name         string
    ReflectType  reflect.Type
    ReflectValue reflect.Value
    Methods      sync.Map // string -> *MethodDescriptor
}

type MethodDescriptor struct {
    Method           reflect.Method
    CallerValue      reflect.Value
    CtxType          reflect.Type
    ArgsType         []reflect.Type
    ReturnValuesType []reflect.Type
}

During the Try phase, RM registers the branch transaction, executes the business logic, and reports the branch status to TC.

2.3 TC (Transaction Coordinator)

TC offers four main APIs: CreateGTX, ReportGTXstatus, CreateBTX, ReportBTXstatus. It stores transaction records in a DB and drives the two‑phase commit/rollback. TC also performs recovery handling for hanging branches.

func (i *TCCTransactionImpl) CreateGTX(...) string {
    db.WithTransaction(ctx, func(ctx context.Context) {
        transRecord := InitGlobalRecord(...)
        InitGlobalInvokeInfo(transRecord.Txid, address)
        repo.Save(transRecord)
    })
    return transRecord.Txid
}

Recovery logic iterates over unfinished branches and invokes Confirm or Cancel as needed.

3. Exception Handling

3.1 Empty Commit

Occurs when a participant never executed Try but receives Confirm. This is prohibited; the system raises an error and alerts operators.

3.2 Empty Rollback

Allowed scenario: a participant never executed Try but receives Cancel. RM records a Canceled branch to prevent hanging and returns success to TC.

3.3 Transaction Hanging

When a global transaction finishes before a participant registers its branch, the branch may become “hanging”. TC rejects new branch registrations after the global transaction ends; RM inserts a Cancel record to block further Try execution.

func (rm *ResourceManager) CheckResourcePhaseTwoTransfer(ctx context.Context, req *ResourceCheckReq) {
    transaction.WithTransaction(ctx, func(ctx context.Context) {
        if branchTransaction == nil {
            if req.TxStatus == Status_Committed {
                // reject empty commit
                panic("empty commit not allowed")
            } else {
                // handle empty rollback
                repo.SaveBranchTransaction(ctx, protectedBranchTrans)
            }
        }
        // further status checks ...
    })
}

3.4 Concurrency Issues

To avoid duplicate branch creation or concurrent Confirm/Cancel calls, RM uses a lock‑check‑update pattern on branch records.

// Pseudocode for lock‑check‑update
lockBranch(txid)
if branch.Status == Expected {
    updateBranch(Status)
}
unlockBranch(txid)

4. Conclusion

The article provides a complete walkthrough of seadt’s TCC implementation, covering design decisions, code integration for initiators and participants, core component interactions, and comprehensive handling of edge cases such as empty commits, empty rollbacks, transaction hanging, and concurrency anomalies. Future work includes adding Saga mode support.