How Vivo’s E‑Commerce Inventory System Scales: Architecture, Challenges, and Solutions

Business Background

Vivo’s mall inventory system was originally coupled with the product service, causing complexity as business logic grew. To support sellable inventory, physical inventory, flash‑sale inventory, logistics timing, shipping restrictions, and multi‑warehouse management, the inventory function was split into an independent service.

System Architecture Design

Overall Inventory Architecture

The inventory domain is divided into three layers:

Warehouse layer : physical warehouses, third‑party warehouses, WMS/ERP.

Scheduling layer : inventory allocation and order fulfillment.

Sales layer : official mall, physical stores, third‑party channels.

Evolution of the Mall Inventory System

Early versions stored inventory redundantly (sellable inventory in the product system, promotional inventory in the marketing system) and only supported simple decrement/release operations. Limitations included:

No fine‑grained, layered management of physical, warehouse, and promotional stocks.

Missing warehouse‑level strategies, preventing pre‑fetch of shipping addresses and logistics timing estimation.

Inability to control shipping by region or product.

Poor real‑time synchronization of physical stock and warehouse policies.

Weak performance and tight coupling, limiting scalability.

In 2021 the first version of the new architecture was released; subsequent iterations produced the current design.

Business Architecture

Inventory Types & Warehouse Management

The system defines four inventory categories:

Sellable inventory : ordinary SKU‑level stock configured by operations.

Physical inventory : stock synchronized from warehouse management systems, detailed per warehouse.

Pre‑reserved inventory : stock reserved when an order is placed and released after shipment; monitors ordered‑but‑unshipped quantities.

Promotional inventory : stock dedicated to flash‑sales, seckill, and other marketing events.

Mall Inventory Flow

Two main operations drive inventory changes: forward deduction and reverse rollback.

Forward Deduction Options

Deduct at order placement – immediate stock reduction avoids payment‑time failures but can be abused; a single user could lock all units of a product.

Deduct at payment – prevents malicious pre‑locking but may block payment for many users when orders exceed stock (e.g., 1,000 orders for 100 units).

Vivo adopts a hybrid approach: deduct at order time with a short hold (e.g., 15 minutes). If payment does not occur within the window, the stock is released, balancing user experience and abuse prevention.

Reverse Rollback Process

Before shipment, a refund request triggers rollback of sellable and pre‑reserved stock, soft‑deletes the deduction log, and adds a rollback log.

After shipment, returns do not roll back inventory because the physical goods have left the warehouse.

Fine‑Grained Shipping Control

Warehouse strategy : ship from the nearest warehouse; if no physical stock, fall back to a backup warehouse.

Shipping restrictions : region‑based bans and time‑window limits (daily, promotional, emergency).

Logistics timing estimation : combine shipping address, warehouse strategy, and delivery windows to predict arrival.

Technical Key Points

Idempotent Deduction to Prevent Duplicate Submissions

Duplicate order submissions can cause double deduction. Considered solutions:

Disable the submit button after click – reduces accidental repeats but does not stop scripted attacks.

Make the deduction API idempotent – include a unique order serial number and rely on a unique DB index.

Token mechanism – verify a one‑time token before allowing submission.

Vivo implements the idempotent API approach: the order serial number is stored with each deduction log; on repeat calls the service checks for an existing log and returns success without re‑deducting.

Preventing Oversell and Handling High Concurrency

Conventional Channels

Direct DB deduction using an UPDATE with a condition (stock ≥ deduction) and optimistic locking:

update store set store = store - #{deductStore} where (store - #{deductStore}) >= 0

Pros: real‑stock deduction, no oversell, optimistic‑lock consistency, transactional rollback.

Cons: row‑level lock becomes a bottleneck under high QPS; each request writes to the DB, limiting throughput.

High‑Concurrency Scenarios (e.g., Seckill)

Cache‑first deduction using Redis + Lua script provides single‑threaded stock updates, dramatically improving performance. Final consistency requires persisting cache changes to the DB.

Two persistence strategies:

Async Redis→DB – periodically flush Redis stock to the database. Simple but risks data loss if Redis crashes.

Redis deduction within DB transaction – insert a “stock task” record (primary key + JSON payload) inside the same transaction that performs the Redis decrement. An asynchronous worker later syncs the task to the inventory table, guaranteeing eventual consistency.

Vivo adopts the second approach for seckill, adding strict pre‑traffic validation and rate limiting to keep traffic controllable.

Hotspot Problems

Hotspot issues arise when a popular product concentrates load on a single DB row or a single Redis shard.

DB hotspot – split the stock row into M rows and randomly pick one for each deduction, breaking the row‑lock bottleneck.

Redis hotspot – distribute keys across multiple shards; Vivo currently relies on rate limiting because traffic has not yet saturated a single shard.

Inventory Synchronization

Two main sync scenarios:

Physical stock sync from warehouse systems (full daily batch or on‑demand SKU update).

Compatibility sync for legacy product‑system sellable stock.

Vivo uses a custom CDC platform (Luban) to keep the product system and inventory system consistent. The pipeline:

Binlog capture detects inventory DB changes.

Custom rules filter relevant events.

Change messages are sent to MQ.

Product system consumes the messages and updates its own stock view.

Summary

The inventory platform has been decoupled from the product service and expanded from simple sellable‑stock deduction to support multiple stock types, fine‑grained warehouse strategies, idempotent deduction, anti‑oversell mechanisms, high‑concurrency handling, hotspot mitigation, and robust synchronization.