How to innovate and solve technical challenges in diverse, complex overseas scenarios? The Shopee Tech Team will explore cutting‑edge technology concepts and applications with you.
SPEC‑RL introduces speculative rollouts that reuse verified historical rollouts as prefixes, cutting rollout time by 2–3× while maintaining or improving performance across various math and reasoning benchmarks, and works seamlessly with PPO, GRPO, DAPO and other on‑policy algorithms.
Shopee’s deployment of StarRocks across DataService, DataGo, and DataStudio demonstrates that its vectorized engine, cost‑based optimizer, and materialized‑view caching can query Hive, Iceberg, Delta Lake and Hudi up to 20,000× faster than Presto, cutting CPU usage and delivering consistently lower latency for complex analytics.
Shopee’s unified Image Service consolidates upload, processing, and delivery into a single platform that overcomes legacy fragmentation and heterogeneous networks through resumable SDKs, adaptive ID schemes, link‑level and HTTP/3 transport optimizations, dynamic CDN scheduling, and device‑aware resizing, boosting upload success to 99.67 %, halving latency, cutting storage by 25 % and bandwidth by 56 % while markedly improving app performance.
The article details Shopee’s seadt‑SAGA framework—a Go‑based distributed transaction solution that implements the SAGA pattern via two modes, orchestration with a central coordinator and collaboration with embedded logic, explains their state‑machine designs, compares rollback ordering using vector clocks versus sequence numbers, and discusses each mode’s trade‑offs.
ShopeeVideo OCR is a multi‑language text‑recognition system for Southeast Asian e‑commerce videos that unifies detection, Transformer‑based recognition, layout analysis, and large‑scale synthetic data generation to handle Indonesian, Filipino, English, Vietnamese, Thai and Chinese scripts, delivering industry‑leading accuracy and winning thirteen ICDAR first‑place awards.
The article describes ShopeePay’s cloud‑native, high‑availability Naming Center built on sRPC, which replaces etcd with an in‑memory, peer‑to‑peer service registry that uses gossip heartbeats, self‑protection, and a coordinator for eventual consistency, rapid recovery, and seamless upgrades.
Shopee’s scalable fund‑routing decision engine combines a configurable Go‑based rule engine with Redis‑backed, Lua‑driven quota control to dynamically match loan orders to external capital providers, enforce multi‑dimensional volume limits, achieve tens of thousands TPS, and provide extensible risk‑management capabilities across markets.
Shopee tackled frequent Flink checkpoint failures caused by back‑pressure by adopting and extending the community’s Unaligned Checkpoint mechanism—adding overdraft buffers, improving legacy sources, introducing an aligned‑checkpoint timeout, enabling output‑buffer switching, merging small HDFS files, and fixing network‑buffer deadlocks—now running hundreds of jobs with stable UC deployment and plans to enable it universally.
Shopee’s off‑platform ads delay service combines Redis Zsets for expiration tracking, HBase for payload storage, and Kafka for queuing to reliably process up to 6 million tasks per minute with minute‑level delays ranging from one minute to thirty days, achieving horizontal scalability, fault tolerance, and a 75 % reduction in Kubernetes resource usage.
Shopee tackled its data‑system bottlenecks by customizing Apache Hudi to provide unified stream‑batch integration, efficient state‑detail snapshots, and low‑latency wide‑table generation, using CDC‑based bootstrapping, COW/MOR tables, savepoints and partial updates, which cut latency to ten minutes, lowered resource use, and yielded several community‑backed enhancements.
