distributed system

Xiaohongshu Tech REDtech

Apr 25, 2023 · Databases

REDtao: A High-Performance Graph Storage System for Social Graph Data

REDtao is a high‑performance graph storage system built for Xiaohongshu that extends Facebook’s Tao architecture with a three‑layer hash structure, decoupled caching, leader‑follower distribution and cross‑cloud availability, delivering over 90% cache hits, 70% MySQL CPU reduction, 150 M QPS on a 16‑core server and seamless migration despite a 250% surge in daily‑active‑user traffic.

cachingcloud nativedata consistency

0 likes · 17 min read

REDtao: A High-Performance Graph Storage System for Social Graph Data

Architecture Digest

Oct 5, 2022 · Backend Development

Designing a High‑Concurrency Ticket Flash‑Sale System with Load Balancing, Nginx Weighted Round‑Robin, and Go

This article explains how to build a high‑concurrency ticket‑seckill system that can handle one million users buying ten thousand tickets simultaneously by using distributed load‑balancing, Nginx weighted round‑robin, Go‑based HTTP services, Redis atomic stock deduction, and practical performance testing.

GoLoad BalancingNginx

0 likes · 19 min read

Designing a High‑Concurrency Ticket Flash‑Sale System with Load Balancing, Nginx Weighted Round‑Robin, and Go

Shopee Tech Team

Sep 28, 2022 · Backend Development

Shopee Off-Platform Ads Delay Service: Architecture and Implementation

Shopee’s off‑platform ads delay service combines Redis Zsets for expiration tracking, HBase for payload storage, and Kafka for queuing to reliably process up to 6 million tasks per minute with minute‑level delays ranging from one minute to thirty days, achieving horizontal scalability, fault tolerance, and a 75 % reduction in Kubernetes resource usage.

HBaseKafkaRedis

0 likes · 17 min read

Shopee Off-Platform Ads Delay Service: Architecture and Implementation

Baidu Geek Talk

Jun 13, 2022 · Backend Development

Baidu Comment Middle Platform: Architecture Design and Implementation

Baidu's Comment Middle Platform evolved from a single service into a robust middleware that delivers stable, high‑performance comment functionality across more than twenty products, handling hundreds of millions of daily requests with 99.995% SLA through graph‑based scheduling, tiered caching, and scalable sorting mechanisms.

BaiduGraph SchedulingService Optimization

0 likes · 17 min read

Baidu Comment Middle Platform: Architecture Design and Implementation

Top Architect

Dec 6, 2021 · Backend Development

Design and Evolution of Baidu Short‑Video Push System

This article details the architecture, data flow, module responsibilities, and successive optimizations of Baidu's short‑video push system, covering personalized timing estimation, user‑group management, frequency‑control redesign, and protobuf‑based compression to handle billion‑scale traffic efficiently.

architecturedistributed systemfrequency control

0 likes · 16 min read

Design and Evolution of Baidu Short‑Video Push System

Baidu Geek Talk

Nov 15, 2021 · Backend Development

Baidu Short Video Push System: Architecture Design and Billion-Level Data Optimization Practice

Baidu’s Short Video Push System is a distributed platform serving hundreds of millions of users across multiple apps, delivering personalized, real‑time notifications via a modular architecture that includes material and user centers, recall, preprocessing, and delivery services, while optimizations such as activity‑based scheduling, bitmap‑based user segmentation, consistent‑hash frequency control, and protobuf compression boost click‑through rates, scalability, and resource efficiency.

BaiduProtobufPush Notification

0 likes · 15 min read

Baidu Short Video Push System: Architecture Design and Billion-Level Data Optimization Practice

FunTester

Jul 30, 2021 · Operations

Distributed QPS Amplifier Design in DCS_FunTester Framework

The article describes the concept, architecture, and Java implementation of a distributed request amplifier in the DCS_FunTester framework that multiplies local QPS to achieve high‑throughput performance testing without affecting the tested service's load.

JavaQPSdistributed system

0 likes · 6 min read

Distributed QPS Amplifier Design in DCS_FunTester Framework

IT Architects Alliance

Dec 10, 2020 · Backend Development

Designing a High‑Concurrency Ticket Spike System: Load Balancing, Stock Deduction, and Go Implementation

This article explores the architecture and implementation of a high‑concurrency ticket‑spike system, covering distributed load‑balancing, Nginx weighted round‑robin configuration, Go‑based local and remote stock deduction with Redis, performance testing, and strategies to avoid overselling and underselling.

GoLoad BalancingRedis

0 likes · 19 min read

Designing a High‑Concurrency Ticket Spike System: Load Balancing, Stock Deduction, and Go Implementation

TAL Education Technology

Apr 2, 2020 · Backend Development

Design and Implementation of the Dandelion Resource Distribution Platform for Dual‑Teacher Classroom

This article describes the challenges of delivering large video lessons and other digital assets to distributed school campuses and explains how the Dandelion release platform uses resumable uploads, cloud OSS, long‑connection messaging, redundancy, and version‑controlled rollout to ensure reliable, low‑latency playback and scalable resource distribution.

Version Managementcloud storagedistributed system

0 likes · 13 min read

Design and Implementation of the Dandelion Resource Distribution Platform for Dual‑Teacher Classroom

Youzan Coder

Mar 25, 2020 · Backend Development

Youzan ID Generator Multi-Room Solution

Youzan redesigned its ID generator to run independent March instances per room, sharing a single etcd cluster but allocating distinct, non‑overlapping ID ranges via boundary, lower, and upper parameters, enabling room‑level high availability, horizontal scaling, and zero‑downtime migration with automated tooling.

ETCDID Generatorarchitecture

0 likes · 7 min read

58 Tech

Jan 6, 2020 · Big Data

Design and Architecture of the 58DP Big Data Platform Task Scheduling System

The article presents a comprehensive overview of the 58DP big data platform's task scheduling system, detailing its background, architecture, high‑availability design, slot‑based resource management, scheduling models, task lifecycle, priority rules, dependency handling, failure recovery, and future enhancements.

Big DataHigh Availabilitydistributed system

0 likes · 14 min read

Design and Architecture of the 58DP Big Data Platform Task Scheduling System

Tencent Cloud Developer

Nov 26, 2019 · Backend Development

TurboSearch: Tencent AI Lab's Next-Generation Large-Scale Search System

TurboSearch is Tencent AI Lab's next-generation large-scale search system, delivering distributed massive indexing, high-performance parallel retrieval, multi-granularity and multi-modal vector indexing, private Docker deployment, integrated NLP query analysis, extensible plugins, and robust operations for massive data and diverse search scenarios.

Inverted IndexNLPQuery Analysis

0 likes · 14 min read

TurboSearch: Tencent AI Lab's Next-Generation Large-Scale Search System

Tencent Cloud Developer

Jul 8, 2019 · Cloud Computing

Overseas Video Cloud Live Streaming System Architecture Practice

Tencent Cloud’s overseas video live‑streaming architecture combines tiered carrier procurement across North America, APAC and Europe, decentralized origin servers with dual‑active synchronization, and network optimizations such as DNS‑based scheduling, QUIC and QTCP, cutting stutter rates while tightly controlling costs.

Live StreamingQUICTencent Cloud

0 likes · 13 min read

Overseas Video Cloud Live Streaming System Architecture Practice

Sohu Tech Products

Dec 5, 2018 · Backend Development

Overview of Web Crawler Types and the Architecture of the Mole Crawler System

This article explains the evolution and classification of web crawlers, describes the design and components of the Mole distributed crawler—including scheduler, fetcher, processor, rate‑limiting, URL deduplication, and Elasticsearch storage optimization—and outlines common anti‑anti‑crawling strategies.

ElasticsearchRate LimitingURL deduplication

0 likes · 12 min read

Overview of Web Crawler Types and the Architecture of the Mole Crawler System

Beike Product & Technology

Jul 9, 2017 · Databases

Understanding Redis Cluster: Architecture, Slot Management, and Fault Tolerance

This article explains Redis Cluster’s distributed architecture, the 16,384 hash slot mechanism, node addition and removal, decentralised request routing, fault‑tolerance through master‑slave replication, slot migration without service interruption, and the internal data structures that enable these features.

ClusterDatabaseHash Slot

0 likes · 10 min read

Understanding Redis Cluster: Architecture, Slot Management, and Fault Tolerance

Architecture Digest

May 7, 2017 · Backend Development

Design and Implementation of a Java-Based Flash Sale (Seckill) System

This article explains the architecture and Java code for a high‑concurrency flash‑sale system, covering request distribution, pre‑processing, queue selection, transaction handling, and database interaction to efficiently manage limited‑stock purchases under massive traffic.

JavaQueuebackend

0 likes · 7 min read

Design and Implementation of a Java-Based Flash Sale (Seckill) System

Architecture Digest

Sep 15, 2016 · Backend Development

Design and Evolution of Xiaomi’s Flash‑Sale System for the 2014 Mi Fan Festival

The article recounts how Xiaomi’s engineering team built, tested, and iteratively improved a high‑concurrency flash‑sale platform—first in PHP with Redis and later in Go—to reliably handle millions of users during the 2014 Mi Fan Festival, detailing architectural choices, performance optimizations, and lessons learned.

GoPHPRedis

0 likes · 15 min read

Design and Evolution of Xiaomi’s Flash‑Sale System for the 2014 Mi Fan Festival