This article outlines a comprehensive series of over 500 planned tutorials covering Java advanced features, distributed theory, Hadoop, Spark, Flink, and various big‑data storage and processing technologies, designed to guide engineers transitioning into big‑data development from fundamentals to expert level.
An extensive roadmap series titled “Big Data Mastery Roadmap” outlines essential topics—from Java advanced features and JVM internals to Hadoop, Spark, Flink, and big-data algorithms—guiding engineers transitioning to big data development with curated references, updates, and author insights.
This article presents a curated collection of Alibaba’s open‑source Java projects, ranging from distributed service frameworks like Spring Cloud Alibaba and Dubbo, database tools such as Druid and TDDL, to monitoring, messaging, and cloud‑native components, each with brief descriptions and GitHub links.
This article explains why chaos engineering is essential for distributed systems, distinguishes it from traditional fault testing, outlines practical inputs and principles, and details Alibaba's multi‑year experience with fault‑drill platforms, automation, and future plans to improve system reliability.
The article summarizes the evolution and redesign of Zhihu's search engine, details 58 Group's high‑performance uesearch architecture, real‑time indexing mechanisms, cloud‑native deployment with Kubernetes, and highlights key technical insights and future directions for large‑scale search systems.
This article explains the concept of state in software, contrasts stateful and stateless designs, and shows how moving state handling to the client or shared storage improves scalability, fault tolerance, and maintainability in distributed backend systems.
This article explains chaos engineering, its origins at Netflix, core principles, practical steps for running experiments, and how organizations can use controlled failure injection to improve system resilience and operational confidence in complex distributed environments.
This article explains the purpose of compensation mechanisms in microservice architectures, compares rollback and retry approaches, outlines their implementation details, discusses idempotency concerns, and provides practical best‑practice recommendations for building resilient distributed systems.
The paper presents a highly scalable, low‑overhead Host‑based Intrusion Detection System architecture designed for hundreds of thousands of servers, emphasizing cluster high‑availability, strong consistency via a CP‑oriented etcd backend, Go‑based agents with efficient resource management, modular sandboxing, and robust process monitoring to ensure reliable, secure operation at massive scale.
This article explains the concept of RPC in Java, compares three popular frameworks—RMI, Hessian, and Dubbo—describes their architectures, and provides complete code samples for interfaces, service implementations, clients, and servers to help developers build scalable distributed applications.
The article examines how architects of large-scale systems like eBay and Amazon forgo traditional ACID transactions in favor of BASE principles, balancing consistency, availability, and scalability through application‑level designs, async processing, and strategic trade‑offs informed by the CAP theorem.
The article describes YouZan's business reconciliation platform for distributed systems, which detects and quantifies data inconsistencies by offering easy plug‑in integration, a four‑step orchestrated workflow, high‑throughput offline processing with Spark, second‑level real‑time event handling, a three‑layer architecture, and health monitoring for transaction chains.
This article provides an in‑depth technical overview of Apache Storm, covering its core architecture, key components such as Nimbus, Supervisor, Worker, Executor, and Task, the role of ZooKeeper, high‑availability setup, API interfaces, code examples, grouping strategies, metrics, back‑pressure handling, and essential configuration parameters for building low‑latency stream processing topologies.
This tutorial explains key concepts such as distributed systems, RPC, and Dubbo, then guides you through installing Zookeeper, creating Maven modules, configuring SpringBoot and Dubbo, implementing service interfaces, providers, and consumers, and testing a simple HelloWorld distributed service.
Message queues serve as essential decoupling and traffic‑control components in distributed systems, and this article outlines their overall design—including producers, brokers, consumers, communication protocols such as JMS, AMQP and Kafka, storage options, consumption relationship handling, and advanced features like ordering, reliability, persistence, and transaction support.
This article explains the concept of Remote Procedure Call (RPC), why distributed systems need remote invocation, the roles of provider, consumer and registry, essential technologies such as dynamic proxies, serialization and NIO, and introduces popular open‑source Java RPC frameworks.
This article explains the mathematical and programming concept of idempotency, provides real‑world examples such as duplicate form submissions and payment requests, and details practical techniques—including query/select operations, unique indexes, token mechanisms, pessimistic and optimistic locks, distributed locks, and API design—to ensure idempotent behavior in backend systems.
MAXIM is Youzan’s distributed full‑link load‑testing engine built on Gatling, featuring a central control center, multiple load injectors, a GUI for test orchestration, data‑parameter binding, real‑time injector monitoring, automated reporting with historical retention, and extensible architecture supporting Dubbo and centralized InfluxDB logging.
This article explains the concept of circuit breaking in distributed systems, outlines a four‑step implementation process with strategies for detecting unhealthy services, cutting off calls, probing recovery, and restoring normal operation, and shares best‑practice tips to minimize downtime and improve resilience.
This article explains the difference between rate limiting and circuit breaking, shows how to determine system capacity, compares fixed‑window, sliding‑window, leaky‑bucket and token‑bucket algorithms with code examples, and offers best‑practice guidance for applying them in distributed backend systems.
The article explains the evolution from classic three‑tier Java web architecture to cluster, distributed, and microservice designs, detailing each model’s components, load‑balancing mechanisms, session sharing, and the trade‑offs of using technologies such as Tomcat, Nginx, Dubbo, and Spring Cloud.
Youzan’s full‑link load‑testing architecture combines a traffic generator, a data‑factory pipeline, and the Maxim platform to replay realistic e‑commerce user actions, tag and isolate test traffic via unified headers, route reads/writes to shadow storage, and integrate Gatling for capacity planning, degradation, alarm, disaster‑recovery and throttling drills.
This article explains the fundamental reasons for building distributed systems, examines the inevitable side‑effects—especially data consistency challenges—analyzes the root causes of inconsistency, and walks through various consistency models from eventual to linearizability with clear examples and illustrations.
This article provides a comprehensive guide to RabbitMQ, covering the fundamentals of message middleware, the P2P and Pub/Sub models, a comparison with Kafka and RocketMQ, detailed explanations of exchanges, queues, channels, and advanced features such as mandatory routing, backup exchanges, TTL, dead‑letter queues, delayed queues, priority queues, and RPC implementations for reliable distributed systems.
This guide outlines the core backend engineering competencies—including Java, Linux, distributed systems, databases, and architectural design—that candidates need to master to succeed in interviews at China’s top tech giants Baidu, Alibaba, and Tencent.
This guide shares actionable strategies for Java backend interview preparation, covering essential Java fundamentals, framework mastery, distributed system concepts, database optimization, Linux troubleshooting, and how to showcase real project experience to stand out from other candidates.
This article examines how Uber built and scaled its real-time ride-sharing platform, detailing the original simple PHP-MySQL architecture, subsequent extensions with message queues, MongoDB, Ringpop storage, TChannel communication, fault-tolerance strategies, latency challenges, and practical tools for distributed system design.
The article explains distributed final consistency challenges, compares heavyweight transaction frameworks with lightweight techniques such as idempotency, retries, state machines, recovery logs, and async verification, and outlines CAP, BASE principles and practical implementation steps for backend systems.
Youzan’s Transparent Multilevel Cache (TMC) adds automatic hotspot detection and a 64 MB local cache to existing distributed caches via a Hermes‑SDK‑augmented Jedis client, delivering transparent Java integration, strong consistency, up to 80 % local‑hit rates, and improved QPS during high‑traffic events.
This article explains how to design and implement a high‑concurrency, Netty‑driven push platform that supports device registration, authentication, message routing, and distributed scaling for millions of IoT connections, covering protocol design, channel management, load balancing, and monitoring.
This article provides a detailed comparison between Apache Pulsar and Apache Kafka, covering their message consumption models (queue vs. stream), subscription types, acknowledgment mechanisms, retention policies, and underlying layered architecture, highlighting Pulsar's unified API and segment‑based storage advantages.
This article explains the causes of hot key problems in high‑traffic scenarios, outlines their potential impact on system performance, and presents multiple mitigation strategies—including server‑side caching, Memcache/Redis, local caches, read‑write separation, and proactive hot‑data detection—while comparing their advantages and trade‑offs.
This article presents a comprehensive overview of constructing a search engine using Elasticsearch, covering architecture components, data read/write mechanisms, shard management, caching strategies, and real‑world case studies that illustrate performance tuning, isolation, and deployment best practices.
Alibaba’s DAMO Academy chief database scientist Li Fei‑Fei was named an ACM Distinguished Scientist in 2018, recognizing his groundbreaking work on next‑generation distributed databases, unstructured data management, and data security that power Alibaba’s massive services and national smart‑city and meteorological platforms.
This article chronicles Alibaba's decade‑long journey of database innovations—from commercial and open‑source systems to a self‑developed distributed engine—highlighting the technical breakthroughs, performance optimizations, and intelligent features that powered successive Double 11 shopping festivals.
This article examines why traditional relational databases like MySQL struggle with analytical workloads, compares ROLAP and MOLAP approaches, explains Spark’s architecture and its advantages for OLAP, and details how Alibaba Cloud’s DRDS HTAP leverages a Spark‑based engine to deliver real‑time distributed query processing.
This article presents a comprehensive case study of JD's B2B platform, detailing its three development phases, the challenges of the initial monolithic architecture, and the step‑by‑step architectural refinements—including service decomposition, database sharding, messaging middleware, and distributed search—that culminated in the current modular 3.0 design.
Ant Group unveiled TRaaS (Technological Risk‑defense as a Service), a comprehensive platform that combines high‑availability, real‑time fund reconciliation and AI‑driven self‑healing capabilities to protect large‑scale financial systems against technical risks.
This article introduces the free classic book “Cloud Design Patterns,” which catalogs 32 design patterns across eight problem domains—availability, data management, design & implementation, messaging, management & monitoring, performance & scalability, resiliency, and security—offering practical solutions and C# examples for modern cloud and distributed systems.
This article uses a vivid allegorical story to introduce the Paxos algorithm, then explains its roles, two-phase protocol, fault assumptions, and why majority and multiple acceptors are essential for achieving reliable consensus in distributed systems.
This article introduces SOFARPC, Ant Financial's financial‑grade Java RPC framework, covering its overall architecture, extension mechanism, link tracing, connection management, synchronous and asynchronous invocation models, thread model, fault‑tolerance, generic invocation, data passthrough, graceful shutdown, routing implementation, and annotation support for building robust microservice systems.
This article introduces the SCF RPC framework developed by 58, covering its overall architecture, synchronous and callback call modes, timeout handling, custom serialization techniques, service registration and discovery using etcd, as well as data collection, storage, and monitoring mechanisms for large‑scale distributed services.
This article presents a curated collection of Alibaba Group’s open‑source projects, spanning frontend design systems, Java libraries, database engines, distributed file and messaging systems, as well as tutorials, highlighting each project's purpose, key features, and GitHub repository links for developers seeking robust, production‑grade solutions.
This article explains the concepts of high concurrency and multithreading, outlines performance metrics such as TPS and QPS, and details essential Java concurrency tools and distributed technologies like Redis, Kafka, and Nginx that enable scalable backend systems.
The article explains how distributed storage systems solve the fundamental problems of data sharding and redundancy by describing three sharding methods (hash, consistent‑hash, and range‑based), the criteria for choosing a shard key, the role of metadata servers, and consistency techniques such as leasing, all illustrated with concrete examples and code snippets.
This article explores the fundamentals and practical details of server‑side caching in distributed web systems, covering local vs. distributed caches, Redis instance planning, key‑value modeling, persistence, eviction policies, and CRUD operations with performance‑focused recommendations.
This article narrates the step‑by‑step evolution of a cross‑border e‑commerce logistics platform, detailing how its backend architecture was repeatedly redesigned—from a simple 1.0 system to a robust, fault‑tolerant 3.0 solution—driven by growing business needs, performance challenges, and reliability lessons.
This article summarizes Guo Dongdong’s presentation on leveraging HBase for a risk‑control platform, detailing its architecture, data import/export mechanisms, indexing, region server recovery challenges, monitoring, SQL interception, dual‑cluster high‑availability, and future enhancements for large‑scale, low‑latency big‑data services.
This article presents a high-availability global unique ordered ID generation solution, Cantor, which addresses the limitations of UUIDs and auto-increment IDs in distributed systems by combining time-based and node-specific sequences for scalability and performance.
The iQIYI video backend team outlines their microservice journey, detailing service decomposition strategies, choosing Spring Cloud for its low migration cost and rich ecosystem, and building a shared platform of registries, configuration, gateways, monitoring, and CI/CD to boost efficiency, reliability, and scalability while planning future adoption of service mesh and domain‑driven design.
This article explains the concept of web sessions, the consistency challenges that arise in distributed server clusters, and evaluates four common solutions—session replication, session affinity, cookie‑based sessions, and dedicated session servers—detailing their use cases, advantages, and drawbacks.
The article introduces common open‑source monitoring tools such as Zabbix and Nagios, explains why distributed systems need proactive health checks, compares features, and provides a detailed Zabbix monitoring workflow including data collection, storage, visualization, alerting, and specific metrics for servers, networks, JVM and MySQL.
This article presents a detailed skill‑tree and practical learning roadmap for software engineers, covering computer fundamentals, data structures, Java core concepts, multithreading, JVM internals, databases, distributed systems, personal branding, and effective study methods such as videos, books, and paid resources.
This article uses a humorous story of two threads battling over inventory updates to illustrate the principles, advantages, and drawbacks of pessimistic and optimistic locking in distributed systems, helping readers understand when to apply each concurrency control method.
This article explains the fundamentals of Remote Procedure Call (RPC), covering its definition, architectural components, communication protocols, serialization process, complete call workflow, and the main technologies and frameworks used in modern backend systems.
This article reviews the evolution of Alibaba's Aliware message engine, analyzes the latency and availability challenges faced during Double‑11, and describes the low‑latency optimizations, capacity‑guarantee strategies, and multi‑replica high‑availability architecture implemented in RocketMQ to sustain trillion‑level message flows.
The article explains how messages travel in monolithic and distributed systems, why network communication is inherently unreliable, and how different delivery semantics—at‑most‑once, at‑least‑once, and exactly‑once—affect reliability, ordering, and protocol choices such as AMQP and MQTT.
This article examines the shortcomings of decentralized and micro‑service‑only approaches for enterprise integration, argues for version‑less services, explains how a centralized ESB and multi‑center architecture can balance security, efficiency, and scalability, and outlines practical design patterns and component roles.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and fingerprint‑based deduplication.
This tutorial walks through building a simple blockchain in Java, covering core concepts, block and transaction structures, proof‑of‑work implementation, and exposing API endpoints via Servlets for creating transactions, mining blocks, viewing the chain, and achieving consensus across nodes.
This article explains the fundamentals of database transactions, the ACID properties, and how they are implemented, then delves into distributed transaction challenges and solutions such as two‑phase commit, three‑phase commit, XA, and Saga patterns, highlighting their trade‑offs and practical usage.
This article details the design, architecture, and implementation of ZanKV—a high‑performance, distributed key‑value store that combines RocksDB storage, etcd‑Raft consensus, and a Redis‑compatible protocol, covering data partitioning, namespace isolation, expiration strategies, cross‑datacenter deployment, and performance tuning.
Zabbix offers a web‑based, enterprise‑grade solution for distributed system and network monitoring; this guide walks Linux ops engineers through why monitoring matters, key availability metrics, what to monitor, step‑by‑step installation, web UI configuration, custom checks, alerting, visualization, template sharing, full‑network scaling, auto‑discovery, proxy deployment, and SNMP integration.
This article provides a comprehensive overview of Spring Cloud, explaining why it’s favored for microservice development, detailing its key features and toolset, and examining essential components such as Eureka, Hystrix, Config, Zuul, and Ribbon with practical insights.
This article provides a comprehensive overview of caching in large distributed systems, covering CDN, reverse‑proxy, local and distributed caches, popular implementations such as Ehcache, Guava, Memcached and Redis, common pitfalls like consistency and avalanche, and practical solutions with real‑world case studies.
Amid rising cloud outages and strict financial regulations, this article examines traditional multi‑active database solutions such as Oracle RAC and IBM GDPS, contrasts them with modern distributed database designs, and details SequoiaDB’s multi‑active architecture and concrete disaster‑recovery procedures for single‑node, site‑wide, and network failures.
This article provides a comprehensive technical overview of Ant Financial’s Service Mesh data‑plane SOFAMosn, covering its architectural background, layered design, core modules, performance optimizations, supported protocols, and future roadmap within a cloud‑native microservices ecosystem.
This article presents a curated knowledge map covering backend architecture, Java best practices, microservice design, distributed consistency, high‑traffic handling, Alibaba frameworks, and design patterns, offering internet high‑concurrency architects a comprehensive reference of essential concepts and skills.
This article explains the fundamentals of database transactions, the ACID properties, the challenges of distributed transactions, and compares various solutions such as 2PC, TCC, local message tables, MQ transactions, and Saga, helping readers choose appropriate approaches based on consistency and availability requirements.
This article presents a chronological overview of Optimistic Concurrency Control (OCC), covering its early theoretical foundations, key research papers, prototype implementations such as MVCC+OCC+2PC and Hekaton, and its adoption in modern distributed NewSQL databases like Megastore, F1, and MaaT, highlighting both advantages and challenges.
This article details Alibaba’s first production use of non‑volatile memory in the Tair MDB cache service, covering deployment, performance challenges such as write imbalance and lock overhead, the optimizations applied, and design guidelines for building NVM‑based caching systems.
Over six years the author chronicles the evolution of a backend system—from its initial three‑day launch, through successive capacity expansions, distributed refactoring, and micro‑service architecture—highlighting the challenges of scaling to millions of users, handling concurrency, and the continuous pursuit of simplicity and resilience.
Elasticsearch is an open‑source, scalable, high‑availability distributed full‑text search engine that operates in near real‑time, using clusters of nodes, indexes, documents, shards and replicas to efficiently store and retrieve large volumes of data.
This article explains how Remote Procedure Call (RPC) underpins micro‑service architectures, connects load balancers like Nginx, powers big‑data frameworks such as Hadoop, fuels AI platforms like TensorFlow, and why mastering RPC is essential for senior backend engineers.
This article traces the step‑by‑step evolution of a web system—from a single‑server LAMP setup through service and data separation, caching, server clusters, read/write splitting, CDN acceleration, distributed databases, NoSQL integration, business splitting, and finally distributed services—highlighting the motivations, characteristics, and practical outcomes of each stage.
This article details the design, architecture, and operational experience of Youzan's Data Platform (DP), covering its scheduling, data‑sync, service, and monitoring modules, the custom Airflow‑based task scheduler, current production metrics, supported task types, and future improvement plans.
This article explores the advantages, architectural layout, network planning, and practical technologies for global server games, emphasizing microservice-based modular design, automated deployment, frame‑synchronization over UDP, centralized versus distributed deployment models, and considerations of latency, data consistency, and blockchain integration.
The article explains the concept of idempotency, why it is critical for data consistency in backend services, and presents practical techniques such as unique business identifiers, locking, optimistic concurrency, deduplication tables, distributed locks, token mechanisms, and payment buffering to achieve reliable idempotent operations.
This article explains the core concepts of distributed architecture, including cohesion and transparency, explores common applications such as distributed file systems, caches, databases, and middleware like MyCat, and outlines the evolutionary stages and challenges of building scalable distributed systems.
This article traces Meituan Dianping's MySQL high‑availability journey, detailing the legacy MMM system, its migration to MHA, integration with Zebra and Proxy middleware, and future architectural ideas such as distributed agents, semi‑sync replication, and MySQL Group Replication.
This article outlines the critical knowledge points for backend developers, covering source code analysis, distributed systems, microservice architecture, performance tuning, Java engineering practices, and comprehensive concurrency programming techniques.
NSQ is a lightweight, high‑performance open‑source message queue written in Go, and this article explains its core components (nsqd, nsqlookupd, nsqadmin, utilities), message flow architecture, configuration details, deployment recommendations, and additional considerations such as persistence, delivery guarantees, and scalability.
This article evaluates the server‑side performance of three popular message middleware platforms—Kafka, RabbitMQ, and RocketMQ—by measuring throughput and latency when sending small 124‑byte messages, revealing that Kafka leads, followed by RocketMQ and then RabbitMQ.
This article introduces ClickHouse as a high‑performance, column‑oriented database designed for real‑time big‑data analytics, outlines its key features, performance characteristics, supported interfaces, differences from Hadoop, and explains its main storage engines—MergeTree and Distributed—while also noting its current limitations.
This article explores the evolution and core concepts of modern distributed architectures—including SOA, microservices, and service mesh—explains key theories such as CAP and BASE, and provides practical guidelines for achieving high availability, scalability, and efficient content delivery through techniques like load balancing, CDN, and gray‑release strategies.
This article provides a step‑by‑step walkthrough of how a Dubbo consumer initiates a remote call, how the request is processed through clustering, load balancing, and invocation layers, and how the provider receives, executes, and returns the result, including the underlying code structures and network interactions.
The article reviews the ten‑year evolution of Ant Financial's message queue, detailing its core reliability, consistency, availability and performance requirements, the architectural mechanisms built to meet them, the shift to pull‑mode and API‑mode designs, and the recent integration of compute capabilities to create a smart data transmission platform.
This article explains core microservice concepts—including componentization, modularization, centralized vs. distributed systems, and the architectural principles behind microservices—while comparing them to SOA and highlighting practical distinctions that help developers design loosely‑coupled, reusable services.
This article explains Kafka's role as a distributed message queue, covering its architecture, replication mechanisms, producer‑consumer workflow, message integrity guarantees, schema management, and performance tuning for high‑throughput, low‑latency backend systems.
This article explains the fundamentals of remote method invocation, compares traditional RMI with modern RPC frameworks, introduces Dubbo's architecture and roles, and provides a step‑by‑step guide to quickly develop, build, and run a complete Dubbo application using API‑centric configuration and multicast registration.
The article explains how the slowdown of Moore's Law drives the shift from monolithic applications to distributed systems, covering load balancing, session handling strategies, read‑write separation, and database sharding to achieve scalable, high‑availability backend architectures.
The article outlines the key characteristics of large-scale websites and traces their architectural evolution from single‑server setups to multi‑tier, cache‑enhanced, clustered, and distributed systems, highlighting strategies such as load balancing, database read/write separation, CDN usage, NoSQL adoption, and service‑oriented decomposition.
This article explains the common misconceptions about software architects, outlines the essential conditions and skills required for senior developers to upgrade, describes practical learning paths, and details the daily responsibilities and interpersonal abilities needed to succeed as an architect.
This article provides a comprehensive technical overview of large distributed website architecture, covering performance, high availability, scalability, security, and agility, and illustrates the evolution, design patterns, and practical optimization techniques for modern e‑commerce platforms.
This article outlines the four major stages of IT architecture evolution—from single‑machine setups, through dual‑machine hot‑standby, multi‑node active clusters, to fully distributed systems—explaining the motivations, challenges, and technologies that drive each transition.
This article introduces core architecture analysis methods—including system engineering, system decomposition, process‑oriented, and object‑oriented approaches—and compares monolithic (ALL IN ONE) and distributed system structures, providing practical examples and visual diagrams.
The article outlines the four major stages of enterprise IT architecture—single‑machine, dual‑machine hot‑standby, multi‑node active‑active, and distributed architectures—explaining their motivations, advantages, limitations, and how businesses should choose the appropriate model based on performance, availability, and scalability requirements.
This article details Meituan's practical approaches to boosting account service reliability, covering MTBF/MTTR metrics, business‑level monitoring, flexible availability with circuit‑breaker patterns, cross‑region active‑active deployment, data synchronization techniques, and the measurable performance gains achieved.
This article explains how Weibo handles massive daily traffic of over a hundred billion requests by employing a five‑layer feed system and a six‑layer cache architecture that evolved from simple KV storage to sophisticated counter and existence‑check services, highlighting design choices, performance optimizations, and future directions.
This article traces the historical development of distributed architecture from early computers to modern large‑scale web systems, explains core concepts such as clustering, replication, and middleware, and outlines eight evolutionary stages that address scalability, reliability, and performance challenges.
