Tagged articles

distributed systems

2274 articles · Page 6 of 23
Ctrip Technology
Ctrip Technology
May 17, 2024 · Backend Development

Comprehensive Log Governance and Mining Solution for Distributed Systems

This article presents a comprehensive log governance and mining solution for distributed systems, covering background challenges, usage scenarios, and detailed strategies such as distributed log integration, front‑back end traceability, standardized log management, large‑payload handling, efficient cleaning, and future plans for componentization and sampling.

Log ManagementTraceabilitydistributed systems
0 likes · 16 min read
Comprehensive Log Governance and Mining Solution for Distributed Systems
Cognitive Technology Team
Cognitive Technology Team
May 16, 2024 · Operations

Guide to Building Stability in Distributed Systems

This guide presents comprehensive principles, best practices, and techniques for designing, deploying, and maintaining stable distributed systems, covering fault tolerance, monitoring, capacity planning, incident response, and operational reliability to help engineers achieve high availability.

distributed systemsoperationsreliability engineering
0 likes · 1 min read
Guide to Building Stability in Distributed Systems
Sohu Tech Products
Sohu Tech Products
May 15, 2024 · Backend Development

Design and Implementation of a Simplified RocketMQ SDK

To simplify RocketMQ usage, I designed an internal SDK that abstracts complex client configurations, offering single‑call ordered message sending, per‑message consumption, unified result handling, and enforced consistent subscription settings, thereby reducing developer effort and preventing common integration errors.

JavaRocketMQSDK
0 likes · 6 min read
Design and Implementation of a Simplified RocketMQ SDK
Top Architect
Top Architect
May 15, 2024 · Backend Development

Implementing a Simple Java RPC Framework: Architecture, Service Registration, Serialization, and Proxy Generation

This article explains the principles and implementation of a Java RPC framework, covering service registration with Zookeeper, serialization, compression, network communication via Netty, dynamic proxy generation using reflection and Javassist, and performance considerations, providing extensive code examples for each component.

JavaJavassistNetty
0 likes · 25 min read
Implementing a Simple Java RPC Framework: Architecture, Service Registration, Serialization, and Proxy Generation
Cognitive Technology Team
Cognitive Technology Team
May 15, 2024 · Fundamentals

The Fallacies of Distributed Systems: Understanding Common Network Assumptions

This article revisits the classic “Fallacies of Distributed Systems” introduced by Peter Deutsch, explaining why assumptions such as reliable networks, zero latency, infinite bandwidth, secure and homogeneous communication are false, and offering practical strategies like retries, caching, batching, and security‑first design to build robust distributed applications.

FallaciesNetwork Reliabilitybandwidth
0 likes · 4 min read
The Fallacies of Distributed Systems: Understanding Common Network Assumptions
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
May 11, 2024 · Big Data

Comprehensive Introduction to Apache Kafka: Architecture, Features, and Use Cases

This article provides a detailed overview of Apache Kafka, covering its core characteristics, distributed architecture, key components such as topics, partitions, brokers, producers, consumers, ZooKeeper, and common application scenarios like log collection, event‑driven architecture, real‑time analytics, and monitoring.

KafkaMessage Queuearchitecture
0 likes · 7 min read
Comprehensive Introduction to Apache Kafka: Architecture, Features, and Use Cases
Architect
Architect
May 10, 2024 · Backend Development

Rate Limiting Algorithms: Fixed Window, Sliding Window, Leaky Bucket, Token Bucket, and Distributed Rate Limiting

This article explains the importance of rate limiting in microservice architectures, introduces four basic algorithms—fixed window, sliding window, leaky bucket, and token bucket—compares their advantages and disadvantages, and presents both single-machine and distributed implementations with Go code examples.

algorithmdistributed systemsgolang
0 likes · 29 min read
Rate Limiting Algorithms: Fixed Window, Sliding Window, Leaky Bucket, Token Bucket, and Distributed Rate Limiting
Architect
Architect
May 1, 2024 · Fundamentals

Why ULID Beats UUID: A Deep Dive into Features, Specs, and Python Usage

This article compares UUID and ULID, explains the limitations of UUID versions, details ULID's timestamp‑based, lexicographically sortable design, presents its binary layout and encoding, and shows how to generate and manipulate ULIDs in Python with concrete code examples.

PythonUUIDdistributed systems
0 likes · 8 min read
Why ULID Beats UUID: A Deep Dive into Features, Specs, and Python Usage
Su San Talks Tech
Su San Talks Tech
May 1, 2024 · Backend Development

7 Common Cache Pitfalls and How to Avoid Them in Backend Systems

This article outlines seven typical cache issues—including cache penetration, breakdown, avalanche, data inconsistency, large keys, hot keys, and low hit rates—explains their causes, and provides practical solutions such as parameter validation, Bloom filters, locking, expiration randomization, key sharding, and cache warming to improve system reliability and performance.

BackendRediscache
0 likes · 20 min read
7 Common Cache Pitfalls and How to Avoid Them in Backend Systems
FunTester
FunTester
Apr 28, 2024 · Backend Development

Tackling Data Consistency: Master‑Slave, Master‑Master & Leaderless Architectures

The article examines why distributed systems inevitably face data‑consistency challenges and breaks down three common service‑architecture patterns—master‑slave, master‑master, and leaderless—detailing their replication mechanisms, advantages, drawbacks, and practical solutions such as synchronous, semi‑synchronous, asynchronous replication, quorum handling, node‑failure recovery, and conflict resolution strategies.

Master‑SlaveReplicationdata consistency
0 likes · 14 min read
Tackling Data Consistency: Master‑Slave, Master‑Master & Leaderless Architectures
AntTech
AntTech
Apr 23, 2024 · Databases

The Cloud Era of Databases: Insights from OceanBase Chief Scientist Yang Zhenkun

In his OceanBase developer conference keynote, chief scientist Yang Zhenkun explains how cloud resource pooling enables distributed databases to achieve elastic compute and storage, discusses the evolution of databases, the challenges of transaction processing, and envisions fully shared, on‑demand cloud database services.

Cloud ComputingOceanBasedistributed systems
0 likes · 7 min read
The Cloud Era of Databases: Insights from OceanBase Chief Scientist Yang Zhenkun
ITPUB
ITPUB
Apr 22, 2024 · Backend Development

How Meta Achieves Near‑Perfect Cache Consistency: Lessons from Polaris

This article explains Meta's approach to cache invalidation and consistency, detailing why ultra‑high consistency matters, how their Polaris monitoring system detects and resolves inconsistencies, and provides a simplified Python example that illustrates the underlying mechanisms and challenges.

BackendConsistencyMeta
0 likes · 12 min read
How Meta Achieves Near‑Perfect Cache Consistency: Lessons from Polaris
21CTO
21CTO
Apr 20, 2024 · Backend Development

Why Backend Development Is Harder Today Than 20 Years Ago

Over the past two decades, backend development has shifted from simple server‑side coding to a complex, multidisciplinary field, driven by soaring user expectations, massive system scale, an overwhelming array of technologies, and the need for cloud, container, and cross‑team collaboration skills.

Backenddevelopment trendsdistributed systems
0 likes · 9 min read
Why Backend Development Is Harder Today Than 20 Years Ago
Architects' Tech Alliance
Architects' Tech Alliance
Apr 17, 2024 · Cloud Native

What Is Microservices? Core Concepts, Architecture, and Implementation Explained

Microservices, also known as microservice architecture, is a cloud‑native approach that breaks a single application into loosely coupled, independently deployable services, each with its own tech stack, communicating via REST APIs, event streams, or message brokers, and organized around business capabilities and bounded contexts.

architecturecloud nativedistributed systems
0 likes · 3 min read
What Is Microservices? Core Concepts, Architecture, and Implementation Explained
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Apr 17, 2024 · Backend Development

In-Depth Analysis of Apache RocketMQ Architecture, Operation Principles, and High‑Throughput Mechanisms

This article provides a comprehensive overview of Apache RocketMQ, detailing its core components, producer and consumer workflows, storage strategies, master‑slave synchronization, Raft‑based half‑write and leader election mechanisms, and best‑practice recommendations for high‑throughput, fault‑tolerant messaging systems.

Message QueueRaftRocketMQ
0 likes · 22 min read
In-Depth Analysis of Apache RocketMQ Architecture, Operation Principles, and High‑Throughput Mechanisms
Architect
Architect
Apr 9, 2024 · Backend Development

Microservice Architecture Design Patterns: Overview, Benefits, Drawbacks, and Usage Guidelines

This article provides a comprehensive overview of microservice architecture, detailing its definition, key characteristics, advantages and disadvantages, and presenting ten essential design patterns—including database per service, event sourcing, CQRS, Saga, BFF, API gateway, Strangler, circuit breaker, externalized configuration, and consumer‑driven contract testing—along with their pros, cons, and appropriate scenarios.

Design Patternsbackend architecturedistributed systems
0 likes · 29 min read
Microservice Architecture Design Patterns: Overview, Benefits, Drawbacks, and Usage Guidelines
FunTester
FunTester
Apr 7, 2024 · Backend Development

Generating Globally Unique Identifiers (GUID) for Performance Testing: UUID, Distributed Services, Snowflake Algorithm, and Thread‑Local Techniques

This article explains why globally unique identifiers are needed in performance testing, compares several common solutions such as Java UUID, Redis/Zookeeper distributed ID generators, the Snowflake algorithm, and thread‑local or shared counters, and provides complete Java code examples for each approach.

GUIDJavaUUID
0 likes · 13 min read
Generating Globally Unique Identifiers (GUID) for Performance Testing: UUID, Distributed Services, Snowflake Algorithm, and Thread‑Local Techniques
Architects' Tech Alliance
Architects' Tech Alliance
Apr 6, 2024 · Artificial Intelligence

How ByteDance Scaled LLM Training to Over 10,000 GPUs: Inside the MegaScale System

The article analyzes ByteDance and Peking University's MegaScale system that enables efficient, stable training of large language models on clusters exceeding ten thousand GPUs, detailing algorithmic tweaks, 3D parallel communication overlap, operator optimizations, data‑pipeline improvements, network tuning, and fault‑tolerance mechanisms that together achieve a 55.2% MFU on a 175B model.

GPU clustersLLM trainingMegaScale
0 likes · 15 min read
How ByteDance Scaled LLM Training to Over 10,000 GPUs: Inside the MegaScale System
Architect
Architect
Apr 4, 2024 · Backend Development

Mastering High Availability: 9 Essential Design Techniques for Scalable Systems

The article walks through nine practical techniques—system splitting, decoupling, asynchronous processing, retry, compensation, backup, multi‑active deployment, rate limiting, circuit breaking, and degradation—explaining why each is needed, how they are implemented in real‑world microservice architectures, and what trade‑offs to consider.

circuit breakerdistributed systemsfault tolerance
0 likes · 13 min read
Mastering High Availability: 9 Essential Design Techniques for Scalable Systems
Practical DevOps Architecture
Practical DevOps Architecture
Apr 4, 2024 · Databases

ClickHouse Training Course Overview and Curriculum

This article introduces a comprehensive ClickHouse training program that covers fundamental concepts, architecture, installation, distributed cluster design, data import, performance tuning, and includes a detailed list of 33 video modules and additional recommended reading resources for large‑scale data analytics.

ClickHouseColumnar DatabaseData Analytics
0 likes · 4 min read
ClickHouse Training Course Overview and Curriculum
Su San Talks Tech
Su San Talks Tech
Mar 30, 2024 · Backend Development

7 Real-World Message Queue Patterns Every Backend Engineer Should Know

From asynchronous decoupling to distributed transactions, this article explores seven classic message‑queue use cases—such as peak‑shaving, event buses, delayed tasks, broadcast consumption, and data hub integration—illustrated with real‑world examples from e‑commerce, ride‑hailing, and lottery systems, plus RocketMQ code snippets.

KafkaMessage QueueRocketMQ
0 likes · 13 min read
7 Real-World Message Queue Patterns Every Backend Engineer Should Know
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Mar 26, 2024 · Fundamentals

Mastering Message Queues: Core Concepts, Designs, and Popular MQs

Message Queues (MQ) are essential for scalable, decoupled systems, offering asynchronous communication, reliability, buffering, and high availability; this guide explains MQ fundamentals, characteristics, architectures like point-to-point and publish/subscribe, core components, design considerations, and reviews popular solutions such as RabbitMQ, Kafka, ActiveMQ, RocketMQ, and Pulsar.

Message Queuearchitectureasynchronous communication
0 likes · 9 min read
Mastering Message Queues: Core Concepts, Designs, and Popular MQs
Tencent Cloud Developer
Tencent Cloud Developer
Mar 26, 2024 · Databases

Data Consistency in Distributed Systems: Master‑Slave, Master‑Master, and Leaderless Architectures

The article compares master‑slave, master‑master, and leaderless distributed architectures, explaining how synchronous, semi‑synchronous, and asynchronous replication affect consistency, latency and scalability, and showing that each pattern trades write throughput, conflict‑resolution complexity, and availability against strong data correctness.

Master‑SlaveReplicationdata consistency
0 likes · 15 min read
Data Consistency in Distributed Systems: Master‑Slave, Master‑Master, and Leaderless Architectures
Java Backend Technology
Java Backend Technology
Mar 24, 2024 · Databases

Beyond Caching: How Redis Powers Real‑World Storage and Services

Redis is widely known as a high‑performance cache, but its rich data structures, persistence options, and distributed features enable it to serve as a primary storage engine for many internet services, from user profiles and leaderboards to rate limiting, messaging, and geo‑location, while presenting scalability and reliability challenges.

CachingData StructuresIn-Memory Database
0 likes · 19 min read
Beyond Caching: How Redis Powers Real‑World Storage and Services
dbaplus Community
dbaplus Community
Mar 21, 2024 · Backend Development

Inside DeWu’s Self‑Built Redis: Architecture, Automation & High‑Availability

This article details DeWu's self‑built high‑performance distributed Redis cache system, covering its Proxy‑based architecture, core components like ConfigServer, Redis‑Proxy and Redis‑Server, the automated operations platform for deployment and scaling, as well as monitoring, alerting, stability measures and future roadmap.

Redisdistributed systems
0 likes · 22 min read
Inside DeWu’s Self‑Built Redis: Architecture, Automation & High‑Availability
FunTester
FunTester
Mar 21, 2024 · Operations

How Dynamic Task‑Grabbing Cuts Distributed Batch Jobs from Hours to Minutes

This article presents a detailed case study of optimizing a distributed batch processing system by replacing static shard‑key concurrency with a dynamic task‑grabbing mechanism, dramatically reducing execution time from several hours to under fifteen minutes while maintaining stable resource usage.

Batch Processingdistributed systemsdynamic concurrency
0 likes · 8 min read
How Dynamic Task‑Grabbing Cuts Distributed Batch Jobs from Hours to Minutes
Cognitive Technology Team
Cognitive Technology Team
Mar 17, 2024 · Backend Development

Understanding Idempotence: Definition, Importance, Common Causes, and Practical Backend Solutions

Idempotence ensures that repeating the same operation yields the same result without side effects, a critical property in distributed and transactional systems; the article defines the concept, explains why it matters, lists common causes of non‑idempotent behavior, and presents several practical backend solutions such as unique indexes, anti‑duplicate tables, optimistic and pessimistic locks, tokens, distributed locks, and state machines.

BackendIdempotencedistributed systems
0 likes · 8 min read
Understanding Idempotence: Definition, Importance, Common Causes, and Practical Backend Solutions
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Mar 17, 2024 · Fundamentals

Implementing Log Snapshotting in Raft: A Step‑by‑Step Guide

This article provides a comprehensive tutorial on adding log snapshotting (snapshotting) to a Raft‑based distributed key‑value store, explaining the motivation, the snapshot mechanism, and detailed Go code for generating, transferring, applying, and persisting snapshots to reduce log size and improve performance.

ConsensusGoLog Compaction
0 likes · 15 min read
Implementing Log Snapshotting in Raft: A Step‑by‑Step Guide
Architecture & Thinking
Architecture & Thinking
Mar 15, 2024 · Backend Development

Why RPC Is Essential for Building Scalable IM Clusters: A Beginner’s Guide

This article explains the fundamentals of Remote Procedure Call (RPC), why it is crucial for developing production‑grade instant‑messaging clusters, how it differs from plain HTTP, the typical RPC call flow, common frameworks such as gRPC, Thrift and Dubbo, and practical considerations for high‑throughput distributed systems.

IMRPCbackend development
0 likes · 12 min read
Why RPC Is Essential for Building Scalable IM Clusters: A Beginner’s Guide
dbaplus Community
dbaplus Community
Mar 12, 2024 · Databases

How Didi Scaled Log Search by Replacing Elasticsearch with ClickHouse

Facing PB‑scale daily logs and costly Elasticsearch bottlenecks, Didi redesigned its log‑search architecture by migrating to ClickHouse, detailing the challenges, storage redesign, cluster upgrades, performance optimizations, stability fixes, and the resulting cost reduction and query speed gains.

ClickHouseLog Storagedistributed systems
0 likes · 15 min read
How Didi Scaled Log Search by Replacing Elasticsearch with ClickHouse
Linux Code Review Hub
Linux Code Review Hub
Mar 11, 2024 · Databases

How Didi Built a Next‑Gen Log Storage System with ClickHouse

Didi migrated its massive PB‑scale log data from Elasticsearch to ClickHouse, redesigning storage with separate Log and Trace clusters, optimizing partition and sorting keys, introducing native TCP connectors, and revamping HDFS cold‑hot separation, achieving up to four‑fold query speed gains and 30% lower hardware costs.

ClickHouseFlinkHDFS
0 likes · 15 min read
How Didi Built a Next‑Gen Log Storage System with ClickHouse
DataFunSummit
DataFunSummit
Mar 8, 2024 · Databases

Ant TuGraph Computing Engine Architecture and Applications

Ant TuGraph’s open‑source graph computing engine, led by Fang Zhihong, will be introduced covering its development history, architectural design, technical principles, integrated stream‑batch‑graph processing capabilities, real‑world large‑scale graph use cases, and future roadmap, offering insights into design, implementation, and value.

TuGraphbig datadistributed systems
0 likes · 2 min read
Ant TuGraph Computing Engine Architecture and Applications
Tencent Cloud Developer
Tencent Cloud Developer
Feb 28, 2024 · Backend Development

Comprehensive Guide to Rate Limiting Algorithms and Distributed Rate Limiting Solutions

This guide explains why rate limiting is essential for micro‑service stability, outlines six design principles, details four classic algorithms—fixed window, sliding window, leaky bucket, and token bucket—and compares centralized Redis, load‑balancer cache, and coordination‑service distributed solutions.

algorithmdistributed systemsgolang
0 likes · 30 min read
Comprehensive Guide to Rate Limiting Algorithms and Distributed Rate Limiting Solutions
Aikesheng Open Source Community
Aikesheng Open Source Community
Feb 27, 2024 · Cloud Native

Detailed Overview of LiteIO Architecture, Components, and Volume Lifecycle

This article provides a comprehensive technical overview of LiteIO, describing its core and CSI components, the complete volume lifecycle within Kubernetes, Disk‑Agent responsibilities, common implementation pitfalls, storage‑pool construction methods, and the design of the node‑disk‑controller, scheduler, and CSI modules.

CSIKubernetesLiteIO
0 likes · 13 min read
Detailed Overview of LiteIO Architecture, Components, and Volume Lifecycle
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Feb 27, 2024 · Cloud Native

Mastering Spring Cloud: A Deep Dive into Microservices Architecture

This article explains the fundamentals of microservices, why monolithic applications become limiting as they grow, and how Spring Cloud and its components—such as Eureka, Ribbon, Feign, Hystrix, Zuul, Config, and Sleuth—provide a complete framework for building scalable, resilient distributed systems.

Spring Cloudcircuit breakerdistributed systems
0 likes · 7 min read
Mastering Spring Cloud: A Deep Dive into Microservices Architecture
Java Architect Essentials
Java Architect Essentials
Feb 25, 2024 · Backend Development

Designing a High‑Concurrency Ticket‑Seckill System: Architecture, Load Balancing, and Go Implementation

The article analyzes the extreme‑traffic challenges of the 12306 ticket‑seckill service, presents a layered load‑balancing architecture, compares inventory‑deduction strategies, and demonstrates a complete Go‑based prototype with Nginx weighted round‑robin, Redis stock management, and ApacheBench performance testing.

GoHigh ConcurrencyNginx
0 likes · 19 min read
Designing a High‑Concurrency Ticket‑Seckill System: Architecture, Load Balancing, and Go Implementation
DataFunSummit
DataFunSummit
Feb 20, 2024 · Big Data

BitSail Open‑Source Data Integration Engine: Architecture, New Features, CDC Solutions and Future Outlook

This article introduces ByteDance's open‑source data integration engine BitSail, covering its background, layered architecture, recent feature enhancements, automated testing framework, CDC‑based full‑library synchronization solutions, and future development plans for connectors and real‑time data consistency.

CDCFlinkbig data
0 likes · 12 min read
BitSail Open‑Source Data Integration Engine: Architecture, New Features, CDC Solutions and Future Outlook
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Feb 20, 2024 · Backend Development

Mastering Dubbo: Deep Dive into Architecture, Components, and Best Practices

This article provides a comprehensive overview of Dubbo, a Java RPC framework, covering its necessity, core components, implementation principles, architecture layers, supported protocols, registry options, load‑balancing strategies, and typical application scenarios for building scalable micro‑services.

DubboJava RPCbackend development
0 likes · 9 min read
Mastering Dubbo: Deep Dive into Architecture, Components, and Best Practices
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Feb 8, 2024 · Backend Development

Core Principles of Message Queues: 12 Key Concepts Explained

This article provides a comprehensive overview of message queue fundamentals, covering producers, consumers, brokers, point-to-point and publish/subscribe models, ordering, ACK mechanisms, eventual consistency, transactions, persistence, high availability, and selection criteria for various MQ technologies.

BackendMQMessage Queue
0 likes · 10 min read
Core Principles of Message Queues: 12 Key Concepts Explained
Sanyou's Java Diary
Sanyou's Java Diary
Feb 1, 2024 · Backend Development

Message Queues Unveiled: From Decoupling to Platformization and Core Architectures

This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.

KafkaMessage QueuePulsar
0 likes · 17 min read
Message Queues Unveiled: From Decoupling to Platformization and Core Architectures
Architect
Architect
Jan 31, 2024 · Backend Development

How to Build Distributed Multi‑Rule Rate Limiting with Redis and Spring AOP

This article explains how to implement multi‑rule rate limiting in a distributed Java application using Redis, covering String‑based counters, Zset timestamp storage, Lua scripts for atomic checks, custom @RateLimiter annotations, key generation logic, and an AOP interceptor that enforces the limits.

AOPJavaLua
0 likes · 12 min read
How to Build Distributed Multi‑Rule Rate Limiting with Redis and Spring AOP
Baidu Geek Talk
Baidu Geek Talk
Jan 22, 2024 · Backend Development

Design and Implementation of Baidu's Unified Long‑Connection Service

Baidu’s Go‑based unified long‑connection service delivers secure, high‑concurrency, low‑latency connections for multiple Baidu apps through a four‑layer architecture (SDK, control, access, routing), employing goroutine pooling, two‑layer connection models and binary routing to support tens of millions of concurrent users and million‑level QPS, while simplifying integration and reducing maintenance costs.

High ConcurrencyLong Connectionbackend architecture
0 likes · 21 min read
Design and Implementation of Baidu's Unified Long‑Connection Service
360 Smart Cloud
360 Smart Cloud
Jan 15, 2024 · Big Data

Design and Optimization of the Ozone Distributed Object Storage System

This article presents a comprehensive overview of Ozone, a Hadoop‑based distributed object storage system, detailing its architecture, metadata management, scalability enhancements, small‑file handling, erasure coding, lifecycle policies, and future improvements aimed at boosting performance and reliability for large‑scale unstructured data workloads.

HadoopObject StorageOptimization
0 likes · 15 min read
Design and Optimization of the Ozone Distributed Object Storage System
dbaplus Community
dbaplus Community
Jan 12, 2024 · Operations

How a Financial Firm Built a Scalable Edge‑Stored APM System for Microservices

This article describes how a securities company tackled the challenges of distributed‑system observability by designing and deploying a self‑developed application performance monitoring platform that supports flexible integration, dynamic metric collection, edge storage, and cross‑center synchronization, delivering measurable improvements in monitoring coverage, alert effectiveness, and bandwidth usage.

APMEdge StorageFinancial Technology
0 likes · 16 min read
How a Financial Firm Built a Scalable Edge‑Stored APM System for Microservices
ITPUB
ITPUB
Jan 12, 2024 · Databases

What the 2023 Chinese Government Database Procurement Standard Means for Vendors

The 2023 Chinese government database procurement standard defines unified requirements for government agencies, covering scope, procurement principles, mandatory indicators, response and acceptance forms, and detailed technical criteria for centralized and distributed databases, offering clear guidance for vendors on functional, reliability, security, compatibility, service, and safety expectations.

Technical Standardsdatabasesdistributed systems
0 likes · 15 min read
What the 2023 Chinese Government Database Procurement Standard Means for Vendors
Senior Tony
Senior Tony
Jan 11, 2024 · Backend Development

10 Powerful Redis Use Cases Beyond Simple Caching

This guide explores ten practical Redis scenarios—including login authentication, counters, fan following, leaderboards, anti‑scraping, message queues, browser history, distributed locks, user sign‑in tracking, and website UV statistics—showcasing commands and patterns to boost performance in real‑world applications.

Caching AlternativesData StructuresRedis
0 likes · 12 min read
10 Powerful Redis Use Cases Beyond Simple Caching
Sohu Tech Products
Sohu Tech Products
Jan 10, 2024 · Databases

An Introduction to Redis: Basics, Performance, and Comparison with Memcached

Redis is an open‑source, in‑memory NoSQL database that provides ultra‑fast key‑value storage, rich data structures, persistence, clustering and extensible modules, making it the preferred distributed cache over Memcached, which lacks these features and is now rarely chosen for new projects.

CachingIn-Memory DatabaseMemcached Comparison
0 likes · 12 min read
An Introduction to Redis: Basics, Performance, and Comparison with Memcached
Goodme Frontend Team
Goodme Frontend Team
Jan 10, 2024 · Frontend Development

Essential Frontend Picks: Canvas Collision Detection, CSS Animation Tricks, Design‑to‑Code & Distributed Theory

This curated newsletter shares the latest frontend news and article recommendations, covering JavaScript rising stars, ESLint upgrades, upcoming jQuery 4.0, V8's Maglev compiler, canvas collision detection, CSS animation control, design‑to‑code workflows, distributed theory basics, version‑update popups, and anti‑mistake design principles.

DesignJavaScriptdistributed systems
0 likes · 6 min read
Essential Frontend Picks: Canvas Collision Detection, CSS Animation Tricks, Design‑to‑Code & Distributed Theory
MaGe Linux Operations
MaGe Linux Operations
Jan 7, 2024 · Backend Development

How Zookeeper Guarantees Reliable Session Management with Heartbeats

This article explains Zookeeper's session management mechanism, detailing why TCP alone is insufficient for client liveness detection, how Zookeeper implements its own heartbeat protocol, and the internal data structures and algorithms—including expiryMap and SessionTracker—that efficiently track and expire sessions.

JavaSession ManagementZooKeeper
0 likes · 12 min read
How Zookeeper Guarantees Reliable Session Management with Heartbeats
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Jan 2, 2024 · Backend Development

Cache Design and Optimization Practices for High‑Concurrency Music Library Service

The article details NetEase Cloud Music’s high‑concurrency cache architecture—using lazy‑load, hole‑wrapped objects for penetration protection, placeholder values for missing data, horizontal and vertical scaling with consistent hashing, and asynchronous binlog‑driven invalidation—to achieve sub‑millisecond reads for a read‑heavy, write‑light music library.

Cache invalidationCachingHigh Concurrency
0 likes · 12 min read
Cache Design and Optimization Practices for High‑Concurrency Music Library Service
Architect
Architect
Dec 28, 2023 · Backend Development

How to Implement Distributed Multi‑Rule Rate Limiting with Redis and Lua

This article explains how to design and implement a distributed rate‑limiting solution that supports multiple concurrent rules—such as per‑minute and per‑hour limits—by analyzing the shortcomings of simple string counters, introducing atomic Lua scripts and Zset structures, and providing complete Java annotation and AOP code examples.

AOPJavaLua
0 likes · 13 min read
How to Implement Distributed Multi‑Rule Rate Limiting with Redis and Lua
Amap Tech
Amap Tech
Dec 28, 2023 · Backend Development

Rate Limiting Algorithms and Their Java Implementations

Rate limiting protects system stability by capping request rates, and this article explains five Java algorithms—Fixed Window, Sliding Window, Leaky Bucket, Token Bucket, and Guava's RateLimiter—showing their principles, pros and cons, and providing sample implementations and a Spring @Limit annotation for practical enforcement.

GuavaJavaalgorithm
0 likes · 17 min read
Rate Limiting Algorithms and Their Java Implementations
macrozheng
macrozheng
Dec 28, 2023 · Databases

Mastering Sharding: 21 Core Concepts for Scalable Database Design

This article introduces the essential concepts of sharding architecture—including data partitioning, logical and physical tables, sharding keys, routing strategies, SQL rewriting, distributed primary keys, data masking, transactions, migration, and shadow databases—providing a comprehensive guide for building high‑performance, horizontally scaled database systems.

Broadcast TableSQL Routingdatabase partitioning
0 likes · 21 min read
Mastering Sharding: 21 Core Concepts for Scalable Database Design
System Architect Go
System Architect Go
Dec 26, 2023 · Cloud Native

Distributed Leader Election and Kubernetes Lease

Distributed leader election ensures a single node performs exclusive tasks in high‑availability systems, with common implementations like Etcd, Kafka, Elasticsearch, and Zookeeper using algorithms such as Paxos, Raft, and ZAB, while Kubernetes provides the Lease resource to manage leader election via a distributed lock.

GoPaxosdistributed systems
0 likes · 7 min read
Distributed Leader Election and Kubernetes Lease
DataFunSummit
DataFunSummit
Dec 23, 2023 · Big Data

Common Optimization Points in Building a User Portrait Platform

The presentation by Zhang Xinglong, head of the user understanding center's portrait platform backend, outlines the platform's overall architecture, core module optimizations, advanced crowd creation techniques, bitmap usage, and industry trends, helping the audience understand platform importance, performance improvements, and future directions.

BackendOptimizationPlatform Architecture
0 likes · 3 min read
Common Optimization Points in Building a User Portrait Platform
Alibaba Cloud Developer
Alibaba Cloud Developer
Dec 21, 2023 · Backend Development

Mastering Rate Limiting: Algorithms, Java Implementations, and Guava Tips

This article explains why rate limiting is essential for high‑traffic systems, defines common rate‑limiting algorithms (fixed window, sliding window, leaky bucket, token bucket), provides complete Java code examples for each, and demonstrates practical usage with Guava's RateLimiter in real‑world applications.

BackendGuavaJava
0 likes · 19 min read
Mastering Rate Limiting: Algorithms, Java Implementations, and Guava Tips
MaGe Linux Operations
MaGe Linux Operations
Dec 18, 2023 · Backend Development

How to Prevent Kafka Message Loss in Critical Transaction Systems

This article explains why Kafka can lose messages in production, broker, and consumer stages, analyzes root causes such as asynchronous batch sends, JVM crashes, and network failures, and provides practical solutions including callbacks, retry mechanisms, replication settings, and manual offset commits to ensure reliable delivery.

KafkaMessage reliabilitybackend development
0 likes · 10 min read
How to Prevent Kafka Message Loss in Critical Transaction Systems
dbaplus Community
dbaplus Community
Dec 18, 2023 · Backend Development

How to Prevent Cache Penetration, Avalanche, Breakdown, Inconsistency, and Concurrency Issues

This guide explains common cache problems such as penetration, avalanche, breakdown, data inconsistency, and concurrent access, and provides practical solutions like Bloom filters, multi‑level caching, random expiration, distributed locks, and transaction mechanisms to keep systems stable and performant.

cachedata consistencydistributed systems
0 likes · 13 min read
How to Prevent Cache Penetration, Avalanche, Breakdown, Inconsistency, and Concurrency Issues
DataFunTalk
DataFunTalk
Dec 15, 2023 · Big Data

Zhihu Bridge Platform: Internal Marketing Architecture, Challenges, and Optimizations

This article presents a comprehensive overview of Zhihu's Bridge Platform internal marketing module, detailing its background, business logic, product components such as CDP, activity and delivery platforms, architectural layers, performance bottlenecks, optimization techniques—including distributed transactions, bitmap indexing, and vectorized query execution—and future directions toward marketing automation and intelligence.

CDPDorisPlatform Architecture
0 likes · 28 min read
Zhihu Bridge Platform: Internal Marketing Architecture, Challenges, and Optimizations
Tencent Cloud Middleware
Tencent Cloud Middleware
Dec 12, 2023 · Cloud Native

How Tencent Cloud Implements Tiered Storage for Kafka: Architecture, Challenges, and Evolution

This article examines the challenges of Kafka's traditional architecture, explains why local‑state heavy deployments cause operational difficulty and resource waste, and details Tencent Cloud's elastic, storage‑compute‑separated designs—including tiered storage, segment state machines, offset constraints, and performance optimizations—while sharing practical implementation insights and future directions.

Data LifecycleElastic ArchitectureKafka
0 likes · 17 min read
How Tencent Cloud Implements Tiered Storage for Kafka: Architecture, Challenges, and Evolution
AntTech
AntTech
Dec 11, 2023 · Artificial Intelligence

Ant Group Open-Sources OpenASCE: A Distributed Full-Stack Causal Learning System Presented at NeurIPS

At NeurIPS 2023, Ant Group unveiled OpenASCE, the industry's first open‑source distributed full‑link causal learning system, detailing its architecture, large‑scale capabilities, and real‑world applications in credit risk, marketing, and recommendation while emphasizing its role in advancing causal AI research.

AIAnt GroupNeurIPS
0 likes · 5 min read
Ant Group Open-Sources OpenASCE: A Distributed Full-Stack Causal Learning System Presented at NeurIPS
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Dec 7, 2023 · Backend Development

In-Depth Analysis of XXL‑RPC Framework: Design, Implementation, and Source Code Walkthrough

This article provides a comprehensive overview of the lightweight XXL‑RPC framework, covering fundamental RPC concepts, the framework's architecture built on Spring and Netty, detailed provider and consumer implementations, various call types, and the service registry‑discovery mechanism, concluding with practical insights for developers.

JavaNettyRPC
0 likes · 17 min read
In-Depth Analysis of XXL‑RPC Framework: Design, Implementation, and Source Code Walkthrough
Alibaba Cloud Native
Alibaba Cloud Native
Dec 6, 2023 · Cloud Native

How RocketMQ Implements Random Indexing for Cloud‑Native Storage

This article explains RocketMQ's random indexing mechanism, detailing its on‑disk three‑segment hash table structure, the compact format conversion process, multi‑threaded write and query workflows, layered system design, crash‑recovery strategy, and comparisons with RocksDB and InnoDB storage engines.

Message IndexingRocketMQcompaction
0 likes · 16 min read
How RocketMQ Implements Random Indexing for Cloud‑Native Storage
Architecture Digest
Architecture Digest
Dec 1, 2023 · Databases

Understanding ULID: A Lexicographically Sortable Unique Identifier and Its Python Implementation

ULID (Universally Unique Lexicographically Sortable Identifier) offers a 128‑bit, time‑based and random identifier that is URL‑safe, sortable, and more collision‑resistant than UUID, with detailed specifications, binary layout, and Python usage examples including generation, conversion, and component extraction.

PythonUUID alternativedistributed systems
0 likes · 7 min read
Understanding ULID: A Lexicographically Sortable Unique Identifier and Its Python Implementation
Architecture Digest
Architecture Digest
Nov 27, 2023 · Databases

Fast Import of 1 Billion Records into MySQL: Design, Performance, and Reliability Considerations

To import one billion 1 KB log records into MySQL efficiently, the article examines data size constraints, B‑tree index limits, batch insertion strategies, storage engine choices, file‑reading techniques, task coordination with Redis, Redisson semaphores, and distributed lock handling to ensure ordered, reliable, high‑throughput loading.

Batch InsertMySQLRedis
0 likes · 18 min read
Fast Import of 1 Billion Records into MySQL: Design, Performance, and Reliability Considerations
Top Architecture Tech Stack
Top Architecture Tech Stack
Nov 27, 2023 · Operations

Designing Multi-Active Cross‑Region Architecture: Scenarios, Patterns, and Practical Techniques

This article explains the motivations, application scenarios, architectural patterns (same‑city, cross‑city, and cross‑country), and concrete design techniques for building multi‑active cross‑region systems that ensure high availability and graceful degradation during extreme failures.

Multi-Activedata synchronizationdisaster recovery
0 likes · 32 min read
Designing Multi-Active Cross‑Region Architecture: Scenarios, Patterns, and Practical Techniques
Top Architecture Tech Stack
Top Architecture Tech Stack
Nov 26, 2023 · Operations

Understanding High Availability and High Performance: Complexity, Redundancy, and Decision Strategies

This article examines the inherent complexity of achieving high availability and high performance in distributed systems, explaining redundancy techniques, storage consistency challenges, various state‑decision models, and the trade‑offs involved in scaling single‑machine and cluster architectures.

High Performancedistributed systemshigh availability
0 likes · 27 min read
Understanding High Availability and High Performance: Complexity, Redundancy, and Decision Strategies
Architect
Architect
Nov 23, 2023 · Databases

Inside Our High‑Performance Self‑Built Redis System: Architecture, Features & Ops

This article details the design and implementation of a self‑managed Redis KV cache system spanning tens of terabytes, covering its Proxy‑based architecture, ConfigServer high‑availability via Raft, Redis‑Proxy slot routing, async‑fork optimizations, data migration strategies, and a comprehensive automation platform for deployment, scaling, monitoring, and stability governance.

Redisautomationdistributed systems
0 likes · 24 min read
Inside Our High‑Performance Self‑Built Redis System: Architecture, Features & Ops
Top Architecture Tech Stack
Top Architecture Tech Stack
Nov 22, 2023 · Operations

Designing Multi‑Active (Active‑Active) Architecture Across Regions: Scenarios, Patterns, and Practical Techniques

This article explains the motivations, application scenarios, architectural patterns, and step‑by‑step design techniques for building geographically distributed active‑active systems that can survive extreme failures while balancing cost, complexity, and data consistency requirements.

Active-Activedata synchronizationdisaster recovery
0 likes · 32 min read
Designing Multi‑Active (Active‑Active) Architecture Across Regions: Scenarios, Patterns, and Practical Techniques
dbaplus Community
dbaplus Community
Nov 19, 2023 · Backend Development

How YouTube Scaled to 100 Million Daily Views with Just 9 Engineers

An in‑depth look at YouTube’s early scalability strategy reveals how a tiny team of nine engineers built a simple yet powerful tech stack—leveraging MySQL, Lighttpd, Python, commodity hardware, stateless design, replication, partitioning, caching, and strategic outsourcing—to handle billions of daily video views.

YouTubedistributed systemsscalability
0 likes · 10 min read
How YouTube Scaled to 100 Million Daily Views with Just 9 Engineers
Architect
Architect
Nov 16, 2023 · Backend Development

Understanding Transactional Messages in Distributed Systems: RocketMQ and Kafka

This article explains the principles of distributed transaction messages, comparing 2PC, TCC, and transactional messaging, and provides detailed walkthroughs of RocketMQ and Kafka implementations, including their two‑phase processes, broker handling, and source‑code insights for ensuring data consistency in asynchronous systems.

2PCMessage QueueRocketMQ
0 likes · 16 min read
Understanding Transactional Messages in Distributed Systems: RocketMQ and Kafka
Didi Tech
Didi Tech
Nov 14, 2023 · Databases

Didi's Multi-Active Redis Architecture: Design, Challenges, and Solutions

To achieve disaster-recovery and cross-data-center resilience, Didi progressed from a simple proxy double-write scheme to a sophisticated MQ-free multi-active Redis design that uses a dedicated syncer, shard-based loop prevention, op-id replay protection, conflict detection, and incremental AOF durability, ensuring low latency, no data loss, and consistent availability.

DidiMulti-ActiveRedis
0 likes · 11 min read
Didi's Multi-Active Redis Architecture: Design, Challenges, and Solutions
AntTech
AntTech
Nov 10, 2023 · Artificial Intelligence

Ant Group and Tsinghua University’s Distributed Collaborative Risk‑Defense System Wins Zhejiang Provincial Science & Technology Progress Award

The award‑winning distributed collaborative risk‑defense system, developed by Ant Group, Tsinghua University and Alipay, leverages AI, privacy‑preserving computing and graph analytics to achieve real‑time, high‑efficiency detection and invisible, precise control of hidden risks in massive digital transactions, earning top provincial honors and extensive industry adoption.

AI securityAwardPrivacy Computing
0 likes · 5 min read
Ant Group and Tsinghua University’s Distributed Collaborative Risk‑Defense System Wins Zhejiang Provincial Science & Technology Progress Award
dbaplus Community
dbaplus Community
Nov 9, 2023 · Fundamentals

Mastering Java Locks: From Pessimistic to Distributed and Optimizations

This article explains the full spectrum of Java locking mechanisms—including pessimistic, optimistic, distributed, reentrant, spin, read/write, fair vs. non‑fair, JVM lock states, and optimization techniques—detailing their principles, use‑cases, SQL/Redis examples, and performance trade‑offs.

JVMLocksconcurrency
0 likes · 16 min read
Mastering Java Locks: From Pessimistic to Distributed and Optimizations
Architecture Digest
Architecture Digest
Nov 6, 2023 · Backend Development

Using Lua Scripts in Spring Boot with Redis: A Comprehensive Guide

This article explains how to integrate Lua scripts into Spring Boot projects for Redis, covering Lua fundamentals, performance benefits, practical use cases, step‑by‑step implementation with code examples, error handling, security considerations, and best‑practice recommendations for backend developers.

LuaRedisSpring Boot
0 likes · 19 min read
Using Lua Scripts in Spring Boot with Redis: A Comprehensive Guide
Selected Java Interview Questions
Selected Java Interview Questions
Nov 5, 2023 · Backend Development

Design and Implementation of a High‑Performance Distributed Reconciliation System for Large‑Scale Payment Orders

This article presents a comprehensive design of a distributed reconciliation system that handles tens of millions of daily payment orders by using a six‑module architecture, Kafka for decoupled state transitions, Hive for large‑scale data processing, and Java‑based plug‑in patterns to achieve six‑nine accuracy and significant operational cost savings.

HiveJavaKafka
0 likes · 15 min read
Design and Implementation of a High‑Performance Distributed Reconciliation System for Large‑Scale Payment Orders
Bilibili Tech
Bilibili Tech
Nov 3, 2023 · Big Data

Comprehensive Governance and Optimization Strategies for Large‑Scale Kafka Clusters

To tame a petabyte‑scale Kafka deployment of over 1,000 brokers, the team built a Raft‑based federation controller (Guardian) that adds per‑partition I/O throttling, disk‑aware automatic balancing, multi‑tenant isolation, cross‑IDC migration, request‑queue splitting, tiered storage, auditing, and fully automated rolling upgrades, enabling stable, self‑healing operations.

Cluster GovernanceKafkabig data
0 likes · 21 min read
Comprehensive Governance and Optimization Strategies for Large‑Scale Kafka Clusters
Architect
Architect
Oct 25, 2023 · Operations

The Importance of Logging and Distributed Log Operations in Modern Architecture

This article explores why logs are essential in software development, outlines when to record them, discusses the value of logging in large-scale distributed systems, and examines the capabilities required of log‑operation tools such as APM, metrics, tracing, ELK, Prometheus, and custom batch querying solutions.

APMELKPrometheus
0 likes · 21 min read
The Importance of Logging and Distributed Log Operations in Modern Architecture
JD Cloud Developers
JD Cloud Developers
Oct 25, 2023 · Backend Development

Master Kafka: Core Concepts, Architecture, and Practical Tips

This article explains Kafka's fundamentals, including topics, partitions, brokers, replication, producer‑consumer workflow, consumer groups, offset management, and common exception handling, while providing code examples and diagrams to help developers understand and effectively use this distributed messaging system.

JavaKafkaMessage Queue
0 likes · 21 min read
Master Kafka: Core Concepts, Architecture, and Practical Tips
JD Tech
JD Tech
Oct 25, 2023 · Backend Development

Design and Implementation of JD Logistics Order System Architecture for High Scalability and Availability

The article details JD Logistics' order system redesign using a four‑layer transaction architecture, describing its decoupled backend, unified data model, high‑availability components such as CQRS, Redis, JMQ, HBase, and Elasticsearch, and outlines design advantages, extensible data modeling, future challenges, and overall performance outcomes.

backend architecturedata modelingdistributed systems
0 likes · 10 min read
Design and Implementation of JD Logistics Order System Architecture for High Scalability and Availability
DataFunSummit
DataFunSummit
Oct 24, 2023 · Databases

OushuDB: A Cloud‑Native Real‑Time Lakehouse Database – Architecture, Evolution and Practice

This article introduces OushuDB, a cloud‑native real‑time lakehouse database, tracing the evolution of cloud‑native lakehouse architectures, detailing OushuDB’s multi‑engine, multi‑storage design, and sharing practical insights on compute‑storage separation, high‑availability, and integration with Hadoop, Hive and Hudi.

cloud-nativedistributed systems
0 likes · 20 min read
OushuDB: A Cloud‑Native Real‑Time Lakehouse Database – Architecture, Evolution and Practice
Su San Talks Tech
Su San Talks Tech
Oct 22, 2023 · Backend Development

Mastering Rate Limiting: Algorithms, Scenarios, and Practical Implementations

Rate limiting controls request flow to protect system stability, covering its definition, motivations, common algorithms such as token bucket, leaky bucket, fixed and sliding windows, their pros and cons, single‑machine vs distributed implementations, and practical component choices for backend services.

Backendalgorithmdistributed systems
0 likes · 17 min read
Mastering Rate Limiting: Algorithms, Scenarios, and Practical Implementations
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Oct 19, 2023 · Backend Development

Understanding RocketMQ Core Architecture and Its Role in High‑Concurrency Systems

This article explains the core components of RocketMQ—including NameServer, Broker, Producer, and Consumer—detailing their functions, deployment diagrams, and how the middleware supports high‑concurrency scenarios such as peak shaving, asynchronous messaging, fault tolerance, and system decoupling.

Message QueueRocketMQbackend architecture
0 likes · 7 min read
Understanding RocketMQ Core Architecture and Its Role in High‑Concurrency Systems