multi-datacenter

DataFunSummit

Feb 6, 2024 · Big Data

Exploring ByteDance's EB‑Scale HDFS: Architecture, Multi‑Datacenter Challenges, Tiered Storage, and Data Protection Practices

This article presents an in‑depth overview of ByteDance's EB‑scale HDFS, covering its new features, multi‑datacenter architecture, tiered storage implementation, data management services, capacity and fault‑tolerance strategies, as well as practical data‑protection mechanisms and related Q&A.

Big DataDistributed StorageHDFS

0 likes · 22 min read

Exploring ByteDance's EB‑Scale HDFS: Architecture, Multi‑Datacenter Challenges, Tiered Storage, and Data Protection Practices

DataFunTalk

Sep 4, 2022 · Big Data

Design and Implementation of Bilibili's Offline Multi‑Datacenter Solution

This article describes Bilibili's offline multi‑datacenter architecture, explaining why a scale‑out approach was chosen over scale‑up, and detailing the unit‑based design, job placement, data replication, routing, versioning, bandwidth throttling, traffic analysis, and the operational results and future directions.

Big DataHDFSJob Scheduling

0 likes · 24 min read

Design and Implementation of Bilibili's Offline Multi‑Datacenter Solution

Bilibili Tech

Jul 5, 2022 · Big Data

Multi‑Datacenter Architecture for Offline Big Data Processing at Bilibili

To overcome rapid data growth and on‑premise capacity limits, Bilibili adopted a scale‑out, unit‑based multi‑datacenter architecture that isolates failures, intelligently places jobs, replicates data via an enhanced DistCp service, routes reads with an IP‑aware HDFS router, and throttles cross‑site traffic, enabling stable offline big‑data processing of hundreds of petabytes while preserving throughput.

Big DataHDFSYARN

0 likes · 28 min read

Multi‑Datacenter Architecture for Offline Big Data Processing at Bilibili

DataFunTalk

Jul 8, 2021 · Big Data

Design and Evolution of ByteDance's Multi‑Datacenter HDFS Architecture

This article explains how ByteDance extended the Apache HDFS architecture with a multi‑datacenter design, introducing components such as DanceNN, NNProxy, and BookKeeper to achieve scalable storage, cross‑datacenter data placement, and rack‑level disaster recovery for petabyte‑scale workloads.

ByteDanceHDFSbig data storage

0 likes · 13 min read

Design and Evolution of ByteDance's Multi‑Datacenter HDFS Architecture

Youzan Coder

Dec 9, 2020 · Operations

A DevOps Engineer's Journey: From Middleware to Business Operations at YouZan

The article chronicles a YouZan DevOps engineer’s five‑year evolution from Alibaba‑based middleware duties to business‑operation leadership, highlighting the relentless pursuit of system stability through the 1‑minute detection, 5‑minute localization, 10‑minute resolution mantra, complex multi‑datacenter integrations, continuous learning, and a mindset of proactive problem‑solving.

DevOpsOperations EngineeringSaaS

0 likes · 7 min read

A DevOps Engineer's Journey: From Middleware to Business Operations at YouZan

Architecture Digest

May 9, 2018 · Operations

High Availability and Disaster Recovery Architecture: The Evolution of Alipay’s System Design

This article examines the importance of high‑availability and disaster‑recovery architectures, tracing Alipay’s evolution from a simple load‑balanced setup through multi‑datacenter, failover, and unit‑based designs that address scalability, data consistency, and continuous service delivery challenges.

FailoverHigh Availabilitydisaster recovery

0 likes · 16 min read

High Availability and Disaster Recovery Architecture: The Evolution of Alipay’s System Design

Efficient Ops

Jan 31, 2016 · Cloud Computing

How Meizu Scales Cloud Sync for Millions: Protocols, Architecture, and Data Strategies

This article details Meizu's cloud synchronization service, covering its custom MZ‑SyncML protocol, semi‑sync mechanisms, file and one‑sync protocols, failure handling, service architecture, modular design, massive data routing, multi‑datacenter deployment, traffic optimization, and practical lessons learned.

cloud syncdata routingdistributed systems

0 likes · 14 min read

How Meizu Scales Cloud Sync for Millions: Protocols, Architecture, and Data Strategies