From Zero to One: Building a Data Center Network—Topology, Equipment, Infrastructure, and Server Selection

A friend from a traditional industry switched to an internet company and was tasked with delivering a 200‑node data center within six months, prompting the need to draft a network architecture diagram.

What a Data Center Network Looks Like

The network interconnects thousands of servers and has four key characteristics: east‑west traffic dominates north‑south traffic, latency‑sensitive workloads (AI training, distributed storage, high‑frequency trading) require microsecond‑level performance, 24/7 availability demands redundancy, and the architecture must scale smoothly from hundreds to thousands of nodes.

Topology: Traditional Three‑Tier vs. Spine‑Leaf

1. Traditional Three‑Tier

Historically, data centers used a core‑aggregation‑access three‑tier design. While simple for small scales, it suffers from high bandwidth convergence ratios, uneven latency (minimum five hops between servers), and poor horizontal scalability because adding a rack requires changes to aggregation, core, STP, and VRRP.

2. Spine‑Leaf (Clos‑derived) Flat Architecture

Modern data centers adopt a two‑layer Spine‑Leaf topology where each leaf (Top‑of‑Rack switch) connects to every spine switch, eliminating hierarchical hops. Benefits include constant two‑hop latency, ample bandwidth (e.g., 4 spines × 32 × 100 Gbps = 12.8 Tbps), and simple expansion by adding leaves or spines.

3. Underlay + Overlay

The physical underlay (IP routing via BGP, OSPF, or IS‑IS) carries traffic between VTEPs, while the overlay (VXLAN + EVPN) provides virtual networks for tenant isolation, cross‑data‑center extension, and SDN‑driven automation.

Network Equipment

Typical devices include:

Core (Spine) Switches : High port density (32 × 100 Gbps or 64 × 400 Gbps), non‑blocking backplane, VXLAN/EVPN support. Examples: Huawei CloudEngine 16800/16800‑X, Cisco Nexus 9500/9300, H3C S12500R, Ruijie RG‑S6510.

Leaf (Access) Switches : ToR devices with 10/25 Gbps downlinks, 4 × 25 Gbps or 2 × 100 Gbps uplinks, VXLAN support. Examples: Huawei CloudEngine 5800, H3C S5560/S6520, Cisco Nexus 9300‑EX/FX, Ruijie RG‑S6500.

Edge Routers : Provide BGP, QoS, traffic cleaning for north‑south traffic. Examples: Huawei NetEngine 8000/AR, Cisco ASR 9000/8000, H3C MSR/SR.

Firewalls : Ensure east‑west micro‑segmentation and north‑south perimeter security. Examples: Huawei USG 6000E/12000, Sangfor SG‑6000, Palo Alto PA‑5200, F5 BIG‑IP.

Load Balancers & SDN Controllers : Distribute inbound requests and automate configuration. Examples: F5 BIG‑IP, A10, Huawei ELB, Cisco ACI, Huawei Agile Controller.

Data Center Infrastructure (the "Water‑Electric‑Gas")

1. Tier Classification

According to TIA‑942, tiers range from I (single power, no redundancy) to IV (2N+1 full redundancy). Most large Chinese internet IDC projects target Tier III or the domestic A‑class standard.

2. UPS

Online double‑conversion UPS with >30% spare capacity, >15 min full‑load runtime, and sufficient kVA rating.

3. Precision Cooling

Cold‑aisle/hot‑aisle containment, cooling capacity >0.9 COP, temperature 18‑27 °C (optimal 22‑25 °C), N+1 redundancy; high‑density racks may use liquid‑cooling plates to achieve PUE < 1.1.

4. Racks and Cabling

Standard 19‑inch × 42U racks with structured cabling: OM3/OM4 multimode or OS2 single‑mode fiber for backbone (10 Gbps‑400 Gbps), DAC or AOC for server‑to‑ToR links, MPO/MTP for spine‑leaf interconnects. Vendors: Corning, CommScope, Panduit, Rosenberger.

5. Monitoring & Fire Suppression

DCIM systems collect UPS, HVAC, PDUs, access control, temperature, humidity, and smoke data, delivering alerts via SMS/WeChat. Gas‑based fire suppression (e.g., FM‑200) protects equipment without damage.

Server Selection

Server choice determines long‑term stability. The article lists major vendors and representative models:

Dell PowerEdge : R760, R770, XE9680 (GPU‑heavy), R660.

HPE ProLiant : DL380 Gen11, DL360 Gen11, DL325 Gen11.

Inspur : NF5280M6, NF5468M6, NF5688M6, NF8260M6.

H3C : UniServer R4900 G5, R5200 G5, R6700 G3.

Huawei : TaiShan 200 (Kunpeng), FusionServer 2288H V5/V6, Atlas 800.

Lenovo : ThinkSystem SR650 V3, SR850 V3.

Sunway & Sugon : I620‑G30/I840‑G30, FusionServer 2288H V6/5288 V6.

Each vendor’s strengths and typical use cases are noted (e.g., Dell’s full product line and iDRAC, HPE’s iLO, Inspur’s price‑performance, Huawei’s full‑stack autonomy).

Conclusion

Building a data‑center network requires coordinated planning of network topology, equipment, power, cooling, cabling, and servers. Spine‑Leaf with VXLAN overlay is the de‑facto standard; equipment choices span multiple vendors; infrastructure components must meet redundancy and efficiency goals; and server selection should align with workload, budget, and long‑term support considerations.