Evolution of Image Server Architecture: From Single‑Node to Distributed File System and CDN

Images are a core component of modern web sites, and large portals inevitably face challenges in storing and serving massive image collections; early architectures often suffered from insufficient capacity planning and limited extensibility.

In the single‑machine era, a simple upload folder under the website directory was used, with relative paths stored in a database (e.g., upload/qa/test.jpg ) and URLs such as http://www.yourdomain.com/upload/qa/test.jpg . This approach is easy to implement but quickly runs into disk‑space limits, cumbersome backup procedures, and synchronization problems when scaling to multiple web servers.

When moving to a clustered environment, a virtual upload directory replaces the physical folder, allowing flexible mapping and basic capacity expansion. However, real‑time file synchronization between nodes becomes a bottleneck; push/pull models or tools like rsync are employed, yet consistency, bandwidth consumption, and latency remain significant concerns.

To avoid continuous sync, the architecture can use shared storage via UNC paths, pointing the virtual directory to a network share. This eliminates inter‑node file replication and enables independent domain names (e.g., http://img.yourdomain.com/upload/qa/test.jpg ), but introduces configuration complexity and a potential single point of failure if the storage lacks redundancy.

Separating the image service onto its own server or domain brings several advantages: it reduces load on web/app servers, simplifies scaling and disaster recovery, improves caching and load‑balancing options, and eases CDN integration.

The current production solution combines a distributed file system (FastDFS) with commercial CDN services. Legacy images are migrated with rsync, old upload endpoints are disabled, and ACL rules on a front‑end load balancer (HAProxy/Nginx) route legacy URLs to dedicated image servers. CDN CNAME records direct traffic to the nearest edge node, which caches images and serves them efficiently.

Key operational considerations highlighted include capacity planning, data synchronization and redundancy, hardware cost versus reliability, appropriate file‑system selection (ext3/4, NFS, GFS, etc.), acceleration strategies (CDN or proxy caches), and maintaining compatibility with existing image paths while ensuring security and performance.