Case Study: Architecture and Scaling Strategies of the Poppen.de Social Platform

Poppen.de is a German social networking site that, while smaller than Facebook or Flickr, employs a sophisticated architecture integrating technologies such as Nginx, MySQL, CouchDB, Erlang, Memcached, RabbitMQ, PHP, Graphite, Red5, and Tsung.

Statistics: 2 million registered users, 20 000 concurrent users, 200 000 private messages per day, 250 000 login attempts daily, supported by a team of 11 developers, 2 designers, and 2 system administrators.

Business model: The service follows a freemium approach where basic features (search, messaging, uploading media, video chat, etc.) are free, while unlimited messaging, image uploads, and other premium services require paid memberships.

Toolbox – Nginx : All services run behind Nginx. Two front‑end Nginx servers handle up to 150 000 requests per minute during peaks, each with a single CPU and 3 GB RAM. Three dedicated image Nginx servers serve *.bilder.poppen.de with 80 000 requests per minute. Caching via Memcached reduces load on PHP back‑ends, with Memcached handling 8 000 requests per minute for user profile pages.

Image handling uses three local Nginx image caches; when an image is not present locally, it is fetched from a central file server, cached, and served, reducing load on primary storage.

PHP‑FPM : The site runs on 28 dual‑CPU, 6 GB RAM PHP machines, each running 100 PHP‑FPM workers with APC‑enabled PHP 5.3.x, achieving over 30 % reduction in CPU and memory usage. The application is built on the Symfony 1.2 framework, facilitating rapid development and onboarding.

Performance profiling is done with XHProf, an open‑source library from Facebook, allowing extensive caching of expensive computations.

MySQL : The primary RDBMS consists of a 4‑CPU, 32 GB server storing user data, with plans to migrate to a shared cluster. Data is partitioned by user ID. A master‑slave‑slave setup provides forum services, handling 2.5 billion messages, while an NDB cluster of four nodes serves write‑intensive statistics. Most tables are MyISAM for fast lookups, though full‑table locks are becoming an issue, prompting consideration of XtraDB.

Memcached : Over 45 GB of cache across 51 nodes stores sessions, view caches, and function results. Automatic cache invalidation updates data when records change, with future improvements potentially using Redis Hash API or MongoDB.

RabbitMQ : Introduced in 2009, RabbitMQ runs on two servers behind LVS, handling asynchronous tasks such as logging, email notifications, system messages, and image uploads. PHP‑FPM uses fastcgi_finish_request() to push responses to the queue, allowing users to receive HTML/JSON without waiting for backend cleanup, enabling up to 1 000 login requests per minute during peaks.

CouchDB : Logs are stored in CouchDB on a dedicated machine, enabling module‑level queries and error analysis, replacing the previous manual log inspection across PHP servers.

Graphite : Real‑time metrics (module hits, Memcached hit/miss rates, RabbitMQ status, Unix load, etc.) are collected by Graphite, updating roughly 4 800 times per minute. It monitors two simultaneous Symfony versions, facilitating performance comparison and issue detection using XHProf.

Red5 : Provides two video services—user‑uploaded video and video chat—delivering about 17 TB of traffic per month as of mid‑2009.

Tsung : An Erlang‑based distributed load testing tool used for HTTP benchmarking and comparative analysis of MySQL/XtraDB performance. Recorded traffic is replayed to simulate thousands of concurrent users in a controlled environment.