Renren’s Middle‑Layer Architecture: Design, Technologies, and Performance Optimizations

Data Layer Renren stores primary data in MySQL while also using Tokyo Cabinet, a key‑value engine, for redundancy. Because Tokyo Cabinet lacks network capabilities, Tokyo Tyrant provides a network interface that forwards API calls to the cabinet for persistent storage.

Server Stack The front‑end uses Nginx as a reverse‑proxy to handle cross‑IDC requests, paired with the Java server Resin (instead of Tomcat) to overcome Resin’s weaker network handling. Squid serves as an image reverse‑proxy cache, and LVS (Linux Virtual Server) provides layer‑4 load balancing, replacing many hardware load balancers.

Frameworks Java web development relies on Struts, while the search service is built on Lucene. Netty, a high‑performance Java network framework, replaces Apache Mina for lightweight services. System monitoring is performed with Ganglia, a scalable distributed monitoring system.

Application‑Level Caching Memcached is used as an in‑memory key‑value cache with the spymemcached Java client. ICE, a cross‑language communication framework, provides richer cache operations: instead of treating a cache entry as an opaque binary blob (as Memcached does), ICE allows field‑level updates, reducing serialization overhead.

Renren’s Middle‑Layer: Problem Overview Renren’s entire stack is built from open‑source software (Nginx, Resin, Memcached, MySQL, Linux). To alleviate database bottlenecks, a middle‑layer service sits between MySQL and Resin, offering high‑concurrency, low‑cost data access.

Database Bottleneck and Optimization Through stages of SQL tuning, business splitting, vertical and horizontal sharding, a single MySQL instance can handle about 3,000 primary‑key queries per second. Adding the middle‑layer further boosts throughput without adding more servers.

Performance Goals and Results The middle‑layer reduces request latency to under 20 ms for 99.9 % of operations. Write traffic for user profiles reaches 500 writes/s and 20,000 reads/s; caching these operations keeps response times well within the target. Friend‑relationship checks (900 reads/s) are served from a 6 GB in‑memory store in ~2 ms. Aggregated page loads stay under 100 ms.

Memory‑Instead‑of‑Disk Strategy By keeping hot data in memory, Renren avoids disk I/O bottlenecks. The middle‑layer pre‑computes and caches results that would otherwise require expensive join queries.

ICE for Communication and Deployment ICE handles service discovery via IceGrid, allowing clients to locate services through a single address and enabling seamless migration of services across servers. Over 500 ICE‑based middle‑layer services are currently running.

Custom In‑Memory Data Structures Renren implements specialized caches, such as a bit‑vector storing user activation states (200 MB holds the full integer range), providing extremely fast queries and updates.

Use Cases The middle‑layer supports multiple friend‑list sorting orders, real‑time online status flags, and other dynamic data that require frequent updates. These use cases illustrate how the layer provides flexible, high‑speed data access across Renren’s services.