Stack Overflow Architecture and Operations: Scaling, Performance, and Infrastructure Overview

Status

110 Stack Exchange sites, growing 3‑4 per month.

4 million users, 8 million questions, 40 million answers.

Peak traffic 2 600‑3 000 requests per second.

25 servers host the entire platform, with 2 TB of SSD‑backed SQL data.

Web servers run IIS; load balancing via HAProxy; 4 active SQL nodes.

ElasticSearch, Redis, and tag‑engine servers support search and caching.

Platform

ElasticSearch

Redis

HAProxy

MS SQL Server

Opserver

TeamCity

Jil – fast .NET JSON serializer

Dapper – micro‑ORM

UI

Inbox notifications via WebSockets backed by Redis.

Search powered by ElasticSearch with a REST API.

Tag‑based recommendation engine to surface relevant questions.

Server‑side templates generate pages.

Servers

25 servers are far from saturated; only 5 are needed for Stack Overflow alone.

Database servers run at ~10 % CPU thanks to 384 GB RAM.

Vertical scaling meets current load; horizontal scaling would require 100‑300 servers.

.NET codebase consists of only 9 projects and ~110 k lines of code.

Data centers: Windows Server 2012/2012 R2 in New York, CentOS 6.4 for Linux.

SSD

Intel 330 SSDs for web tier, Intel 520 for middle‑tier writes, Intel 710/S3700 for data tier.

RAID 1 and RAID 10 used; thousands of 2.5" SSDs with spare drives.

ElasticSearch benefits heavily from all‑SSD storage.

High Availability

Active‑passive data centers (New York & Oregon) with replicated services.

Redis, SQL, Tag Engine, and ElasticSearch each have multiple nodes.

SSL termination via Nginx, then HAProxy.

Database

MS SQL Server per site, with primary‑read‑only replica in each data center.

Schema changes require coordinated multi‑step migrations.

Tag Engine runs as a dedicated Windows service with low CPU usage.

Dapper provides fast, lightweight data access.

Coding

Developers work remotely, compile quickly, and run minimal tests.

Feature flags hide new functionality until validated.

Heavy use of static classes/methods for performance.

Multiple monitors boost developer productivity.

Cache

Five‑tier caching strategy: browser/CDN, .NET HttpRuntime, Redis, SQL Server cache, SSD.

Static methods and Dapper back the cache layer.

Deployment

Five deployments per day, automated via Puppet/DSC.

Rolling updates performed by disabling a server in HAProxy, copying files with Robocopy, then re‑enabling.

Collaboration

SRE (5), Core Dev (6‑7), Mobile Core (6), Careers team (7).

DevOps tightly integrated with developers; most staff remote.

Budgeting

Budget focuses on infrastructure; many servers are legacy purchases with low utilization.

Testing

Fast iteration, limited unit tests due to static code base.

Integration and UI tests run on meta.stackexchange before public release.

Regular disaster‑recovery drills using redundant systems.

Monitoring / Logging

Logstash under evaluation; syslog forwarded to SQL.

Opserver and Realog (Go‑based) display metrics and logs.

HAProxy forwards logs via syslog, not IIS.

About Cloud

Stack Overflow prefers on‑prem hardware for cost and performance reasons.

Cloud would increase expense for comparable performance.

Performance First

Home page loads in ~28 ms; target <50 ms.

CPU utilization stays below 15 % on web servers and 10 % on SQL servers.

Low resource usage leaves ample headroom for upgrades and failures.

Lessons Learned

Choose the right tool for the job (e.g., Redis on Linux, IIS on Windows).

Over‑provisioning for rare peaks provides safety.

All‑SSD storage eliminates latency.

Understand read/write patterns to size hardware appropriately.

Efficient code reduces hardware needs.

Custom tag engine enables complex queries.

Do only what is necessary; avoid unnecessary abstraction.

Focus on low‑GC, static‑heavy code for performance.

Continuously improve tooling to reduce friction.