Designing a Scalable Email Management System: Core Principles and Implementation

System Positioning and Core Goals

The target scenario includes enterprise email platforms and public services (e.g., NetEase Mail, QQ Mail) that must support billions of daily send/receive operations and massive concurrent users. Key hard requirements are millisecond‑level response for all operations, no loss or duplicate delivery, seamless cross‑provider delivery, efficient historical mail storage, spam/virus filtering, peak‑time resilience, and full feature stability (attachments, drafts, archives, etc.).

Overall Design Philosophy (Four Underlying Logics)

Read‑write separation: write‑heavy paths (sending, ingestion, storage) run on dedicated clusters, while read‑heavy paths (list, detail retrieval) run on independent clusters.

Asynchronous decoupling: the client receives an immediate success response; actual delivery, persistence, and push are processed in the background.

Hot‑cold data tiering: recent three‑month mails reside on high‑performance disks; older archives move to cheap, high‑capacity storage.

Multi‑level queue throttling: bursty send requests are queued to prevent database overload.

Modular decomposition: sending, receiving, storage, search, anti‑spam, attachment handling, and scheduling are split into independent services for selective scaling.

End‑to‑End Business Flow (Logistics Analogy)

User composes and clicks send (like dropping a parcel at a post office).

Gateway validates format, size, and sensitive content (initial security check).

Message is placed into a queue; the client is immediately told "sent".

Delivery service dequeues the mail, performs deep spam and virus scanning.

Two delivery paths:

Internal: store directly into the recipient’s mailbox repository.

External: forward via standard SMTP to the recipient’s provider.

Recipient pulls new mail or receives a push notification.

Mail lands in hot storage; after a defined period it is migrated to cold archive.

User actions (view, delete, move, search, download) go through the query service.

All operation logs and delivery records are stored asynchronously for later troubleshooting.

Step‑by‑Step Implementation Blueprint

Step 1 – Gateway Layer and Rate Limiting

All web, app, and client (Outlook, Foxmail) requests pass through a unified gateway.

Basic interception includes:

Rate limiting per account (e.g., max emails per minute).

Format validation (email address syntax, total size limits).

Black/white lists for malicious accounts and domains.

Purpose: block malicious traffic at the perimeter.

Step 2 – Microservice Decomposition (7 Services)

Email Sending Service – receives user send requests and pushes them to the queue.

Delivery Scheduling Service – consumes the queue, handles internal routing and external SMTP forwarding.

Anti‑spam & Risk Control Service – spam detection, virus scanning, sensitive‑content filtering.

Email Storage Service – persists mail bodies and metadata in sharded storage.

Email Query Service – provides list, detail, folder filtering, and full‑text search.

Attachment Storage Service – stores attachments separately, supports resumable upload/download and deduplication.

Scheduled Mail Service – manages delayed, recurring, and batch mail tasks.

Step 3 – Queue‑Based Peak‑Shaving

Use high‑throughput MQ (RocketMQ or Kafka) with two queues:

Normal send queue for regular mail.

Dead‑letter queue for failed deliveries (rejected by remote server, timeout).

After user clicks send, only lightweight validation occurs; the mail is enqueued for background processing.

Benefits: simultaneous morning bursts (e.g., 9 am) do not overwhelm the database.

Failure handling: automatic retry up to three times, then move to dead‑letter queue for manual investigation.

Step 4 – Multi‑Tier Storage Architecture

Metadata (sender, receiver, timestamps, folder, read flag, unique ID) stored in a MySQL cluster with sharding by user‑ID hash (e.g., 100 databases × 100 tables each).

Mail body stored in distributed object storage:

Hot data (≤3 months) on high‑performance nodes (MinIO/OSS).

Cold data (>6 months) on low‑cost archival storage, compressed.

Each mail body gets a unique ID; metadata references this ID for retrieval.

Attachments stored independently, supporting resumable upload, download acceleration, and hash‑based deduplication.

Cache layer (Redis cluster):

Cache recent 200‑mail list per user for fast homepage load.

Cache high‑frequency mutable fields (read status, folder flags) and sync asynchronously to MySQL.

Local JVM cache for hot active users.

Full‑text search via Elasticsearch: asynchronous sync of subject and body on new mail, periodic sync for cold archives; search returns matching mail IDs which are then fetched from storage.

Step 5 – Anti‑Spam & Risk Control

Three‑layer interception:

Rule‑based blacklist/keyword filtering.

Bayesian model that learns from user‑marked spam.

Third‑party RBL (real‑time blackhole list) integration.

Step 6 – External SMTP Gateway

Dedicated SMTP cluster with multiple IPs rotating to avoid single‑IP blacklisting.

Per‑provider configuration for delivery rules and anti‑spam policies.

Record delivery receipts and bounce reasons; surface bounce info to the user’s spam folder.

Step 7 – Push Notification & Read‑State Sync

Long‑connection push for online users (real‑time new‑mail alerts).

Offline users fetch unread list on next login.

Read, move, delete actions first update Redis, then asynchronously persist to MySQL.

Step 8 – Scheduled Tasks & Background Services

Scheduled mail execution via a task queue.

Automatic archival: move mails older than three months to cold storage during low‑traffic windows.

Automatic cleanup: delete trash after 30 days.

Log persistence for all send, delivery, and delete events for traceability.

Step 9 – High‑Availability Safeguards

All services run in clustered mode; node failure is tolerated.

MQ persistence ensures no loss of enqueued mails on restart.

MySQL master‑slave setup: writes to master, reads from slaves.

Degradation plan: if Elasticsearch fails, fallback to basic list view without full‑text search.

Technology Stack Summary

Gateway & Rate Limiting: Nginx.

Asynchronous Throttling: RocketMQ/Kafka.

Data Storage:

Metadata – MySQL sharding.

Body & Attachments – Distributed object storage (MinIO/OSS).

Cache – Redis cluster.

Search – Elasticsearch.

Business Services: Spring Cloud microservices.

SMTP Delivery: Independent SMTP gateway cluster.

Security: Anti‑spam service, virus scanning, IP black/white lists.

Reliability: Master‑slave DB, dead‑letter queue, retry mechanisms, audit logs.

Typical High‑Frequency Issues and Solutions

Peak‑time concurrency spikes – queue all send requests, enforce per‑account rate limits, scale delivery nodes horizontally, schedule capacity expansion during morning peaks.

Mail loss – generate a global unique ID per mail, log every stage, move failed mails to dead‑letter queue with 7‑day retention, provide one‑click resend based on ID.

Slow pagination for massive history – hot‑cold tiered storage, per‑user sharding, Redis cache for recent list, archive queries only when needed, folder‑specific indexes.

Spam influx – three‑layer filtering (rules, Bayesian model, RBL), user‑driven labeling to improve model.

External IP blacklisting – pool multiple outbound IPs, rotate sending, integrate with anti‑spam alliances, switch to backup IPs on high bounce rates.

Large attachment latency – separate attachment storage with resumable upload, hash‑based deduplication, temporary links that expire after 30 days, asynchronous upload processing.

Full‑text search slowdown – dedicated Elasticsearch cluster, time‑based index shards, exclude cold archives from real‑time index unless explicitly requested.

MySQL table bloat – hash‑based sharding, read‑write splitting, archive old metadata to a separate archive DB.

Duplicate delivery – idempotent identifiers in queue, unique constraints on metadata insertion, spaced retry intervals.

Scheduled‑mail burst – time‑slice execution, persist scheduled tasks, queue them like normal sends during peak periods.