How Qihoo 360 Scaled MongoDB to 1500 Instances and 200 Billion Daily Queries

360, as the largest domestic MongoDB user, now operates over 2400 instances, 150 TB of data, and 30 billion daily accesses, with large‑scale MongoDB 3.x deployments.

100+ applications, 1500+ instances, 200 billion queries per day

Qihoo 360 is China’s leading Android mobile distribution platform and a top malware protection company, serving both web and mobile. Since adopting MongoDB in 2011, it has built more than 100 applications across 1500 instances, supporting roughly 200 billion queries daily.

Interview with senior DBA Yang Yanjie reveals why and how they use MongoDB, their scaling best practices, and advice for newcomers.

Introduce Qihoo 360. By June 2014 the company had about 500 million monthly active PC internet users and over 640 million mobile users, offering cloud‑based security products and services.

Its market position includes:

Top three Chinese internet companies by user base.

Largest Android mobile distribution platform in China.

Largest internet and mobile malware protection provider.

Second‑largest PC search engine.

When did Qihoo start using MongoDB? Early adoption in 2011, initially version 1.8.

How is MongoDB used today? MongoDB is now a standard modern database platform supporting over 100 applications, both external services and internal business systems.

In total, the internal “HULK” cloud platform runs more than 1500 MongoDB instances handling 200 billion queries per day.

Three especially critical applications are:

Location‑based mobile search. Uses MongoDB geospatial indexes to serve 1.2 billion queries daily, delivering nearby restaurant, shop, or dealer results based on user location.

User authentication cache. Supports single sign‑on for millions of concurrent users, handling 30 k operations per second and 1.8 billion queries daily.

Log analysis platform. Streams logs from Linux, Apache, and Tomcat servers into MongoDB. Stores 2.5 billion documents across 18 shards, processes nearly 3 billion queries and 1 billion writes daily.

Other databases used? MongoDB is one of three databases in the company; MySQL handles relational data and Redis serves specific caching needs. Several projects have migrated from MySQL and Redis to MongoDB.

Factors driving migrations? For MySQL, scalability and developer productivity; MongoDB’s automatic sharding and flexible data model enable horizontal scaling and faster iteration. For Redis, cost and flexibility; MongoDB meets low‑latency cache requirements while offering richer document features.

MongoDB platform details. Most applications are PHP‑based, running on x86 hardware with CentOS and standardized SSD storage. Versions 2.4, 2.6, and anticipation of 3.0 are in use.

Configuration. Both single replica sets and sharded clusters are deployed, depending on the application. Data centers are distributed nationwide, primarily in Beijing, with private‑cloud deployments for disaster recovery and latency reduction.

Management. The in‑house “HULK” cloud platform provides a centralized web interface for engineers to control critical infrastructure. It automates deployment, upgrades, monitoring, and backup, allowing a small DBA team to manage over 1500 instances.

Backup methods. Depending on RPO/RTO goals, they use:

Filesystem snapshots by stopping a secondary replica set member.

Incremental replication using an oplog‑tracking tool for faster restores.

Delayed replication for additional safety.

Scaling best practices.

DBAs should understand application usage patterns and add roughly 50 % capacity beyond current metrics.

Address performance issues first with hardware upgrades, such as moving to SSDs.

For write‑intensive workloads, regularly monitor storage fragmentation and compact as needed.

Measuring business impact. MongoDB enabled rapid response to the 2014 Yunnan earthquake: an application was designed in the morning, coded by afternoon, and released that evening—only a single workday from concept to production.

Only MongoDB could support that development speed.

Expectations for MongoDB 3.0. Anticipate document‑level concurrency control, improved write scalability, and compression benefits that reduce storage costs on standardized SSDs.

Advice for new MongoDB users. Leverage the flexible document model wisely: avoid mixing many document types in one collection; instead, separate them into distinct collections and use tools to scan and sample documents for variance, alerting developers when schemas drift.

Thank you to Mr. Yang for sharing insights with the MongoDB community.

Original links: Chinese | English