Facebook Configurator: Managing Configuration as Code

Facebook uses a suite of internal tools to manage configuration for a wide range of scenarios, including feature flags, experiments, traffic control, topology, monitoring, and machine‑learning model updates.

The core toolset consists of Configerator (providing version control, authoring, code review, automated canary testing, and distribution), Gatekeeper (rolling out new features), Package Vessel (distributing large files such as GB‑scale models), and Sitevars (a thin shim for PHP‑based front‑end products).

Configerator treats configuration as code. Engineers write Python files such as create_job.cinc and cache_job.cconf to manipulate Thrift objects, then export them as JSON. A validator written in Python ( job.thrift-cvalidator ) checks invariants before the JSON is generated.

All configuration source files and generated JSON are stored in a version‑control system (e.g., Git). Engineers edit the source on a development server, invoke Configerator to compile JSON, and can also trigger changes via a Web UI or an API provided by the mutator component.

Multiple teams (scheduler, cache, security) share reusable modules like app.cconf , firewall.cconf , and app_port.cinc . When a shared dependency changes, Configerator automatically recompiles dependent configurations and commits the updated JSON files, ensuring consistency.

Maintaining configuration code is easier than editing raw JSON because code can be modularized and reused. Dependency declarations (e.g., import thrift() , import python() ) allow the Dependency Service to rebuild affected configurations automatically.

For simple configurations, Configerator offers a UI that generates the necessary code behind the scenes, while Sitevars provides a lightweight key‑value interface for PHP products, including optional PHP‑based checkers to enforce invariants.

To prevent configuration errors, Facebook employs a comprehensive strategy: validators, code review, manual testing, automated integration tests, and canary testing. Engineers can manually deploy a new configuration to a subset of servers for quick verification, then submit the source, JSON, and test results to Phabricator for review.

Approved changes are rolled out via the Canary Service, which runs the new configuration against real traffic in a controlled subset of production machines. Multiple testing phases (e.g., 20 machines, then the full cluster) evaluate health‑check metrics such as click‑through rate before the change is merged into the master repository.