Scaling DaVinci Resolve’s Database to Millions of Projects: Evolution and Lessons

Overview

DaVinci Resolve uses a PostgreSQL‑backed Project Library to allow multiple users to collaborate on video editing, color grading, and audio processing in real time.

Evolution Path

2019 – DaVinci Resolve Project Server : Monolithic architecture caused single‑point failures, no automated backups, difficult permission control, and maintenance challenges.

April 2023 – Docker containerization : Docker introduced new drawbacks; after six months the team evaluated alternatives and moved to a high‑availability cluster.

January 2024 – High‑availability cluster : After comparative selection, the open‑source Pigsty solution was adopted. The cluster runs one primary and two replicas; on primary failure a replica is promoted and the failed node isolated. Dual‑server virtual machines provide hardware redundancy, enabling automatic failover within milliseconds, perceived by users as a brief interruption.

Technical Challenges

DaVinci Resolve is a closed‑source application; internal logic cannot be modified, so optimization must occur at the database layer. The software omits advanced PostgreSQL features such as connection pooling and read‑only permissions, complicating management.

Point‑In‑Time Recovery (PITR)

Implemented ability to restore the database to any point within the past week. Editors provide the desired timestamp; DBA rolls back the data to a dedicated recovery database within minutes.

Permission Governance

LDAP integration with the company’s SSO aligns DaVinci Resolve accounts with corporate identities. Credentials match, reducing separate password maintenance. Regular roles follow the least‑privilege principle, allowing only connect, insert, update, delete. All actions are audited for traceability.

Incident Experience

On 2024-05-08 at 11:35 AM, multiple post‑production staff reported loss of database connectivity.

11:36 – All nodes offline; LDAP and other dependencies healthy.

11:43 – Logs showed numerous etcd “capacity full” and service‑stop alerts.

11:46 – Manual fragment cleanup performed to free space.

11:50 – etcd and HAProxy services restarted.

11:52 – Nodes came back online and service recovered.

Root cause: etcd storage quota exhausted because automatic compaction was disabled and the default capacity was only 2 GB. The control plane failed, but the data plane remained unaffected.

etcd is a distributed, reliable key‑value store used by Pigsty as the DCS (distributed configuration store) for PostgreSQL high‑availability state.

Launch Benefits

Zero data loss.

99.99 % service availability.

Approximately ten successful point‑in‑time rollbacks of accidental operations.