From Google’s Graphd to Dgraph: Building Distributed Graph Database Systems

Author ManishRai Jain, founder of Dgraph Labs, shares his experience building graph database systems from his time at Google to the creation of Dgraph.

He explains why Google needed a new graph database service capable of handling knowledge‑graph data and high‑throughput search queries, describing the limitations of existing systems such as Bigtable, SSTable, and Borg.

Google’s internal project Graphd, originally powering Freebase, ran as a single‑process daemon with all data in memory and required machines with >64 GB RAM. Attempts to rewrite Graphd for distributed operation led to projects like MindMeld and eventually the Dgraph system.

Dgraph was designed as a distributed graph‑database service that stores SPO triples, shards data by predicate, and executes arbitrary‑depth traversals with at most two network round‑trips, eliminating the fan‑out broadcast problem that plagues many non‑native graph stores.

The article also describes the “Cerebro” knowledge‑graph engine built on top of Google’s search index, its ability to assign IDs to triples, generate multiple interpretations of queries, and provide rich filtering and sorting based on entity types.

It discusses the connection‑depth issue in distributed queries, the latency impact of broadcasting across many servers, and how Dgraph’s predicate‑based sharding mitigates this problem.

Additional projects such as Plasma (a real‑time graph index unifying Google OneBox services) and the challenges of management turnover are covered, illustrating the evolution from internal prototypes to the open‑source Dgraph product.

Overall, the piece offers a historical and technical perspective on the design decisions that shaped modern distributed graph databases.