Understanding Snuba’s Data Model: Logical vs Physical Architecture

Snuba Data Organization

Snuba stores data in two orthogonal layers: a logical model that the Snuba client sees via the query language, and a physical model that maps to underlying database concepts such as tables and views.

This separation lets Snuba expose a stable interface while internally performing complex mappings to improve query performance for the client.

Dataset

A Dataset is a namespace for Snuba data, providing its own schema and being independent of other datasets in both logical and physical models.

Examples of datasets are discover, outcomes, and sessions, which have no relationship to each other.

Entity and Entity Types

In the logical model, the fundamental building block is an Entity, representing an instance of an abstract concept such as a transaction or error. An Entity corresponds to a row in a database table.

An Entity Type is the class of entities (e.g., Errors, Transactions). Each entity type has a schema defined by a list of fields with associated abstract data types.

The logical model consists of a set of Entity Types and their relationships.

Relationships Between Entity Types

Entity Set Relationship : mimics foreign keys and currently supports one‑to‑one and one‑to‑many connections.

Inheritance Relationship : models sub‑typing; a group of entity types can share a parent type and inherit its schema.

Entity Types and Consistency

An Entity Type is the largest unit in Snuba that can provide strong consistency guarantees, such as Serializable Consistency. Consistency does not extend across multiple entity types, which then fall back to eventual consistency. This also affects subscription queries, which can operate on only a single entity type.

Note: The consistency unit may be even smaller, depending on how ingestion topics are partitioned (e.g., by project_id ).

Storage

Storage

defines the physical data model for a Dataset. Each storage maps to a concrete database concept such as a table or materialized view and has a schema that reflects the underlying DB structure. It can generate DDL statements to create the tables.

Every Entity Type must be associated with at least one readable storage (which can be queried) and exactly one writable storage (used for ingestion). A storage supports only one entity type, while an entity type may be backed by multiple storages to enable query optimization.

Examples

Single‑Entity Dataset

The Outcomes dataset contains a single entity type representing outcomes. Raw outcome queries are slow, so two storages are used: a raw storage for ingested data and a materialized view that aggregates data hourly for faster queries. The query planner selects the appropriate storage based on whether the query can be satisfied by the aggregated view.

Multi‑Entity‑Type Dataset

The Discover dataset includes three entity types: Errors, Transactions, and a parent Events type from which the others inherit. Queries on the Events entity type return the union of Transactions and Errors, but only fields common to both are available.

For performance, the Errors entity type is backed by two storages: a primary errors storage for ingestion and a read‑only view that reduces ClickHouse load at the cost of weaker consistency. The Transactions entity type has a single storage and a merge table that serves the Events view.

Connected Entity Types

A simple dataset example shows how multiple entity types can be joined in a query. For instance, GroupedMessage and GroupAssignee can be left‑joined with Errors as part of the query, while the rest of the query follows the same pattern described earlier.