Sentry Contributor Guide: Essential Testing Techniques

Setting Up Devservices

Acceptance and Python tests require a set of functional devservices. Use the devservices command to start and stop the required containers. The --project flag prefixes container names with test_ so they are isolated from local development services.

# Shut down local test services.
sentry devservices down

# Bring up services with a test prefix.
sentry devservices up --project test

# Verify containers are up.
docker ps --format '{{.Names}}'

# After tests, restart normal services.
sentry devservices down --project test && sentry devservices up

Python Testing

Python tests run with pytest together with Django's test utilities. Core test cases live in sentry.testutils.cases. Endpoint integration tests cover customers, integrations, API behavior across user roles, organizations, and invalid‑data scenarios.

Running pytest

Run tests at different scopes:

# Run an entire directory.
pytest tests/sentry/api/endpoints/

# Run files matching a pattern.
pytest tests/sentry/api/endpoints/test_organization_*.py

# Run a single file.
pytest tests/sentry/api/endpoints/test_organization_group_index.py

# Run a single test method.
pytest tests/snuba/api/endpoints/test_organization_events_distribution.py::OrganizationEventsDistributionEndpointTest::test_this_thing

# Run all tests matching a substring.
pytest tests/snuba/api/endpoints/test_organization_events_distribution.py -k method_name

Common options: -k filters tests by a substring of the method or class name. -s disables stdout capture.

Creating Test Data

Factory helpers in sentry.testutils.factories build organizations, projects, and other Postgres‑backed entities. Store events with store_event() in a test class derived from SnubaTestCase. Always set a past timestamp; otherwise the default "now" may cause boundary‑related selection failures.

from sentry.testutils.helpers.datetime import before_now
from sentry.utils.samples import load_data

def test_query(self):
    data = load_data("python", timestamp=before_now(minutes=1))
    event = self.store_event(data, project_id=self.project.id)

Setting Options and Feature Flags

Use the helper methods self.feature() and self.options() to toggle feature flags and configuration options for a test.

def test_success(self):
    with self.feature('organization:new-thing'):
        with self.options({'option': 'value'}):
            # test logic with feature enabled
    # Disable the feature.
    with self.feature({'organization:new-thing': False}):
        # test logic with feature disabled

Mocking External Services

Use the responses library to stub outbound API requests, allowing simulation of success and failure scenarios.

Reliable Time Handling

Event‑related tests must stay within a 30‑day window. Instead of freezing time arbitrarily, generate timestamps relative to the current moment with helpers.

from sentry.testutils.helpers.datetime import before_now, iso_format

five_min_ago = before_now(minutes=5)
iso_timestamp = iso_format(five_min_ago)

Checking SQL Queries

Add a log_sql fixture to conftest.py to reset queries, enable DEBUG, and print aggregated SQL and execution time after each test.

import itertools
from django.conf import settings
from django.db import connection, connections, reset_queries
from django.template import Template, Context
import pytest

@pytest.fixture(scope="function", autouse=True)
def log_sql():
    reset_queries()
    settings.DEBUG = True
    yield
    time = sum([float(q["time"]) for q in connection.queries])
    tmpl = Template("{% for sql in sqllog %}{{sql.sql|safe}}{% if not forloop.last %}

{% endif %}{% endfor %}")
    queries = list(itertools.chain.from_iterable([conn.queries for conn in connections.all()]))
    log = tmpl.render(Context({"sqllog": queries, "count": len(queries), "time": time}))
    print(log)

Use pytest -k to filter tests and -s to view the printed SQL.

Acceptance Testing

Acceptance tests use selenium and chromedriver to simulate full‑stack user interactions and generate visual‑regression snapshots via GitHub Actions.

Running Acceptance Tests

When running acceptance tests, webpack rebuilds static assets. After modifying JavaScript files, delete .webpack.meta to force a rebuild.

# Run a single acceptance test.
pytest tests/acceptance/test_organization_group_index.py -k test_with_onboarding

# Run with a visible browser.
pytest tests/acceptance/test_organization_group_index.py --no-headless=true -k test_with_onboarding

# Capture visual snapshots.
SENTRY_SCREENSHOT=1 VISUAL_SNAPSHOT_ENABLE=1 pytest tests/acceptance/test_organization_group_index.py -k test_with_onboarding

If you see a warning about a non‑W3C command, webpack failed to compile resources.

Locating Elements

Because the UI uses emotion, class names are unstable. Use stable data-test-id attributes as hooks for Selenium and Jest.

Handling Asynchronous Actions

All data loads asynchronously; use Selenium's wait_until* helpers to poll the DOM until elements appear, avoiding sleep().

Visual Regression

Pixel‑perfect snapshots catch unexpected rendering changes. Screenshots are compared against approved baselines during acceptance runs. Known blind spots include hover cards, modal windows, and chart visualizations, which require extra care.

Dealing with Dynamic Data

Time‑series data can cause flaky snapshots. Replace time‑dependent content with fixed placeholders using the getDynamicText helper.

Jest Testing

The Jest suite provides functional and unit tests for frontend components. Preference is given to interaction‑driven tests that exercise navigation and API calls rather than shallow prop/state checks.

# Run Jest in interactive mode
yarn test

# Run a single test
yarn test tests/js/spec/views/issueList/overview.spec.js

API Fixtures

Jest tests run without a real API; fixture builders in tests/js/sentry-test/fixtures/ generate mock responses. Use MockApiClient.addMockResponse() to define expected API payloads; missing mocks cause test failures.

Kafka Testing in CI

The Snuba test suite ( .github/workflows/snuba-integration-test.yml) is the only CI test that runs a real Kafka broker. Place any test requiring Kafka under tests/snuba/. Otherwise the job will time out and be cancelled by GitHub Actions.

Additional Resources

https://develop.sentry.dev/services/devservices/

https://docs.pytest.org/en/latest/

https://develop.sentry.dev/frontend/#testing