Panel Discussion on Large‑Scale Event‑Driven Architectures and Practical Lessons

Reisz welcomes the audience to a panel on large‑scale event‑driven architectures, prompting participants to reflect on the benefits of scale, performance, and flexibility. The discussion features Wes Reis, a VMware platform architect, Gwen Shapira from Confluent, Thomas from Sky Bet, and Matthew Clark from the BBC.

Gwen explains how Confluent adopted event‑driven design after experimenting with other approaches, emphasizing that events eliminate inter‑team blame, provide immutable audit trails, and enable replay for debugging and recovery.

Thomas describes Sky Bet’s evolution from a monolithic Informix database to Kafka‑backed services, highlighting the need for real‑time event handling in sports betting and the challenges of scaling and state management.

Matthew shares the BBC’s perspective, noting that event‑driven architecture fits high‑throughput services like search and recommendation engines, but must be applied where its advantages outweigh the added complexity.

The panel stresses the importance of a solid domain model when adopting event‑driven systems, warning that architects must understand the business semantics behind events to avoid mis‑alignment.

Common surprises include dealing with unordered events, ensuring idempotency, handling replay, and managing partitioning and ordering guarantees in Kafka topics.

When discussing choreography versus orchestration, the experts agree that both have places; choreography shines in loosely coupled systems, while explicit orchestration is useful for complex, multi‑step workflows.

Gwen and Thomas outline best practices for separating events and designing Kafka topics, such as grouping high‑frequency events, respecting ordering constraints, and avoiding overly large event payloads.

Day‑two operational concerns raised include load testing, chaos engineering, AIOps monitoring, version control of event schemas, and strategies for rolling upgrades without service disruption.

Observability is highlighted as critical: tracing event flow across microservices, using tools like Kibana and AWS X‑Ray, and sampling events to detect anomalies while managing data volume.

War stories illustrate real‑world failures, such as mismatched partition hashing between Node.js and Kotlin producers, and the impact of oversized events on system stability.

Advice for long‑term success includes keeping systems simple, avoiding over‑engineering, ensuring idempotent processing, and carefully evaluating when event‑driven architecture is truly needed versus traditional request‑response models.