Master Distributed System Interview Questions: CAP, Redis, Zookeeper, Kafka and More

Source: cnblogs.com/expiator/p/10201004.html

Distributed System Theory

Q: What are the main theories of distributed systems?

A: CAP and BASE. In any distributed system you can satisfy at most two of Consistency, Availability, and Partition tolerance; Partition tolerance is mandatory, so systems are usually CP or AP.

Q: How do you understand distributed consistency?

A: Consistency means the logical relationship between related data is correct and complete. In a distributed setting it refers to whether replicated data across nodes remains identical.

Consistency can be strong, weak, or eventual. Strong consistency guarantees immediate synchronization, while eventual consistency ensures convergence after some time.

Distributed Transactions

Q: What is a distributed transaction and what protocols exist?

A: A transaction that spans multiple databases to maintain data consistency. Common protocols are Two‑Phase Commit (2PC) and Three‑Phase Commit (3PC).

2PC: Prepare (voting) phase and Commit phase.

3PC: CanCommit, PreCommit, DoCommit phases.

Q: What are common solutions for distributed transactions?

A: Compensation (TCC), XA, and Message‑Queue based approaches.

TCC (Try‑Confirm‑Cancel)

Try : Lock resources and set a provisional state (e.g., order status → "PAYING", reserve stock, pre‑add points).

Confirm : After all Try steps succeed, the transaction manager moves to the Confirm phase and permanently applies the changes (e.g., order status → "PAID", deduct stock, add points).

Cancel : If any Try fails, the framework rolls back by executing the Cancel logic (e.g., revert provisional states).

To avoid a single point of failure, deploy multiple transaction managers; if one crashes, others continue operating.

Redis

Q: What are Redis's main advantages?

In‑memory speed.

Rich data types: string, list, set, sorted set, hash.

Atomic operations and transaction support.

Versatile features: caching, messaging, key expiration.

Single‑threaded I/O multiplexing.

Q: How does Redis store data internally?

Strings use a Simple Dynamic String (SDS) structure.

Lists use ZipList for small lists, otherwise a doubly‑linked list.

Q: What persistence mechanisms does Redis provide? AOF (Append‑Only File): logs every write command; fast writes, slower recovery; supports rewriting to shrink size. RDB (Redis Database): snapshot of the entire dataset; fast recovery.

Q: How to set a key’s expiration?

Use EXPIRE key seconds (returns 1 on success, 0 if the key does not exist).

Q: What are Redis’s eviction policies?

volatile‑lru, allkeys‑lru, volatile‑random, allkeys‑random, volatile‑ttl, noeviction.

Q: How to implement a distributed lock with Redis?

Use SET key value EX seconds NX which sets the key only if it does not exist and assigns an expiration atomically. SET name fenglin EX 100 NX Store lock metadata as JSON (e.g., thread ID, count) to support re‑entrancy and safe release.

Zookeeper

Q: How does Zookeeper achieve distributed locking?

Create an ephemeral sequential znode under a lock path; the client with the smallest sequence number holds the lock.

Other clients watch the predecessor node; when it disappears they re‑evaluate.

Release the lock by deleting the created znode.

Q: What is Zookeeper’s consistency protocol?

It uses the ZAB protocol (Zookeeper Atomic Broadcast) with recovery (leader election) and broadcast (state synchronization) modes.

Q: Typical Zookeeper use cases?

Service registry and discovery.

Configuration management.

Distributed coordination such as locks.

Kafka

Q: What are Kafka’s key characteristics?

Durable log‑based storage with high throughput.

Zero‑copy data transfer and batch processing.

Partitioned topics for scalability and ordering guarantees per partition.

Replication and ISR (in‑sync replica) mechanism for high availability.

Q: How does Kafka ensure message ordering?

Messages within the same partition are strictly ordered; producers can use a key‑ordering strategy to route related messages to the same partition.

Q: What is the ACK configuration? acks=0: fire‑and‑forget (no guarantee). acks=1: leader acknowledgment only. acks=all: all in‑sync replicas must acknowledge (strongest durability).

Q: How does Kafka handle consumer offset and duplicate consumption?

Consumers record the physical offset; disabling auto‑commit and committing offsets only after successful processing prevents duplicates. Idempotent handling (e.g., using unique IDs) further ensures exactly‑once semantics.

High‑Concurrency Design Patterns

Common techniques for scenarios such as flash sales, attendance punching, and short‑URL services include:

Using Redis for caching, rate limiting (counter or token‑bucket), and atomic list operations.

Employing message queues (Kafka, RabbitMQ, RocketMQ) for asynchronous processing and traffic shaping.

Applying database sharding, read‑write separation, and master‑slave architectures.

Designing idempotent APIs and compensating transactions (TCC) to maintain data consistency.