Spring Boot 3 + WebFlux High-Concurrency API Architecture: From Reactive Theory to Production

1. Core concepts of WebFlux

WebFlux is not just about Mono and Flux. Its real value comes from three underlying mechanisms:

Event‑loop driven I/O – a small number of threads handle network events and dispatch callbacks.

Reactive Streams protocol – defines Publisher, Subscriber, Subscription and Processor with explicit back‑pressure via Subscription.request(n).

Resource‑reuse for high‑latency I/O – the same threads can serve thousands of waiting requests.

Consequences:

CPU‑bound workloads gain little.

Heavy blocking JDBC/SDK calls erase the benefit.

Teams without reactive experience may face higher complexity.

2. Why the traditional blocking model fails under load

In a Servlet/MVC model each request binds to a worker thread. When the request spends most of its time waiting for DB, remote services or cache, the thread stays occupied, causing:

Explosion of worker threads

More context switches

Higher memory consumption

Growing queue latency

Thus a system that looks fine at low concurrency collapses as soon as the number of concurrent requests grows.

3. What WebFlux actually solves

WebFlux uses a non‑blocking event‑loop model:

Threads never block while waiting for I/O.

Pending I/O is represented as a callback chain.

When I/O completes the event loop resumes the chain.

This allows a tiny thread pool to serve a huge number of requests that spend most of their time waiting. The advantage is stability and resource efficiency, not raw speed.

4. Reactive Streams and back‑pressure

Reactive Streams defines four roles:

Publisher

Subscriber

Subscription

Processor

The key method Subscription.request(n) lets a consumer tell the producer how many items it can handle, preventing overload.

Example of a reactive chain (cold flow)

return repository.findById(id)
    .switchIfEmpty(Mono.error(new BizException("message not found")))
    .flatMap(message -> publisher.publish(message).thenReturn(message))
    .map(MessageResponse::from);

The chain is only executed when WebFlux subscribes to it.

5. Reactor Netty thread model

Two kinds of threads are used:

Event‑loop threads – handle connection events, read/write network data and dispatch callbacks.

Few auxiliary threads – e.g., DNS resolution.

Any long‑running blocking operation (blocking DB, large file I/O, heavy CPU work, encryption, etc.) must never run on an event‑loop thread, otherwise a single slow operation can cripple a batch of connections.

6. When to adopt WebFlux

Strongly recommended for:

Real‑time push platforms

Long‑connection gateways (SSE / WebSocket / RSocket)

High‑concurrency API aggregation layers

I/O‑intensive middle‑tier services

Event‑driven log or message streams

Avoid for:

Low‑concurrency pure CRUD services

CPU‑intensive compute services

Batch‑oriented offline jobs

Systems heavily dependent on blocking SDKs without reactive clients

7. Real‑world case: Enterprise‑grade real‑time message push platform

Business background

User message center

Risk‑control alerts

Order status notifications

IoT device event delivery

Admin‑side broadcast messages

Overall layered architecture

Ingress layer – authentication, rate‑limiting, protocol handling, SSE/WebSocket connection management.

Application layer – validation, idempotency, orchestration, aggregation, retry, degradation.

Data layer – R2DBC PostgreSQL for persistence, Reactive Redis for hot‑cache and Pub/Sub.

Asynchronous dispatch layer – Kafka decouples write from delivery.

Observability layer – metrics, tracing, logging, alerts.

Why the architecture fits high concurrency

The design separates the write path from the delivery path:

Strong‑consistent write path – minimal required data is persisted first.

Eventually‑consistent delivery path – Kafka asynchronously distributes the message.

Long‑connection push path – Redis Pub/Sub or Stream broadcasts to online nodes.

Benefits:

Write requests are not blocked by downstream latency.

Downstream jitter does not directly affect ingress latency.

Message dispatch can be scaled independently.

Connection management is decoupled from business logic.

8. Engineering design for production‑grade high concurrency

Four dimensions to consider: throughput, latency, stability, recoverability. Typical production problems are “passes load test but unstable in production”. Required mechanisms:

Rate limiting

Back‑pressure

Isolation (blocking calls on dedicated schedulers)

Timeouts

Retries with exponential back‑off

Degradation

Idempotency

Observability

Rate‑limit filter example

package com.example.push.support.ratelimit;

import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.core.Ordered;
import org.springframework.http.HttpStatus;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebFilter;
import org.springframework.web.server.WebFilterChain;
import reactor.core.publisher.Mono;

import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;

@Component
public class SimpleRateLimitFilter implements WebFilter, Ordered {
    private final Map<String, WindowCounter> counters = new ConcurrentHashMap<>();
    private final Counter rejectCounter;

    public SimpleRateLimitFilter(MeterRegistry registry) {
        this.rejectCounter = registry.counter("http_rate_limit_reject_total");
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
        String path = exchange.getRequest().getPath().value();
        String clientKey = exchange.getRequest().getRemoteAddress() == null ? "unknown"
                : exchange.getRequest().getRemoteAddress().getAddress().getHostAddress();
        String key = clientKey + ":" + path;
        WindowCounter counter = counters.computeIfAbsent(key, k -> new WindowCounter(Duration.ofSeconds(1)));
        if (!counter.tryAcquire(200)) {
            rejectCounter.increment();
            ServerHttpResponse response = exchange.getResponse();
            response.setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
            byte[] body = "{\"code\":\"RATE_LIMITED\",\"message\":\"too many requests\"}".getBytes(StandardCharsets.UTF_8);
            return response.writeWith(Mono.just(response.bufferFactory().wrap(body)));
        }
        return chain.filter(exchange);
    }

    @Override
    public int getOrder() { return -200; }

    private static final class WindowCounter {
        private final Duration window;
        private volatile long windowStart;
        private final AtomicInteger value = new AtomicInteger();
        WindowCounter(Duration window) { this.window = window; this.windowStart = System.currentTimeMillis(); }
        boolean tryAcquire(int limit) {
            long now = System.currentTimeMillis();
            if (now - windowStart >= window.toMillis()) {
                synchronized (this) {
                    if (now - windowStart >= window.toMillis()) {
                        windowStart = now;
                        value.set(0);
                    }
                }
            }
            return value.incrementAndGet() <= limit;
        }
    }
}

Blocking‑call isolation

Bad example (still runs on the event‑loop):

public Mono<String> badCase() {
    return Mono.fromCallable(() -> legacyHttpClient.call());
}

Correct isolation using a dedicated scheduler:

import reactor.core.scheduler.Scheduler;
import reactor.core.scheduler.Schedulers;

public class LegacyAdapter {
    private final Scheduler blockingScheduler = Schedulers.newBoundedElastic(50, 10_000, "legacy-blocking");

    public Mono<String> invoke() {
        return Mono.fromCallable(this::callBlockingSdk)
                .subscribeOn(blockingScheduler);
    }

    private String callBlockingSdk() { return "result"; }
}

Reactive transaction configuration

package com.example.push.infrastructure.config;

import io.r2dbc.spi.ConnectionFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.transaction.ReactiveTransactionManager;
import org.springframework.transaction.reactive.TransactionalOperator;
import org.springframework.r2dbc.connection.R2dbcTransactionManager;

@Configuration
public class TransactionConfig {
    @Bean
    public ReactiveTransactionManager reactiveTransactionManager(ConnectionFactory connectionFactory) {
        return new R2dbcTransactionManager(connectionFactory);
    }

    @Bean
    public TransactionalOperator transactionalOperator(ReactiveTransactionManager transactionManager) {
        return TransactionalOperator.create(transactionManager);
    }
}

Guidelines:

Only wrap truly atomic DB operations in a transaction.

Do not force remote calls, Redis broadcast or Kafka publish into the same transaction.

Prefer outbox/event‑driven patterns for cross‑resource consistency.

Global exception handling

package com.example.push.api.advice;

import com.example.push.api.dto.ApiResponse;
import com.example.push.support.exception.BizException;
import org.springframework.http.HttpStatus;
import org.springframework.web.ErrorResponseException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import reactor.core.publisher.Mono;

@RestControllerAdvice
public class GlobalExceptionHandler {
    @ExceptionHandler(BizException.class)
    @ResponseStatus(HttpStatus.BAD_REQUEST)
    public Mono<ApiResponse<Void>> handleBiz(BizException ex) {
        return Mono.just(ApiResponse.fail(ex.getCode(), ex.getMessage()));
    }

    @ExceptionHandler(ErrorResponseException.class)
    public Mono<ApiResponse<Void>> handleHttp(ErrorResponseException ex) {
        String code = "HTTP_" + ex.getStatusCode().value();
        return Mono.just(ApiResponse.fail(code, ex.getMessage()));
    }

    @ExceptionHandler(Throwable.class)
    @ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
    public Mono<ApiResponse<Void>> handleUnknown(Throwable ex) {
        return Mono.just(ApiResponse.fail("INTERNAL_ERROR", "system busy"));
    }
}

9. Observability and metrics

Essential metrics groups:

Ingress layer : QPS, active connections, P50/P95/P99 latency, 4xx/5xx ratios, rate‑limit rejections.

Middleware : R2DBC pool usage, DB slow queries, Redis command latency, Kafka consumer lag.

Business : message write success, delivery success rate, retry attempts, dead‑letter count, online user count.

Simple metric‑recording component

package com.example.push.infrastructure.config;

import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.time.Duration;

@Configuration
public class MetricsConfig {
    @Bean
    public PushMetrics pushMetrics(MeterRegistry registry) {
        return new PushMetrics(
                registry.counter("push_message_create_total"),
                registry.counter("push_dispatch_success_total"),
                registry.counter("push_dispatch_failed_total"),
                registry.timer("push_dispatch_latency")
        );
    }
}

class PushMetrics {
    private final Counter createCounter;
    private final Counter dispatchSuccessCounter;
    private final Counter dispatchFailedCounter;
    private final Timer dispatchLatency;

    PushMetrics(Counter createCounter, Counter dispatchSuccessCounter, Counter dispatchFailedCounter, Timer dispatchLatency) {
        this.createCounter = createCounter;
        this.dispatchSuccessCounter = dispatchSuccessCounter;
        this.dispatchFailedCounter = dispatchFailedCounter;
        this.dispatchLatency = dispatchLatency;
    }

    void recordDispatch(Duration duration, boolean success) {
        dispatchLatency.record(duration);
        if (success) {
            dispatchSuccessCounter.increment();
        } else {
            dispatchFailedCounter.increment();
        }
    }
}

10. Scaling design – not just adding instances

Capacity must be evaluated for three dimensions:

API instance capacity (CPU, thread pool).

Connection‑node capacity (how many long‑lived connections a node can hold).

Middleware capacity (DB, Redis, Kafka).

Example calculation: a single node can stably hold 50 000 SSE connections. For a peak of 800 000 online users with 30 % headroom, required nodes ≈ 800 000 / 50 000 × 1.3 ≈ 21.

Database connection pool must be sized together with instance count. If each instance needs 40 connections and you run 12 instances, total connections = 480; a DB that only allows 300 connections would be a configuration error.

11. Performance testing methodology

Three‑layer testing approach:

Pure ingress capability – measure framework and network stack limits.

Full business path – include cache, DB and async dispatch.

Long‑connection scenario – test connection establishment, keep‑alive, fan‑out and reconnection.

Write‑API load test (wrk)

wrk -t8 -c400 -d60s --latency \
    -s post-message.lua http://127.0.0.1:8080/api/v1/messages

Lua script (post‑message.lua):

wrk.method = "POST"
wrk.body = '{"tenantId":"t1","messageKey":"k-' .. os.time() .. '","userId":"u1001","channel":"SSE","title":"Order Update","content":"Order shipped"}'
wrk.headers["Content-Type"] = "application/json"

SSE long‑connection test

curl -N http://127.0.0.1:8080/api/v1/stream/tenants/t1/users/u1001

Metrics to record during load tests: P50/P95/P99 latency, error rate, GC pauses, CPU user/system, connection‑pool wait time, Kafka lag, Redis latency.

Interpreting results

If QPS stalls while CPU stays low → downstream waiting or insufficient connection pool.

P99 spikes but average is normal → queue buildup, hotspot, or retry storm.

CPU spikes together with RT rise → event‑loop blocked or heavy serialization.

Redis latency jitter causing overall jitter → cache or broadcast bottleneck.

Multiple test rounds, different traffic models and fault injection are required for production‑grade conclusions.

12. Real‑world scenario: Order‑status real‑time notification

Order system emits a "paid" event.

Message platform creates a message (idempotent via unique (tenantId, messageKey) constraint).

Message is persisted.

Async dispatch service consumes the event.

Broadcast to online users via SSE/WebSocket.

If the user is offline, status stays STORED.

When the user reconnects, a pull API fetches recent messages.

Compensation query API

package com.example.push.api.controller;

import com.example.push.api.dto.ApiResponse;
import com.example.push.domain.model.PushMessage;
import com.example.push.domain.repository.PushMessageRepository;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;

@RestController
@RequestMapping("/query")
@RequiredArgsConstructor
public class MessageQueryController {
    private final PushMessageRepository repository;

    @GetMapping("/tenants/{tenantId}/users/{userId}/latest")
    public Flux<ApiResponse<PushMessage>> latest(@PathVariable String tenantId,
                                                @PathVariable String userId) {
        return repository.findLatestMessages(tenantId, userId, 50)
                .map(ApiResponse::ok);
    }
}

13. Common pitfalls and concrete solutions

Treating WebFlux as a performance magic wand – only I/O‑wait‑bound workloads benefit; blocking code nullifies gains.

Unbounded buffers – e.g., Sinks.many().multicast().onBackpressureBuffer() can cause OOM. Use a bounded buffer, e.g. Sinks.many().multicast().onBackpressureBuffer(1024, false), and define overflow policies.

Calling block() inside a reactive pipeline – blocks the event‑loop and defeats non‑blocking design. Use only in tests or at the very edge of adaptation layers.

Missing concurrency limit on flatMap – default may flood downstream. Specify concurrency, e.g. .flatMap(this::dispatch, 64).

Logs without business tags – include tenantId, userId, messageId, traceId in every log line for troubleshooting.

14. Production‑grade checklist

Architecture layer

Separate write, dispatch and connection handling.

Make the core path asynchronous; avoid synchronous chaining of long‑running calls.

Prefer event‑driven coordination over distributed two‑phase commits.

Code layer

Identify cold vs hot streams; use bounded buffers for hot streams.

Specify concurrency on flatMap.

Isolate every blocking call onto a dedicated scheduler.

Handle exceptions per layer; avoid default 500 leakage.

Middleware layer

Calculate DB pool size together with instance count.

Set timeouts and failure alerts on Redis broadcast.

Match Kafka consumer concurrency with partition count.

Operations layer

Enable graceful shutdown.

Monitor P99 latency, not just averages.

Maintain dead‑letter queues and manual compensation procedures.

Include fault‑injection scenarios in load tests.

15. How to explain WebFlux value to a team

Ask three concrete questions:

Is our current bottleneck caused by thread‑and‑I/O waiting?

Do we need to support many connections, long‑lived streams, or bidirectional communication?

Can we reliably manage blocking dependencies, rate limiting, observability and error handling?

If the answer to all three is “yes”, WebFlux provides a relevant solution.

16. Final takeaway

Spring Boot 3 + WebFlux gives a programming and runtime model tailored for:

Non‑blocking I/O that maximizes thread utilization.

Reactive Streams back‑pressure that protects the system from overload.

Composable asynchronous pipelines that can be made resilient, idempotent and observable.

To turn this into a production‑grade high‑concurrency system you must also add:

Idempotent design (unique keys, DB constraints).

Rate limiting and overload protection.

Explicit isolation of blocking calls.

Reactive transaction boundaries.

Comprehensive observability (metrics, tracing, structured logs).

Horizontal scaling model with proper capacity planning.

Dead‑letter handling and compensation paths.

WebFlux solves “how to efficiently handle many waiting requests”; a production architecture solves “how to keep the system stable and recoverable under high concurrency”. Combining both yields a truly robust high‑throughput service.