Spring Boot 3 Enterprise Development Guide: From Monolith to High‑Concurrency Distributed Architecture

Motivation for Rewriting Spring Boot Articles

Many existing tutorials stop at annotations and simple CRUD, ignoring real‑world concerns such as massive traffic, transaction boundaries, idempotency, observability, and fault tolerance. The guide therefore answers critical questions for an e‑commerce order center, e.g., how to prevent overselling, duplicate submissions, and message loss under high load.

Business Scenario and Core Challenges

The example implements an order center responsible for creating orders, locking inventory, calculating discounts, initiating payment, closing timed‑out orders, driving state transitions, and notifying downstream systems (fulfillment, loyalty, marketing). Expected load:

≈800 000 orders per day

Normal peak QPS > 3 000

Promotional peak QPS > 12 000

Flash‑sale bursts > 100 000 QPS

Key technical challenges:

High‑concurrency writes

Strong consistency for inventory and payment

Idempotent processing of duplicate requests

Graceful degradation when downstream services fail

Spring Boot 3 Key Changes (vs 2.x)

Java baseline : 8/11 → 17+ (records, sealed classes, pattern matching)

Servlet/JPA packages : javax.* → jakarta.* (higher migration cost but unified ecosystem)

Auto‑configuration : spring.factories → AutoConfiguration.imports (clearer conditional loading)

Observability : Sleuth → Micrometer Tracing (unified metrics, logs, traces)

Native image support : experimental → officially supported (faster cold start, lower memory for serverless)

Automatic Configuration Mechanism

SpringApplication.run()
  → Create ApplicationContext
  → Scan @Configuration / @Component
  → Import auto‑configuration via @EnableAutoConfiguration
  → Read META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
  → Apply @ConditionalOnClass / @ConditionalOnBean / @ConditionalOnProperty
  → Register BeanDefinitions and finish container initialization

Enterprise‑Level Layered Architecture

┌───────────────────────────────────────┐
│ Interface Layer (REST / MQ / Scheduler) │
├───────────────────────────────────────┤
│ Application Layer (use‑case orchestration, │
│ transaction boundary, permission, idempotency) │
├───────────────────────────────────────┤
│ Domain Layer (aggregates, entities, domain services, │
│ domain events) │
├───────────────────────────────────────┤
│ Infrastructure Layer (MySQL, Redis, Kafka, Feign, │
│ Search, OSS) │
└───────────────────────────────────────┘

Design principles: high cohesion, replaceability of infrastructure, isolated thread pools, and a clear evolution path from monolith to microservices.

Multi‑Module Maven Project

order-center/
├── pom.xml
├── order-boot-starter-common/
├── order-common/
├── order-domain/
├── order-application/
├── order-infrastructure/
├── order-interfaces/
├── order-start/
└── deploy/
    ├── docker/
    └── k8s/

The parent pom.xml centralises versions (Java 17, Spring Boot 3.3.2, Spring Cloud 2023.0.3, MyBatis‑Plus 3.5.7, Redisson 3.32.0, MapStruct 1.5.5.Final) and manages dependencies via dependencyManagement.

Domain Model (Core Aggregates)

public class Order {
    private Long id;
    private String orderNo;
    private Long userId;
    private OrderStatus status;
    private BigDecimal totalAmount;
    private BigDecimal discountAmount;
    private BigDecimal payAmount;
    private LocalDateTime expireTime;
    private final List<OrderItem> items = new ArrayList<>();

    public static Order create(String orderNo, Long userId, List<OrderItem> items, LocalDateTime expireTime) {
        if (items == null || items.isEmpty()) {
            throw new IllegalArgumentException("Order items cannot be empty");
        }
        Order order = new Order();
        order.orderNo = orderNo;
        order.userId = userId;
        order.status = OrderStatus.PENDING_PAYMENT;
        order.expireTime = expireTime;
        order.items.addAll(items);
        order.totalAmount = items.stream()
            .map(OrderItem::subtotal)
            .reduce(BigDecimal.ZERO, BigDecimal::add);
        order.discountAmount = BigDecimal.ZERO;
        order.payAmount = order.totalAmount.subtract(order.discountAmount);
        return order;
    }

    public void applyDiscount(BigDecimal discountAmount) {
        if (discountAmount == null || discountAmount.compareTo(BigDecimal.ZERO) < 0) {
            throw new IllegalArgumentException("Invalid discount amount");
        }
        this.discountAmount = discountAmount;
        this.payAmount = this.totalAmount.subtract(discountAmount);
        if (this.payAmount.compareTo(BigDecimal.ZERO) < 0) {
            throw new IllegalStateException("Pay amount cannot be negative");
        }
    }

    public void pay(LocalDateTime now) {
        if (status != OrderStatus.PENDING_PAYMENT) {
            throw new IllegalStateException("Order not payable in current state");
        }
        if (expireTime != null && now.isAfter(expireTime)) {
            throw new IllegalStateException("Order expired");
        }
        this.status = OrderStatus.PAID;
    }
    // getters omitted for brevity
}

public class OrderItem {
    private Long productId;
    private String productName;
    private Integer quantity;
    private BigDecimal unitPrice;

    public OrderItem(Long productId, String productName, Integer quantity, BigDecimal unitPrice) {
        if (quantity == null || quantity <= 0) {
            throw new IllegalArgumentException("Quantity must be > 0");
        }
        if (unitPrice == null || unitPrice.compareTo(BigDecimal.ZERO) <= 0) {
            throw new IllegalArgumentException("Unit price must be > 0");
        }
        this.productId = productId;
        this.productName = productName;
        this.quantity = quantity;
        this.unitPrice = unitPrice;
    }

    public BigDecimal subtotal() {
        return unitPrice.multiply(BigDecimal.valueOf(quantity));
    }
}

Production‑Grade Order Creation Flow

Client → Gateway → OrderController → OrderApplicationService → IdempotentService → InventoryGateway (pre‑reserve) → OrderRepository (order table) → OutboxRepository (local message table) → Commit Tx → MessageRelay (asynchronous Kafka) → Inventory / Coupon / Fulfillment downstream

Idempotency First

Before any business logic runs, the service checks an idempotent table keyed by userId + operation + X‑Idempotency‑Key. If a record exists, the previously created OrderDTO is returned, guaranteeing exactly‑once semantics.

Outbox Pattern for Reliable Messaging

Instead of calling kafkaTemplate.send() inside the transaction, the order and a serialized OrderCreatedEvent are persisted together. A scheduled job reads pending outbox rows, publishes them to Kafka, and marks success or retries with exponential back‑off.

CREATE TABLE order_outbox_event (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    aggregate_type VARCHAR(64) NOT NULL,
    aggregate_id VARCHAR(64) NOT NULL,
    event_type VARCHAR(64) NOT NULL,
    payload JSON NOT NULL,
    status VARCHAR(32) NOT NULL,
    retry_count INT NOT NULL DEFAULT 0,
    next_retry_time DATETIME DEFAULT NULL,
    created_at DATETIME NOT NULL,
    updated_at DATETIME NOT NULL,
    KEY idx_status_next_retry_time(status, next_retry_time)
);

High‑Concurrency Safeguards

Duplicate submission prevention : clients send X‑Idempotency‑Key; the server stores a record per user and operation.

Flash‑sale hot‑item flow : Nginx/Gateway rate limiting → Redis atomic stock decrement → MQ enqueue → Asynchronous order creation.

Redis Lua stock deduction (atomic) :

local stock = tonumber(redis.call('GET', KEYS[1])
local qty = tonumber(ARGV[1])
if stock == nil then return -1 end
if stock < qty then return 0 end
redis.call('DECRBY', KEYS[1], qty)
return 1

Thread‑pool isolation : separate executors for core order processing, async tasks, batch exports, and MQ consumption.

Resilience4j circuit breaker, timeout, retry :

resilience4j:
  circuitbreaker:
    instances:
      inventoryService:
        slidingWindowType: COUNT_BASED
        slidingWindowSize: 20
        minimumNumberOfCalls: 10
        failureRateThreshold: 50
        waitDurationInOpenState: 10s
  timelimiter:
    instances:
      inventoryService:
        timeoutDuration: 500ms
  retry:
    instances:
      inventoryQuery:
        maxAttempts: 3
        waitDuration: 100ms
        enableExponentialBackoff: true

Distributed Transaction & Consistency Strategy

The system adopts local transaction + outbox + compensation :

Order service writes orders and order_outbox_event in one DB transaction.

Outbox relay publishes the event; downstream services (inventory, loyalty, fulfillment) consume and update their own stores.

Compensation tasks (order timeout release, payment‑failure rollback) are idempotent, retryable, and auditable.

Example of optimistic‑lock update for payment:

UPDATE orders
SET status = 'PAID',
    version = version + 1,
    updated_at = NOW()
WHERE order_no = ?
  AND status = 'PENDING_PAYMENT'
  AND version = ?;

Cache Design and Consistency

A three‑level cache (Caffeine → Redis → MySQL) is used for read‑heavy, relatively static data such as product info or order read‑only views.

Cache‑aside pattern with proper write‑through order:

Update the database.

Delete the cache entry.

Optionally delay a second delete for eventual consistency.

Typical pitfalls and mitigations:

Cache penetration → cache empty values or use Bloom filter.

Cache breakdown → mutex lock on miss or keep hot keys warm.

Cache avalanche → random TTL jitter, multi‑level cache, circuit breaking.

Database & Data‑Access Optimisations

Key table orders includes fields for order number, user ID, status, monetary amounts, idempotency key, version (optimistic lock), and timestamps. Indexes for the most frequent query pattern:

CREATE INDEX idx_user_status_time ON orders(user_id, status, created_at);
CREATE INDEX idx_status_expire_time ON orders(status, expire_time);

Slow‑SQL governance: set a 200 ms threshold, weekly TOP‑SQL review, and mandatory EXPLAIN before schema changes.

Observability Stack

Spring Boot 3 natively exposes Micrometer metrics. Recommended metrics include JVM, thread‑pool, datasource, HTTP latency, Kafka producer/consumer, and business‑level counters (order success/failure, inventory lock failures).

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus
  metrics:
    tags:
      application: ${spring.application.name}
      env: ${spring.profiles.active}
    observations:
      key-values:
        service: ${spring.application.name}

Custom business metric component (code shown in the article) records success counters, failure counters, and latency timers.

Logging pattern must include trace identifiers and business keys:

logging:
  pattern:
    console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%X{traceId:-},%X{spanId:-}] %logger{36} - %msg%n"

Security, Configuration & Release Management

Spring Security 6 is configured for stateless JWT authentication, role‑based access, CORS, and CSRF disabled. Secrets (DB passwords, Redis credentials, JWT private keys) are stored in Kubernetes Secrets, external configuration centers (Nacos, Apollo) with encryption, or Vault.

@Configuration
public class SecurityConfig {
    @Bean
    public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
        http.csrf(csrf -> csrf.disable())
            .cors(Customizer.withDefaults())
            .sessionManagement(sess -> sess.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
            .authorizeHttpRequests(auth -> auth
                .requestMatchers("/actuator/health/**").permitAll()
                .requestMatchers("/api/admin/**").hasRole("ADMIN")
                .anyRequest().authenticated())
            .oauth2ResourceServer(oauth2 -> oauth2.jwt(Customizer.withDefaults()));
        return http.build();
    }
}

Containerisation & Kubernetes Deployment

Multi‑stage Dockerfile builds the application with Maven, then runs a lightweight JRE image as a non‑root user. Container‑aware JVM flags are used.

FROM maven:3.9.0-eclipse-temurin-17 AS builder
WORKDIR /workspace
COPY pom.xml .
COPY order-common order-common
COPY order-domain order-domain
COPY order-application order-application
COPY order-infrastructure order-infrastructure
COPY order-interfaces order-interfaces
COPY order-start order-start
RUN mvn -T 1C -DskipTests clean package -pl order-start -am

FROM eclipse-temurin:17-jre-jammy
ENV TZ=Asia/Shanghai
ENV JAVA_OPTS="-XX:+UseContainerSupport -XX:MaxRAMPercentage=75 -XX:InitialRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=100"
RUN groupadd -g 1001 app && useradd -u 1001 -g app -s /sbin/nologin app
WORKDIR /app
COPY --from=builder /workspace/order-start/target/*.jar app.jar
USER app
EXPOSE 8080
ENTRYPOINT ["sh","-c","java $JAVA_OPTS -jar /app/app.jar"]

Kubernetes deployment includes liveness, readiness, and startup probes, resource requests/limits, and graceful shutdown configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
  namespace: production
spec:
  replicas: 4
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
      - name: order-service
        image: registry.example.com/order-service:1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: prod
        - name: JAVA_OPTS
          value: "-XX:+UseContainerSupport -XX:MaxRAMPercentage=75"
        resources:
          requests:
            cpu: "500m"
            memory: "1024Mi"
          limits:
            cpu: "2"
            memory: "2048Mi"
        livenessProbe:
          httpGet:
            path: /actuator/health/liveness
            port: 8080
          initialDelaySeconds: 40
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 8080
          initialDelaySeconds: 20
          periodSeconds: 5
        startupProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          failureThreshold: 30
          periodSeconds: 5

Evolution Roadmap: From Monolith to Microservices

Monolith Phase : single codebase with clean domain layers.

Vertical Split Phase : extract order, inventory, payment, product services when domain boundaries become clear.

Microservice Governance Phase : add service discovery, centralized configuration, tracing, circuit breaking, and gray‑release pipelines.

Data‑Split Phase : introduce read/write separation, then vertical sharding, and finally horizontal sharding only when load justifies the added complexity.

Production Incident Post‑Mortems (Five Common Failures)

DB Connection Pool Exhaustion : caused by slow SQL, long‑running transactions, or batch jobs holding connections. Fix by shortening transaction scope, moving heavy remote calls out of transactions, and enforcing a slow‑SQL alerting system.

Overselling : caused by read‑then‑update patterns or local locks across instances. Remedy with atomic DB updates, Redis Lua stock deduction, and separating pre‑reserve from final deduction.

Duplicate Payment / Callback : caused by third‑party retries or user re‑submits. Solve with state‑machine‑driven idempotent updates and persisting callback logs.

Message Backlog : caused by heavy consumer logic, single hot partition, or downstream blockage. Mitigate by IO‑free consumer design, better partition keys, increased parallelism, and dead‑letter handling.

Cache Avalanche : simultaneous key expiry or Redis outage. Counter with random TTL jitter, local cache fallback, hot‑key pre‑warming, and circuit‑breaker protection for DB.

Best‑Practice Checklist

Layered architecture with clear domain boundaries.

Controllers only handle protocol, validation, and identity injection.

All write APIs are idempotent.

Use local transaction + outbox for reliable event publishing.

Apply Redis atomic stock deduction for flash sales.

Isolate critical thread pools and configure circuit breakers.

Prefer conditional DB updates or optimistic locking for state changes.

Expose Micrometer metrics, Prometheus, and Grafana dashboards.

Structure logs with traceId, spanId, orderNo, and errorCode.

Run containers as non‑root, configure health probes, and enable graceful shutdown.

Adopt blue‑green or canary releases with quick rollback.

Conclusion

Spring Boot 3 becomes a true enterprise backbone when it is used to enforce domain‑driven design, layered architecture, high‑concurrency safeguards, eventual consistency via outbox, comprehensive observability, and cloud‑native deployment. By following the patterns and practices outlined above, teams can evolve from a simple monolith to a resilient, scalable microservice ecosystem capable of handling massive traffic while maintaining data integrity and operational visibility.

References

Spring Boot Reference Documentation

Spring Framework 6 Documentation

Micrometer Documentation

Resilience4j Documentation

Kubernetes Documentation

Domain‑Driven Design Reference