Spring Cloud Microservices Series #10: Key Takeaways and Best Practices

Series Review

The author revisits the ten‑episode journey from a monolithic blog system to a complete microservices architecture, summarizing each episode:

Episode 1 – Monolithic blog (pain point: tight coupling, hard to extend)

Episode 2 – Nacos service registry (solution: service discovery)

Episode 3 – OpenFeign (solution: elegant remote calls)

Episode 4 – Spring Cloud Gateway (solution: unified entry, authentication)

Episode 5 – Nacos configuration center (solution: centralized, dynamic config)

Episode 6 – Sentinel (solution: rate limiting, circuit breaking, degradation)

Episode 7 – SkyWalking (solution: distributed tracing)

Episode 8 – Seata (solution: cross‑service data consistency)

Episode 9 – Docker + K8s (solution: unified deployment, elastic scaling)

Episode 10 – Summary and best practices (current article)

Production Best Practices

1. Service Splitting Principles

Single responsibility – each service does one thing (e.g., user service handles only users)

Clear boundaries – services communicate via APIs, no shared databases

Independent deployment – each service can be released separately

Data isolation – each service owns its own database (e.g., separate user and article databases)

2. Configuration Management

# Configuration hierarchy
bootstrap.yml      # Fixed config (e.g., Nacos address)
application.yml    # Default config
application-{env}.yml  # Environment‑specific config (dev/test/prod)
Nacos               # Dynamic config (feature flags, thresholds)

# Sensitive information
spring:
  datasource:
    password: ${DB_PASSWORD}

# Naming convention
business:
  enable-comment: true   # Feature flag
  comment-timeout: 30     # Timeout (seconds)
  max-batch-size: 100    # Batch size

3. Service Call Best Practices

// 1. Set reasonable timeout
@FeignClient(name = "user-service", configuration = FeignConfig.class)
public interface UserClient {}

@Configuration
public class FeignConfig {
    @Bean
    public Request.Options options() {
        return new Request.Options(3000, 5000); // connect 3s, read 5s
    }
}

// 2. Implement fallback
@FeignClient(name = "user-service", fallbackFactory = UserFallbackFactory.class)

// 3. Batch call instead of loop
// Wrong
for (Long id : ids) {
    userClient.getUser(id);
}
// Correct
userClient.batchGetUsers(ids);

// 4. Parallel calls
CompletableFuture.allOf(future1, future2).join();

4. Rate Limiting and Circuit Breaking

Normal API – QPS 100‑1000

Login API – hotspot limiting, 10 requests per minute per user

Slow API – thread‑count limiting, 5‑10 concurrent threads

Dependent services – circuit breaking at 50% failure rate

# Sentinel configuration suggestions
user-service:
  flow:
    count: 200               # 200 QPS
    controlBehavior: 0       # fast fail
  degrade:
    slowRatioThreshold: 0.5 # 50% slow requests trigger circuit break
    maxRt: 500               # >500 ms considered slow
    timeWindow: 30           # break for 30 s

5. Database Best Practices

-- 1. Index design (left‑most prefix)
CREATE INDEX idx_user_status ON user(status, create_time);

-- 2. Cursor pagination
SELECT * FROM article WHERE id < #{lastId} ORDER BY id DESC LIMIT 20;

-- 3. Avoid N+1 queries
-- Wrong: looped queries
-- Correct: batch query with IN clause

-- 4. Read‑write separation
@DataSource("slave")
public List<Article> listArticles() { ... }

6. Logging Best Practices

// 1. Log level conventions
// ERROR – system error, needs manual intervention
// WARN  – recoverable exception
// INFO  – key business flow
// DEBUG – debugging info (disabled in production)

// 2. Log key information
log.info("User login succeeded, userId={}, ip={}", userId, ip);

// 3. Log TraceId (integrated with SkyWalking)
log.info("Processing order, traceId={}", TraceContext.traceId());

// 4. Exception logging
log.error("Failed to call user service, userId={}", userId, e);

7. Monitoring and Alerting Best Practices

Key metrics: P99 response time < 500 ms, success rate > 99.9 %, QPS within normal range, DB connections < 80 %, Redis memory < 80 %, JVM memory < 80 %, GC < 5 times per hour, each < 100 ms

Alert levels:

P0 (5 min): service unavailable, core API success rate < 95 %

P1 (15 min): response time > 2 s, success rate < 99 %

P2 (1 h): CPU > 80 %, memory > 85 %

Pitfalls

Infrastructure Pitfalls

Nacos data loss – after restart configuration disappears → store in MySQL

Sentinel rule loss – after restart rules disappear → enable persistence

Seata transaction not rolled back – exception committed → ensure exception is thrown, not swallowed

SkyWalking performance impact – service slows down → reduce sampling rate to 10 %

Service Call Pitfalls

Circular dependency – service A calls B and B calls A → redesign or use a message queue

Unreasonable timeout – frequent timeouts → adjust per‑business, configure slow APIs separately

Thread pool exhaustion – service becomes unavailable → isolate with thread pools

Load imbalance – one instance overloaded → check load‑balancing strategy

Database Pitfalls

Connection pool leak – connections keep growing → enable leak‑detection‑threshold

Deadlock – transactions block each other → enforce a unified lock order

Slow queries – API slows down → optimize indexes and use read‑write separation

Distributed transaction performance degradation – QPS drops → adopt eventual consistency

Deployment Pitfalls

Large Docker images – slow deployment → multi‑stage build with slim base images

Health‑check failures – service restarts frequently → increase initialDelaySeconds

Service discovery failure – calls error → use service name instead of localhost

Log disk full – service errors → enable log rotation and periodic cleanup

Performance Optimization Checklist

1. High‑Concurrency Optimizations

// Multi‑level cache
Local cache (Caffeine) → Distributed cache (Redis) → Database

// Parallel calls
CompletableFuture.allOf(future1, future2, future3).join();

// Batch processing
List<User> users = userClient.batchGet(userIds);

// Asynchronous processing
@Async
public void sendNotification() { ... }

2. JVM Tuning

java -jar app.jar \
  -Xms2g -Xmx2g \
  -XX:+UseG1GC \
  -XX:MaxGCPauseMillis=200 \
  -XX:+PrintGCDetails \
  -XX:+PrintGCDateStamps \
  -Xloggc:/var/log/gc.log \
  -XX:+UseContainerSupport \
  -XX:InitialRAMPercentage=50 \
  -XX:MaxRAMPercentage=75

3. Database Connection Pool

spring:
  datasource:
    hikari:
      maximum-pool-size: 50
      minimum-idle: 10
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000
      leak-detection-threshold: 60000  # enable leak detection

Pros and Cons of Microservices

Advantages

Independent deployment – each service can be released without affecting others

Technology heterogeneity – different services may use different stacks

Strong isolation – failure of one service does not cascade

Elastic scaling – only bottleneck services need to be scaled

Team autonomy – each team owns its service lifecycle

Disadvantages

High complexity – service discovery, configuration, tracing, distributed transactions, etc.

Operational cost – dozens of services require monitoring and maintenance

Network latency – local calls become remote calls

Data consistency – distributed transactions are hard

Debugging difficulty – issues span multiple services

When to Use Microservices

Large projects with teams > 50 people

Complex business requiring independent iteration

High concurrency demanding elastic scaling

Diverse technology stacks across components

When Not to Use Microservices

Small projects, team size < 10

Simple, stable business with little change

Limited resources, no dedicated ops team

Early‑stage startups needing rapid validation

Future Directions

1. Service Mesh

Traditional microservices:
Business code + service discovery + circuit breaking + tracing + ...

Service Mesh:
Business code (lightweight)
    ↓
Sidecar proxy (all governance capabilities offloaded)

2. Serverless

# Function compute definition
functions:
  user-service:
    runtime: java11
    handler: com.laok.UserHandler
    triggers:
      - http:
          path: /user/{id}

3. Cloud Native

Full containerization

GitOps – declarative deployment

Chaos engineering – fault injection drills

FinOps – cost optimization

Resources

# GitHub repository (one tag per episode)
git clone https://github.com/laok/blog-micro

# Checkout this episode
git checkout episode-10-summary

# Quick start (requires Docker)
docker-compose up -d

# Access URLs
Nacos:      http://localhost:8848/nacos
Sentinel:   http://localhost:8858
SkyWalking: http://localhost:8088
Seata:      http://localhost:7091
Gateway:    http://localhost:8080