Why Splitting into Microservices Can Drop Performance by 300%: 10 Common Anti‑Patterns to Avoid

Problem Overview

After splitting a daily‑hundreds‑of‑thousands‑order transaction system from a single monolith into 13 microservices, response time grew from 300 ms to 1.2 s, P99 exceeded 8 s, throughput halved, and both the database connection pool and Tomcat thread pool became saturated.

The root cause is not the decision to split but an oversimplified justification and a failure to anticipate the new distributed‑system problems introduced by the split.

When to Split – Signals for Migration

Release frequency is limited because every change requires a full‑system deployment.

Hot modules cannot be scaled independently during traffic spikes.

Cross‑module queries (e.g., an order query slows down login and homepage).

Team size grows beyond ~10 developers, causing coordination bottlenecks.

Business boundaries are already clear (user, order, inventory, payment domains).

Avoid splitting when the team is very small (3‑5 engineers), the domain is still in rapid experimentation, the current bottleneck is a slow SQL or missing index, or CI/CD, monitoring, automated testing, and gray‑release pipelines are not in place.

10 Fatal Anti‑Patterns and Correct Practices

Anti‑Pattern 1 – Split the Database First

Wrong approach: Create a separate database for each new service at the beginning, moving 200+ tables into many databases.

Why it hurts:

Cross‑database joins become inter‑service calls.

Local transactions are replaced by distributed transactions.

Data‑migration windows (schema changes, dual‑write periods) add risk.

Correct approach:

First split business boundaries.

Then split deployment units.

Finally split databases.

During transition, multiple services may share the same database instance, but must obey two constraints:

Logical table‑access isolation per service.

No service writes another service’s tables.

Risk‑mitigation techniques:

Separate DB accounts with per‑service table permissions.

Enforce DAO boundaries in code.

Use change‑review processes to block cross‑service writes.

Anti‑Pattern 2 – Split by Controller/Interface Instead of Business Capability

Copying the monolith’s layered structure into services (e.g., UserController → user‑service) creates a “distributed monolith”: no independent data ownership, blurred boundaries, and strong cross‑service coupling.

Correct practice: Split by bounded context. Example mapping:

user‑service : registration, login, profile, address, security settings – owns user master data.

order‑service : create/cancel order, order state machine – owns order master table and order items.

inventory‑service : reserve, deduct, release, replenish inventory – owns inventory ledger.

payment‑service : payment initiation, callback, reconciliation – owns payment records and channel logs.

promotion‑service : coupon calculation, lock, redemption – owns coupon instances and promotion rules.

A capability is a good split candidate when it has clear boundaries, clear data ownership, low coupling, and does not require strong cross‑service strong consistency.

Anti‑Pattern 3 – Deploy Without Automated Tests

Microservices add interaction surfaces: API contracts, timeouts/retries, message publishing, idempotency, version compatibility. Without a test pyramid, hidden failures appear (missing fields causing downstream 500, response‑structure changes breaking old clients, duplicate message consumption, schema incompatibility after rollback).

Production‑grade test pyramid:

Unit tests – verify domain logic.

Contract tests – ensure API compatibility.

Component tests – run against real middleware (cache, message broker, DB).

End‑to‑end tests – cover core business flows.

Example contract test for order service (JUnit + Mockito):

@Test
public void should_create_order_when_user_and_stock_are_valid() {
    UserDTO user = new UserDTO(1001L, "alice", UserStatus.ACTIVE);
    StockCheckResult stock = new StockCheckResult("SKU-1", true);
    when(userClient.getUser(1001L)).thenReturn(user);
    when(stockClient.check("SKU-1", 2)).thenReturn(stock);
    Order order = orderApplicationService.createOrder(new CreateOrderCommand(1001L, "SKU-1", 2));
    assertThat(order.getStatus()).isEqualTo(OrderStatus.CREATED);
}

Anti‑Pattern 4 – Manual Deployment After Splitting

Continuing to use local packaging, manual upload, manual replace, and manual restart leads to long release windows, environment drift, slow rollbacks, and complex start‑order dependencies.

Engineering upgrade: Adopt a full CI/CD pipeline – Git branching, automated builds, Docker images, image registry, Kubernetes rolling updates, automatic rollback.

Typical Kubernetes Deployment snippet:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
      - name: order-service
        image: registry.local/order-service:2.3.1
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 8080
          initialDelaySeconds: 20
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /actuator/health/liveness
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10

Anti‑Pattern 5 – Treat Inter‑Service Calls as Local Calls Without Fault‑Tolerance

Calling inventory via Feign without timeout, circuit‑breaker, isolation, or retry causes thread‑pool exhaustion and retry storms.

Production‑grade communication principles (Resilience4j example):

resilience4j:
  circuitbreaker:
    instances:
      stockService:
        slidingWindowType: COUNT_BASED
        slidingWindowSize: 20
        minimumNumberOfCalls: 10
        failureRateThreshold: 50
        waitDurationInOpenState: 10s
  timelimiter:
    instances:
      stockService:
        timeoutDuration: 800ms
  bulkhead:
    instances:
      stockService:
        maxConcurrentCalls: 50
        maxWaitDuration: 50ms

Timeout budgeting example for a 300 ms order request:

Gateway: 30 ms

Order service orchestration: 80 ms

Inventory RPC: 80 ms

User RPC: 50 ms

Redundant budget: 60 ms

Anti‑Pattern 6 – Multiple Services Directly Share One Database Instance

Sharing a DB instance creates hidden coupling: index changes, lock contention, and multi‑service writes break clear ownership and make schema changes a coordination nightmare.

Correct practice:

Who owns data writes it.

Cross‑service reads must go through APIs or events.

Aggregations are handled by a read model, search index, or dedicated aggregation service.

Anti‑Pattern 7 – Long Synchronous Call Chains

A typical order request traverses six services synchronously, causing latency accumulation, thread‑pool consumption, and fault propagation.

Engineering upgrade – short sync + async coordination:

Order request enters order service.

Local validation of user and product.

Write order draft with status “pending”.

Publish OrderCreated event to a message queue.

Inventory, promotion, points services consume the event asynchronously.

Result is fed back via status flow or notification.

Transactional method with outbox pattern:

@Transactional
public OrderDraft createOrder(CreateOrderCommand command) {
    OrderDraft draft = OrderDraft.create(command.getUserId(), command.getItems(), command.getRequestId());
    orderDraftRepository.save(draft);
    outboxRepository.save(OutboxMessage.forEvent(
        "OrderCreated",
        draft.getOrderId(),
        JsonUtils.toJson(new OrderCreatedEvent(draft.getOrderId(), command.getItems()))));
    return draft;
}

Key points:

Order draft and outbox message are persisted in the same local transaction.

Actual MQ delivery is performed asynchronously by an outbox relay.

Anti‑Pattern 8 – Enforce Strong Distributed Transactions

Introducing XA or two‑phase commit after splitting dramatically increases lock hold time, reduces throughput, and makes failure recovery complex.

Recommended model: Choose consistency per business need. Use Saga with compensation for order creation:

Create pending order.

Publish OrderCreated event.

Inventory service reserves stock.

Coupon service locks coupon.

If all succeed, order moves to CONFIRMED.

On any failure, trigger compensations: release stock, unlock coupon, close order.

Compensating consumer (Kafka) with idempotency:

@KafkaListener(topics = "order-events", groupId = "inventory-service")
public void onOrderCreated(OrderCreatedEvent event) {
    if (consumeLogRepository.existsByMessageId(event.getMessageId())) {
        return;
    }
    inventoryDomainService.reserve(event.getOrderId(), event.getItems());
    consumeLogRepository.save(new ConsumeLog(event.getMessageId(), "inventory-service"));
}

Anti‑Pattern 9 – No Observability

Logs remain on individual machines, making root‑cause analysis expensive.

Production‑grade observability “three‑horse‑carriage”:

Centralised, structured log collection.

Metrics (QPS, latency, error rates, thread‑pool, connection‑pool, message backlog, JVM).

Distributed tracing for full request flow.

Business‑level metrics are equally important (order success rate, inventory reservation success, payment callback latency, message backlog duration, order‑state dead‑locks).

Anti‑Pattern 10 – No Canary or Traffic‑Shading – Full‑Traffic Switch

Deploying a new service directly to 100 % traffic hides protocol incompatibilities, cache‑key mismatches, message‑schema changes, gateway routing errors, and configuration drift.

Correct practice – weight‑based rollout with header‑based shading (Istio VirtualService example):

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service
spec:
  hosts:
  - order-service
  http:
  - match:
    - headers:
        x-canary:
          exact: "true"
    route:
    - destination:
        host: order-service
        subset: v2
  - route:
    - destination:
        host: order-service
        subset: v1
      weight: 95
    - destination:
        host: order-service
        subset: v2
      weight: 5

Structured Migration Roadmap

Foundation : unified logging, monitoring, TraceId propagation, CI/CD, Docker, config centre, service discovery, automated test suite.

Boundary First : split the clearest capability domains (user centre, product centre, order query, inventory, core transaction).

Service First, Database Later : deploy services while keeping a shared database; split DB only after API stability and data ownership are settled.

Async‑ify Long Chains : convert synchronous chains to event‑driven flows.

High‑Concurrency Controls : caching, rate‑limiting, thread‑pool isolation, HPA/KEDA, idempotency, back‑pressure.

High‑Concurrency Design Principles

Cache everything possible.

Make asynchronous everything possible.

Apply traffic‑shaping (sharding, rate‑limiting, queueing).

Isolate failures so they do not cascade.

Typical tech stack:

Redis – hot cache, idempotency token, limited distributed lock.

Kafka / RocketMQ – peak‑shaving, async coordination, delayed compensation.

MySQL sharding – final transaction persistence.

Read model / Elasticsearch – cross‑domain query and complex list rendering.

Production‑Ready Code Samples

Order API with Idempotency and 202 Accepted

@RestController
@RequestMapping("/orders")
public class OrderController {
    private final OrderApplicationService orderApplicationService;
    public OrderController(OrderApplicationService orderApplicationService) {
        this.orderApplicationService = orderApplicationService;
    }
    @PostMapping
    public ResponseEntity<CreateOrderResponse> create(
            @RequestHeader("X-Request-Id") String requestId,
            @RequestBody CreateOrderRequest request,
            @RequestHeader("X-User-Id") Long userId) {
        CreateOrderCommand command = new CreateOrderCommand(requestId, userId, request.getItems(), request.getAddressId());
        OrderDraft draft = orderApplicationService.accept(command);
        return ResponseEntity.accepted().body(new CreateOrderResponse(draft.getOrderId(), draft.getStatus().name()));
    }
}

Key points:

Use X-Request-Id as idempotency key.

Return 202 to indicate acceptance, not immediate success.

Separate acceptance from final fulfillment.

Idempotent Acceptance Logic

public OrderDraft accept(CreateOrderCommand command) {
    Optional<OrderDraft> existing = orderDraftRepository.findByRequestId(command.getRequestId());
    if (existing.isPresent()) {
        return existing.get();
    }
    OrderDraft draft = createOrder(command);
    return draft;
}

Database unique indexes on request_id and order_no provide a second line of defence.

Outbox Relay for Reliable Event Delivery

@Scheduled(fixedDelay = 1000)
public void relay() {
    List<OutboxMessage> messages = outboxRepository.findTop100ByStatusOrderByIdAsc(MessageStatus.NEW);
    for (OutboxMessage message : messages) {
        try {
            kafkaTemplate.send(message.getTopic(), message.getAggregateId(), message.getPayload()).get();
            message.markSent();
        } catch (Exception ex) {
            message.markRetry(ex.getMessage());
        } finally {
            outboxRepository.save(message);
        }
    }
}

Feign Client with Fallback

@FeignClient(name = "user-service", fallbackFactory = UserClientFallbackFactory.class)
public interface UserClient {
    @GetMapping("/users/{id}")
    UserDTO getUser(@PathVariable("id") Long id);
}

@Component
public class UserClientFallbackFactory implements FallbackFactory<UserClient> {
    @Override
    public UserClient create(Throwable cause) {
        return id -> UserDTO.degraded(id, "UNKNOWN");
    }
}

Fallback returns a degraded but meaningful business object; not every interface should be retried, and not every failure should be swallowed.

Idempotent Consumer Example

@KafkaListener(topics = "order-events", groupId = "inventory-service")
public void onOrderCreated(OrderCreatedEvent event) {
    if (consumeLogRepository.existsByMessageId(event.getMessageId())) {
        return;
    }
    inventoryDomainService.reserve(event.getOrderId(), event.getItems());
    consumeLogRepository.save(new ConsumeLog(event.getMessageId(), "inventory-service"));
}

Consumers must be idempotent because the message system guarantees at‑least‑once delivery.

Migration Checklist (Converted from Table)

Independent deployment : CI/CD, containerisation, GitOps, canary release.

Independent scaling : HPA, KEDA, thread‑pool isolation, rate‑limiting.

Failure isolation : timeout, circuit‑breaker, bulkhead, retry budget.

Data decoupling : Outbox, Saga, idempotency, compensation.

Team autonomy : API specifications, contract testing, version management.

Distributed scaling : logs, metrics, tracing, SLO alerts.

Recommended Process for a Microservice Refactor

Current‑state diagnosis : identify bottleneck modules, dependency graph, DB access map.

Domain delimitation : define bounded contexts, data ownership, prioritize split candidates.

Infrastructure foundation : set up CI/CD, observability, service registry, config centre.

In‑monolith convergence : refactor the monolith to enforce module boundaries before extracting services.

Independent deployment : package each module, publish Docker images, register with gateway, establish contract tests.

Canary verification : gradually route a small traffic slice, compare new vs. old metrics, roll back if needed.

Data migration : use read‑write dual‑write, outbox, and eventual cut‑over of tables.

Continuous evolution : incrementally async‑ify, add elasticity, push governance deeper.

Conclusion

Microservice transformation is not a silver bullet. Success depends on understanding why the change is needed and having the engineering capability to solve the new distributed problems it introduces. A pragmatic path starts with solid governance foundations, then splits clear‑boundary modules (user centre is a typical first win), defers database separation, and continuously validates observability and delivery pipelines before scaling the number of services.