Why Circular Microservice Calls Trigger SocketTimeoutException and How to Resolve Them

Preliminary Analysis

After a recent iteration test, the system began to throw many java.net.SocketTimeoutException: Read timed out errors. Restarting the services only postponed the problem.

Note: Restarting in a test environment is not a good practice because it can mask the root cause and may lead to production incidents.

Tracing the call chain revealed a circular dependency between two microservices, Foo and Boo, both communicating over HTTP:

Client calls

Foo.hello()

Foo.hello()

invokes

Boo.boo()

Boo.boo()

calls back Foo.another() The real scenario involved a longer chain, but the essential problem was the circular call, which led to timeouts despite the operations being simple queries with negligible data size.

Removing the circular dependency eliminated the SocketTimeoutException, confirming the suspicion.

Exploring the Root Cause

To verify that the circular call caused the timeout, a more detailed diagram of the Foo service container was created:

If all threads in Foo's thread pool are waiting for Boo's response, and Boo simultaneously waits for Foo's another(), the threads become deadlocked. When the client request rate exceeds the processing capacity of this loop, the services stall, producing the observed timeouts.

Verification

Key code snippets (full project available at gitee.com/donghbcn/CircularDependency ) illustrate the setup.

Eureka Server

A minimal Eureka server is started to enable service discovery.

Service Foo

Spring Boot application that calls Boo via a Feign client. The Tomcat thread pool is limited to 16 threads to reproduce the deadlock quickly.

spring.application.name=demo-foo
server.port=8000
eureka.client.serviceUrl.defaultZone=http://localhost:8080/eureka
server.tomcat.threads.max=16

package com.cd.demofoo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class FooController {
    @Autowired
    BooFeignClient booFeignClient;

    @RequestMapping("/hello")
    public String hello() {
        long start = System.currentTimeMillis();
        System.out.println("[" + Thread.currentThread() + "] foo:hello called, call boo:boo now");
        booFeignClient.boo();
        System.out.println("[" + Thread.currentThread() + "] foo:hello called, call boo:boo, total cost:" + (System.currentTimeMillis() - start));
        return "hello world";
    }

    @RequestMapping("/another")
    public String another() {
        long start = System.currentTimeMillis();
        try {
            // simulate a slow call
            Thread.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println("foo:another called, total cost:" + (System.currentTimeMillis() - start));
        return "another";
    }
}

Service Boo

Spring Boot application that calls Foo's another() via a Feign client.

package com.cd.demoboo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class BooController {
    @Autowired
    FooFeignClient fooFeignClient;

    @RequestMapping("/boo")
    public String boo() {
        long start = System.currentTimeMillis();
        fooFeignClient.another();
        System.out.println("boo:boo called, call foo:another, total cost:" + (System.currentTimeMillis() - start));
        return "boo";
    }
}

JMeter Test

A JMeter test with 30 concurrent threads continuously invokes the client endpoint, quickly exhausting Foo's thread pool.

Foo's logs become blocked, and Boo soon starts throwing SocketTimeoutException:

Thread Dump (jstack)

The jstack output shows all Foo threads stuck inside hello(), confirming the deadlock.

Conclusion

Circular dependencies between microservices act like class-level cyclic dependencies: they can cause severe deadlocks and effectively turn a microservice architecture into a "distributed monolith" with tight coupling. Avoiding such loops is essential for maintaining the independence and resilience of each service.