Optimizing ServiceComb Communication: Reactive vs Sync Thread Models

Overall Introduction

ServiceComb's underlying communication framework relies on Vert.x, which uses a high‑performance reactive model.

Business logic runs directly in the EventLoop without thread switches, and all waiting logic is asynchronous, fully utilizing system resources.

Vert.x provides reactive wrappers for common components (JDBC, Zookeeper, MQ, etc.), but reactive mode demands non‑blocking business code. To simplify development, ServiceComb adds a synchronous API on top of Vert.x.

Some security components only offer synchronous mode (e.g., Redis, Zookeeper).

Legacy code may need low‑cost migration to sync mode.

Developers may lack skills to handle reactive logic.

Therefore, ServiceComb extends Vert.x with support for both reactive and synchronous modes.

Synchronous Mode Thread Model

Key points:

Each microservice process creates an independent Vert.x instance for transport.

EventLoop is Vert.x’s network/task thread.

Default EventLoop count = 2 × Runtime.getRuntime().availableProcessors().

Service Consumer Side

The consumer focuses on efficiently pushing requests to providers and receiving responses.

Single‑Connection Model

1. Simple single‑connection model

All consumer threads sending to the same target compete for resources; Connection.send directly calls Vert.x socket.write, which locks the socket, causing contention under high concurrency.

Socket.write invokes Netty channel.write; if the calling thread is not the EventLoop, a task is queued and the EventLoop may be woken up, increasing CPU usage.

2. Optimized single‑connection model

Improvements:

Each TcpClientConnection gets an extra CAS queue to reduce enqueue contention.

Connection.send no longer calls Vert.x write directly; messages are stored in the CAS queue.

Only when the CAS queue is empty is a write task submitted to the EventLoop, minimizing wake‑ups.

Write tasks batch multiple requests into a composite buffer, reducing kernel calls and improving TCP efficiency.

Code references:

io.servicecomb.foundation.vertx.client.tcp.TcpClientConnection.packageQueue
io.servicecomb.foundation.vertx.client.tcp.TcpClientConnection.send(...)
io.servicecomb.foundation.vertx.tcp.TcpConnection.write(ByteBuf)
io.servicecomb.foundation.vertx.tcp.TcpConnection.writeInContext()

Performance comparison (single‑connection):

Scenario

TPS

Latency (ms)

CPU Consumer

CPU Producer

TPS ↑

Latency ↓

Before optimization

81986

1.22

290%

290%

77.31%

43.61%

After optimization

145369

0.688

270%

270%

Multi‑Connection Model

When hardware (multi‑core CPUs, 10 Gbps NICs with RSS) can support it, multiple connections between a consumer and a producer fully exploit resources.

Configure multiple EventLoop threads.

Configuration example (microservice.yaml):

cse:
  highway:
    client:
      thread-count: <span>...</span>
    server:
      thread-count: <span>...</span>

Service Provider Side

Providers mainly wait for requests, process them, and return responses, focusing on efficient data reception and handling.

In synchronous mode, business logic and I/O are separated; each operation runs in an isolated thread pool ("warehouse" principle) to ensure stability.

Thread‑Pool Strategies

1. Single ThreadPool (ThreadPoolExecutor)

Bottlenecks: all EventLoops submit tasks to a single blocking queue, and all worker threads compete for tasks.

2. Multiple ThreadPools (ThreadPoolExecutor)

EventLoop threads are evenly bound to separate thread pools, reducing lock contention and isolating queues per pool.

ServiceComb uses a default FixedThreadExecutor with two real thread pools; the number of threads per pool can be configured via servicecomb.executor settings.

Configuration example:

servicecomb:
  executor:
    default:
      group: <span>number of real thread pools</span>
      thread-per-group: <span>threads per pool</span>

3. Warehouse (Isolation) Model

Different business operations are assigned to separate thread pools to prevent slow requests from blocking fast ones.

4. Flexible Thread‑Pool Strategy

Operations can be bound to specific thread pools during startup based on configuration keys such as cse.executors.Provider.[schemaId].[operationId]. If not set, defaults are used.

To create a custom thread pool, implement java.util.concurrent.Executor, register it as a bean, and bind it in microservice.yaml.

Summary of Thread‑Pool Model

The flexible thread‑pool configuration allows isolation of business logic, preventing a failure in one operation from affecting others. However, care must be taken to avoid configuring a single thread pool for all operations, which would re‑introduce bottlenecks.

Final Note : ServiceComb is not only used in Huawei Cloud’s microservice engine but also entered the Apache Incubator in December 2017, inviting the community to contribute and discuss implementation details.