Asynchronous Refactoring of an HTTP Service Using a Graph‑Based Execution Engine

Background

Our entry‑level HTTP service handled each user request synchronously, occupying a container thread while performing multiple remote calls. This caused three major problems: downstream service timeouts could exhaust the thread pool, serial remote calls increased response latency, and long‑running threads limited overall throughput.

Solution

We evaluated three asynchronous programming models: lightweight user‑level threads (fibers), the Actor model, and Rx (reactive extensions). Fibers such as libco or Quasar can schedule thousands of lightweight tasks but lack JVM support; the Actor model separates state via message passing but requires substantial code changes; Rx provides a functional, observable‑based API but also demands significant refactoring.

Considering our stack and migration cost, we chose a lightweight asynchronous redesign that combines Servlet 3.0 asynchronous support, Spring MVC async modules, and a custom Graph‑Based Execution Engine to manage complex service dependencies.

Principle Analysis

Servlet 3.0 introduced an asynchronous API, and Spring MVC has supported it since version 3.2. The following code illustrates the core idea:

AsyncContext context = HttpServletRequest.startAsync(req, response);

After initiating an asynchronous RPC call, we register a listener on the returned Future. When the RPC completes, the listener sets the result on a DeferredResult. Spring MVC detects the DeferredResult and continues processing, eventually invoking: context.dispatch(); This releases the original container thread while the RPC executes, and later re‑enters the Spring MVC interceptor chain to produce the final response.

Graph‑Based Execution Engine

Real‑world order processing involves multiple services (user, pricing, payment, discounts, etc.) with complex dependencies. Some services can run in parallel, while others must wait for predecessors. We model these dependencies as a directed acyclic graph (DAG) where each node represents a unit of work that returns a ListenableFuture.

Key features of the engine:

Dependency management: nodes declare their parent nodes; execution proceeds only when all parents have completed.

Unified callback registration: each node overrides a callback to handle its result or exceptions, managed centrally by the executor.

Asynchronous event‑driven execution: the engine triggers callbacks when conditions are met, eliminating nested callbacks in business code.

The engine builds the execution graph from configuration files (e.g., Spring bean definitions) that specify each node’s parents. A GraphBasedExecutor walks the graph, invoking apply(node, context) recursively; after a node finishes, the executor checks whether dependent nodes can now run.

By separating business logic from low‑level async mechanisms, developers focus on the core functionality of each node while the engine handles scheduling, error propagation, and resource utilization.

Postscript

During migration we encountered practical issues: using Dubbo’s async RPC, upgrading Tomcat to version 7 for Servlet 3.0 support, configuring <async-supported>true</async-supported> in web.xml and all filters, handling ThreadLocal context loss by passing data via request attributes, and adapting exception handling because asynchronous futures wrap exceptions in ExecutionException. The Graph‑Based Execution Engine can centralize exception conversion and propagation.

For further details, refer to the diagrams and code snippets embedded in the original article.