Mastering Graceful Shutdown in Dubbo: From 2.5.x to 2.7.x

1 Introduction

A year ago the author wrote about Linux kill signals and Spring Boot graceful shutdown; this piece uses Dubbo as a case study to discuss architecture, source code, and service‑governance details required for true graceful shutdown.

Dubbo 2.7.x and 2.6.x are the recommended releases; 2.7.x is the Apache‑donated version with many new features, while 2.6.x is maintenance‑only. Dubbo 2.5.x is no longer maintained and contains bugs that affect graceful shutdown.

Graceful shutdown is essential for full application lifecycle management, preventing incomplete requests and ensuring system robustness.

OS level: kill -9 (SIGKILL) and kill -15 (SIGTERM)

JVM level: shutdown hooks

Framework level: Spring Boot actuator and ContextClosedEvent

Container level: Docker stop (SIGTERM then SIGKILL after 10 s) and K8s pre‑stop hook

Application architecture level: different deployment strategies for monoliths vs microservices

The focus is on framework and architecture layers, using Dubbo as the representative microservice framework.

3 Graceful Shutdown Initial Scheme – 2.5.x

Dubbo 2.5.x implements a simple graceful shutdown via a JVM shutdown hook registered in AbstractConfig that calls ProtocolConfig.destroyAll().

3.1 Entry Class: AbstractConfig

public abstract class AbstractConfig implements Serializable{ static { Runtime.getRuntime().addShutdownHook(new Thread(new Runnable(){ public void run(){ ProtocolConfig.destroyAll(); }}, "DubboShutdownHook")); } }

The hook triggers ProtocolConfig.destroyAll().

3.2 ProtocolConfig

public static void destroyAll(){ if(!destroyed.compareAndSet(false,true)) return; AbstractRegistryFactory.destroyAll(); Thread.sleep(ConfigUtils.getServerShutdownTimeout()); ExtensionLoader<Protocol> loader = ExtensionLoader.getExtensionLoader(Protocol.class); for(String protocolName : loader.getLoadedExtensions()){ try{ Protocol protocol = loader.getLoadedExtension(protocolName); if(protocol != null){ protocol.destroy(); } }catch(Throwable t){ logger.warn(t.getMessage(), t); } } }

The method performs three steps: deregister from the registry, wait for registry notification (default 10 s), and destroy each loaded protocol.

3.3 Registry Deregistration Logic

public abstract class AbstractRegistryFactory implements RegistryFactory{ public static void destroyAll(){ LOCK.lock(); try{ for(Registry registry : getRegistries()){ try{ registry.destroy(); }catch(Throwable e){ LOGGER.error(e.getMessage(), e); } } REGISTRIES.clear(); }finally{ LOCK.unlock(); } } }

This removes the provider’s address from the registry.

3.4 Protocol/Process Deregistration Logic

ExtensionLoader<Protocol> loader = ExtensionLoader.getExtensionLoader(Protocol.class); for(String protocolName : loader.getLoadedExtensions()){ try{ Protocol protocol = loader.getLoadedExtension(protocolName); if(protocol != null){ protocol.destroy(); } }catch(Throwable t){ logger.warn(t.getMessage(), t); } }

Dubbo loads two protocols: DubboProtocol and Injvm. The focus is on DubboProtocol ’s destroy() method.

public class DubboProtocol extends AbstractProtocol{ public void destroy(){ for(String key : new ArrayList<String>(serverMap.keySet())){ ExchangeServer server = serverMap.remove(key); if(server != null){ server.close(ConfigUtils.getServerShutdownTimeout()); } } for(String key : new ArrayList<String>(referenceClientMap.keySet())){ ExchangeClient client = referenceClientMap.remove(key); if(client != null){ client.close(ConfigUtils.getServerShutdownTimeout()); } } for(String key : new ArrayList<String>(ghostClientMap.keySet())){ ExchangeClient client = ghostClientMap.remove(key); if(client != null){ client.close(ConfigUtils.getServerShutdownTimeout()); } } stubServiceMethodsMap.clear(); super.destroy(); } }

The shutdown sequence first deregisters servers (providers) then clients (consumers) to avoid new requests reaching a shutting‑down provider.

public void close(final int timeout){ startClose(); if(timeout > 0){ long max = timeout; long start = System.currentTimeMillis(); if(getUrl().getParameter(Constants.CHANNEL_SEND_READONLYEVENT_KEY, true)){ sendChannelReadOnlyEvent(); } while(HeaderExchangeServer.this.isRunning() && System.currentTimeMillis() - start < max){ try{ Thread.sleep(10); }catch(InterruptedException e){ logger.warn(e.getMessage(), e); } } } doClose(); server.close(timeout); }

The method waits for in‑flight requests to finish, stops heartbeat, and finally closes Netty resources.

Dubbo 2.5.3 has known concurrency bugs; upgrade is recommended.

4 Graceful Shutdown under Spring

When Dubbo runs inside Spring, both Dubbo and Spring register shutdown hooks, which can interfere. Dubbo introduced ShutdownHookListener (implements ApplicationListener) in 2.6.3 to handle ContextClosedEvent and invoke DubboShutdownHook.destroyAll().

private static class ShutdownHookListener implements ApplicationListener{ @Override public void onApplicationEvent(ApplicationEvent event){ if(event instanceof ContextClosedEvent){ DubboShutdownHook shutdownHook = DubboShutdownHook.getDubboShutdownHook(); shutdownHook.destroyAll(); } } }

Both ServiceBean and ReferenceBean trigger this listener.

Spring also registers its own JVM shutdown hook via AbstractApplicationContext.registerShutdownHook(), leading to two hooks.

4.2 Spring Container Close Events

Three ways to execute cleanup in Spring:

Implement DisposableBean Use @PreDestroy Register an ApplicationListener for ContextClosedEvent When using plain Spring (non‑Boot) or external Tomcat, context.registerShutdownHook() must be called explicitly.

4.3 Mid‑Level Solution Summary

Dubbo 2.6.x works well in non‑Spring, Spring Boot, and Spring + ContextClosedEvent environments.

5 Dubbo 2.7 Final Solution

public class SpringExtensionFactory implements ExtensionFactory{ public static void addApplicationContext(ApplicationContext context){ CONTEXTS.add(context); if(context instanceof ConfigurableApplicationContext){ ((ConfigurableApplicationContext) context).registerShutdownHook(); DubboShutdownHook.getDubboShutdownHook().unregister(); } BeanFactoryUtils.addApplicationListener(context, SHUTDOWN_HOOK_LISTENER); } }

This code registers Spring’s shutdown hook, unregisters the JVM hook when possible, and adds the listener, solving the two earlier concerns.

6 Conclusion

Graceful shutdown appears simple but becomes complex in a general framework like Dubbo due to diverse deployment scenarios. Understanding the evolution from 2.5.x to 2.7.x reveals many subtle issues and fixes, emphasizing the importance of iterative development and community contributions.

References to relevant pull requests are listed for further reading.