During rolling upgrades or node restarts, ApsaraMQ for RocketMQ uses graceful shutdown to drain active connections and persist in-flight messages before a storage node goes offline. This prevents message loss, avoids client errors, and keeps producers and consumers running with minimal disruption.
How it works
When a storage node needs to restart, ApsaraMQ for RocketMQ coordinates the shutdown in two phases:
Write suspension
The broker stops routing new write operations to the target storage node before the restart begins. Messages already submitted by producers are fully persisted before the node goes offline. This guarantees zero message loss during planned maintenance because no new data lands on a node that is about to shut down.
Connection draining via GOAWAY
The broker sends a GOAWAY frame (defined in the HTTP/2 specification) to every client connected to the target node. When a client receives this signal, it:
Completes any in-progress requests on the existing connection.
Opens a new connection to a different available node.
Because clients are notified in advance rather than disconnected abruptly, in-flight requests finish normally and no work is lost.
GOAWAY handling is built into the ApsaraMQ for RocketMQ client SDKs. If your SDK version meets the minimum requirements listed below, reconnection happens automatically with no application-level code changes.
Effects on producers and consumers
| Role | Behavior during graceful shutdown |
|---|---|
| Producer | All submitted messages are processed successfully. The write suspension phase guarantees that every message is persisted before the node goes offline. |
| Consumer | Most messages are consumed without interruption. In rare cases, a small number of messages on the restarting node may be redelivered because consumer offset commitment fails during the brief transition period. |
SDK version requirements
Graceful shutdown requires client SDK support for the GOAWAY frame. Verify that your SDK version meets or exceeds the minimums listed below.
| Language | Minimum version | Reference |
|---|---|---|
| Java (gRPC protocol) | 5.0.7 | SDK for Java |
| Java (Remoting protocol) | 5.3.1 | SDK for Java |
| Go | 5.1.0-rc.1 | SDK for Go |
| Python | 5.0.4 | SDK for Python |
| C++ | 5.0.2 | SDK for C++ |
| C# | 5.1.0 | SDK for C# |
If your SDK version is older than the minimum listed above, the client cannot process GOAWAY frames. Connections from outdated SDKs are terminated abruptly during node restarts, which may cause transient errors or message redelivery.
Usage notes
No client-side configuration required. Graceful shutdown is a server-side feature. Upgrading to a supported SDK version is the only action needed.
Design consumers for idempotency. Because consumer offset commitment may fail during the brief transition period, a small number of messages can be redelivered. Make sure your consumer logic can safely process the same message more than once.
Monitor SDK versions proactively. When you add new services or update dependencies, verify that the RocketMQ client SDK still meets the minimum version requirements to maintain graceful shutdown support.