By Jingquan
When an AI agent invokes a tool, the call may return in a few hundred milliseconds—or run for several minutes, for example, to search a large body of material, generate lengthy content, or execute a batch job. Keeping the request open while waiting for the result ties up connections and resources for extended periods and does not scale well to high concurrency.
Model Context Protocol (MCP) is widely used for tool invocation by AI agents. Through the Tasks extension, it defines an asynchronous task mechanism that eliminates the need for requests to wait in place, but leaves a more important implementation question unanswered: where exactly is task state stored?
This article explains what the problem is, why it must be solved, and how we solve it with RocketMQ LiteTopic.
The official name of this mechanism is the Tasks extension (SEP-2663, now Final).
Without the Tasks extension, MCP tool calls (tools/call) are typically synchronous: a request is sent, the tool runs to completion, and the result returns over the same path. The longer the tool runs, the longer the call occupies connections and resources.
The Tasks extension changes this. When the server identifies a call as a long-running task, it does not return the result immediately. Instead, it first returns a task handle—essentially a task ID (taskId).
Once it has the handle, the client has two official ways to obtain the final result:
● Polling tasks/get: repeatedly ask whether the task is done using the taskId;
● Subscribing to notifications/tasks: whenever task state changes, the server pushes an update, so the client does not need to keep asking.
The protocol itself is cleanly designed. But it defines only what a handle looks like and how to retrieve the result; it leaves one more important question unanswered—where the task state from **working to **completed is actually stored.
The simplest approach is to store task state in the memory of the service replica handling the request.
That works for a single-node demo, but once the system goes into production with multiple backend replicas, it immediately runs into three pitfalls:
● Switch replicas and the task can no longer be found. If the client's second poll is load-balanced to another replica, that replica has no record of the task in memory. The protocol therefore requires the client to send the Mcp-Name header for routing affinity, attempting to direct requests for the same task to the same replica. But affinity is a gateway behavior: scaling, restarts, or a gateway that does not recognize the header will all cause the query to fail.
● If a replica fails, its state is lost. The task may still be executing and may still produce a result, but the replica that remembered it is gone. The client presents a valid handle only to be told that the task does not exist.
● The client cannot prove otherwise. It cannot distinguish between a task that truly does not exist and a request that merely reached the wrong replica, so it can only stop polling and abandon the task.
During the same period, MCP established another requirement: servers must be stateless (SEP-2575, “Make MCP Stateless”)—any replica must be able to process any request, and state cannot be tied to a particular process. Workloads requiring persistence were explicitly directed to use Tasks instead.
The contradiction is now clear: every replica is expected to answer any request, yet task state lives only in the memory of one replica.
This is an intentional gap in the MCP protocol: MCP defines only how tasks are created, queried, and reported. Where state is stored—and on which machine—is explicitly left to implementers. The official SDK also states that it provides no cross-process message bus.
This gap is best filled by messaging infrastructure—durability, ordering, retry on failure, and reachability across replicas are general requirements in agent scenarios and native capabilities of a message queue.
Our approach is to entrust task state to RocketMQ. We call this design Task as LiteTopic: each task receives a dedicated LiteTopic channel, and the sequence of messages in that channel becomes the task's state ledger.
LiteTopic (a lightweight topic) is a queue form introduced by RocketMQ for AI workloads: a single Broker can support millions of independent channels, created on demand and automatically reclaimed when their TTL expires, with almost no additional maintenance. It can comfortably assign every task a channel named by its taskId.
The core lifecycle of a task has three steps:
working plus the creation time), then returns the task handle to the client;The messages themselves are the state; RocketMQ is the state store. The “task disappears on another replica” problem is solved because task state no longer exists on only one replica. The “state is lost when a replica fails” problem is also solved because state lives in the messaging system and does not disappear when a replica exits.
● Ordered within the channel; the last message is the current state
A channel maps uniquely to a single queue on one Broker. Write order is therefore read order, so the last record is the task's current state. No ordered replay or reduction by state priority is required. The protocol allows a task to move from “waiting for user input” back to “in progress”; in this ledger, that transition is simply another appended record.
● One write, two consumption planes
A LiteTopic message is naturally indexed twice in the broker core—the parent Topic's own queue and the LiteTopic channel queue. The same message can therefore serve two groups of consumers without a second Topic or redundant snapshots:
| Consumption plane | Consumer | Isolation method |
|---|---|---|
| Parent Topic plane | Execution pool (shared consumer group, competing consumption; each task runs once) | Use TAG filtering to accept only submission commands and block result writes; otherwise, the execution pool would treat its own output as a new task and feed itself indefinitely |
| Channel plane | Per-replica state following | Subscribe precisely by channel name; use an independent consumer group for each replica. A shared group would make replicas compete for messages and reset one another's offsets, causing intermittent missing reads |
● No task loss: the task handle is returned only after the submission message has been durably written to RocketMQ, so restarts and scale-in do not lose tasks.
● Automatic retries: a task message whose execution fails is automatically redelivered; the client does not need to submit it again. This semantic is impossible when state is kept only in memory. The prerequisite is tool idempotency: under at-least-once delivery, rerunning the task should only append a record with the same content. For tools with external side effects, such as transferring funds or sending email, the application must enforce idempotency.
● Migration across replicas: the shared consumer group decides which replica executes the task. The execution replica is fully decoupled from the replica that created the task, and execution capacity scales horizontally with Workers.
● Addressability: results remain permanently in the task's channel. Any replica can answer a query for the task, and replicas do not need to store task state themselves.
This solution is open-sourced in Apache RocketMQ's AI repository: apache/rocketmq-ai.
The mcp-tasks-rocketmq/ directory contains the Python implementation of the MCP Tasks extension (SEP-2663) on RocketMQ LiteTopic described here: a RocketMQ execution pool runs tasks, while each task maps to a LiteTopic channel that serves as its state ledger.
For usage details, see README.md in that directory. A runnable end-to-end example is available under examples/.
MCP defines the semantics of asynchronous tasks precisely while leaving the concrete but critical question of where state lives to implementers, drawing a clear boundary between the protocol layer and the infrastructure layer.
Our approach is straightforward: give each task a dedicated channel, make its lifecycle messages its own state ledger, and entrust that ledger to RocketMQ.
As a result, MCP server replicas remain stateless and can restart or scale freely. Task dispatch and state storage both rest on the same messaging foundation—a foundation that has already supported tens of thousands of concurrent AI inference instances. Connecting it to MCP requires only a protocol adapter: no protocol changes, no additional semantics, and only the extension points already defined by the specification.
For RocketMQ, this is another step in its evolution from messaging middleware into collaboration infrastructure for AI agents: LiteTopic has been merged into the Apache RocketMQ main branch, and the MCP implementation has been open-sourced in apache/rocketmq-ai.
When agents truly enter production—where tasks run for hours or even days and sessions number in the thousands—reliably carrying these messages, states, and events still requires a messaging foundation repeatedly proven at production scale.
Inside STAROps: An Intelligent Log Inspection Architecture for Surfacing Unknown Anomalies
The AI Agent Era Doesn't Need More Data — It Needs a Semantic Layer
789 posts | 60 followers
FollowAlibaba Cloud Native Community - June 8, 2026
Alibaba Cloud Native Community - September 16, 2026
Alibaba Cloud Native Community - May 14, 2026
Alibaba Cloud Native Community - October 22, 2025
Alibaba Cloud Native Community - May 19, 2026
Alibaba Cloud Native Community - September 30, 2026
789 posts | 60 followers
Follow
ApsaraMQ for RocketMQ
ApsaraMQ for RocketMQ is a distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications.
Learn More
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn More
Cloud-Native Applications Management Solution
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn More
Application Real-Time Monitoring Service
Build business monitoring capabilities with real time response based on frontend monitoring, application monitoring, and custom business monitoring capabilities
Learn MoreMore Posts by Alibaba Cloud Native Community