ApsaraMQ for RocketMQ retains messages for a configurable period called the storage duration. After this period expires, messages are automatically cleaned up regardless of whether they have been consumed.
Understanding message storage and cleanup helps you with the following aspects:
SLA for storage: Storage duration defines the time period in which consumers can obtain messages. This is relevant for scenarios that require long consumption periods, message accumulation handling, and fault recovery.
Storage cost evaluation: ApsaraMQ for RocketMQ stores messages on disks. You can evaluate storage space and reserve storage resources in advance.
How it works
Messages arrive at the broker and are appended to queues in the order they are received. In theory, the number of messages that a queue can store is unlimited. However, because the physical storage space of a broker is limited, messages cannot be permanently stored. Each message has a storage duration that starts from the moment it is produced and is not related to consumption status.
Messages within the storage duration are retained.
Messages past the storage duration are cleaned up, whether or not a consumer has read them.
The following sections explain three aspects of this mechanism: why time-based retention is used, why storage duration is managed per node, and why retention does not depend on consumption status.
Why time-based retention
Storage duration -- rather than message count or total size -- determines how long messages are kept. Time-based retention provides a predictable lifecycle: every message follows the same retention rule regardless of topic volume or consumer throughput.
Storage duration also acts as a Service Level Agreement (SLA) for consumers. Consumers must read their messages before the storage duration expires, or those messages become unavailable.
Why storage duration is managed per node
Storage duration is managed at the broker node level, not per topic or per queue. This design follows from the internal storage architecture:
Two-level data organization: ApsaraMQ for RocketMQ uses a unified two-level organization method that consists of physical log queues and lightweight logical queues. This structure delivers ordered read and write operations, high throughput, and high performance but does not support per-topic or per-queue retention policies.
Shared storage medium: Even when independent storage files are generated for different topics or queues, they share the same underlying storage medium. Per-topic retention cannot guarantee SLA if the cluster runs low on capacity.
To apply different retention periods to different workloads, deploy them on separate clusters with different storage duration configurations.
Retention does not depend on consumption status
Messages may accumulate in a queue when consumers are inactive or consuming slowly. Retaining all unconsumed messages indefinitely would exhaust storage and degrade read/write performance for new messages.
ApsaraMQ for RocketMQ applies a unified retention policy: all messages follow the same storage duration, consumed or not. During the storage duration, consumers can read a message at any time or re-read it multiple times by using the Reset a consumer offset feature.
Guaranteed retention on Alibaba Cloud
ApsaraMQ for RocketMQ is built on Alibaba Cloud cloud-native storage, which decouples storage capacity from local disk constraints. All messages are retained for their full specified storage duration. Forced deletion due to insufficient disk space does not occur.
Note: In open-source Apache RocketMQ, messages are stored on local disks. When disk space runs low, the system forcibly deletes messages to maintain stability, even if the configured storage duration has not expired. This limitation does not apply to ApsaraMQ for RocketMQ.
Usage notes
We recommend that you specify a longer storage duration based on your business requirements. A longer storage duration gives you more room for:
Emergency fault recovery
Emergency troubleshooting
Message backtracking