You can modify the stream-node-max-bytes parameter to specify the maximum memory that can be used by each macro node in streams. You can also modify the stream-node-max-entries parameter to specify the maximum number of stream entries that can be stored on each macro node.

Prerequisites

The engine version of the ApsaraDB for Redis instance is Redis 5.0 and later.

Relationship between Redis streams and macro nodes

Stream is a new data type introduced with Redis 5.0. Redis streams are represented as delta-compressed macro nodes that are linked together by the radix tree. Each macro node stores multiple stream entries. You can use this data structure to access random elements, obtain elements in a specified range, and create capped streams with high efficiency. This data structure also optimizes memory usage.

You can specify the stream-node-max-entries parameter to limit the maximum stream entries supported by each macro node. You can specify the stream-node-max-bytes parameter to limit the maximum memory consumed by each macro node.

  • stream-node-max-entries: The default value is 100, which specifies that each macro node can store up to 100 stream entries. Valid values: 0 to 999,999,999,999,999. A value of 0 specifies that each macro node can store an unlimited number of stream entries. If the number of stream entries stored in a macro node reaches the upper limit, new stream entries are stored on a new macro node.
  • stream-node-max-bytes: Unit: bytes. The default value is 4096, which specifies that each macro node can consume up to 4,096 bytes of the memory. Valid values: 0 to 999,999,999,999,999. A value of 0 specifies that each macro node can store an unlimited number of stream entries.

Scenarios

You can specify the stream-node-max-entries parameter to adjust the length deviation of a fixed-length message queue.

If your application does not need to store messages permanently, you can use the MAXLEN parameter to specify the maximum number of messages in a stream when you run the XADD command to add a message to the stream. Example:

XADD cappedstream MAXLEN 5000 * field value5001 // Add a value value5001 to field1 of cappedstream and set the maximum number of messages to 5,000.
When the number of messages in the stream reaches the upper limit, the earliest message is deleted each time a new message is added. No matter how many messages are added, the maximum length of the stream remains unchanged. In addition, the memory consumed by a message is released after the message is deleted.
Note When you delete a message from a macro node, the message is marked as deleted, but the memory consumed by the message is not released immediately. ApsaraDB for Redis deletes a macro node and releases the consumed memory only when all messages on the macro node are marked as deleted.

However, if you set the maximum queue length to an exact value such as 5,000 messages, precise control is achieved at the sacrifice of performance. To optimize memory usage, Redis streams are represented as delta-compressed macro nodes that are linked together by a radix tree. Each time ApsaraDB for Redis deletes a message, it must search the macro node for the message and mark the message as deleted. This mechanism is not optimal for high-throughput ApsaraDB for Redis services where messages are deleted and added frequently. The performance of ApsaraDB for Redis deteriorates if it deletes messages frequently. Therefore, we recommend that you add a tilde (~) in the XADD command to specify an approximate maximum length. Example:

XADD cappedstream MAXLEN ~ 5000 * field value1 // Add a value value5001 to field1 of cappedstream and set the maximum number of messages to approximately 5,000.

The actual length of the stream can be an approximate value greater than or equal to the specified value. For example, the stream may contain 5000, 5050, or 5060 messages. The deviation from 5000 depends on the number of macro nodes in the stream and the maximum number of messages that can be stored in each macro node. ApsaraDB for Redis calculates the approximate value based on the stream-node-max-entries parameter. This parameter specifies the maximum number of messages that can be stored on each macro node. If the number of messages stored in the stream exceeds this approximate value, ApsaraDB for Redis deletes the macro node that stores the earliest messages, instead of deleting specific messages.

The value of the stream-node-max-entries parameter determines the length deviation of a fixed-length message queue. To reduce the deviation, you can set the parameter to a proper smaller value.

Specify the parameter in the ApsaraDB for Redis console

  1. Log on to the ApsaraDB for Redis console.
  2. In the upper-left corner of the page, select the region where the instance is deployed.
  3. On the Instances page, click the instance ID or click Manage in the Actions column.
  4. On the Instance Information page, click System Parameters in the left-side navigation pane.
    Note The following steps describe how to specify the stream-node-max-bytes parameter.
  5. On the page that appears, find the stream-node-max-bytes parameter and click Modify in the Actions column.
  6. In the dialog box that appears, follow these steps to specify the parameter.
    1. Change the value of the stream-node-max-bytes parameter based on your business requirements.
    2. Click OK.

Call API operations to set parameters

You can also call the ModifyInstanceConfig operation to set parameters.