You can modify the stream-node-max-bytes parameter to specify the maximum memory that can be used by each macro node in streams. You can also modify the stream-node-max-entries parameter to specify the maximum number of stream entries that can be stored on each macro node.

Prerequisites

The engine version of the ApsaraDB for Redis instance is Redis 5.0 or later.

Relationship between Redis streams and macro nodes

Stream is a new data type introduced with Redis 5.0. A Redis stream stores data on delta-compressed macro nodes that are connected in a radix tree. Each macro node stores multiple stream entries. You can use this data structure to access random elements, obtain elements within a specified range, and create capped streams with high efficiency. This data structure also optimizes memory usage.

You can specify the stream-node-max-entries parameter to limit the maximum stream entries supported by each macro node. You can specify the stream-node-max-bytes parameter to limit the maximum memory consumed by each macro node.

  • stream-node-max-entries: The default value is 100, which specifies that each macro node can store up to 100 stream entries. Valid values: 0 to 999,999,999,999,999. A value of 0 specifies that each macro node can store an unlimited number of stream entries. If the number of stream entries stored in a macro node reaches the upper limit, new stream entries are stored on a new macro node.
  • stream-node-max-bytes: Unit: bytes. The default value is 4096, which specifies that each macro node can consume up to 4,096 bytes of memory. Valid values: 0 to 999,999,999,999,999. A value of 0 specifies that each macro node can store an unlimited number of stream entries.

Typical scenarios

You can specify the stream-node-max-entries parameter to adjust the length deviation of a fixed-length message queue.

If your application does not need to store messages permanently, you can use the MAXLEN parameter to specify the maximum number of messages in a stream when you run the XADD command to add a message to the stream. Example:

XADD cappedstream MAXLEN 5000 * field value5001 //Add a value of value5001 to field1 of cappedstream and set the maximum number of messages to 5,000.
When the number of messages in the stream reaches the upper limit, the earliest message is deleted each time a new message is added. The maximum length of the stream remains unchanged regardless of the number of messages that are added. In addition, the memory consumed by a message is released after the message is deleted.
Note When you delete a message from a macro node, the message is marked as deleted, but the memory consumed by the message is not immediately released. ApsaraDB for Redis deletes a macro node and releases the consumed memory only when all messages on the macro node are marked as deleted.

If you set the maximum queue length to an exact value such as 5,000 messages, precise control is achieved, but performance is degraded. To optimize memory usage, a Redis stream stores data on delta-compressed macro nodes that are connected in a radix tree. Each time ApsaraDB for Redis deletes a message, it must search the macro node for the message and mark the message as deleted. This mechanism is not optimal for high-throughput ApsaraDB for Redis services where messages are frequently added and deleted. The performance of ApsaraDB for Redis degrades if it frequently deletes messages. Therefore, we recommend that you add a tilde (~) in the XADD command to specify an approximate maximum length. Example:

XADD cappedstream MAXLEN ~ 5000 * field value1 //Add a value of value5001 to field1 of cappedstream and set the maximum number of messages to approximately 5,000.

The actual length of the stream can be an approximate value greater than or equal to the specified value. For example, the stream may contain 5000, 5050, or 5060 messages. The deviation from 5000 depends on the number of macro nodes in the stream and the maximum number of messages that can be stored on each macro node. ApsaraDB for Redis calculates the approximate value based on the stream-node-max-entries parameter. This parameter specifies the maximum number of messages that can be stored on each macro node. If the number of messages stored in the stream exceeds this approximate value, ApsaraDB for Redis deletes the macro node that stores the earliest messages, instead of deleting specific messages.

The value of the stream-node-max-entries parameter determines the length deviation of a fixed-length message queue. To reduce the deviation, you can set the parameter to a proper smaller value.

Procedure

  1. Log on to the ApsaraDB for Redis console.
  2. On the top of the page, select the region where the instance is deployed.
  3. On the Instance List page, click the Instance ID of the instance.
  4. In the left-side navigation pane, click System Parameters.
    Note The following steps describe how to specify the stream-node-max-bytes parameter.
  5. On the page that appears, find the stream-node-max-bytes parameter and click Modify in the Actions column.
  6. In the dialog box that appears, perform the following steps:
    1. Change the value of the stream-node-max-bytes parameter based on your business requirements.
    2. Click OK.

Related operations

Operation Description
DescribeParameters Queries the configuration and operational parameters of an ApsaraDB for Redis instance.
ModifyInstanceConfig Modifies parameters of an ApsaraDB for Redis instance.