All Products
Search

This Product

    Microservices Engine:Configure message canary release

    Document Center

    Microservices Engine:Configure message canary release

    Last Updated:Mar 11, 2026

    Enable message canary release for scenarios such as end-to-end canary release and development environment isolation. Message canary release extends canary semantics into RocketMQ by tagging each message with its environment context, so canary and baseline consumers process only the messages intended for them.

    Use message canary release to:

    • Verify consumption logic changes safely. Route a subset of messages to canary consumers to validate new logic before it handles production traffic.

    • Prevent traffic leakage in end-to-end canary releases. When message consumers generate downstream RPCs, untagged messages can escape canary routing and reach production nodes. Message tagging keeps the entire call chain within the correct environment.

    MSE supports message canary release for RocketMQ only, including both open-source and Alibaba Cloud commercial editions.

    How it works

    When a producer sends a message, the MSE Agent attaches the current environment context to the message attributes. During consumption, the agent filters messages based on this context so that each consumer receives only messages intended for its environment.

    Canary message classification

    A message is classified as a canary message in either of these cases:

    1. Produced by a canary node. Any message sent from a producer running in a canary environment.

    2. Produced by a baseline node handling canary traffic. When no canary producer exists, canary traffic falls back to baseline nodes. Messages produced under this fallback are also classified as canary messages.

    Default consumption behavior

    After you enable message canary release:

    • Baseline messages are consumed only by baseline consumers.

    • Canary messages are consumed by both baseline consumers and their corresponding canary consumers. Both groups independently consume the same canary message.

    To restrict canary messages to canary consumers only, see Configure canary message consumers.

    Compatibility

    RocketMQ Server

    EditionServer-side filteringClient-side filtering
    Open-source RocketMQ (4.5.0 or later)Supported (requires enablePropertyFilter=true in broker.conf)Supported
    Alibaba Cloud RocketMQ 4.xPlatinum Edition onlyAll editions
    Alibaba Cloud RocketMQ 5.xSupportedSupported

    RocketMQ Client

    ClientMinimum versionFiltering support
    Ons Client (Alibaba Cloud)1.8.0.Final or later (1.x series)--
    org.apache.rocketmq:rocketmq-client4.2.0 or later (4.x series)Server-side and client-side
    org.apache.rocketmq:rocketmq-client-java5.x series (requires MSE Agent 4.2.2+)Server-side only

    For differences between the open-source RocketMQ SDKs, see the RocketMQ official documentation.

    Note

    The MSE Agent does not support transactional messages.

    Limitations

    • Message canary release takes effect only when enabled for both producers and consumers in MSE service governance.

    • After you enable message canary release, the MSE Agent modifies the Consumer Group of each canary consumer by appending _<tag> to the original group name, where <tag> is the environment tag.

      For example, if the original Consumer Group is group1 and the environment tag is gray, the Consumer Group changes to group1_gray.

      Important

      If you use Alibaba Cloud RocketMQ, create the modified Consumer Group (for example, group1_gray) before you enable message canary release.

    • MSE uses SQL-92 filtering for server-side filtering. If you use open-source RocketMQ 4.x, set enablePropertyFilter=true in broker.conf to enable SQL-92 support.

    • If your setup does not support SQL-92 filtering, select client-side filtering when you enable message canary release. Client-side filtering requires each environment to process all messages, which increases the load on both producers and consumers. Avoid client-side filtering in production environments.

    Enable message canary release

    Before you start, make sure you have:

    • An MSE-managed application with both message producers and consumers connected to microservice governance

    • A supported RocketMQ setup (see Compatibility)

    • (If using Alibaba Cloud RocketMQ) A pre-created Consumer Group for each canary environment

    To enable message canary release:

    1. Log on to the MSE console, and select a region in the top navigation bar.

    2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

    3. On the Application list page, click the resource card of the desired application. In the left-side navigation pane, click Traffic management.

    4. Click the Message grayscale tab.

    5. Next to Open message grayscale, click Edit. Configure the filtering method and consumption rules, then click OK.

      Note

      After you change message canary release settings, restart all nodes of the application for the changes to take effect. This applies to both server-side and client-side filtering.

      Note

      Message canary release is a sub-feature of end-to-end canary release. For best results, also configure the related end-to-end canary release capabilities.

    Configure canary message consumers

    By default, both baseline and canary consumers receive canary messages. To restrict canary messages to canary consumers only, clear the Baseline Consumer checkbox for the corresponding tag.

    After this change, baseline consumers process only baseline messages and any canary messages for which the baseline consumer is explicitly selected.

    1. Log on to the MSE console, and select a region in the top navigation bar.

    2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

    3. On the Application list page, click the resource card of the desired application. In the left-side navigation pane, click Traffic management.

    4. Click the Message grayscale tab.

    5. In the Message Canary Release Configuration section, click Edit. Clear the Baseline Consumer checkbox for the corresponding tag, then click OK.

    Note

    Changes to consumption rules take effect dynamically without an application restart. However, if all canary consumers go offline while the Baseline Consumer checkbox is cleared, canary messages with that tag will not be consumed.

    Canary release workflows

    Permanent canary consumers

    Use this workflow when canary consumers run continuously alongside baseline consumers. A single configuration keeps canary messages routed to canary consumers and baseline messages routed to baseline consumers.

    Note

    If your messages are not idempotent, make sure canary and baseline consumers process only their respective messages. Follow the steps below.

    Before you start, make sure you have:

    • Both the message producer and consumer applications connected to microservice governance

    • Message canary release enabled, and all application nodes restarted

    Steps:

    1. Configure the consumption rule for the consumer application. For the canary tag, clear the Baseline Consumer checkbox.

    2. Wait 1 to 3 minutes for the subscription relationship on the RocketMQ side to update.

    3. Bring the canary consumer nodes online.

    4. Bring the canary producer nodes online to start sending canary messages.

    Dynamic canary consumers

    Use this workflow when canary nodes are temporary and tied to a specific release iteration. Because canary consumers may go offline after verification, adjust baseline consumer filtering to prevent message accumulation or loss.

    Release procedure:

    1. Configure the consumption rule for the consumer application. For the canary tag, clear the Baseline Consumer checkbox so that canary messages are consumed only by canary consumers. Wait 1 to 3 minutes for the subscription relationship on the RocketMQ side to update.

    2. Bring the canary consumer nodes online.

    3. Send canary messages from the canary producer nodes.

    4. Verify the canary messages.

    5. Take the canary producer nodes offline.

    6. Compare consumer group offsets. Check that the offset of the baseline consumer_group is less than or equal to the offset of the canary consumer_group. Use logs and monitoring to confirm that all canary messages have been consumed.

      Important

      If the baseline consumer_group offset is less than or equal to the canary consumer_group offset, baseline consumers can safely pick up any remaining canary messages. Some messages may be consumed twice -- make sure your business logic handles idempotency.

      If the baseline offset is greater than the canary offset, proceeding to the next step causes canary messages to be lost. Wait until the offsets align.

    7. Re-enable baseline consumption. For the canary tag, select the Baseline Consumer checkbox so that baseline consumers can process any remaining canary messages. Wait 1 to 3 minutes for the subscription relationship to update.

    8. Take the canary consumer nodes offline and delete the canary consumer_group.

      Important

      If you do not delete the canary consumer_group, messages accumulate in that group. The next time canary consumers come online, they resume consuming from the last recorded offset rather than the latest position.