All Products
Search
Document Center

ApsaraMQ for RocketMQ:Troubleshooting

Last Updated:Jul 09, 2026

Message Queue for Apache RocketMQ provides a troubleshooting feature that analyzes exceptions from log files, runs backend service diagnostics on instances, and provides actionable suggestions. The diagnostic process may probe your instance for analysis but does not affect your instance's configuration or services.

Region limitations

The troubleshooting feature is available only in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and China (Chengdu).

Prerequisites

A RAM user must have the required permissions to use the troubleshooting feature. For more information, see Example 4: Grant a RAM user all permissions on instance diagnostics.

Log diagnostics

This feature diagnoses issues from log files with the .log extension and provides suggestions in a diagnostic report. For information about specific issues and their suggested resolutions, see Log diagnostics suggestions.

Procedure

  1. Log on to the Message Queue for Apache RocketMQ console. In the top navigation bar, select a region, for example, China (Hangzhou).

  2. In the navigation pane on the left, choose Troubleshooting.

  3. On the Troubleshooting page, click the Log Diagnostics tab. In the Log section, upload a log file with the .log extension, and then click Submit Diagnostics.

    Note

    The uploaded log file cannot exceed 64 MB.

  4. On the Diagnostic Report page, wait a few minutes for the task to finish, and then view the results. Alternatively, you can click View Later and Exit. On the Diagnostic History page, after the status of the diagnostic task that you just created changes to Complete, click Details in the Actions column to return to the Diagnostic Report page and view the diagnostic results.

Backend service diagnostics

This feature diagnoses an instance's backend service and provides suggestions in a diagnostic report. For information about specific issues and their suggested resolutions, see Backend service diagnostics suggestions.

Procedure

  1. Log on to the Message Queue for Apache RocketMQ console. In the top navigation bar, select a region, for example, China (Hangzhou).

  2. In the navigation pane on the left, choose Troubleshooting.

  3. On the Troubleshooting page, click the Backend Service Diagnostics tab. Configure the parameters, and then click Submit Diagnostics.

    5.x instances

    Parameter

    Description

    Example

    Instance

    Select the ID of the Message Queue for Apache RocketMQ instance.

    rmq-cn-vkl42*****

    Time Range

    Select the time range for the diagnostics.

    2024-12-24 10:30:31 - 2024-12-24 11:30:31

    4.x instances

    Parameter

    Description

    Example

    Instance

    Select the ID of the Message Queue for Apache RocketMQ instance.

    rmq-cn-vkl42*****

    Topic Name

    Select the topic to diagnose.

    test_topic

    SDK Type

    Select the SDK type.

    • HTTP SDK

    • TCP SDK

    TCP SDK

    Time Range

    Select the time range for the diagnostics.

    2024-12-24 10:30:31 - 2024-12-24 11:30:31

  4. On the Diagnostic Report page, wait a few minutes for the task to finish, and then view the results. Alternatively, you can click View Later and Exit. On the Diagnostic History page, after the status of the diagnostic task that you just created changes to Complete, click Details in the Actions column to return to the Diagnostic Report page and view the diagnostic results.

Network Connectivity Troubleshooting

If you experience connection issues between your client and the ApsaraMQ for RocketMQ instance, use the following scenario-based steps for targeted diagnosis.

VPC peering connection — Check whether the VPC route table of the ECS instance contains a route entry with the RocketMQ instance IP range as the destination and the peering connection ID as the next hop. Also verify that /etc/resolv.conf on the ECS instance is configured with Alibaba Cloud internal DNS servers (100.100.2.136 and 100.100.2.138).

Public network RemotingConnectException — Run nc -vz <endpoint> 8080 to test basic network connectivity. If the network is reachable but the error persists, compare your client code against the official SDK sample code to rule out configuration issues.

VPC internal network RemotingConnectException (Serverless instance) — Check whether the security group associated with the RocketMQ instance allows inbound traffic on ports 8080 and 8081. Also verify that the ECS security group's outbound rules allow traffic on those ports.

General port check — Verify the security group configurations of both the RocketMQ instance and the ECS instance, and confirm that the MQ allow-list includes the relevant ECS IP addresses.

Topic access topology

This feature shows the producers and consumers that access a specific topic on an instance during a selected time range.

Note

This feature is supported only for Message Queue for Apache RocketMQ 5.x instances.

Procedure

  1. Log on to the Message Queue for Apache RocketMQ console. In the top navigation bar, select a region, for example, China (Hangzhou).

  2. In the navigation pane on the left, choose Troubleshooting.

  3. On the Troubleshooting page, click the Topic Access Topology tab. Configure the parameters, and then click Submit Diagnostics.

    Parameter

    Description

    Example

    Instance

    Select the ID of the Message Queue for Apache RocketMQ instance.

    rmq-cn-vkl42*****

    Topic Name

    Select the topic to query.

    testTopic

    Time Range

    Select the time range for the diagnostics.

    2024-12-24 10:30:31 - 2024-12-24 11:30:31

  4. On the Diagnostic Report page, wait a few minutes for the task to finish, and then view the results. Alternatively, you can click View Later and Exit. On the Diagnostic History page, after the status of the diagnostic task that you just created changes to Complete, click Details in the Actions column to return to the Diagnostic Report page and view the diagnostic results.

Troubleshooting suggestions

Log diagnostics

Issue

Suggestion

Unrecorded Logs Exist

The diagnostic system did not record certain logs. For assistance, submit a ticket.

Signature Algorithm Not Found

Check your Java Development Kit (JDK) and operating system versions, and check for dependency conflicts.

Consumption Exception

Check your consumer code. If the issue persists, submit a ticket.

Messages not consumed (console shows "Processing" but no client-side logs) — Add a log statement at the consumer entry point to print the message ID, confirming whether messages are reaching the client. If the console shows no delivery trace and the client has no logs, focus your investigation on the client-side receive layer.

Ordered messages stuck (manual ACK ineffective) — Ordered messages are processed sequentially within a queue. If one message is stuck, all subsequent messages in that queue are blocked. The server does not support manual ACK for ordered messages. Investigate your consumer-side business logic for excessive processing time or deadlocks.

Ordered consumption repeated pull — This typically occurs when the consumer fails to process a message (throws an exception or does not return ACK before timeout), triggering a client-side retry. Check your local client logs at /{user.home}/logs/rocketmq/rocketmq-client.log to confirm failure records, and investigate client-side performance issues (CPU, memory, full GC, and so on).

consumeTimeout parameter not taking effect — Verify that your SDK version supports the consumeTimeout parameter and that the configured value complies with the specification requirements.

Failed to Send Heartbeat

Verify that the topic and group resources exist. If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. If the issue persists, submit a ticket.

Failed to Acknowledge Consumption

If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. If the issue persists, submit a ticket.

Failed to Pull Message

Verify that the topic and group resources exist. If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. If the issue persists, submit a ticket.

Failed to Send Message

Verify that the topic resource exists. If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. If the issue persists, submit a ticket.

Frequent producer instance creation causing send failures — If the error contains checkCurrentState or a connection timeout, the likely cause is that your code (for example, inside a retry loop) calls buildProducer() to create a new instance on each invocation. A producer is a heavyweight object and must be maintained as a global singleton. Avoid repeatedly creating and destroying producer instances, and ensure your application has fully started before sending messages.

ONSClientException error — In addition to checking security group ports, the MQ allow-list, and telnet connectivity, verify that your code correctly sets the Namespace parameter: properties.setProperty(PropertyKeyConst.Namespace, "Instance ID");

Failed to Obtain Route Information

Verify that the topic resource exists. If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. If the issue persists, submit a ticket.

Failed to Disconnect Client

If this issue is accompanied by a network or backend service exception, follow the suggestions for those issues. You can ignore this error if it occurs when the application process terminates.

Throttling Triggered on Broker

Log on to the Message Queue for Apache RocketMQ console. On the instance dashboard, check the producer and consumer TPS and the number of throttled requests to determine if an instance upgrade is needed. For more information, see Upgrade or downgrade an instance.

Consumer Group Not Found

Verify that the group for the client subscription exists. Log on to the Message Queue for Apache RocketMQ console. On the Instances page, click the instance that the client accesses. On the Groups page, check if the group exists.

Invalid AccessKey ID

Verify that the authentication credentials in your client are correct.

  • For 5.x instances:

    The instance username. You can obtain this on the Intelligent Authentication tab of the Access Control page in the Message Queue for Apache RocketMQ console.

  • For 4.x instances:

    The AccessKey ID, which is the identifier for your Alibaba Cloud account. For more information, see Create an AccessKey pair.

Invalid AccessKey Secret

Verify that the authentication credentials in your client are correct.

  • For 5.x instances:

    The instance password. You can obtain this on the Intelligent Authentication tab of the Access Control page in the Message Queue for Apache RocketMQ console.

  • For 4.x instances:

    The AccessKey Secret, which is the key for your Alibaba Cloud account. For more information, see Create an AccessKey pair.

Invalid Scheduled Time for Scheduled Messages

Adjust the message delivery delay or upgrade your instance. For information about the supported delivery delay for each instance edition, see Quotas and limits.

Invalid Message Attribute

Review the error log and check for conflicting message properties. For more information, see Internal message properties.

Message Type Mismatch

Ensure that the message type configured for the topic in the console matches the message type your code sends.

Full Local Message Cache

This indicates that the message consumption rate is too low. Review your consumption logic.

Network Error

A network error occurred. For assistance, submit a ticket.

No Online Consumer Client

Log on to the Message Queue for Apache RocketMQ console. On the Instances page, click the instance that the client accesses. Then, on the Groups page, click the group ID used by the client.

  • For 5.x instances:

    On the Group Details page, click the Running Information tab. In the Client Connection section, verify that the client is listed and check its consumption status.

  • For 4.x instances:

    In the Client Connection Information section, verify that the client is listed and check its consumption status.

Messages not consumed (console shows "Processing" but no client-side logs) — Add a log statement at the consumer entry point to print the message ID, confirming whether messages are reaching the client. If the console shows no delivery trace and the client has no logs, focus your investigation on the client-side receive layer.

Ordered messages stuck (manual ACK ineffective) — Ordered messages are processed sequentially within a queue. If one message is stuck, all subsequent messages in that queue are blocked. The server does not support manual ACK for ordered messages. Investigate your consumer-side business logic for excessive processing time or deadlocks.

Ordered consumption repeated pull — This typically occurs when the consumer fails to process a message (throws an exception or does not return ACK before timeout), triggering a client-side retry. Check your local client logs at /{user.home}/logs/rocketmq/rocketmq-client.log to confirm failure records, and investigate client-side performance issues (CPU, memory, full GC, and so on).

consumeTimeout parameter not taking effect — Verify that your SDK version supports the consumeTimeout parameter and that the configured value complies with the specification requirements.

Local Request Missing

Possible causes:

  1. The client is stuck due to an issue such as a full garbage collection (GC).

  2. The client retransmitted the request due to a network issue. You can ignore occasional occurrences.

Backend Service Exception

A broker-side exception occurred. For assistance, submit a ticket.

Inconsistent Subscriptions

Log on to the Message Queue for Apache RocketMQ console. On the Instances page, click the instance that the client accesses. On the Groups page, click the group ID that the client uses. On the Group Details page, check the Subscriptions section for inconsistencies. Ensure all consumers in a group subscribe to the same topics. For more information, see Subscriptions.

Subscription Does Not Exist

Check whether the consumer client is online and the network is functioning correctly.

Topic Not Found

Verify that the topic your client uses exists. Log on to the Message Queue for Apache RocketMQ console. On the Instances page, click the instance that the client accesses. On the Topics page, check if the topic exists.

Request Code Not Supported

A common example is error code 320, which indicates that batch messaging is not supported.

Backend service diagnostics

Check item

Issue

Suggestion

Instance

Instance Does Not Exist

Check the instance name.

Instance Not Running

Check the instance status.

Topic

Topic Does Not Exist

Check if the topic was created in the console.

Broker

Machine Failure

A machine in the backend broker cluster has failed. This may cause a temporary network error, which you can ignore.

Broker Update Released

The backend broker cluster is undergoing maintenance, which might cause a brief network disruption and temporary errors in client logs. You can ignore these errors. If you did not receive a maintenance notification, submit a ticket.

FAQ

Q: When the Broker processes heartbeat data (HeartbeatData), does it authenticate each consumer group independently, or does it authenticate all groups atomically?

A: The documentation does not commit to the specific atomicity or independence of the Broker's internal heartbeat processing logic. Use actual business behavior as the authoritative reference. If you encounter heartbeat-related exceptions, compare your client implementation against the standard sample code provided in the official documentation. This helps rule out issues caused by non-standard client implementations or differences in integration method.