ApsaraMQ for RocketMQ uses the metric storage and visualization capabilities of Managed Service for Prometheus and Managed Service for Grafana from Application Real-Time Monitoring Service (ARMS) to provide a real-time statistics dashboard. This feature lets you collect and monitor metrics to quickly understand your business's operational state. This topic describes the use cases, billing, available metrics, and how to use the dashboard.
Use cases
Receive immediate alerts for message consumption anomalies to quickly locate and resolve issues and ensure timely message processing.
Troubleshoot issues with specific orders, such as an abnormal status, by reviewing the message trace to verify proper message transmission throughout the workflow.
Analyze message traffic trends, distribution patterns, and overall volume to forecast business trends and plan resources.
Visualize and analyze your application dependency topology to understand upstream and downstream dependencies, simplifying planning for architectural upgrades, optimizations, and refactoring.
Prerequisites
Create a service-linked role.
Role name: AliyunServiceRoleForOns
Role policy name: AliyunServiceRolePolicyForOns
Permission description: Allows Message Queue for RocketMQ to assume the role and access CloudMonitor and ARMS for monitoring, alerting, and dashboard features.
For details, see Service-linked roles.
Billing
The dashboard metrics for ApsaraMQ for RocketMQ are basic metrics in Managed Service for Prometheus. Since these metrics are free, the dashboard feature is also free of charge.
For more information, see Metrics and Pay-as-you-go.
Key concepts
Before you view dashboard metrics, you need to understand the following concepts related to consumer lag.
The preceding figure shows the state of messages in a queue for a specific topic. Message Queue for RocketMQ collects statistics on the number and processing time of messages at different stages. These metrics directly reflect the message processing rate and consumer lag. Monitoring these metrics helps you quickly identify potential anomalies in message consumption.
Category | Metric | Description | Calculation formula |
Message count metrics | inflight messages | Messages that consumer clients are processing but have not yet acknowledged. | offset of the latest pulled message - offset of the latest committed message |
ready messages | Messages on the Message Queue for RocketMQ server that are ready for consumption. | Maximum message offset - offset of the latest pulled message | |
consumer lag | The total number of unprocessed messages. | Number of inflight messages + Number of ready messages | |
Message latency metrics | ready time |
| Not applicable |
ready message queue time | The difference between the current time and the ready time of the oldest ready message. This metric shows how quickly consumers pull messages. | Current time - Ready time of the oldest ready message | |
consumer lag time | The difference between the current time and the ready time of the oldest unacknowledged message. This metric shows how quickly consumers process messages. | Current time - Ready time of the oldest unacknowledged message |
Metrics
The dashboard for Message Queue for Apache RocketMQ provides the following metrics:
Producer: View metrics for a topic, such as the number of sent messages, send success rate, and send message response time.
Consumer: View metrics related to a group's subscription to a specific topic, such as the number of received messages, consumption success rate, and consumer lag.
Instance top 20 overview: View the top 20 values for specific metrics within an instance, along with their corresponding topic or group.
Billing metric overview: View metrics for a specific instance, such as send and receive TPS, number of API calls, and average message size. These metrics can help you estimate billing costs.
All metrics are collected at a 1-minute interval. You can query data from the last 15 days in Message Queue for Apache RocketMQ. The maximum time range for a single query is 24 hours.
Producer
Metric | Description |
Send message rate | The rate of messages sent and API calls made to a topic. Units:
|
Max send message rate | The maximum send message rate. Unit: messages/second. |
Total sent messages | The total number of messages sent in the specified instance. Unit: messages. |
Send API call success rate | The percentage of successful API calls for sending messages to a topic. |
Send message response time | The time it takes to send a message to a topic. Unit: ms. |
Consumer
Metric | Description |
Average consumption success rate | The percentage of messages that are successfully consumed by a group. |
Consumer lag (ready + inflight) | The total number of backlogged messages for a group. Unit: messages. |
Inflight messages | The number of messages that have been delivered to consumer clients but have not yet been acknowledged. Unit: messages. |
Ready messages | The number of messages on the Message Queue for Apache RocketMQ server that are available for consumption. This metric indicates the volume of messages that consumers have not yet started to process. Unit: messages. |
Ready message queueing time | The time elapsed since the oldest ready message became available for consumption. This metric is a key indicator of message processing delays, making it critical for time-sensitive applications. The overview value shows the average ready message queueing time for the instance, while the chart shows the ready message queueing time for a specific group subscribing to a specific topic. Unit: ms. |
Receive message rate | The rate at which a group consumes messages. Unit: messages/second |
Max receive message rate | The maximum message consumption rate. Unit: messages/second |
Total received messages | The total number of messages consumed by a group. Unit: messages. |
Consumer lag | The number of backlogged messages for a group. This includes both ready messages and inflight messages. Unit: messages. |
Message processing time | The time it takes for a group to process a message, from consumption start to completion. Unit: ms. |
Consumer-side wait time | The amount of time from when a message arrives at the consumer client until consumption starts. Unit: ms. |
Consumption success rate | The success rate of message consumption. |
Consumer client protocol ratio | The proportion of messages consumed by each client protocol. |
Instance Top 20
Metric | Description |
Top 20 topics by send message rate | The top 20 topics with the highest send message rate. Unit: messages/second. |
Top 20 groups by receive message rate | The top 20 groups with the highest message consumption rate. Unit: messages/second. |
Top 20 groups by ready messages | The top 20 groups with the largest number of ready messages. Unit: messages. |
Top 20 groups by ready message queueing time | The top 20 groups with the longest ready message queueing time. Unit: ms. |
Top 20 groups by consumer lag (ready + inflight) | The top 20 groups with the largest consumer lag. Unit: messages. |
Billing metrics
The following billing metrics account for multipliers for large messages and advanced features.
Large message multiplier: An API call is counted for every 4 KB of message size. For example, sending a 16 KB message is counted as four API calls (16 KB / 4 KB = 4).
Advanced feature message multiplier: API calls for advanced features, such as ordered, scheduled/delayed, and transactional messages, are counted at five times the rate of those for normal messages.
Metric | Description |
Max send TPS | The maximum transactions per second (TPS) for message production. This metric helps you estimate the peak TPS specification required for your instance. Unit: TPS. |
Max receive TPS | The maximum TPS for message consumption. This metric helps you estimate the peak TPS specification required for your instance. Unit: TPS. |
Max TPS | The maximum combined TPS for both message production and consumption. This metric helps you estimate the peak TPS specification required for your instance. Unit: TPS. |
Total API calls | The total number of API calls. This metric helps you estimate the API call usage for billing purposes. Unit: calls. |
Average message size | The average size of all produced messages. Unit: bytes. |
Send and receive TPS | The combined TPS for both message production and consumption. Unit: TPS. |
Total API calls per day | The daily total of API calls for both message production and consumption. Unit: calls. |
Metrics
All message throughput (TPS), request, and message count metrics use a 4 KB normal message as a baseline. The final value is multiplied by factors based on the actual message size and advanced message type.
The following table describes the fields in the metrics.
Field | Value |
Metric type | Gauge: A metric that can increase or decrease. It represents an instantaneous measurement, such as the TPS of API calls. |
Label |
|
Server-side metrics
Metric type | Metric name | Unit | Description | Label |
Gauge | rocketmq_instance_requests_threshold | count/s | The throttling threshold for the instance. |
|
Gauge | rocketmq_instance_requests_max | count/s | The maximum transactions per second (TPS) of the instance, excluding throttled requests. This value is the peak TPS recorded within a one-minute interval. |
|
Producer metrics
Metric type | Metric name | Unit | Description | Label |
Gauge | rocketmq_producer_requests (Billable metric) | count | The number of billable API calls for sending messages. |
|
Gauge | rocketmq_producer_messages | message | The number of sent messages. |
|
Gauge | rocketmq_producer_message_size_bytes | byte | The total size of sent messages. |
|
Gauge | rocketmq_producer_send_success_rate | % | The success rate for sending messages. |
|
Gauge | rocketmq_producer_failure_api_calls | count | The number of failed API calls for sending messages. |
|
Gauge | rocketmq_producer_send_rt_milliseconds_avg | ms | The average response time for sending messages. |
|
Gauge | rocketmq_producer_send_rt_milliseconds_min | ms | The minimum response time for sending messages. |
|
Gauge | rocketmq_producer_send_rt_milliseconds_max | ms | The maximum response time for sending messages. |
|
Gauge | rocketmq_producer_send_rt_milliseconds_p95 | ms | The 95th percentile of response time for sending messages. |
|
Gauge | rocketmq_producer_send_rt_milliseconds_p99 | ms | The 99th percentile of response time for sending messages. |
|
Consumer metrics
Metric type | Metric name | Unit | Description | Label |
Gauge | rocketmq_consumer_requests | count | The number of API calls for consuming messages. |
|
Gauge | rocketmq_consumer_send_back_requests | count | The number of API calls to send back messages after a consumption failure. |
|
Gauge | rocketmq_consumer_send_back_messages | message | The number of messages sent back after a consumption failure. |
|
Gauge | rocketmq_consumer_messages | message | The number of consumed messages. |
|
Gauge | rocketmq_consumer_message_size_bytes | byte | The total size of consumed messages. |
|
Gauge | rocketmq_consumer_ready_and_inflight_messages | message | The total consumer lag, calculated as the sum of ready messages and in-flight messages. |
|
Gauge | rocketmq_consumer_ready_messages | message | The number of messages in the queue waiting to be pulled by consumers. Calculated as: |
|
Gauge | rocketmq_consumer_inflight_messages | message | The number of messages pulled by consumers but not yet acknowledged. Calculated as: |
|
Gauge | rocketmq_consumer_queue_time_milliseconds | ms | The message queuing time. |
|
Gauge | rocketmq_consumer_message_await_time_milliseconds_avg | ms | The average time a message waits in the consumer client's buffer before being processed. |
|
Gauge | rocketmq_consumer_message_await_time_milliseconds_min | ms | The minimum time a message waits in the consumer client's buffer before being processed. |
|
Gauge | rocketmq_consumer_message_await_time_milliseconds_max | ms | The maximum time a message waits in the consumer client's buffer before being processed. |
|
Gauge | rocketmq_consumer_message_await_time_milliseconds_p95 | ms | The 95th percentile of the time a message waits in the consumer client's buffer before being processed. |
|
Gauge | rocketmq_consumer_message_await_time_milliseconds_p99 | ms | The 99th percentile of the time a message waits in the consumer client's buffer before being processed. |
|
Gauge | rocketmq_consumer_message_process_time_milliseconds_avg | ms | The average time a consumer spends processing a message. |
|
Gauge | rocketmq_consumer_message_process_time_milliseconds_min | ms | The minimum time a consumer spends processing a message. |
|
Gauge | rocketmq_consumer_message_process_time_milliseconds_max | ms | The maximum time a consumer spends processing a message. |
|
Gauge | rocketmq_consumer_message_process_time_milliseconds_p95 | ms | The 95th percentile of message processing time. |
|
Gauge | rocketmq_consumer_message_process_time_milliseconds_p99 | ms | The 99th percentile of message processing time. |
|
Gauge | rocketmq_consumer_consume_success_rate | % | The success rate for consuming messages. |
|
Gauge | rocketmq_consumer_failure_api_calls | count | The number of failed API calls for consuming messages. |
|
Gauge | rocketmq_consumer_to_dlq_messages | message | The number of messages sent to the dead-letter queue. |
|
Dashboard
Sign in to the Message Queue for Apache RocketMQ console. From the left navigation pane, click Instances.
In the top menu bar, select a region, such as China (Hangzhou). Then, in the instance list, click the target instance.
You can view the dashboard in any of the following ways:
Instance Details page: On the Instance Details page, click the Dashboard tab.
Dashboard tab: In the left navigation pane, click Dashboard.
Topic Details page: From the left navigation pane, click Topics. In the topic list, click the target topic. On the Topic Details page, click the Topic Overview tab.
Group Details page: From the left navigation pane, click Groups. In the group list, click the target group. On the Group Details page, click the Group Overview tab.
Dashboard FAQ
Access dashboard metric data
Use your root account to log in to the ARMS console.
In the left-side navigation pane, click Access Center.
On the Access Center page, enter
RocketMQin the search box and click the search icon.In the search results, select the cloud service that you want to integrate, such as Alibaba Cloud Message Queue for RocketMQ. For more information about the integration procedure, see Step 1: Integrate monitoring data from a cloud service.
After the integration is successful, click Integration Management in the left-side navigation pane.
In the Cloud Service Environment list, click the name of the target environment to open its details page.
On the Component Management tab, in the Basic Information section, click the cloud service region next to Prometheus instance.
This page provides different methods to access the data.{key, select, RAM {RAM} Overview {Overview} Identities {Identities} Users {Users} Groups {Groups} Roles {Roles} Settings {Settings} Permissions {Permissions} Grants {Grants} Troubleshoot {Troubleshooting} Policies {Policies} Integrations {Integrations} Providers {SSO} Applications {OAuth (Preview)} CloudSSO {CloudSSO} AccessAnalyzing {Access Analysis} Analyzers {Analyzers} Findings {Findings} Beta {Beta} Governance {Cloud Governance} other {{key}} }
Import data into a self-managed Grafana
All metric data for Alibaba Cloud Message Queue for RocketMQ is stored in your Alibaba Cloud Prometheus Service. Follow the steps in How do I get dashboard metric data? to integrate the cloud service and get the environment name and HTTP API endpoint. Then, use the API to import this data into your self-managed Grafana. For more information, see Use an HTTP API endpoint to connect to Grafana or a self-managed application to access Prometheus data.
Average and maximum TPS
average TPS = Total number of requests in a 1-minute period / 60 seconds
maximum TPS: The system samples the TPS value once per second over a 1-minute period. The maximum TPS is the highest value among the 60 samples.
For example:
Assume an instance produces 60 messages in one minute (all are normal messages of 4 KB each).
average TPS of the instance = 60 requests / 60 seconds = 1 request/second
The maximum TPS of the instance is calculated as follows:
If all 60 messages are sent within the first second, the TPS for each second of that minute is 60, 0, 0, ..., 0.
The maximum TPS of the instance is 60 requests/second.
If 40 messages are sent in the first second and 20 messages are sent in the second second, the TPS for each second of that minute is 40, 20, 0, 0, ..., 0.
The maximum TPS of the instance is 40 requests/second.