Alibaba Cloud Prometheus Service enables you to collect, store, and view monitoring data of Alibaba Cloud services in a unified manner. Prometheus Service also provides Grafana dashboards and alerts for cloud services.
This topic describes the metrics supported in Prometheus Service and how to use Prometheus Service with ApsaraMQ for Kafka. You can view monitoring data from the ApsaraMQ for Kafka console, the Managed Service for Grafana console, or a self-managed Grafana system.
Metrics
Metrics are organized by scope: instance, topic, and consumer group. The following tables list each metric name, description, and unit.
Instance metrics
| Metric name | Description | Unit |
|---|---|---|
instance_message_input | Inbound message throughput (production traffic) | bytes/s |
instance_message_output | Outbound message throughput (consumption traffic) | bytes/s |
instance_reqs_input | Rate of produce requests | requests/s |
instance_message_num_input | Rate of messages produced | messages/second |
instance_reqs_output | Rate of consume requests | requests/s |
instance_disk_capacity | Disk usage | % |
instance_tcp_num | TCP connections per node | Item |
instance_tcp_num_sum | Total TCP connections across all nodes | unit |
instance_public_tcp_num | Internet TCP connections per node | Item |
instance_public_tcp_num_sum | Total Internet TCP connections across all nodes | Unit |
Topic metrics
| Metric name | Description | Unit |
|---|---|---|
topic_message_input | Inbound message throughput (production traffic) | bytes/s |
topic_message_output | Outbound message throughput (consumption traffic) | bytes/s |
topic_reqs_input | Rate of produce requests | requests/s |
topic_reqs_output | Rate of consume requests | requests/s |
topic_message_num_input | Rate of messages produced | messages/second |
Consumer group metrics
| Metric name | Description | Unit |
|---|---|---|
message_accumulation | Total number of accumulated messages | Items |
message_accumulation_onetopic | Number of accumulated messages in a topic | Items |
rebalance_generation | Number of rebalances triggered | Count |
group_message_num_output | Total consumption rate | messages/second |
group_message_num_output_onetopic | Consumption rate for a single topic | messages/second |
group_message_num_output_onetopic_onepartition | Consumption rate for a single partition | messages/second |
View monitoring data in the ApsaraMQ for Kafka console
Log on to the ApsaraMQ for Kafka console.
On the Overview page, select a region in the Resource Distribution section.
On the Instances page, click the name of the target instance.
In the left-side navigation pane, choose Observability > Prometheus Monitoring.
On the Prometheus Monitoring page, select a time range in the upper-right corner.
To view metrics for a specific topic or consumer group, go to the details page of that topic or consumer group and click the Prometheus Monitoring tab.
View monitoring data in Grafana
Log on to the Managed Service for Grafana console.
In the left-side navigation pane, click Workspace Management.
On the Workspace Management page, click the URL of the target workspace to open Grafana.
Locate and open the ApsaraMQ for Kafka dashboard. The navigation path depends on your Grafana version.
Grafana 9.x
In the left-side navigation pane, click the
icon, and then click Browse.On the Browse tab, click a dashboard in the target folder.
Example: To view instance-level data for the China (Hangzhou) region, open the
kafka-instancedashboard in the folder namedcloud-product-prometheus_cn-hangzhou.
Grafana 10.x
In the upper-left corner of the Grafana homepage, click the
icon.In the left-side navigation pane, click Dashboard, and then click a dashboard in the target folder.
Example: To view instance-level data for the China (Hangzhou) region, open the
kafka-instancedashboard in the folder namedcloud-product-prometheus_cn-hangzhou.
Export metrics to a self-managed Grafana system
All ApsaraMQ for Kafka metric data is stored in Alibaba Cloud Managed Service for Prometheus. To import this data into a self-managed Grafana system, use the API operations provided by Managed Service for Prometheus.
For detailed steps, see Import data from Managed Service for Prometheus to a local Grafana system.