Managed Service for Prometheus actively pulls performance data from your Kafka cluster based on the specified parameters. This way, you can monitor the cluster in real time and flexibly analyze the monitoring data. This topic describes how to monitor a Kafka cluster.
Prerequisites
Container cluster
Managed Service for Prometheus is activated. For more information, see Billing overview.
A Container Service for Kubernetes (ACK) cluster is created. For more information, see Create an ACK managed cluster.
Alibaba Cloud Resource Center is activated. For more information, see Activate Resource Center.
ECS instance
Managed Service for Prometheus is activated. For more information, see Billing overview.
An Elastic Compute Service (ECS) instance is created. For more information, see Create and manage an ECS instance by using the ECS console (express version).
Alibaba Cloud Resource Center is activated. For more information, see Activate Resource Center.
Procedure
Log on to the ARMS console. In the left-side navigation pane, click Integration Center.
Click Kafka and set the parameters as prompted. The following tables describe the key parameters.
Container cluster
Parameter
Description
Pod Tag
The tag key and value configured for the pod when the Java Management Extensions (JMX) agent was deployed. Managed Service for Prometheus uses this tag for service discovery. For more information, see How do I deploy and configure a Kafka JMX agent?
Metric collection interval
The interval for collecting monitoring data. The default value is 15.
ECS instance
Paramter
Description
Kafka cluster name
The name of the Kafka cluster. The name must be unique to prevent duplicate metric collection.
Service address
The IP or DNS address of the Kafka broker. Separate multiple addresses with commas (,) or semicolons (;).
Example: 192.168.0.1:9092,10.0.11.123:9092.
Enable SASL
Specify whether to enable Simple Authentication and Security Layer (SASL) on the brokers.
SASL Username
Enter the SASL username if SASL is enabled.
SASL Password
Enter the SASL password if SASL is enabled.
SASL Mechanism
Select an authentication method. Valid values: plain, scram-sha512, and scram-sha256.
Enable TLS
Specify whether to enable Transport Layer Security (TLS) on the brokers.
Ignore TLS security verification
Turn on the switch if TLS is enabled and a self-signed certificate is used.
Metric collection interval
The interval for collecting monitoring data. The default value is 15.
Custom Labels
Add tags to the metrics of the ECS instance. Then, you can monitor, manage, query, or analyze the metric data in a fine-grained manner when you use tools such as Grafana. For more information, see Add tags to the metrics of an ECS instance using a VPC.
NoteA tag key cannot contain special characters used in the Prometheus functionality, such as hyphens (-), periods (.), or percent signs (%).
Tags must be specified in the key-value format required by Prometheus.
View and manage the monitoring data of the Kafka cluster
The installed component is displayed on the Integration Management page in the ARMS console. The Integration Management page consists of the Integrated Environments, Integrated Addons, and Query Dashboards tabs, where you can view information such as targets, metrics, dashboards, and alerts.
Targets
In the Targets section of the Self-Monitoring tab, you can view the list of targets discovered by the default jobs.
Metric Scraping
On the Metric Scraping tab, you can view or discard metrics. For more information, see Discard metrics.
Dashboards
In the Dashboards section of the Component Management tab, you can click a dashboard name to go to Grafana and view more dashboard data.
References
You can configure alert rules for the cluster to monitor the health status of key metrics in real time. This way, you can discover and handle exceptions in a timely manner. For more information, see Create an alert rule for a Prometheus instance.
For more information about how to monitoring a Kafka cluster in Managed Service for Prometheus, see Use Managed Service for Prometheus to monitor self-managed Kafka clusters and ApsaraMQ for Kafka instances and .