Alibaba Cloud Managed Service for Prometheus allows you to install and configure Kafka exporters for self-managed Kafka instances. Managed Service for Prometheus also provides out-of-the-box dashboards that you can use to monitor your applications.
Background information
Alibaba Cloud Managed Service for Prometheus allows you to install and configure Kafka exporters for self-managed Kafka clusters deployed in Kubernetes clusters or on Elastic Compute Service (ECS) instances. The following types of Kubernetes clusters are supported: Alibaba Cloud Container Service for Kubernetes (ACK), ACK Serverless (ASK), and registered Kubernetes clusters. Managed Service for Prometheus provides Kafka exporters of the following editions:
Kafka (Basic Edition): Kafka (Basic Edition) exporters collect basic metrics such as the number of brokers, partitions of topics, and message group latency. You do not need to configure or restart the Kafka server. For more information about how to connect to Kafka (Basic Edition) exporters, see the Add a Kafka (Basic Edition) exporter section of this topic.
Kafka (Advanced Edition): Kafka (Advanced Edition) exporters have the capabilities of Kafka (Basic Edition) exporters and can collect important metrics of producers, brokers, consumers, and their internal modules based on a JMX agent. You can monitor the entire process of Kafka messages from the perspective of an expert by using the metrics. You need to manually deploy the JMX agent and restart the Kafka broker process. For more information about how to connect to Kafka (Advanced Edition) exporters, see the Add a Kafka (Advanced Edition) exporter section of this topic.
Go to the Integration Center page of Managed Service for Prometheus
- Log on to the Managed Service for Prometheus console.
- In the left-side navigation pane, click Monitoring List. The Prometheus Service page appears.
Click the name of the Prometheus instance instance that you want to manage to go to the Integration Center page.
Go to the Integration Center page of ARMS
Log on to the Application Real-Time Monitoring Service (ARMS) console.
In the left-side navigation pane, click Integration Center. On the Integration Center page, click Add on the Kafka(Basic Edition) or Kafka(Advanced Version) card in the Application Components section. In the panel that appears, configure the required parameters to add a Kafka exporter.
Add a Kafka (Basic Edition) exporter
In this example, a Kafka (Basic Edition) exporter is added on the Integration Center page of the ARMS console.
Step 1: Add a Kafka (Basic Edition) exporter
Log on to the ARMS console.
In the left-side navigation pane, click Integration Center. On the Integration Center page, click Add on the Kafka(Basic Edition) card in the Application Components section. In the panel that appears, perform the following steps to add a Kafka (Basic Edition) exporter:
In the STEP1 section, select the environment in which you want to deploy the Kafka exporter.
In the STEP2 section, select the Prometheus instance on which you want the Kafka exporter to reside.
On the configuration tab in the STEP3 section, configure the parameters and click OK.
Parameter
Description
Exporter Name
The unique name of the Kafka exporter.
kafka address
The endpoint of the self-managed Kafka broker. Separate multiple broker addresses with commas (,) or semicolons (;).
If your Kafka instance is deployed in a container service environment, you can enter the IP address or service address of the Kafka broker.
If your Kafka instance is deployed in an ECS environment, you can enter the IP address or domain name system (DNS) address of the Kafka broker.
Metrics space interval (seconds)
The interval at which you want the service to collect monitoring data.
kafka version
The version number of the Kafka broker. The latest version is V3.2.0.
SASL enabled
Specifies whether to enable the Simple Authentication and Security Layer (SASL) feature on the Kafka broker.
SASL username
The username of the SASL account. If you enable the SASL feature, you must specify the parameter.
SASL password
The password of the SASL account. If you enable the SASL feature, you must specify the parameter.
SASL mechanism
The SASL mechanism. The following authentication mechanisms are supported: PLAIN, SCRAM-SHA-512, and SCRAM-SHA-256.
TLS enabled
Specifies whether to enable the Transport Layer Security (TLS) feature on the Kafka broker.
insecure skip TLS verify
Specifies whether to skip TLS security verification. If TLS is enabled on the Kafka broker and a self-signed certificate is used, you can skip the TLS security verification.
Step 2: View the status of the Kafka (Basic Edition) exporter
- Log on to the Managed Service for Prometheus console.
- In the left-side navigation pane, click Monitoring List. The Prometheus Service page appears.
- Click the name of the Prometheus instance instance that you want to manage to go to the Integration Center page.
Click the Kafka(Basic Edition) card in the Installed section. In the panel that appears, you can view information on the following tabs:
Targets: You can view the list of targets discovered by default jobs on the Targets tab.
Metrics: You can view information about metrics and discard metrics on the Metrics tab. For more information, see Configure metrics.
Dashboards: You can click a dashboard thumbnail to view the Grafana dashboard on the Dashboards tab.
The following figure shows the information that is displayed on the Kafka (Basic Edition) dashboard.
The number of Kafka brokers.
The number of partitions in each topic.
The numbers of inbound messages, outbound messages, and accumulated messages in each topic.
The number of in-sync replicas (ISRs) in each topic.
Alerts: You can view Prometheus alerts on the Alerts tab. The following figure shows the key metrics of Prometheus alerts. You can add alert rules based on your business requirements. For more information about how to create a Prometheus alert rule, see Create an alert rule for a Prometheus instance.
Service Discovery Configurations: You can manage the default basic monitoring jobs on the Service Discovery Configurations tab. For example, you can modify data extraction intervals or disable collection jobs.
Exporter: You can view the version number of
node-exporter
that is installed by default on the Exporter tab.
Add a Kafka (Advanced Edition) exporter
In this example, a Kafka (Advanced Edition) exporter is added on the Integration Center page of the ARMS console.
Step 1: Add a Kafka (Advanced Edition) exporter
Log on to the ARMS console.
In the left-side navigation pane, click Integration Center. On the Integration Center page, click Add on the Kafka(Advanced Version) card in the Application Components section. In the panel that appears, perform the following steps to add a Kafka (Advanced Version) exporter:
In the STEP1 section, select the environment in which you want to deploy the Kafka exporter cluster.
In the STEP2 section, select the Prometheus instance on which you want the Kafka exporter to reside.
On the configuration tab in the STEP3 section, configure the parameters and click OK.
Parameter
Description
Instance name
The unique name of the Kafka exporter.
Kafka instance name
The name of the Kafka instance that you want to monitor. You can specify an instance name on the dashboard to view the producer, broker, and consumer of a Kafka cluster.
Pod/ECS Label Key (service discovery)
The key and value that are specified for the pod or ECS instance when the JMX agent is deployed. Managed Service for Prometheus uses this key-value pair for service discovery.
Pod/ECS Label value
JMX Agent listening port
The listening port that is specified when the JMX agent is deployed.
Metrics path
The HTTP path that is used by Prometheus to collect monitoring data from the JMX agent. Default value:
/metrics
.Metrics space interval (seconds)
The interval at which you want the service to collect monitoring data.
Step 2: View the status of the Kafka (Advanced Version) exporter
- Log on to the Managed Service for Prometheus console.
- In the left-side navigation pane, click Monitoring List. The Prometheus Service page appears.
- Click the name of the Prometheus instance instance that you want to manage to go to the Integration Center page.
Click the Kafka(Advanced Version) card in the Installed section. In the panel that appears, you can view information on the following tabs:
Targets: You can view the list of targets discovered by default jobs on the Targets tab.
Metrics: You can view information about metrics and discard metrics on the Metrics tab. For more information, see Configure metrics.
Dashboards: You can click a dashboard thumbnail to view the Grafana dashboard on the Dashboards tab. The Kafka (Advanced Edition) exporter provides dashboards based on instances and topics.
Instance dashboard
The following metrics of Kafka brokers are displayed on the dashboard:
Core metrics: the numbers of brokers, offline partitions, under-replicated partitions, and controllers, and information about CPUs and networks
Java Virtual Machine (JVM) metrics: the key information about the JVM memory and garbage collection (GC)
Partition metrics: the partition information, such as the partition quantity, ISR, unclean leader election, replica lag, offline partitions, and under-replicated partitions
Time metrics: the time metrics in the Produce, Request, and Fetch phases
Cluster traffic metrics: the overall traffic metrics of the cluster
Broker traffic metrics: broker-specific traffic details
Topic dashboard
The following end-to-end metrics of each Kafka topic are displayed on the dashboard:
Producer: the key metrics of the producer, including the message sending rate, message compression ratio, and message sending latency
Server (Kafka broker): the number of partitions in a topic, and the rates and traffic volumes of inbound messages and outbound messages
Consumer: the message consumption rate, message consumption latency, and rebalance information
Alerts: You can view Prometheus alerts on the Alerts tab. Managed Service for Prometheus provides alert metrics for self-managed Kafka producers, instances, and consumers.
Producer: Three alert metrics are provided, including the message sending failure rate, message sending duration, and message sending retry rate. You can use the metrics to identify exceptions on the producer.
Instance: Thirteen alert metrics are provided, including topics with excessive partitions, offline partitions, unclean leader election, under-replicated partitions, a decrease in effective brokers, the number of effective controllers, the number of rejected messages, numbers of inbound messages and outbound messages of the instance, and numbers of inbound messages and outbound messages of topics. You can use the metrics to identify exceptions on the broker.
Consumer: The alert metric for message accumulation is provided. You can use this metric to identify exceptions on consumption.
You can also create alert rules based on your business requirements. For more information about how to create a Prometheus alert rule, see Create an alert rule for a Prometheus instance.
Service Discovery Configurations: You can manage the default basic monitoring jobs on the Service Discovery Configurations tab. For example, you can change data extraction intervals or disable collection jobs.
Exporter: You can view the version number of
node-exporter
that is installed by default on the Exporter tab.