All Products
Search
Document Center

:Monitor a Kafka cluster

Last Updated:Dec 13, 2024

Managed Service for Prometheus actively pulls performance data from your Kafka cluster based on the specified parameters. This way, you can monitor the cluster in real time and flexibly analyze the monitoring data. This topic describes how to monitor a Kafka cluster.

Prerequisites

Container cluster

ECS instance

Procedure

  1. Log on to the Managed Service for Prometheus console. In the left-side navigation pane, click Integration Center.

  2. Click Kafka and set the parameters as prompted. The following tables describe the key parameters.

    Container cluster

    Parameter

    Description

    Pod Tag

    The tag key and value configured for the pod when the Java Management Extensions (JMX) agent was deployed. Managed Service for Prometheus uses this tag for service discovery. For more information, see How do I deploy and configure a Kafka JMX agent?

    Metric collection interval

    The interval for collecting monitoring data. The default value is 15.

    ECS instance

    Paramter

    Description

    Kafka cluster name

    The name of the Kafka cluster. The name must be unique to prevent duplicate metric collection.

    Service address

    The IP or DNS address of the Kafka broker. Separate multiple addresses with commas (,) or semicolons (;).

    Example: 192.168.0.1:9092,10.0.11.123:9092.

    Enable SASL

    Specify whether to enable Simple Authentication and Security Layer (SASL) on the brokers.

    SASL Username

    Enter the SASL username if SASL is enabled.

    SASL Password

    Enter the SASL password if SASL is enabled.

    SASL Mechanism

    Select an authentication method. Valid values: plain, scram-sha512, and scram-sha256.

    Enable TLS

    Specify whether to enable Transport Layer Security (TLS) on the brokers.

    Ignore TLS security verification

    Turn on the switch if TLS is enabled and a self-signed certificate is used.

    Metric collection interval

    The interval for collecting monitoring data. The default value is 15.

    Custom Labels

    Add tags to the metrics of the ECS instance. Then, you can monitor, manage, query, or analyze the metric data in a fine-grained manner when you use tools such as Grafana. For more information, see Add tags to the metrics of an ECS instance using a VPC.

    Note
    • A tag key cannot contain special characters used in the Prometheus functionality, such as hyphens (-), periods (.), or percent signs (%).

    • Tags must be specified in the key-value format required by Prometheus.

View and manage the monitoring data of the Kafka cluster

The installed component is displayed on the Integration Management page in the Managed Service for Prometheus console. The Integration Management page consists of the Integrated Environments, Integrated Addons, and Query Dashboards tabs, where you can view information such as targets, metrics, dashboards, and alerts.

Targets

In the Targets section of the Self-Monitoring tab, you can view the list of targets discovered by the default jobs.

85.jpg

Metric Scraping

On the Metric Scraping tab, you can view or discard metrics. For more information, see Discard metrics.

84.jpg

Dashboards

In the Dashboards section of the Component Management tab, you can click a dashboard name to go to Grafana and view more dashboard data.

83.jpg

References