This topic provides an overview of the monitoring for the Kafka service.

Prerequisites

A Kafka cluster is created.

Enter the KAFKA Monitoring page

  1. Log on to the Alibaba Cloud E-MapReduce console.
  2. Click the Monitor tab.
  3. In the left-side navigation pane, click Cluster Monitoring.
  4. On the Cluster Status page that appears, find the target Kafka cluster and click Details in the Action column.
  5. In the left-side navigation pane, choose Service Monitoring > Kafka to enter the KAFKA Monitoring page.

KAFKA Monitoring page

The KAFKA Monitoring page displays basic metric charts, events and alerts on the current day, and statuses of Kafka broker nodes.

The Kafka Broker section lists the CPU utilization and usage of memory, heap memory, non-heap memory, and file descriptors of instances where the broker nodes reside. Click an instance name to access the monitoring details page.

Monitoring details page of Kafka broker nodes

  • Broker Process JVM Indicators section: displays heap and non-heap memory usage.Kafka_jvm
  • Broker Process File Descriptor section: displays the maximum number of file descriptors that a broker process can use and the number of file descriptors that are in use.Kafka_broker
  • Broker Core Indicators section
    • Failed Requests: includes Fetch requests and Produce requests failed per second.
    • Broker Packet Rate: includes inbound and outbound broker traffic.
    • Offline Count: includes offline log directories, replicas, and partitions.
    • Replica Manager Disk Usage: includes the maximum value, minimum value, average value, and standard deviation.
    • Message: indicates the number of inbound messages per second.
    • Total Fetch Requests: indicates the total number of Fetch requests generated per second.
  • Broker Process History sectionBroker_history

    For more information about the parameters, see the following table.

    Parameter Description
    Date The time when an operation is performed.
    Start/Restart/Stop The operation type, which can be start, stop, or restart.
    Auto Resume Whether an operation is automatically resumed by the keepalive mechanism of EMR. The EMR agent automatically resumes components that exit abnormally to ensure service availability.
    Started By The Linux user who performs an operation. This parameter is left empty for a process in the Stop state.
    PID The ID of a process generated in an operation. This parameter is left empty for a process in the Stop state.
    PPID The ID of a parent process generated in an operation. This parameter is left empty for a process in the Stop state.
    Startup Parameters The detailed start parameters for a process generated in an operation. This parameter is left empty for a process in the Stop state.