All Products
Search

This Product

    E-MapReduce:Kafka Manager

    Document Center

    E-MapReduce:Kafka Manager

    Last Updated:Mar 26, 2026

    Kafka Manager is a web-based cluster management tool for Kafka. Use it to manage your Dataflow Kafka clusters in E-MapReduce (EMR) in a convenient manner.

    Prerequisites

    Before you begin, ensure that you have:

    • A Dataflow cluster with Kafka selected during creation. For more information, see Create a cluster.

    Note

    Selecting Kafka during cluster creation automatically installs Kafka Manager with authentication enabled.

    Collect the following values from the EMR console before you start. In the EMR console, go to EMR on ECS, find your cluster, and click Services in the Actions column.

    Value Where to find it
    kafka.manager.authentication.username Services tab → find Kafka Manager → Configure → Configure tab
    kafka.manager.authentication.password Services tab → find Kafka Manager → Configure → Configure tab
    zookeeper.connect Services tab → find Kafka → Configure → Configure tab → server.properties tab

    Limitations

    Kafka Manager does not support throttling when reassigning partitions. To enable throttling, run kafka-configs.sh to configure throttling-related parameters. For more information, see Throttle the O&M traffic of Kafka brokers.

    Access Kafka Manager and add a cluster

    1. Open an SSH tunnel to the cluster, then go to http://localhost:8085 in your browser. For instructions, see Create an SSH tunnel to access web UIs of open source components.

      Important

      Use the SSH tunnel rather than exposing port 8085 directly. If you must access Kafka Manager via a public address, configure an IP address whitelist to prevent data leaks.

      Note

      Change the default password the first time you log in.

    2. On the login page, enter the kafka.manager.authentication.username and kafka.manager.authentication.password values you collected in the Prerequisites.

    3. Choose Cluster > Add Cluster.

    4. On the Add Cluster page, configure the following parameters and click Save.

      Parameter Description
      Cluster Name The name of the Dataflow cluster.
      Cluster Zookeeper Hosts The zookeeper.connect value you collected in the Prerequisites.
      Kafka Version The Kafka version deployed in the Dataflow cluster. If Kafka Manager does not support your cluster's Kafka version, select the latest version that Kafka Manager supports.
      Enable JMX Polling (Set JMX_PORT env variable before starting kafka server) Enables Java Management Extensions (JMX) monitoring.
      brokerViewThreadPoolSize The number of threads used to fetch broker data. Set this to a value greater than 2.

      Add Kafka

    After adding the cluster, you can use the common features of Kafka.

    Kafka

    What's next

    Use Kafka Manager to manage topics, reassign partitions, and monitor broker metrics for your Dataflow cluster.