All Products
Search

This Product

    E-MapReduce:Access Kafka over the public network

    Document Center

    E-MapReduce:Access Kafka over the public network

    Last Updated:Mar 26, 2026

    A Dataflow cluster runs inside a virtual private cloud (VPC), which blocks direct public access. To reach Kafka from the Internet, you need to assign public IP addresses to the broker nodes and open the required port on the security group.

    Two scenarios are covered:

    Prerequisites

    Before you begin, make sure that you have:

    • An E-MapReduce (EMR) Dataflow cluster with Kafka deployed

    • Permission to modify security groups in the EMR console

    • Permission to manage elastic IP addresses (EIPs) in the ECS console (required for the EIP method)

    Enable internet-based access when you create a cluster

    When creating a Dataflow cluster in the EMR console, turn on Assign Public Network IP for the core node group.

    After the cluster is created, open port 9093 on the security group. Kafka is then accessible via port 9093 of the node's public IP address. For instructions, see Manage security groups.

    Enable internet-based access for an existing cluster

    Choose a method based on your network requirements:

    Method When to use
    Associate EIPs You want a direct, self-managed public IP on each broker node.
    Deploy Express Connect You need to establish a connection between the Internet and your VPC.

    Method 1: Associate EIPs

    Step 1: Open port 9093 on the security group

    1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

    2. In the top navigation bar, select the region where your cluster resides, and select a resource group.

    3. Find the cluster and click its name.

    4. On the Basic Information tab, click the link next to Cluster Security Group in the Security section.

    5. Add an inbound rule to allow TCP traffic on port 9093. Restrict the source IP range to your client addresses rather than opening access to all IPs (0.0.0.0/0). For detailed steps, see Manage security groups.

    Limiting the source IP range to known client addresses reduces exposure. Avoid using 0.0.0.0/0 in production environments.

    Step 2: Associate an EIP with each broker node

    Associate an EIP with every Elastic Compute Service (ECS) instance that runs a Kafka broker.

    1. On the cluster detail page, click the Nodes tab.

    2. Find a broker node and click its ID in the Node Name/ID column to open the ECS console.

    3. Associate an EIP with the ECS instance. For instructions, see Associate an EIP with an instance.

    4. Repeat steps 2–3 for all remaining broker nodes.

    Step 3: Synchronize public IP addresses to the cluster

    In the upper-right corner of the Nodes tab, choose All Operation > Synchronize Host Information.

    This pushes the newly assigned public IP addresses into the Dataflow cluster configuration.

    Step 4: Enable public access in Kafka configuration

    1. Navigate to the Kafka service page and click the Configure tab.

    2. Set `kafka.public-access.enable` to true.

    3. Click Deploy Client Configuration.

    4. In the dialog box, fill in the Execution Reason field and click OK.

    5. In the Confirm message, click OK.

    Step 5: Restart the Kafka service

    1. On the Kafka service page, click the Status tab.

    2. In the Components section, find KafkaBroker and click Restart in the Actions column.

    3. Fill in the Execution Reason field and click OK.

    4. In the Confirm message, click OK.

    Method 2: Deploy Express Connect

    Deploy Express Connect to establish a connection between the Internet and your VPC. For more information, see What is Express Connect.

    Access Kafka

    After setup, connect to a broker using its public IP address on port 9093:

    <Public IP address of the broker>:9093

    For example, if a broker's public IP address is 211.90.**.**, the connection address is:

    211.90.**.**:9093

    What's next

    References