E-MapReduce:Enable Kafka in Ranger and configure related permissions

Prerequisites

• A Dataflow cluster is created and the Kafka, Ranger, and Ranger-plugin services are selected for the cluster. For more information about how to create a cluster, see Create a Dataflow Kafka cluster.

• The Simple Authentication and Security Layer (SASL) feature is enabled for the cluster. For more information, see Log on to a Kafka cluster by using SASL.

  Important

  • E-MapReduce (EMR) Ranger grants all permissions on a default service that is automatically generated for the Kafka cluster to the kafka user. When you use the SASL mechanism for identity authentication, you must configure logon permissions for the kafka user.

  • If you want to create a custom administrator account, you must add a configuration item whose key is super.users on the server.properties tab for the Kafka service. This way, the account can be granted the superuser permissions.

    For more information about how to add a configuration item, see Manage configuration items.

Limits

This topic is applicable to only EMR clusters of a minor version later than EMR V3.45.0.

Procedure

1. Go to the Services tab.

   1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

   2. On the EMR on ECS page, find the desired cluster and click Services in the Actions column.

2. Enable Kafka in Ranger.

   1. On the Services tab of the page that appears, click Status in the Ranger-plugin section.

   2. In the Service Overview section of the Status tab, turn on enableKafka.

   3. In the message that appears, click OK.

3. Restart Kafka.

   1. On the Services tab, click the icon and select Kafka.

   2. In the upper-right corner, choose More > RESTART.

   3. In the dialog box that appears, configure the Execution Reason parameter and click OK.

   4. In the Confirm message, click OK.

4. View Kafka service information on the web UI of Ranger.

   1. Access the web UI of Ranger. For more information, see Access the web UIs of open source components.

   2. Click emr-kafka.

      When you enable Kafka in Ranger, a Kafka service named emr-kafka is automatically added.

      

   3. You can configure policies based on your business requirements. The following figure shows the default configurations of the Kafka service.

      

      Important

      By default, the system uses the kafka user as the administrator to ensure that Dataflow Kafka can run as expected. Make sure that the kafka user has all required permissions.

References

• If your cluster does not contain required services, you can add services to the cluster. For more information about how to add a service, see Add services.

• If the version of your cluster is EMR V3.45.0 or an earlier minor version, you must manually integrate Kafka with Ranger. For more information, see Manually integrate the Ranger Kafka plug-in.