All Products
Search

This Product

    E-MapReduce:Enable YARN in Ranger and configure related permissions

    Document Center

    E-MapReduce:Enable YARN in Ranger and configure related permissions

    Last Updated:Jan 29, 2024

    This topic describes how to enable YARN in Ranger and how to configure the related permissions.

    Background information

    If YARN is integrated with Ranger, you can use Ranger to configure permissions only on scheduler queues. Permissions on fair queues cannot be configured. The permissions that you configured on YARN queues by using Ranger and the Capacity Scheduler configurations of YARN take effect at the same time. The following figure shows the authentication process. YARN

    Prerequisites

    A cluster of a version that is earlier than EMR V5.11.0 or EMR V3.45.0 is created, and Ranger is selected for the cluster. For more information about how to create a cluster, see Create a cluster.

    Note

    For clusters of EMR V5.11.0 or a later minor version and clusters of EMR V3.45.0 or a later minor version, RangerUserSync automatically connects to an LDAP server if OpenLDAP is installed in the cluster. You can search for the ranger.usersync.sync.source configuration item on the Configure tab of the Ranger service page to view the user source (UNIX or LDAP) of RangerUserSync.

    Limits

    Capacity Scheduler must be used as the scheduler of YARN, and the access control list (ACL) feature of YARN must be enabled. You must make sure that no unnecessary permissions are granted. For more information, see YARN schedulers and High security feature of YARN.

    Make sure that the following requirements are met before you enable YARN in Ranger:

    • On the yarn-site.xml tab of the YARN service page, the yarn.resourcemanager.scheduler.class parameter is set to org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.

    • On the yarn-site.xml tab of the YARN service page, the yarn.acl.enable parameter is set to true. This indicates that the ACL feature of YARN is enabled.

    • On the capacity-scheduler.xml tab of the YARN service page, the yarn.scheduler.capacity.root.acl_submit_applications parameter is set to a single space, and the yarn.scheduler.capacity.root.acl_administer_queue parameter is set to hadoop (a space is added before hadoop). This indicates that the hadoop user group has the permissions to manage all queues.

    • The acl_submit_applications or acl_administer_queue configuration item is not configured for queues other than the root queue. This prevents the ACL feature of YARN from affecting Ranger authentication.

    Procedure

    1. Go to the Services tab.

      1. Log on to the EMR console.

      2. In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.

      3. On the EMR on ECS page, find the desired cluster and click Services in the Actions column.

    2. Enable YARN in Ranger.

      1. On the Services tab of the page that appears, click Status in the Ranger-plugin section.

      2. In the Service Overview section of the Status tab, turn on enableYARN.

      3. In the message that appears, click OK.

    3. Restart YARN ResourceManager.

      1. On the Services tab, click the More icon and select YARN.

      2. In the Components section of the Status tab, find ResourceManager and click Restart in the Actions column.

      3. In the dialog box that appears, configure the Execution Reason parameter and click OK.

      4. In the Confirm message, click OK.

    Configure permissions in Ranger

    After you enable YARN in Ranger, the YARN service is added to the web UI of Ranger. You can use the YARN service to grant queue management permissions to the user emr-user.

    1. Access the web UI of Ranger. For more information, see Overview.

    2. Click emr-yarn.

      emr-yarn

    3. In the upper-right corner of the page that appears, click Add New Policy.

    4. Configure the parameters based on your business requirements. The following table describes the parameters.

      Create Policy

      Parameter

      Description

      Policy Name

      The name of the policy. You can specify a custom name.

      Queue

      The name of the queue. Example: root.default.

      recursive

      Specifies whether the child queue inherits the permissions.

      Select Group

      The user group to which you want to attach the policy.

      Select User

      The user to whom you want to attach the policy. Example: emr-user.

      Permissions

      The permissions that you want to grant. Example: admin-queue.

    5. Click Add.

      After the policy is added, the related permissions are granted to the user emr-user. The user emr-user can submit jobs to the root.default queue.

      Note

      After you add, remove, or modify a policy, it takes about one minute for the configuration to take effect.