This topic describes how to integrate YARN with Ranger and how to configure permissions on YARN queues.

Prerequisites

A cluster is created in E-MapReduce (EMR) V3.34.0 or a later minor version, or in EMR V4.8.0 or a later minor version, and Ranger is selected from the optional services when you create the cluster. For more information, see Create a cluster.

Background information

In Ranger YARN, you can configure permissions on Scheduler queues instead of on Fair queues. The permissions you configured on YARN queues by using Ranger take effect together with the Capacity Scheduler configuration provided by YARN, but the priority of the configured permissions is lower than that of the Capacity Scheduler configuration. The permissions you configured on YARN queues by using Ranger are verified only if the Capacity Scheduler configuration fails to be verified. The following figure shows the authentication process. Authentication process

Integrate YARN with Ranger

Notice Make sure that no YARN jobs are submitted in the cluster when you perform the following steps. After you enable YARN in Ranger, you must grant the user that needs to submit a YARN job the permissions on the required queues. Otherwise, the user cannot submit the YARN job.
  1. Enable YARN.
    1. Log on to the Alibaba Cloud EMR console.
    2. In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.
    3. Click the Cluster Management tab.
    4. On the Cluster Management page, find your cluster and click Details in the Actions column.
    5. In the left-side navigation pane, choose Cluster Service > RANGER.
    6. On the page that appears, choose Actions > EnabledYARN in the upper-right corner.
    7. In the Cluster Activities dialog box, specify Description and click OK.
    8. In the Confirm message, click OK.
      1. In the Cluster Activities dialog box, specify Description and click OK.
      2. In the Confirm message, click OK.
      3. Click History in the upper-right corner to view the task progress.
  2. Add the YARN service on the web UI of Ranger.
    1. Log on to Ranger. For more information, see Overview.
    2. On the Ranger web UI, click the Add icon in the row in which YARN is located to add the YARN service.
      YARN
    3. Configure the parameters.
      YARN
      Parameter Description
      Service Name The value is emr-yarn and cannot be changed.
      Username The value is hadoop and cannot be changed.
      Password You can customize a password.
      Authentication Type
      • Set the parameter to Simple for a common cluster.
      • Set the parameter to Kerberos for a high-security cluster.
      YARN REST URL Set the parameter to http://emr-header-1:8088.
      Add New Configurations
      • Set Name in the first row to policy.download.auth.users.
      • Set Value in the second row to yarn.
      • Set Name in the third row to hadoop.http.user.name.
      • Set Value in the fourth row to hadoop.
    4. Click Add.
  3. Restart YARN ResourceManager.
    1. On the YARN service page of the cluster, choose Actions > Restart ResourceManager in the upper-right corner.
    2. In the Cluster Activities dialog box, specify Description and click OK.
    3. In the Confirm message, click OK.
      1. In the Cluster Activities dialog box, specify Description and click OK.
      2. In the Confirm message, click OK.
      3. Click History in the upper-right corner to view the task progress.
  4. Modify the Capacity Scheduler configuration.
    1. On the YARN service page of the cluster, click the Configure tab.
    2. In the Service Configuration section, click the capacity-scheduler tab.
    3. Modify the content in the xml-direct-to-file-content field.
      Note We recommend that you copy the content in the xml-direct-to-file-content field to a text editor to edit the content.
      1. Delete the following information:
        <property>
          <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
          <value>*</value>
          <description>The ACL of who can submit jobs to the default queue.</description>
        </property>
        <property>
          <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
          <value>*</value>
          <description>The ACL of who can administer jobs on the default queue.</description>
        </property>
      2. Add the following information:
        <property>
          <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
          <value> </value>
          <description>The ACL of who can submit jobs to the root queue.</description>
        </property>
        <property>
          <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
          <value> </value>
          <description>The ACL of who can administer jobs on the root queue.</description>
        </property>
        Note In the added information, a space exists between <value> and </value>, which indicates that no one can submit jobs to the root queue or manage the root queue.
  5. Save the configuration.
    1. In the upper-right corner of the Service Configuration section, click Save.
    2. In the Confirm Changes dialog box, specify Description and turn on Auto-update Configuration.
    3. Click OK.
  6. Refresh queues.
    1. On the YARN service page of the cluster, choose Actions > Refresh Queues in the upper-right corner.
    2. In the Cluster Activities dialog box, specify Description and click OK.
    3. In the Confirm message, click OK.
      1. In the Cluster Activities dialog box, specify Description and click OK.
      2. In the Confirm message, click OK.
      3. Click History in the upper-right corner to view the task progress.

Configure permissions

Perform the following steps to grant the test user the permissions to submit a job to the default queue.

  1. Log on to Ranger. For more information, see Overview.
  2. Click emr-yarn.
    Configure permissions on YARN queues
  3. Click Add New Policy in the upper-right corner.
  4. Configure permissions on YARN queues.
    Parameter Description
    Policy Name The name of the policy. You can customize a name.
    Queue The name of a queue, such as root.default.
    recursive Specifies whether a subqueue inherits the same permissions.
    Select Group The user group to which you want to add this policy.
    Select User The user to whom you want to add this policy.
    Permissions The permissions that you want to grant.
  5. Click Add.
    After the policy is added, the test user is granted the permissions. The test user can submit jobs to the default queue.
    Note After you add, remove, or modify a policy, it takes about one minute for the configuration to take effect.