Control YARN Queue Access with Ranger Permissions - E-MapReduce

How it works

When YARN is integrated with Ranger, Ranger enforces access control on Capacity Scheduler queues only. Fair Scheduler queue permissions cannot be managed through Ranger.

Ranger queue policies and YARN's own Capacity Scheduler access control list (ACL) configurations take effect simultaneously. The following diagram shows the authentication flow.

Prerequisites

Before you begin, make sure that you have:

An EMR cluster earlier than V5.11.0 or V3.45.0 with Ranger selected during cluster creation. For more information, see Create a cluster.

For clusters running EMR V5.11.0 or later, and EMR V3.45.0 or later, RangerUserSync automatically connects to an LDAP server if OpenLDAP is installed. To check the user source (Unix or LDAP) of RangerUserSync, search for the ranger.usersync.sync.source configuration item on the Configure tab of the Ranger service page.

Limitations

Ranger controls YARN queue permissions only when Capacity Scheduler is used and the YARN ACL feature is enabled. Verify the following configuration requirements before enabling YARN in Ranger:

Config file	Parameter	Required value
`yarn-site.xml`	`yarn.resourcemanager.scheduler.class`	`org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler`
`yarn-site.xml`	`yarn.acl.enable`	`true`
`capacity-scheduler.xml`	`yarn.scheduler.capacity.root.acl_submit_applications`	Single space ( )
`capacity-scheduler.xml`	`yarn.scheduler.capacity.root.acl_administer_queue`	`hadoop` (with a leading space, granting the `hadoop` group permission to manage all queues)

Important

Do not set acl_submit_applications or acl_administer_queue on any queue other than the root queue. Setting these parameters on non-root queues overrides Ranger policies and breaks Ranger authentication.

For more information, see YARN schedulers and High security feature of YARN.

Enable YARN in Ranger

Log on to the EMR console.
In the top navigation bar, select the region where your cluster resides and select a resource group.
On the EMR on ECS page, find the target cluster and click Services in the Actions column.
On the Services tab, click Status in the Ranger-plugin section.
In the Service Overview section, turn on enableYARN and click OK in the confirmation dialog box.
Restart ResourceManager to apply the change:
1. On the Services tab, click the icon and select YARN.
2. In the Components section of the Status tab, find ResourceManager and click Restart in the Actions column.
3. In the dialog box, fill in the Execution Reason field and click OK.
4. In the Confirm dialog box, click OK.

Configure queue permissions in Ranger

After you enable YARN in Ranger, the YARN service is added to the web UI of Ranger. The following steps grant queue permissions to the user emr-user as an example.

In the Ranger web UI, click emr-yarn. For more information, see Overview.
Click Add New Policy in the upper-right corner.

Configure the policy details.

Policy details

Parameter	Description
Policy Name	A unique name for the policy.
Queue	The target queue. For example, `root.default`.
recursive	Whether child queues inherit this policy's permissions.

Allow conditions

Parameter	Description
Select Group	The user group to which this policy applies.
Select User	The user to which this policy applies. For example, `emr-user`.
Permissions	The permissions to grant. For example, `admin-queue`.

Create Policy

Click Add. The policy takes effect within about one minute. After it applies, emr-user can submit jobs to the root.default queue.