Answers to frequently asked questions about Ranger user sync in E-MapReduce.
RangerUserSync fails to sync Linux/LDAP users
-
Cause: This issue occurs in E-MapReduce versions 3.23.0 and earlier.
-
Solution: Log on to the emr-header-1 node of the cluster. Manually modify the /etc/ecm/ranger-usersync-conf/ranger-ugsync-site.xml file to set the ranger.usersync.enabled parameter to
true. Then, in the EMR console, restart the RangerUserSync component.
How to sync Unix users to Ranger
Ranger does not support syncing both LDAP and Unix users in the same cluster. The method for syncing Unix users depends on whether OpenLDAP is deployed in your cluster.
Without OpenLDAP
-
On the master node of your E-MapReduce cluster, run the following command to add a user.
sudo adduser <username> -
In the EMR console, navigate to the Ranger service. On the Status tab, find RangerUserSync in the Components list and click Restart in the Actions column.
-
Verify the sync result.
-
Log on to the Ranger web UI. For more information, see Access the web UIs of open-source components.
-
Choose .
-
On the Users tab, verify that the new Unix user appears.
For example, the
testuser created in step 1 now appears.
-
With OpenLDAP
If OpenLDAP is deployed in your cluster, Ranger syncs LDAP users by default. To switch to Unix user sync, modify the following Ranger parameters:
-
On the master node of your E-MapReduce cluster, run the following command to add a user.
sudo adduser <username> -
In the EMR console, navigate to the Ranger service. On the Configure tab, find and modify the following parameters:
-
Set the
ranger.usersync.sync.sourceparameter tounix. -
Set the
ranger.usersync.source.impl.classparameter toorg.apache.ranger.unixusersync.process.UnixUserGroupBuilder.
-
-
When prompted by the console, save the configuration and restart the Ranger service. After the service restarts, Unix users are synced automatically.
-
Verify the sync result.
-
Log on to the Ranger web UI. For more information, see Access the web UIs of open-source components.
-
Choose .
-
On the Users tab, verify that the new Unix user appears.
For example, the
testuser created in step 1 now appears.
-
What do I do if Hive on Spark jobs take too long due to too many small files in EMR?
When a large number of small files exist in OSS, Ranger authorization checks on OSS significantly increase latency, affecting the performance of HiveServer2.
Solution 1: Disable Ranger OSS authorization
-
Log on to the EMR console and go to the cluster details page.
-
On the Cluster Services tab, find the
Ranger-pluginservice. -
Click the Status tab of the
Ranger-pluginservice to go to the Service Overview section. -
In the Service Overview section, find the
enableOSSswitch and turn it off. -
Restart resident components such as
HiveServer2and SparkThriftServer.
Solution 2: Optimize Hive small file merge parameters
In the Hive service configuration, set the following parameters:
-
hive.merge.mapfiles=true(enables small file merging in the Map stage) -
hive.merge.mapredfiles=true(enables small file merging in the Reduce stage) -
hive.merge.size.per.task=256000000(target file size after merging)