edit-icon download-icon

RAM authentication

Last Updated: Mar 23, 2018

RAM ID authentication

The Kerberos server in the EMR cluster supports not only the authentication method compatible with MIT Kerberos, but also the identity authentication by using RAM as the identity information.

RAM product supports creating/managing subaccounts and using subaccounts to implement access control for various resources on the cloud.

Administrator of the master account may create a subaccount on the RAM user management page (subaccount name must comply with Linux username specifications) and download the subaccount AccessKey for the corresponding developer. The developer can then configure the AccessKey to pass Kerberos authentication and access the cluster service.

Unlike using the first type MIT Kerberos authentication, RAM identity authentication does not require adding principle to the Kerberos server in advance.

The following example uses subaccount test that has already been created to access the Gateway:

  • Add the test subaccount into the EMR cluster

    The EMR security cluster’s yarn uses LinuxContainerExecutor. Running the yarn job on a cluster requires all cluster nodes to add the user account that is going to run the job. LinuxContainerExecutor conducts the related permission validation based on the user account during the execution process.

    The EMR cluster administrator executes the following code on the EMR cluster's master node:

    1. sudo su hadoop
    2. sh adduser.sh test 1 2

    Attachment: adduser.sh code

    1. # Username
    2. user_name=$1
    3. # Master node count in the cluster. For example, the HA cluster has two master nodes.
    4. master_cnt=$2
    5. # Worker node count in the cluster
    6. worker_cnt=$3
    7. for((i=1;i<=$master_cnt;i++))
    8. do
    9. ssh -o StrictHostKeyChecking=no emr-header-$i sudo useradd $user_name
    10. done
    11. for((i=1;i<=$worker_cnt;i++))
    12. do
    13. ssh -o StrictHostKeyChecking=no emr-worker-$i sudo useradd $user_name
    14. done
  • The Gateway administrator adds the test user account on the Gateway machine

    1. useradd test
  • The Gateway administrator configures the basic Kerberos environment

    1. sudo su root
    2. sh config_gateway_kerberos.sh 10.27.230.10 /pathto/emrheader1_pwd_file
    3. # Ensures the value of the /etc/ecm/hadoop-conf/core-site.xml file on the Gateway is true
    4. <property>
    5. <name>hadoop.security.authentication.use.has</name>
    6. <value>true</value>
    7. </property>

    Attachment: config_gateway_kerberos.sh script

    1. # IP address of the emr-header-1 in the EMR cluster
    2. masterip=$1
    3. # Saves the corresponding root logon password file for masterip
    4. masterpwdfile=$2
    5. if ! type sshpass >/dev/null 2>&1; then
    6. yum install -y sshpass
    7. fi
    8. ## Kerberos conf
    9. sshpass -f $masterpwdfile scp root@$masterip:/etc/krb5.conf /etc/
    10. mkdir /etc/has
    11. sshpass -f $masterpwdfile scp root@$masterip:/etc/has/has-client.conf /etc/has
    12. sshpass -f $masterpwdfile scp root@$masterip:/etc/has/truststore /etc/has/
    13. sshpass -f $masterpwdfile scp root@$masterip:/etc/has/ssl-client.conf /etc/has/
    14. # Modifies Kerberos client configuration, changing the default auth_type from EMR to RAM
    15. # This file can be manually modified
    16. sed -i 's/EMR/RAM/g' /etc/has/has-client.conf
  • test user logs on to Gateway and configures AccessKey

    1. Log on the test account of Gateway
    2. # Run the script
    3. sh add_accesskey.sh test

    Attachment: add_accesskey.sh script (modify the AccessKey)

    1. user=$1
    2. if [[ `cat /home/$user/.bashrc | grep 'export AccessKey'` == "" ]];then
    3. echo "
    4. # Change to the test user's AccessKeyId/AccessKeySecret
    5. export AccessKeyId=YOUR_AccessKeyId
    6. export AccessKeySecret=YOUR_AccessKeySecret
    7. " >>~/.bashrc
    8. else
    9. echo $user AccessKey has been added to .bashrc
    10. fi
  • Test user executes the command

    After taking the preceding steps, the test user is now able to execute the relevant commands to access the cluster service.

    Execute HDFS commands

    1. [test@gateway ~]$ hadoop fs -ls /
    2. 17/11/19 12:32:15 INFO client.HasClient: The plugin type is: RAM
    3. Found 4 items
    4. drwxr-x--- - has hadoop 0 2017-11-18 21:12 /apps
    5. drwxrwxrwt - hadoop hadoop 0 2017-11-19 12:32 /spark-history
    6. drwxrwxrwt - hadoop hadoop 0 2017-11-18 21:16 /tmp
    7. drwxrwxrwt - hadoop hadoop 0 2017-11-18 21:16 /user

    Run the assignments job

    1. [test@gateway ~]$ hadoop jar /usr/lib/hadoop-current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar pi 10 1

    Run the spark job

    1. [test@gateway ~]$ spark-submit --conf spark.ui.view.acls=* --class org.apache.spark.examples.SparkPi --master yarn-client --driver-memory 512m --num-executors 1 --executor-memory 1g --executor-cores 2 /usr/lib/spark-current/examples/jars/spark-examples_2.11-2.1.1.jar 10
Thank you! We've received your feedback.