Kerberos in E-MapReduce supports cross-region access, meaning that different Kerberos clusters can access each other. This section describes cross-region access using cluster A and cluster B as an example.
- Hostname of emr-header-1 in clusterA → emr-header-1.cluster-1234. Region → EMR.1234.COM
- Hostname of emr-header-1 in clusterB → emr-header-1.cluster-6789. Region → EMR.6789.COM
Note
- The hostname can be obtained by executing the hostname command on emr-header-1.
- The region can be obtained in /etc/krb5.conf on emr-header-1.
Add principal
The emr-header-1 nodes in clusterA and clusterB both run the following command:
# root account
sh /usr/lib/has-current/bin/hadmin-local.sh /etc/ecm/has-conf -k /etc/ecm/has-conf/admin.keytab
HadminLocalTool.local: addprinc -pw 123456 krbtgt/EMR.6789.COM@EMR.1234.COM 6789. COM@EMR. 1234. Com
- The password is 123456. This can be modified.
- The region of cluster B is EMR.6789.COM. This is the region of the cluster being accessed.
- The region of cluster A is EMR.1234.COM. This is the region of the cluster that initiates access.
Configure /etc/krb5.conf for clusterA
Configure [regions]/[domain_region]/[capaths] on clusterA as follows:
[libdefaults]
kdc_realm = EMR. 1234. COM
default_realm = EMR. 1234. COM
udp_preference_limit = 4096
kdc_tcp_port = 88
kdc_udp_port = 88
dns_lookup_kdc = false
[realms]
EMR. 1234. COM = {
kdc = 10.81.49.3:88
}
EMR. 6789. COM = {
kdc = 10.81.49.7:88
}
[domain_realm]
.cluster-1234 = EMR. 1234. COM
.cluster-6789 = EMR. 6789. COM
[capaths]
EMR. 1234. COM = {
EMR. 6789. COM = .
}
EMR. 6789. COM = {
EMR. 1234. COM = .
}
Synchronize /etc/krb5.conf to all clusterA nodes.
Copy the binding information (only the long domain name emr-xxx-x.cluster-xxx is required) from cluster B's /etc/hosts file to /etc/hosts for all clusterA nodes.
10.81.45.89 emr-worker-1.cluster-xxx
10.81.46.222 emr-worker-2.cluster-xx
10.81.44.177 emr-header-1.cluster-xxx
- If you want to run a job on clusterA to access clusterB, you must first restart YARN.
- Configure cluster B's host binding information for all clusterA nodes.
Access services in clusterB
You can use cluster A's Kerberos keytab file /ticket as a cache on cluster A to access services in clusterB.
For example, access the HDFS service in clusterB as follows:
su has;
hadoop fs -ls hdfs://emr-header-1.cluster-6789:9000/
Found 4 items
-rw-r----- 2 has hadoop 34 2017-12-05 18:15 hdfs://emr-header-1.cluster-6789:9000/abc
drwxrwxrwt - hadoop hadoop 0 2017-12-05 18:32 hdfs://emr-header-1.cluster-6789:9000/spark-history
drwxrwxrwt - hadoop hadoop 0 2017-12-05 17:53 hdfs://emr-header-1.cluster-6789:9000/tmp
drwxrwxrwt - hadoop hadoop 0 2017-12-05 18:24 hdfs://emr-header-1.cluster-6789:9000/user