E-MapReduce中的Kerberos支援跨域訪問(cross-realm),即不同的Kerberos叢集之間可以互相訪問。

下面以Cluster-A跨域去訪問Cluster-B中的服務為例:

  • Cluster-A的emr-header-1的hostname -> emr-header-1.cluster-1234 ; realm -> EMR.1234.COM
  • Cluster-B的emr-header-1的hostname -> emr-header-1.cluster-6789 ; realm -> EMR.6789.COM
  • 说明
    • hostname可以在emr-header-1上面執行命令hostname擷取
    • realm可以在emr-header-1上面的/etc/krb5.conf擷取

添加principal

Cluster-A和Cluster-B兩個叢集的emr-header-1節點分別執行以下完全一樣的命令:

# root帳號
       sh /usr/lib/has-current/bin/hadmin-local.sh /etc/ecm/has-conf -k /etc/ecm/has-conf/admin.keytab
       HadminLocalTool.local: addprinc -pw 123456 krbtgt/EMR.6789.COM@EMR.1234.COM
说明
  • 123456是密碼,可自行修改
  • EMR.6789.COM是Cluster-B的realm,即被訪問的叢集的realm
  • EMR.1234.COM是Cluster-A的realm,即發起訪問的叢集realm

配置Cluster-A的/etc/krb5.conf

在Cluster-A叢集上配置[realms]/[domain_realm]/[capaths],如下所示:

[libdefaults]
    kdc_realm = EMR.1234.COM
    default_realm = EMR.1234.COM
    udp_preference_limit = 4096
    kdc_tcp_port = 88
    kdc_udp_port = 88
    dns_lookup_kdc = false
[realms]
    EMR.1234.COM = {
                kdc = 10.81.49.3:88
    }
    EMR.6789.COM = {
                kdc = 10.81.49.7:88
    }
[domain_realm]
    .cluster-1234 = EMR.1234.COM
    .cluster-6789 = EMR.6789.COM
[capaths]
    EMR.1234.COM = {
       EMR.6789.COM = .
    }
    EMR.6789.COM = {
       EMR.1234.COM = .
    }
將上述/etc/krb5.conf同步到Cluster-A所有節點

將Cluster-B節點的/etc/hosts檔案中綁定資訊(只需要長網域名稱emr-xxx-x.cluster-xxx)拷貝到Cluster-A的所有節點/etc/hosts

10.81.45.89  emr-worker-1.cluster-xxx
 10.81.46.222  emr-worker-2.cluster-xx
 10.81.44.177  emr-header-1.cluster-xxx
说明
  • Cluster-A上面如果要跑作業訪問Cluster-B,需要先重啟yarn
  • Cluster-A的所有節點配置Cluster-B的host綁定資訊

訪問Cluster-B服務

在Cluster-A上面可以用Cluster-A的Kerberos的keytab檔案/ticket緩衝,去訪問Cluster-B的服務。

如訪問Cluster-B的hdfs服務:

su has;
hadoop fs -ls hdfs://emr-header-1.cluster-6789:9000/
Found 4 items
-rw-r-----   2 has    hadoop         34 2017-12-05 18:15 hdfs://emr-header-1.cluster-6789:9000/abc
drwxrwxrwt   - hadoop hadoop          0 2017-12-05 18:32 hdfs://emr-header-1.cluster-6789:9000/spark-history
drwxrwxrwt   - hadoop hadoop          0 2017-12-05 17:53 hdfs://emr-header-1.cluster-6789:9000/tmp
drwxrwxrwt   - hadoop hadoop          0 2017-12-05 18:24 hdfs://emr-header-1.cluster-6789:9000/user