All Products
Search
Document Center

E-MapReduce:FAQ about AccessKey pairs when you access OSS or OSS-HDFS

Last Updated:May 08, 2024

This topic provides answers to some frequently asked questions about AccessKey pairs when you use E-MapReduce (EMR) to access Alibaba Cloud Object Storage Service (OSS) or OSS-HDFS.

How do I access OSS or OSS-HDFS in an EMR cluster in password-free mode?

When an EMR cluster is created, an application role of Elastic Compute Service (ECS) is associated with the cluster. By default, the application role is AliyunECSInstanceForEMRRole. JindoSDK uses the role to obtain the security token to access OSS or OSS-HDFS.1

To access OSS or OSS-HDFS in password-free mode, perform the following steps: On the Configure tab of the Hadoop-Common service page, click the core-site.xml tab and check whether the value of the fs.oss.credentials.provider parameter contains com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider. If the parameter value does not contain com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider, add com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider to the value of the parameter and click Save.

How do I check whether password-free access to OSS or OSS-HDFS is available in an EMR cluster?

  • Method 1: Run the following command to query the credentials of the RAM role:

    curl http://100.100.100.200/latest/meta-data/Ram/Security-credentials/AliyunECSInstanceForEMRRole

    If the following result is returned, password-free access to OSS or OSS-HDFS is available:

     {
      "AccessKeyId" : "STS.NUreXXXXXX",
      "AccessKeySecret" : "BsmbnDoXXXXXXXX",
      "Expiration" : "2022-11-22T11:27:39Z",
      "SecurityToken" : "CAISlwJ1q6FXXXXXXX",
      "LastUpdated" : "2022-11-22T05:27:39Z",
      "Code" : "Success"
      }
  • Method 2: Configure EcsStsCredentialsProvider to access OSS or OSS-HDFS in password-free mode.

    1. On the Configure tab of the Hadoop-Common service page, click the core-site.xml tab and check whether the value of the fs.oss.credentials.provider parameter contains com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider. If the parameter value does not contain com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider, add com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider to the value of the parameter and click Save.

    2. Use shell commands of Hadoop Distributed File System (HDFS) to access OSS or OSS-HDFS. If you can access OSS or OSS-HDFS, password-free access to OSS or OSS-HDFS is available.

What do I do if I am unable to access OSS or OSS-HDFS by using HiveServer2 or Presto every 5 to 6 hours?

In some versions of JindoData, some known issues about password-free access exist. For more information, see Known issues in JindoData 4.X.

What do I do if an error indicating that the OSS or OSS-HDFS path contains an AccessKey pair is reported?

  • Error message:

    The Filesystem URI contains login details. This authentication mechanism is no longer supported.
  • Cause:

    To prevent security issues such as the leak of AccessKey pairs, JindoSDK V4.0.0 and later versions do not allow you to include an AccessKey pair in the access path.

  • Solutions:

    • Method 1: Delete the AccessKey pair from the access path.

    • Method 2: If you want to include the AccessKey pair in the access path, go to the Configure tab of the Hadoop-Common service page, click the core-site.xml tab. On the tab, click Add Configuration Item to add the following configuration item.

      Configuration item

      Value

      fs.oss.uri-with-secrets.enable

      true

      After you add the configuration item, click Save.