This topic provides answers to some frequently asked questions about AccessKey pairs when you use E-MapReduce (EMR) to access Alibaba Cloud Object Storage Service (OSS) or OSS-HDFS.
How do I access OSS or OSS-HDFS in an EMR cluster in password-free mode?
When an EMR cluster is created, an application role of Elastic Compute Service (ECS) is associated with the cluster. By default, the application role is AliyunECSInstanceForEMRRole. JindoSDK uses the role to obtain the security token to access OSS or OSS-HDFS.
To access OSS or OSS-HDFS in password-free mode, perform the following steps: On the Configure tab of the Hadoop-Common service page, click the core-site.xml tab and check whether the value of the fs.oss.credentials.provider parameter contains com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider. If the parameter value does not contain com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider, add com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider to the value of the parameter and click Save.
How do I check whether password-free access to OSS or OSS-HDFS is available in an EMR cluster?
Method 1: Run the following command to query the credentials of the RAM role:
curl http://100.100.100.200/latest/meta-data/Ram/Security-credentials/AliyunECSInstanceForEMRRoleIf the following result is returned, password-free access to OSS or OSS-HDFS is available:
{ "AccessKeyId" : "STS.NUreXXXXXX", "AccessKeySecret" : "BsmbnDoXXXXXXXX", "Expiration" : "2022-11-22T11:27:39Z", "SecurityToken" : "CAISlwJ1q6FXXXXXXX", "LastUpdated" : "2022-11-22T05:27:39Z", "Code" : "Success" }Method 2: Configure EcsStsCredentialsProvider to access OSS or OSS-HDFS in password-free mode.
On the Configure tab of the Hadoop-Common service page, click the core-site.xml tab and check whether the value of the fs.oss.credentials.provider parameter contains com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider. If the parameter value does not contain com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider, add com.aliyun.jindodata.oss.auth.EcsStsCredentialsProvider to the value of the parameter and click Save.
Use shell commands of Hadoop Distributed File System (HDFS) to access OSS or OSS-HDFS. If you can access OSS or OSS-HDFS, password-free access to OSS or OSS-HDFS is available.
What do I do if I am unable to access OSS or OSS-HDFS by using HiveServer2 or Presto every 5 to 6 hours?
In some versions of JindoData, some known issues about password-free access exist. For more information, see Known issues in JindoData 4.X.
Method 1: Use a fixed AccessKey pair.
Configure SimpleCredentialsProvider. For more information, see Configure a credential provider for OSS or OSS-HDFS.
Method 2: Upgrade JindoSDK to the latest version.
For more information about how to upgrade JindoSDK in the old EMR console, see Upgrade JindoSDK in EMR clusters in the old EMR console.
For more information about how to upgrade JindoSDK in the new EMR console, see Upgrade JindoSDK in EMR clusters in the new EMR console.
For more information about how to upgrade JindoSDK in an EMR cluster of V5.5.0, V3.39.0, or a minor version earlier than V5.5.0 or V3.39.0, see Update JindoSDK in EMR clusters.
What do I do if an error indicating that the OSS or OSS-HDFS path contains an AccessKey pair is reported?
Error message:
The Filesystem URI contains login details. This authentication mechanism is no longer supported.Cause:
To prevent security issues such as the leak of AccessKey pairs, JindoSDK V4.0.0 and later versions do not allow you to include an AccessKey pair in the access path.
Solutions:
Method 1: Delete the AccessKey pair from the access path.
Method 2: If you want to include the AccessKey pair in the access path, go to the Configure tab of the Hadoop-Common service page, click the core-site.xml tab. On the tab, click Add Configuration Item to add the following configuration item.
Configuration item
Value
fs.oss.uri-with-secrets.enable
true
After you add the configuration item, click Save.