JindoFS OSS credential providers store encrypted AccessKey credentials in a configuration file, preventing them from being exposed in plaintext. Starting from SmartData 3.4.0, you can configure one or more providers globally (for all buckets) or per OSS bucket.
Version requirements
| Provider | Minimum SmartData version |
|---|---|
| TemporaryAliyunCredentialsProvider | 3.4.0 |
| SimpleAliyunCredentialsProvider | 3.4.0 |
| EnvironmentVariableCredentialsProvider | 3.4.0 |
| JindoCommonCredentialsProvider | 3.4.0 |
| EcsStsCredentialsProvider | 3.4.0 |
| JindoRangerCredentialsProvider | 3.8.0 |
| AssumeRoleStsCredentialsProvider | 3.8.0 |
Choose a credential provider
Pick the provider that matches your authentication method:
| Provider | When to use |
|---|---|
TemporaryAliyunCredentialsProvider |
Short-lived AccessKey pair plus a security token |
SimpleAliyunCredentialsProvider |
Long-lived AccessKey pair |
EnvironmentVariableCredentialsProvider |
Credentials stored as environment variables |
JindoCommonCredentialsProvider |
Shared credentials used by both JindoOSS and JindoFS |
EcsStsCredentialsProvider |
Password-free access from an ECS instance (no AccessKey required) |
JindoRangerCredentialsProvider |
Apache Ranger-controlled access to OSS (SmartData 3.8.0+) |
AssumeRoleStsCredentialsProvider |
Temporary credentials via Security Token Service (STS) by assuming a Resource Access Management (RAM) role (SmartData 3.8.0+) |
You can specify multiple providers as a comma-separated list. The system tries each provider in order and uses the first one that returns valid credentials. For example:
com.aliyun.emr.fs.auth.TemporaryAliyunCredentialsProvider,com.aliyun.emr.fs.auth.SimpleAliyunCredentialsProvider,com.aliyun.emr.fs.auth.EnvironmentVariableCredentialsProvider
Configure a credential provider
Prerequisites
Before you begin, ensure that you have:
-
An EMR cluster running SmartData 3.4.0 or later
-
Access to the Alibaba Cloud EMR console
Step 1: Open the SmartData Configure tab
-
Log on to the Alibaba Cloud EMR console.
-
In the top navigation bar, select the region where your cluster resides and select a resource group.
-
Click the Cluster Management tab.
-
Find your cluster and click Details in the Actions column.
-
In the left-side navigation pane, choose Cluster Service > SmartData.
-
Click the Configure tab.
Step 2: Add the provider configuration
Choose the configuration scope:
-
Global (all buckets): In the Configuration Filter section on the smartdata-site tab, find the
fs.jfs.cache.oss.credentials.providerparameter and append the provider class name to its value. -
Bucket-level: In the upper-right corner of the smartdata-site tab, click Custom Configuration. In the Add Configuration Item dialog box, set Key to
fs.jfs.cache.oss.bucket.<bucket-name>.credentials.providerand Value to the provider class name.NoteReplace
<bucket-name>with the name of the OSS bucket.ImportantIf you use
JindoRangerCredentialsProvider, add the configuration on the namespace tab instead of the smartdata-site tab.
Then add any required parameters for the provider. See Provider reference for the full parameter list.
Step 3: Save the configuration
-
In the upper-right corner of the Service Configuration section, click Save.
-
In the Confirm Changes dialog box, enter a description and turn on Auto-update Configuration.
-
Click OK.
Provider reference
TemporaryAliyunCredentialsProvider
Use for short-lived AccessKey pairs and security tokens.
Class name: com.aliyun.emr.fs.auth.TemporaryAliyunCredentialsProvider
Add the following parameters on the smartdata-site tab:
Global configuration
| Parameter | Description |
|---|---|
fs.jfs.cache.oss.accessKeyId |
AccessKey ID |
fs.jfs.cache.oss.accessKeySecret |
AccessKey secret |
fs.jfs.cache.oss.securityToken |
Temporary security token |
Bucket-level configuration
| Parameter | Description |
|---|---|
fs.jfs.cache.oss.bucket.<bucket-name>.accessKeyId |
AccessKey ID |
fs.jfs.cache.oss.bucket.<bucket-name>.accessKeySecret |
AccessKey secret |
fs.jfs.cache.oss.bucket.<bucket-name>.securityToken |
Temporary security token |
SimpleAliyunCredentialsProvider
Use for long-lived AccessKey pairs.
Class name: com.aliyun.emr.fs.auth.SimpleAliyunCredentialsProvider
Add the following parameters on the smartdata-site tab:
Global configuration
| Parameter | Description |
|---|---|
fs.jfs.cache.oss.accessKeyId |
AccessKey ID |
fs.jfs.cache.oss.accessKeySecret |
AccessKey secret |
Bucket-level configuration
| Parameter | Description |
|---|---|
fs.jfs.cache.oss.bucket.<bucket-name>.accessKeyId |
AccessKey ID |
fs.jfs.cache.oss.bucket.<bucket-name>.accessKeySecret |
AccessKey secret |
EnvironmentVariableCredentialsProvider
Use when credentials are stored as environment variables on the cluster nodes.
Class name: com.aliyun.emr.fs.auth.EnvironmentVariableCredentialsProvider
Set the following environment variables:
| Variable | Description |
|---|---|
ALIYUN_ACCESS_KEY_ID |
AccessKey ID |
ALIYUN_ACCESS_KEY_SECRET |
AccessKey secret |
ALIYUN_SECURITY_TOKEN |
Temporary security token. Required only when the token has a validity period. |
JindoCommonCredentialsProvider
A general-purpose provider that works across JindoOSS and JindoFS.
Class name: com.aliyun.emr.fs.auth.JindoCommonCredentialsProvider
Add the following parameters on the smartdata-site tab. These parameters apply to both global and bucket-level configuration scopes.
| Parameter | Description |
|---|---|
jindo.common.accessKeyId |
AccessKey ID |
jindo.common.accessKeySecret |
AccessKey secret |
jindo.common.securityToken |
Temporary security token |
EcsStsCredentialsProvider
Use for password-free access from an ECS instance. No AccessKey pair is needed.
Class name: com.aliyun.emr.fs.auth.EcsStsCredentialsProvider
No additional parameters are required.
JindoRangerCredentialsProvider
Use when Apache Ranger controls access to OSS. Requires SmartData 3.8.0 or later.
Class name: com.aliyun.emr.fs.auth.JindoRangerCredentialsProvider
On the namespace tab, add the following custom parameter:
| Parameter | Value |
|---|---|
namespace.oss.permission.method |
ranger |
After adding the parameter, restart Jindo Namespace Service. See Restart Jindo Namespace Service.
AssumeRoleStsCredentialsProvider
Use to access OSS with temporary credentials obtained by assuming a RAM role through STS. Requires SmartData 3.8.0 or later.
Class name: com.aliyun.emr.fs.auth.AssumeRoleStsCredentialsProvider
Add the following parameters on the smartdata-site tab:
| Parameter | Description |
|---|---|
assume.role.sts.accessKeyId |
AccessKey ID obtained from STS |
assume.role.sts.accessKeySecret |
AccessKey secret obtained from STS |
assume.role.sts.endpoint |
STS endpoint. See Endpoints. |
assume.role.roleArn |
Alibaba Cloud Resource Name (ARN) of the RAM role, in the format acs:ram::<accountID>:role/<roleName>. To view the ARN, see How do I view the ARN of a RAM role? |
assume.role.roleSessionName |
Name of the role session. Set this to any identifier, such as a username. |
Restart Jindo Namespace Service
If you use JindoRangerCredentialsProvider, restart Jindo Namespace Service after adding the configuration.
-
On the Configure tab of the SmartData service page, click Actions in the upper-right corner and select Restart Jindo Namespace Service.
-
In the Cluster Activities dialog box, enter a description and click OK.
-
In the confirmation message, click OK.