Configure a credential provider for an OSS or OSS-HDFS bucket - E-MapReduce

When your EMR cluster accesses multiple OSS or OSS-HDFS buckets that belong to different accounts, or have different security requirements, you can assign a separate credential provider to each bucket. The bucket-level setting overrides the global credential configuration for that specific bucket and leaves all other buckets unaffected.

How it works

Set fs.oss.bucket.<bucket-name>.credentials.provider to assign a credential provider to a specific bucket. Replace <bucket-name> with the actual bucket name.

The property accepts a comma-separated list of provider class names. The system tries each provider in order and uses the first one that returns valid credentials.

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.credentials.provider</name>
        <value>com.aliyun.jindodata.oss.auth.SimpleAliyunCredentialsProvider,com.aliyun.jindodata.oss.auth.EnvironmentVariableCredentialsProvider,com.aliyun.jindodata.oss.auth.CommonCredentialsProvider</value>
        <description>Specify the credential provider implementation classes. Separate multiple classes with commas. The system reads credential values in sequence until a valid credential is found.</description>
    </property>
</configuration>

In the preceding code, XXX indicates the name of an OSS or OSS-HDFS bucket.

Choose a credential provider

Select a provider based on your use case:

Provider	Full class name	Use when
TemporaryCredentialsProvider	`com.aliyun.jindodata.oss.auth.TemporaryCredentialsProvider`	Accessing OSS or OSS-HDFS with a time-limited AccessKey pair and security token
SimpleCredentialsProvider	`com.aliyun.jindodata.oss.auth.SimpleCredentialsProvider`	Accessing OSS or OSS-HDFS with a permanently valid AccessKey pair
EnvironmentVariableCredentialsProvider	`com.aliyun.jindodata.oss.auth.EnvironmentVariableCredentialsProvider`	Reading the AccessKey pair from environment variables
CommonCredentialsProvider	`com.aliyun.jindodata.oss.auth.CommonCredentialsProvider`	General-purpose scenarios

TemporaryCredentialsProvider

Use this provider when accessing OSS or OSS-HDFS with a time-limited AccessKey pair and a security token.

Configure the provider

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.credentials.provider</name>
        <value>com.aliyun.jindodata.oss.auth.TemporaryCredentialsProvider</value>
    </property>
</configuration>

Configure the credentials

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.accessKeyId</name>
        <value>The AccessKey ID of the OSS or OSS-HDFS bucket</value>
    </property>
    <property>
        <name>fs.oss.bucket.XXX.accessKeySecret</name>
        <value>The AccessKey secret of the OSS or OSS-HDFS bucket</value>
    </property>
    <property>
        <name>fs.oss.bucket.XXX.securityToken</name>
        <value>The security token of the OSS or OSS-HDFS bucket</value>
    </property>
</configuration>

SimpleCredentialsProvider

Use this provider when accessing OSS or OSS-HDFS with a permanently valid AccessKey pair.

Configure the provider

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.credentials.provider</name>
        <value>com.aliyun.jindodata.oss.auth.SimpleCredentialsProvider</value>
    </property>
</configuration>

Configure the credentials

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.accessKeyId</name>
        <value>The AccessKey ID of the OSS or OSS-HDFS bucket</value>
    </property>
    <property>
        <name>fs.oss.bucket.XXX.accessKeySecret</name>
        <value>The AccessKey secret of the OSS or OSS-HDFS bucket</value>
    </property>
</configuration>

EnvironmentVariableCredentialsProvider

Use this provider when the AccessKey pair is injected through environment variables.

Configure the provider

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.credentials.provider</name>
        <value>com.aliyun.jindodata.oss.auth.EnvironmentVariableCredentialsProvider</value>
    </property>
</configuration>

Configure the environment variables

Set the following variables in your environment variable file:

Environment variable	Description
`OSS_ACCESS_KEY_ID`	The AccessKey ID used to access OSS or OSS-HDFS
`OSS_ACCESS_KEY_SECRET`	The AccessKey secret used to access OSS or OSS-HDFS
`OSS_SECURITY_TOKEN`	The security token used to access OSS or OSS-HDFS. Required only when using a time-limited security token.

CommonCredentialsProvider

Use this provider for general-purpose scenarios where credentials are shared across multiple configurations.

Configure the provider

<configuration>
    <property>
        <name>fs.oss.bucket.XXX.credentials.provider</name>
        <value>com.aliyun.jindodata.oss.auth.CommonCredentialsProvider</value>
    </property>
</configuration>

Configure the credentials

<configuration>
    <property>
        <name>jindo.common.accessKeyId</name>
        <value>The AccessKey ID of the OSS or OSS-HDFS bucket</value>
    </property>
    <property>
        <name>jindo.common.accessKeySecret</name>
        <value>The AccessKey secret of the OSS or OSS-HDFS bucket</value>
    </property>
    <property>
        <name>jindo.common.securityToken</name>
        <value>The security token of the OSS or OSS-HDFS bucket. Required only when using a time-limited security token.</value>
    </property>
</configuration>