Configure RootPolicy to access OSS-HDFS with a custom prefix - Object Storage Service

If your Hadoop jobs use hdfs:// URIs and you want to run them on OSS-HDFS without modifying any job code or configuration, use RootPolicy. RootPolicy maps a custom hdfs:// namespace to an OSS-HDFS bucket, so existing hdfs:// paths resolve transparently to OSS-HDFS at runtime.

Prerequisites

Before you begin, make sure you have:

A Hadoop environment, Hadoop cluster, or Hadoop client. For setup instructions, see Step 2: Create a Hadoop runtime environment
OSS-HDFS enabled on the target bucket. See Enable OSS-HDFS and grant access permissions
JindoSDK 4.5.0 or later installed and configured. See Connect non-EMR clusters to OSS-HDFS

Set up RootPolicy

Step 1: Configure environment variables

Connect to an Elastic Compute Service (ECS) instance. See Connect to an ECS instance.
Download the JindoFS command line interface.
Go to the bin directory of the JindoFS JAR package. Replace x.x.x with the actual version number.
```
   cd jindofs-sdk-x.x.x-linux/bin/
```
Create a configuration file named jindofs.cfg in the bin directory. Add the AccessKey pair for your Alibaba Cloud account or a Resource Access Management (RAM) user with the required permissions.
```
   [client]
   fs.oss.accessKeyId = <your-access-key-id>
   fs.oss.accessKeySecret = <your-access-key-secret>
```
Set the JINDOSDK_CONF_DIR environment variable to the absolute path of jindofs.cfg.
```
   export JINDOSDK_CONF_DIR=<absolute-path-to-jindofs-cfg>
```

Step 2: Register a RootPolicy

Run the SetRootPolicy command to map a custom hdfs:// namespace to your OSS-HDFS bucket.

./jindofs admin -setRootPolicy oss://<bucket_name>.<dls_endpoint>/ hdfs://<your_ns_name>/

Parameter	Description	Example
`bucket_name`	The name of the bucket with OSS-HDFS enabled	`my-bucket`
`dls_endpoint`	The OSS-HDFS endpoint for the bucket's region	`cn-hangzhou.oss-dls.aliyuncs.com`
`your_ns_name`	A non-empty string used as the custom namespace name	`test`

The current version supports mapping only the root directory.

Avoid repeating the endpoint in every command

To avoid specifying <dls_endpoint> each time you run a RootPolicy command, add one of the following properties to the core-site.xml file of Hadoop:

Apply to all buckets:

  <configuration>
      <property>
          <name>fs.oss.endpoint</name>
          <value><dls_endpoint></value>
      </property>
  </configuration>

Apply to a specific bucket:

  <configuration>
      <property>
          <name>fs.oss.bucket.<bucket_name>.endpoint</name>
          <value><dls_endpoint></value>
      </property>
  </configuration>

Step 3: Configure access policy discovery and scheme classes

Add the following properties to core-site.xml. These tell Hadoop where to discover RootPolicy mappings and which FileSystem implementation to use for hdfs:// URIs.

Select the fs.AbstractFileSystem.hdfs.impl and fs.hdfs.impl values based on your Hadoop version. Hadoop 2.x uses the v28 classes; Hadoop 3.x uses the v3 classes.

<configuration>
    <property>
        <!-- The OSS-HDFS bucket that stores RootPolicy mappings -->
        <name>fs.accessPolicies.discovery</name>
        <value>oss://<bucket_name>.<dls_endpoint>/</value>
    </property>
    <property>
        <!-- AbstractFileSystem implementation for hdfs:// URIs -->
        <!-- Hadoop 2.x: com.aliyun.jindodata.hdfs.v28.HDFS -->
        <!-- Hadoop 3.x: com.aliyun.jindodata.hdfs.v3.HDFS -->
        <name>fs.AbstractFileSystem.hdfs.impl</name>
        <value>com.aliyun.jindodata.hdfs.v3.HDFS</value>
    </property>
    <property>
        <!-- FileSystem implementation for hdfs:// URIs -->
        <!-- Hadoop 2.x: com.aliyun.jindodata.hdfs.v28.JindoDistributedFileSystem -->
        <!-- Hadoop 3.x: com.aliyun.jindodata.hdfs.v3.JindoDistributedFileSystem -->
        <name>fs.hdfs.impl</name>
        <value>com.aliyun.jindodata.hdfs.v3.JindoDistributedFileSystem</value>
    </property>
</configuration>

To configure multiple buckets, separate them with commas in the fs.accessPolicies.discovery value:

<value>oss://<bucket1_name>.<dls_endpoint>/, oss://<bucket2_name>.<dls_endpoint>/</value>

Step 4: Verify the configuration

Run the following command to list the contents of the mapped namespace:

hadoop fs -ls hdfs://<your_ns_name>/

If RootPolicy is working correctly, the command returns the root directory listing of your OSS-HDFS bucket. This confirms that hdfs://<your_ns_name>/ now routes to oss://<bucket_name>.<dls_endpoint>/ based on the fs.accessPolicies.discovery property you set in Step 3.

drwxr-x--x   - hdfs  hadoop          0 2025-06-30 12:27 hdfs://<your_ns_name>/apps
drwxrwxrwx   - spark hadoop          0 2025-06-30 12:27 hdfs://<your_ns_name>/spark-history
drwxrwxrwx   - hdfs  hadoop          0 2025-06-30 12:27 hdfs://<your_ns_name>/tmp
drwxrwxrwx   - hdfs  hadoop          0 2025-06-30 12:27 hdfs://<your_ns_name>/user

Step 5: Access OSS-HDFS using the custom namespace

Restart services such as Hive and Spark, then use hdfs://<your_ns_name>/ in place of oss://<bucket_name>.<dls_endpoint>/ in your jobs and configurations.

Manage RootPolicy mappings

List registered mappings

To see all hdfs:// namespaces registered for a bucket, run:

./jindofs admin -listAccessPolicies oss://<bucket_name>.<dls_endpoint>/

Delete a mapping

To remove a registered mapping, run:

./jindofs admin -unsetRootPolicy oss://<bucket_name>.<dls_endpoint>/ hdfs://<your_ns_name>/

What's next

Jindo CLI user guide