All Products
Search

This Product

    E-MapReduce:Rewrite Jindo HDFS client paths

    Document Center

    E-MapReduce:Rewrite Jindo HDFS client paths

    Last Updated:Sep 16, 2026

    SmartData 3.1.x and later supports path rewriting at the Jindo HDFS client level, simplifying path changes during cluster migration. For example, rewriting an HDFS path to an OSS path lets you access data migrated to OSS without changing your application's data paths.

    Limitations

    This feature is supported on Hadoop 2.x, but not on Hadoop 3.x.

    Enable path rewriting

    1. Go to the SmartData service.
      1. Log on to the EMR console.
      2. In the top navigation bar, select a region and a resource group as needed.
      3. Click the Clusters tab.
      4. On the Clusters page, find your cluster and click Details in the Actions column.
      5. In the navigation pane on the left, choose Services > SmartData.
    2. Click the Configure tab.
    3. In the Service Configuration section, click the smartdata-site tab.
    4. In the upper-right corner of the Service Configuration section, click Custom Configuration.
    5. In the Add Configuration Item dialog box, add a configuration item, setting Key to fs.hdfs.impl and Value to com.aliyun.emr.fs.hdfs.JindoHdfsShimsFileSystem.
    6. Click OK.
    7. Save the configurations.
      1. In the upper-right corner, click Save.
      2. In the Confirm dialog box, enter a reason in the Description field and enable Auto-update Configuration.
      3. Click OK.

    Configure path rewriting

    1. Go to the SmartData service.
      1. Log on to the EMR console.
      2. In the top navigation bar, select a region and a resource group as needed.
      3. Click the Clusters tab.
      4. On the Clusters page, find your cluster and click Details in the Actions column.
      5. In the navigation pane on the left, choose Services > SmartData.
    2. Click the Configure tab.
    3. In the Service Configuration section, click the smartdata-site tab.
    4. In the upper-right corner of the Service Configuration section, click Custom Configuration.
    5. In the Add Configuration Item dialog box, add the following two configuration items.
      Parameter Description Value
      fs.jindo.shim.path-rewrite.<RULE-NAME>.source The source path to rewrite. This path acts as a mount point.
      • HA cluster

        hdfs://emr-cluster/<osspath>

      • non-HA cluster

        hdfs://<your_hostname>:9000/<osspath>

        Note To obtain your hostname, log on to the master node by using SSH and run the hostname command. For more information, see Log on to a cluster.
      fs.jindo.shim.path-rewrite.<RULE-NAME>.target The destination path after the rewrite. oss://<your_bucket>/<testpath>

      You must customize <RULE-NAME>.

    6. Click OK.
    7. Save the configurations.
      1. In the upper-right corner, click Save.
      2. In the Confirm dialog box, enter a reason in the Description field and enable Auto-update Configuration.
      3. Click OK.

    Example

    For an HA cluster, after you add the following parameters on the smartdata-site tab, requests to hdfs://emr-cluster/osspath are rewritten to oss://jindo-bucket/<testpath>.
    Parameter Value
    fs.jindo.shim.path-rewrite.testrule.source hdfs://emr-cluster/osspath
    fs.jindo.shim.path-rewrite.testrule.target oss://jindo-bucket/<testpath>
    Log on to the master node of the cluster by using SSH and run the following command to verify the configuration:
    hadoop fs -ls /
    The following output indicates that osspath is mounted to the root directory. 
    [root@emr-header-1 ~]# hadoop fs -ls /
Found 7 items
drwxr-xr-x   - hadoop    hadoop          0 2020-11-24 21:12 /apps
drwxrwxrwx   - flowagent hadoop          0 2020-11-24 21:11 /emr-flow
drwxr-x--x   - root      hadoop          0 2020-11-24 21:12 /emr-sparksql-udf
drwxrwxrwx   -                            0 2020-11-25 16:55 /osspath
drwxr-x-x    - hadoop    hadoop          0 2020-11-25 16:56 /spark-history
drwxrwxrwx   - root      hadoop          0 2020-11-25 16:55 /tmp
drwxr-x--t   - hadoop    hadoop          0 2020-11-24 21:14 /user