In SmartData 3.1.X, you can change the HDFS client path for the Jindo system. This way, you do not need to change the path when you migrate cluster data. For example, after you change the HDFS client path for the Jindo system to an Object Storage Service (OSS) path, you can access the data that you migrated from HDFS to OSS. You do not need to change the data storage path that is defined in the business logic.

Limits

This feature is supported only in Hadoop 2.x.

Enable the path change feature

  1. Go to the SmartData service.
    1. Log on to the Alibaba Cloud EMR console.
    2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.
    3. Click the Cluster Management tab.
    4. On the Cluster Management page that appears, find the target cluster and click Details in the Actions column.
    5. In the left-side navigation pane, click Cluster Service and then SmartData.
  2. Click the Configure tab.
  3. In the Service Configuration section, click the smartdata-site tab.
  4. In the Service Configuration section, click Custom Configuration in the upper-right corner.
  5. In the Add Configuration Item dialog box, add a configuration item whose Key is set to fs.hdfs.impl and Value is set to com.aliyun.emr.fs.hdfs.JindoHdfsShimsFileSystem.
    add-fs
  6. Click OK.
  7. Save the configuration.
    1. In the upper-right corner of the Service Configuration section, click Save.
    2. In the Confirm Changes dialog box, specify Description and turn on Auto-update Configuration.
    3. Click OK.

Change the HDFS client path for the Jindo system

  1. Go to the SmartData service.
    1. Log on to the Alibaba Cloud EMR console.
    2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.
    3. Click the Cluster Management tab.
    4. On the Cluster Management page that appears, find the target cluster and click Details in the Actions column.
    5. In the left-side navigation pane, click Cluster Service and then SmartData.
  2. Click the Configure tab.
  3. In the Service Configuration section, click the smartdata-site tab.
  4. In the Service Configuration section, click Custom Configuration in the upper-right corner.
  5. In the Add Configuration Item dialog box, add the configuration items that are described in the following table.
    Parameter Description Value
    fs.jindo.shim.path-rewrite.<RULE-NAME>.source The HDFS client path that you want to change.
    • High availability (HA) cluster

      hdfs://emr-cluster/<osspath>

    • Non-HA cluster

      hdfs://<your_hostname>:9000/<osspath>

      Note To obtain the hostname that you can use to replace <your_hostname> in the parameter value, you can log on to the master node of your EMR cluster in SSH mode and run the hostname command. For more information, see Log on to a cluster.
    fs.jindo.shim.path-rewrite.<RULE-NAME>.target The path you actually access. oss://<your_bucket>/<testpath>

    You must specify RULE-NAME.

  6. Click OK.
  7. Save the configuration.
    1. In the upper-right corner of the Service Configuration section, click Save.
    2. In the Confirm Changes dialog box, specify Description and turn on Auto-update Configuration.
    3. Click OK.

Example

For an HA cluster, you can access data that is stored in oss://jindo-bucket/<testpath> instead of hdfs://emr-cluster/osspath after you add the parameters on the smartdata-site tab. The following table describes the parameters.
Parameter Value
fs.jindo.shim.path-rewrite.testrule.source hdfs://emr-cluster/osspath
fs.jindo.shim.path-rewrite.testrule.target oss://jindo-bucket/<testpath>
To view the path change, you can log on to the master node of your EMR cluster in SSH mode and run the following command: 
hadoop fs -ls /
The following output shows that osspath is mounted on the root directory. demo