All Products
Search

This Product

    E-MapReduce:Use JindoFS in block storage mode

    Document Center

    E-MapReduce:Use JindoFS in block storage mode

    Last Updated:Mar 26, 2026

    JindoFS block storage mode stores data as blocks in OSS and caches them on local cluster disks for faster access. Jindo Namespace Service manages metadata and ensures high metadata accessibility — unlike querying OSS directly, which can cause instability when data and metadata are frequently accessed.

    How it works

    CapabilityDescription
    Scalable storageOSS serves as the storage backend, so storage capacity scales independently of your EMR cluster. Scale the cluster in or out without worrying about storage limits.
    Read accelerationJindoFS caches a subset of data on local disks, boosting throughput for Write Once Read Many (WORM) workloads.
    Fast metadataNamespace Service handles metadata operations similar to Hadoop Distributed File System (HDFS). This avoids the latency and instability that come with frequent data and metadata access against OSS directly.
    Data localityJindoFS schedules jobs close to cached data, reducing network traffic and improving read performance.

    Prerequisites

    Before you begin, ensure that you have:

    • An EMR cluster with the SmartData service installed

    • An OSS bucket to use as the storage backend

    • (If the OSS bucket is in a different region or under a different account than your EMR cluster) An AccessKey ID and AccessKey secret for the OSS bucket

    If the OSS bucket is in the same region and under the same account as your EMR cluster, the cluster can access it without credentials. You can skip the AccessKey parameters when configuring the namespace.

    Configure block storage mode

    JindoFS supports multiple namespaces. The steps below use test as the example namespace name.

    1. Go to the SmartData service.

      1. Log on to the EMR console.

      2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.

      3. Click the Cluster Management tab.

      4. On the Cluster Management page, find the target cluster and click Details in the Actions column.

      5. In the left-side navigation pane, click Cluster Service and then SmartData.

    2. Open the bigboot configuration.

      1. Click the Configure tab.

      2. In the Service Configuration section, click the bigboot tab. bigboot

    3. Set the namespace list. Set jfs.namespaces to test. To configure multiple namespaces, separate them with commas (,).

    4. Add the namespace parameters. In the upper-right corner of the Service Configuration section, click Custom Configuration. In the Add Configuration Item dialog box, add the following parameters:

      ParameterDescriptionExample
      jfs.namespaces.test.oss.uriThe OSS storage backend for the test namespace. Set this to a directory inside your bucket — JindoFS stores data blocks there.oss://<OSS_BUCKET>/<OSS_DIR>/
      jfs.namespaces.test.modeThe storage mode. Must be set to block.block
      jfs.namespaces.test.oss.access.keyThe AccessKey ID of the OSS bucket. Skip this if the bucket is in the same region and account as your EMR cluster.<OSS_ACCESS_KEY_ID>
      jfs.namespaces.test.oss.access.secretThe AccessKey secret of the OSS bucket. Skip this if the bucket is in the same region and account as your EMR cluster.<OSS_ACCESS_KEY_SECRET>

    5. Click OK.

    6. In the upper-right corner of the Service Configuration section, click Save.

    7. From the Actions drop-down list in the upper-right corner, select Restart Jindo Namespace Service.

    Verify the configuration

    After Jindo Namespace Service restarts, access files in JindoFS using the following URI pattern:

    jfs://test/<path_to_file>

    Replace test with your namespace name and <path_to_file> with the path to your file inside JindoFS.

    Manage local disk cache usage

    JindoFS automatically deletes cold data from local disks when they fill up. Two watermark parameters control when this happens:

    ParameterDescriptionDefault
    storage.watermark.high.ratioWhen local disk usage from JindoFS data reaches this ratio, automatic deletion starts.0.4
    storage.watermark.low.ratioDeletion continues until disk usage drops to this ratio.0.2

    Both values are decimals between 0 and 1. The high ratio must be greater than the low ratio.

    To adjust the watermarks:

    1. In the Service Configuration section for the SmartData service, click the storage tab and update the parameters.

      storage

    2. In the upper-right corner of the Service Configuration section, click Save.

    3. In the Confirm Changes dialog box, enter a description and turn on Auto-update Configuration. Click OK.

    4. From the Actions drop-down list in the upper-right corner, select Restart Jindo Storage Service.

    5. In the Cluster Activities dialog box, set the required parameters and click OK.

    6. In the Confirm message, click OK.