All Products
Search

This Product

    E-MapReduce:Use JindoFS in block storage mode

    Document Center

    E-MapReduce:Use JindoFS in block storage mode

    Last Updated:Mar 26, 2026

    JindoFS block storage mode stores data as blocks in OSS and caches frequently accessed data on local cluster disks. Namespace Service manages metadata to ensure high metadata accessibility. Use this mode when you need scalable OSS-backed storage with local caching acceleration and HDFS-compatible metadata performance.

    How block storage mode works

    Block storage mode provides the following advantages:

    • Scalable storage: OSS acts as the storage backend, so storage capacity grows independently of your EMR cluster size. Scale the cluster in or out based on compute requirements.

    • Accelerated reads: JindoFS caches data on local disks to boost read throughput, especially for Write Once Read Many (WORM) workloads.

    • Fast metadata queries: Namespace Service delivers metadata performance comparable to Hadoop Distributed File System (HDFS), avoiding the latency and instability that occur when data and metadata are accessed concurrently at high frequency.

    • Maximized data locality: Jobs run on nodes that hold cached copies of the data they need, reducing network traffic and improving read performance.

    Configure block storage mode

    Prerequisites

    Before you begin, ensure that you have:

    • An active EMR cluster with the SmartData service installed

    • An OSS bucket to use as the storage backend

    If your OSS bucket is in the same region and under the same account as your EMR cluster, the bucket allows password-free access from the cluster. You do not need to configure the AccessKey ID or AccessKey secret. Skip the jfs.namespaces.test.oss.access.key and jfs.namespaces.test.oss.access.secret parameters in step 3.

    Steps

    1. Go to the SmartData service in the EMR console.

      1. Log on to the EMR console.

      2. In the top navigation bar, select the region where your cluster resides. Select a resource group as needed. By default, all resources in the account are shown.

      3. Click the Cluster Management tab.

      4. Find your cluster and click Details in the Actions column.

      5. In the left navigation pane, click Cluster Service, then click SmartData.

    2. Open the namespace configuration tab.

      1. Click the Configure tab.

      2. In the Service Configuration section, click the namespace tab. namespace

    3. Configure the namespace parameters. JindoFS supports multiple namespaces. The following example uses a namespace named test.

      1. Set jfs.namespaces to test. To configure multiple namespaces, separate names with commas (,).

      2. In the upper-right corner of the Service Configuration section, click Custom Configuration. In the Add Configuration Item dialog box, add the following parameters:

        ParameterDescriptionExample
        jfs.namespaces.test.oss.uriThe OSS storage backend for the test namespace. Set this to a directory path — the namespace stores data blocks in that directory.oss://<oss_bucket>/<oss_dir>/
        jfs.namespaces.test.modeThe storage mode. Set to block for block storage mode.block
        jfs.namespaces.test.oss.access.keyThe AccessKey ID of the OSS bucket. Not required if the bucket is in the same region and account as the cluster.xxxx
        jfs.namespaces.test.oss.access.secretThe AccessKey secret of the OSS bucket. Not required if the bucket is in the same region and account as the cluster.—

      3. Click OK.

    4. In the upper-right corner of the Service Configuration section, click Save.

    5. From the Actions drop-down list in the upper-right corner, select Restart Jindo Namespace Service.

    After Namespace Service restarts, access files using the path jfs://test/<path_of_file>.

    Control disk space usage

    JindoFS uses OSS as the primary storage backend and caches data on local cluster disks for faster access. Because local disk capacity is finite, JindoFS automatically evicts cold data when disk usage crosses the high watermark, continuing until usage falls to the low watermark.

    Use the storage.watermark.high.ratio and storage.watermark.low.ratio parameters to control how much local disk space JindoFS can use. Both parameters accept a decimal value between 0 and 1.

    Important

    The high watermark value must be greater than the low watermark value.

    Steps

    1. In the Service Configuration section for the SmartData service, click the storage tab and set the watermark parameters.

      ParameterDescriptionDefault
      storage.watermark.high.ratioUpper limit of local disk usage. When JindoFS data exceeds this ratio, automatic eviction begins.0.4
      storage.watermark.low.ratioLower limit of local disk usage. Eviction continues until disk usage falls to this ratio.0.2

      storage

    2. Save the configuration.

      1. In the upper-right corner of the Service Configuration section, click Save.

      2. In the Confirm Changes dialog box, enter a description and turn on Auto-update Configuration.

      3. Click OK.

    3. Restart Jindo Storage Service to apply the changes.

      1. In the upper-right corner, choose Actions > Restart Jindo Storage Service.

      2. In the Cluster Activities dialog box, set the required parameters.

      3. Click OK.

      4. In the confirmation message, click OK.