E-MapReduce:JindoFuse User Guide

Prerequisites

Before you begin, make sure you have:

• JindoSDK installed and configured

  • In an E-MapReduce (EMR) cluster: JindoSDK is pre-installed. To access OSS-HDFS, use EMR V3.42.0 or later, or EMR V5.8.0 or later.

  • In a non-EMR environment: install JindoSDK first. See Deploy JindoSDK in an environment other than EMR. To access OSS-HDFS, JindoSDK V4.X or later is required.

• The required FUSE (Filesystem in Userspace) dependencies installed (see Install dependencies)

• An OSS bucket or OSS-HDFS path to mount

• Valid Alibaba Cloud credentials (AccessKey ID and AccessKey Secret) with read or write permissions on the target bucket

Install dependencies

Clusters running EMR V3.44.0 or later, EMR V5.10.0 or later, or JindoSDK V4.6.2 or later already have the required dependencies. Skip this section if your cluster meets any of these conditions.

For older clusters, the required dependencies differ by JindoSDK version:

JindoSDK 4.5.0 or earlier — install fuse3:

# CentOS
sudo yum install -y fuse3 fuse3-devel

# Debian
sudo apt install -y fuse3 libfuse3-dev

JindoSDK 4.5.1 or later — build and install libfuse 3.7 or later. The following commands compile fuse-3.11:

# Build dependencies (Debian: apt install -y pkg-config meson ninja-build)
sudo yum install -y meson ninja-build

# CentOS only: install a newer C++ compiler from devtoolset-8
sudo yum install -y scl-utils
sudo yum install -y alinux-release-experimentals
sudo yum install -y devtoolset-8-gcc devtoolset-8-gdb devtoolset-8-binutils devtoolset-8-make devtoolset-8-gcc-c++
sudo su -c "echo 'source /opt/rh/devtoolset-8/enable' > /etc/profile.d/g++.sh"
source /opt/rh/devtoolset-8/enable
sudo ln -s /opt/rh/devtoolset-8/root/bin/gcc /usr/local/bin/gcc
sudo ln -s /opt/rh/devtoolset-8/root/bin/g++ /usr/local/bin/g++

# Download and compile libfuse
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.11.0/fuse-3.11.0.tar.xz
xz -d fuse-3.11.0.tar.xz
tar xf fuse-3.11.0.tar
cd fuse-3.11.0/
mkdir build; cd build
meson ..
sudo ninja install

Mount JindoFuse

1. Create a local mount point:

   mkdir -p <mount_point>

   Replace <mount_point> with a local path, for example /mnt/oss.

2. Mount the OSS or OSS-HDFS path:

   jindo-fuse <mount_point> -ouri=<oss_path>

   Replace <oss_path> with the OSS or OSS-HDFS path to map. The path can be the root directory or a subdirectory of a bucket, for example oss://examplebucket.cn-shanghai.oss-dls.aliyuncs.com/subdir/. OSS and OSS-HDFS mount commands are identical — only the endpoint in the path differs. This command starts a daemon process in the background.

3. Verify that JindoFuse is running:

   ps -ef | grep jindo-fuse

   If the jindo-fuse process is listed with the expected startup parameters, the mount is active.

Access files through the mount point

With JindoFuse mounted at /mnt/oss/, use standard Linux commands to work with your files:

# List the top-level contents
ls /mnt/oss/

# Create a directory
mkdir /mnt/oss/dir1

# Write a file
echo "hello world" > /mnt/oss/dir1/hello.txt

# Read the file back
cat /mnt/oss/dir1/hello.txt
# Output: hello world

# Delete a directory and its contents
rm -rf /mnt/oss/dir1/

Unmount JindoFuse

To unmount a mount point, run:

umount <mount_point>

Alternatively, pass -oauto_unmount when mounting. With this option, sending SIGINT to the jindo-fuse process automatically unmounts the mount point before the process exits:

killall -9 jindo-fuse

Supported POSIX operations

Most standard file system operations work on both OSS and OSS-HDFS. The table below lists all supported operations and highlights where OSS differs from OSS-HDFS.

Advanced usage

jindo-fuse <mount_point> -ouri=[<oss_path>] -ofs.oss.endpoint=[<endpoint>] -ofs.oss.accessKeyId=[<access_key_id>] -ofs.oss.accessKeySecret=[<access_key_secret>]

FAQ