Object Storage Service:Accelerate OSS access by using EFC

Runtime environment

Hardware resource requirements

Plan memory and disk before deployment based on whether caching is enabled.

Caching disabled:

• Memory: Resident usage is typically under 1 GiB. Reserve at least 5 GiB to handle burst loads.

• Disk: No special requirement beyond space for operational logs.

Caching enabled:

Resource usage includes cache media plus index overhead.

• Memory:

  • Base usage: same as caching-disabled mode.

  • Memory cache: consumes memory equal to the configured cache size.

  • Index overhead: approximately 0.1% of total cache capacity, regardless of cache type.

  • Formula: Total memory ≈ Base runtime memory + Memory cache size + (Memory cache size + Disk cache size) × 0.001

• Disk: Disk cache consumes space equal to the configured size (1 TiB configured = 1 TiB used).

Feature limits

• Access mode: Read-only. After mounting, all files are owned by root. chmod and chown are not supported.

• POSIX operations: Supported: open, close, read, readdir, readdirplus, lookup, getattr. Not supported: readlink, write, rename, setattr, link, symlink.

• Storage class: Supports Standard and Infrequent Access only. Archive, Cold Archive, and Deep Cold Archive objects are not accessible.

• Path restrictions:

  • If both a/b (file) and a/b/ (directory) exist in OSS, only the directory a/b/ is accessible after mounting.

  • Objects whose keys start with /, contain consecutive //, or include . or .. are not accessible.

• Required OSS permissions: oss:GetBucketStat, oss:ListObjects, and oss:GetObject.

• Cache availability: EFC is a read-only cache and does not guarantee data high availability. Cached data can be lost on hardware failure, node replacement, or decommissioning, causing a cache miss and an origin fetch on the next access.

Step 1: Install the EFC client

Download and install two packages on each compute node.

1. Download the packages:

      # alinas-utils package
      wget https://aliyun-alinas-eac-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/cache/aliyun-alinas-utils.amd64.rpm
   
      # EFC client
      wget https://aliyun-alinas-eac-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/cache/alinas-efc-latest.amd64.rpm

2. Install the packages:

      sudo rpm -ivh aliyun-alinas-utils.amd64.rpm
      sudo rpm -ivh alinas-efc-latest.amd64.rpm

After installation, the EFC service registers automatically but does not start. Complete credential configuration before mounting.

Step 2: Configure access credentials

EFC supports two authentication methods. Use Security Token Service (STS) temporary credentials to avoid exposing long-term AccessKeys.

# Create the STS credential file
cat > /etc/passwd-sts << EOF
{
    "SecurityToken": "YourSecurityToken",
    "AccessKeyId": "YourSTSAccessKeyId",
    "AccessKeySecret": "YourSTSAccessKeySecret",
    "Expiration": "YourExpiration"
}
EOF

# Restrict access to root only
chmod 600 /etc/passwd-sts

# Create the password file
echo "YourAccessKeyId:YourAccessKeySecret" > /etc/passwd-oss

# Restrict access to root only
chmod 600 /etc/passwd-oss

Step 3: Mount the OSS bucket

Choose the configuration that matches your workload. Start with the basic mount to verify access, then add caching.

# Create the mount point
mkdir -p /mnt/oss_data

# Mount using STS credentials (recommended)
mount -t alinas \
  -o efc,protocol=oss,g_oss_STSFile=/etc/passwd-sts \
  your-bucket.oss-cn-hangzhou.aliyuncs.com:/ /mnt/oss_data

# Mount using AccessKey
mount -t alinas \
  -o efc,protocol=oss,passwd_file=/etc/passwd-oss \
  your-bucket.oss-cn-hangzhou.aliyuncs.com:/ /mnt/oss_data

# Check that the mount succeeded
df -h | grep /mnt/oss_data

# List files in the bucket
ls /mnt/oss_data

# Create the disk cache directory
mkdir -p /mnt/cache/

# Mount with single-node cache (using STS credentials)
mount -t alinas \
  -o efc,protocol=oss,g_oss_STSFile=/etc/passwd-sts \
  -o g_tier_EnableClusterCache=true \
  -o g_tier_DadiIsDistributed=false \
  -o g_tier_DadiMemCacheCapacityMB=1024 \
  -o g_tier_DadiDiskCacheCapacityMB=10240 \
  -o g_tier_DadiDiskCachePath=/mnt/cache/ \
  -o g_server_Port=17871 \
  your-bucket.oss-cn-hangzhou.aliyuncs.com:/ /mnt/oss_data

# Format: <unique-id>:<node-private-IP>:<P2P-port>
1:192.168.1.1:17980
2:192.168.1.2:17980
3:192.168.1.3:17980

# Create the disk cache directory
mkdir -p /mnt/cache/

# Mount with distributed cache (using STS credentials)
mount -t alinas \
  -o efc,protocol=oss,g_oss_STSFile=/etc/passwd-sts \
  -o g_tier_EnableClusterCache=true \
  -o g_tier_DadiIsDistributed=true \
  -o g_tier_DadiAddr=/etc/efc/rootlist \
  -o g_tier_DadiMemCacheCapacityMB=1024 \
  -o g_tier_DadiDiskCacheCapacityMB=10240 \
  -o g_tier_DadiDiskCachePath=/mnt/cache/ \
  -o g_server_Port=17871 \
  your-bucket.oss-cn-hangzhou.aliyuncs.com:/ /mnt/oss_data

mount -t alinas \
  -o efc,protocol=oss,g_oss_STSFile=/etc/passwd-sts \
  -o g_tier_EnableClusterCache=true \
  -o g_tier_DadiIsDistributed=true \
  -o g_tier_DadiRootClientType=2 \
  -o g_tier_DadiAddr=/etc/efc/rootlist \
  -o g_server_Port=17871 \
  your-bucket.oss-cn-hangzhou.aliyuncs.com:/ /mnt/oss_data

Prefetch data

Prefetching loads data from OSS into cache before it is read, eliminating first-access latency. This is useful for model inference deployments, where model files should be warm before traffic arrives.

• Prefetch paths must be relative paths under the mount point. For example, use model/weights.bin, not /mnt/oss_data/model/weights.bin.

• Task statuses: running, completed, canceled, failed.

• Completed, canceled, and failed tasks are auto-evicted when total historical tasks exceed 10,000 or task age exceeds 7 days. Running tasks can only be stopped with the cancel command.

curl -s "http://localhost:17871/v1/warmup/load?target_path=file100G"

{
    "ErrorCode": 0,
    "ErrorMessage": "Request processed",
    "Results": [{
        "ErrorCode": 0,
        "ErrorMessage": "The warm up (file100G) is processing in the background. Use the 'stat' command to get status.",
        "Location": "127.0.0.1:17871",
        "Path": "file100G"
    }]
}

curl -s "http://localhost:17871/v1/warmup/stat?target_path=file100G"

{
    "ErrorCode": 0,
    "ErrorMessage": "Request processed",
    "Results": [{
        "ErrorCode": 0,
        "ErrorMessage": "Successfully stat the warm up",
        "Location": "127.0.0.1:17871",
        "Path": "file100G",
        "TaskInfos": [{
            "CompletedSize": 13898874880,
            "ErrorCode": 0,
            "ErrorMessage": "",
            "IsDir": false,
            "Path": "file100G",
            "Pattern": "",
            "Status": "running",
            "SubmitTime": 1765274023424073,
            "TotalSize": 107374182400
        }]
    }]
}

curl -s "http://localhost:17871/v1/warmup/cancel?target_path=file100G"

/usr/bin/aliyun-alinas-efc-cli -m /mnt/oss_data -r warmup/load --path "testDir" --pattern ".*_2$"

/usr/bin/aliyun-alinas-efc-cli -m /mnt/oss_data -r warmup/stat --path "testDir"

/usr/bin/aliyun-alinas-efc-cli -m /mnt/oss_data -r warmup/cancel --path "testDir"

Monitor cache performance

Track cache hit rate and throughput to tune performance. Enable the management port during mounting to access these metrics.

View metrics:

# HTTP (returns JSON)
curl -s "http://localhost:17871/v1/tier"

# CLI (returns plain text)
aliyun-alinas-efc-cli -m <mount_point> -r tier

Reset metrics counters:

# HTTP
curl -s "http://localhost:17871/v1/tier/clear"

# CLI
aliyun-alinas-efc-cli -m <mount_point> -r tier -c

The following metrics describe possible read paths when client A accesses the cache provided by client B:

Scale the cluster

Update mount parameters without remounting

Use the Python runtime tool to update mount parameters without manually unmounting. The tool updates EFC's state file and restarts the EFC process with the new parameters.

Download the tool:

wget https://aliyun-alinas-eac-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/cache/efc_runtime_ops.latest.py

Update parameters:

python3 efc_runtime_ops.py update /mnt/oss_data \
  -o g_tier_EnableClusterCache=true,g_tier_DadiP2PPort=23456 \
  -o g_tier_BlockSize=1048576

Roll back if the update causes issues:

The tool automatically backs up the previous state file. Restore it with:

python3 efc_runtime_ops.py rollback /mnt/oss_data

Manage the directory cache

EFC caches directory listings by default to accelerate ls operations. If OSS directory structure changes are not reflected immediately, clear the directory cache manually.

Directory cache parameters (set at mount time):

Clear the directory cache:

# HTTP
curl -s "http://localhost:17871/v1/action/clear_readdir_cache"

# CLI
aliyun-alinas-efc-cli -m <mount_point> -r action -k clear_readdir_cache

Basic parameters

Cache configuration parameters

Metadata and directory cache parameters

Operations parameters

AI model training

AI training jobs typically do repeated, sequential reads of large datasets, often with many small files.

• Configure a disk cache large enough to hold the entire hot dataset—this prevents repeated origin fetches across epochs.

• Run data prefetch on your training data before starting the job to front-load the initial read latency.

AI model inference

Inference workloads have many compute nodes simultaneously loading the same model files.

• Always use distributed cluster mode so the P2P network handles model file distribution. This avoids concentrating origin fetch traffic on OSS or a single node.

• Prefetch model files when deploying or updating a service, before traffic arrives.

Security

• Set credential file permissions to 600.

• Use STS temporary credentials rather than long-term AccessKeys.

• Open P2P port 17980 only between cluster nodes. Restrict management port 17871 to trusted hosts.

Data consistency

• Enable metadata cache (g_metadata_CacheEnable=true) only for workloads where object metadata rarely changes, such as read-only model repositories. Understand that OSS updates will not be visible until the cache is cleared.

• For workloads that update OSS frequently and require immediate visibility, integrate cache-clearing calls into your update pipeline.