Object Storage Service:ossfs FAQ

A 403 error occurs when you run the touch command after a successful mount

The "Operation not permitted" error occurs when you run the rm command to delete a file

Cause: When you run the `rm` command to delete a file, the `DeleteObject` API operation is called. If you mounted the bucket as a Resource Access Management (RAM) user, check whether the RAM user has permission to delete files.

Solution: Grant the required permissions to the RAM user. For more information, see RAM policies and Common examples of RAM policies.

The "The bucket you are attempting to access must be addressed using the specified endpoint" error occurs

Solution: Confirm that your configuration is correct and make changes if necessary.

Does ossfs support custom domain names for mounting buckets?

No, ossfs does not currently support custom domain name scenarios.

Mounting fails when you use an ECS RAM role with ossfs 1.91.7 or later

Solution:

1. Run the curl http://100.100.100.200/latest/meta-data/ram/security-credentials/[your-ecs-ram-role] command to check for connectivity.

2. If the `curl` command runs successfully, add the -o disable_imdsv2 option when you mount the bucket.

The "ossfs: unable to access MOUNTPOINT /tmp/ossfs: Transport endpoint is not connected" error occurs during mounting

Cause: This error occurs because the mount point directory does not exist.

Solution: Create the mount point directory before you mount the bucket.

The "fusermount: failed to open current directory: Permission denied" error occurs during mounting

Cause: This is a bug in Filesystem in Userspace (FUSE). The current user must have read permissions on the current working directory, which is different from the mount directory.

Solution: Run the `cd` command to switch to a directory for which you have read permissions, and then run the `ossfs` command.

The "ossfs: Mountpoint directory /tmp/ossfs is not empty. if you are sure this is safe, can use the 'nonempty' mount option" error occurs during mounting

Cause: By default, ossfs can be mounted only to an empty directory. This error occurs when you try to mount a bucket to a non-empty directory.

Solution: Switch to an empty directory and mount the bucket again. If you must mount the bucket to the non-empty directory, add the `-ononempty` parameter to the mount command.

The "ops-nginx-12-32 s3fs[163588]: [tid-75593]curl.cpp:CurlProgress(532): timeout now: 1656407871, curl_times[curl]: 1656407810, readwrite_timeout: 60" error occurs during mounting

Cause: The ossfs mount operation timed out.

Solution: ossfs uses the readwrite_timeout option to specify the timeout period for read and write requests. The unit is seconds, and the default value is 60. You can increase this value as needed.

The "ossfs: credentials file /etc/passwd-ossfs should not have others permissions" error occurs during mounting

Cause: The permissions for the /etc/passwd-ossfs file are incorrect.

Solution: The /etc/passwd-ossfs file stores access credentials. To protect these credentials, you must prevent other users from accessing this file. Run the chmod 640 /etc/passwd-ossfs command to modify the file's access permissions.

The "operation not permitted" error occurs when you run the ls command on a folder after a successful mount

Cause: This error can occur if your bucket contains objects whose names include invisible characters. File systems have strict limits on file and directory names.

Solution: Use another tool to rename these objects. After you rename them, the `ls` command can correctly display the directory content.

The "fuse: device not found, try 'modprobe fuse'" error occurs during mounting

Cause: The "fuse: device not found, try 'modprobe fuse'" error can occur when you try to mount with ossfs in a Docker container. This error is usually caused by the container lacking the permissions required to access or load the FUSE kernel module.

Solution: When you run ossfs in a Docker container, you can add the --privileged=true parameter to grant higher permissions to the container. This allows processes within the container to perform operations similar to those on the host, including using the FUSE file system. The following command is an example of how to start a container with the --privileged flag:

docker run --privileged=true -d your_image

The "ossfs: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory" error occurs during mounting

Cause: The version of the installation package does not match the version of the operating system.

Solution: To resolve this issue, download the installation package that corresponds to your operating system.

How to avoid costs from background programs scanning files when OSS is mounted on an ECS instance

Cause: When a program scans a directory mounted by ossfs, the scan is converted into requests to OSS. Many requests can incur fees.

ossfs disconnects intermittently

Why does ossfs fill up the disk space?

Cause: To improve performance, ossfs uses as much disk space as possible by default to save temporary data for uploads and downloads. This can cause the disk to become full.

Solution: You can use the -oensure_diskfree option to specify the amount of disk space to reserve. For example, to reserve 20 GB of disk space, add the option to your mount command:

ossfs examplebucket /tmp/ossfs -o url=http://oss-cn-hangzhou.aliyuncs.com -oensure_diskfree=20480

After mounting with ossfs, why does the df command show a disk space of 256 TB?

The disk space shown by the df command is for display purposes only. It does not represent the actual capacity of the OSS bucket. The `Size` (total disk space) and `Avail` (free disk space) values are fixed at 256 TB. The `Used` (used disk space) value is fixed at 0 TB.

OSS provides unlimited bucket capacity. Your bucket's storage usage depends on your actual usage. For more information about how to query bucket usage, see Query bucket-level usage.

The "input/output error" occurs when you copy data using the cp command

Cause: An input/output error is reported when a system disk error is detected. Check whether the disk is experiencing a high read/write load when the error occurs.

Solution: You can add sharding parameters to control file reads and writes. Run the ossfs -h command to view the available sharding parameters.

The "input/output error" occurs when you use rsync for synchronization

Cause: Using ossfs with `rsync` for synchronization can cause issues. In this case, a user ran the `cp` command on a 141 GB file. This put the disk in a high-load state for read and write operations, which caused the error.

Solution: If you want to download files from OSS to a local ECS instance or upload local files to an ECS instance, use the multipart upload or download feature of ossutil.

The "there is no enough disk space for used as cache(or temporary)" error occurs when you upload a large file

• Cause

  The available disk space is less than multipart_size × parallel_count.

  `multipart_size` specifies the part size in MB, and `parallel_count` specifies the number of concurrent upload parts. The default value for `parallel_count` is 5.

• Analysis

  By default, ossfs uses multipart upload to upload large files. During the upload, ossfs writes temporary cache files to the /tmp folder. Before writing, ossfs checks if the free space on the disk that contains the /tmp folder is less than multipart_size × parallel_count. If the free disk space is greater than multipart_size × parallel_count, the file is written normally. If the free disk space is less than multipart_size × parallel_count, an error that indicates insufficient local disk space is returned.

  For example, assume the free disk space is 300 GB and the file to be uploaded is 200 GB. However, `multipart_size` is set to 100000 (which is 100 GB), and the number of concurrent upload parts is kept at the default value of 5. In this case, ossfs determines that the required upload space is 100 GB × 5 = 500 GB, which exceeds the available local disk space.

• Solution

  If the number of concurrent upload parts is kept at the default value of 5, set `multipart_size` to a reasonable value:

  • If the free disk space is 300 GB and the file to be uploaded is 200 GB, set `multipart_size` to 20.

  • If the free disk space is 300 GB and the file to be uploaded is 500 GB, set `multipart_size` to 50.

The "fuse: warning: library too old, some operations may not work" error occurs during ossfs installation

Cause: This error occurs because the `libfuse` version used at compile-time for ossfs is higher than the `libfuse` version linked at runtime. This often happens when a user installs `libfuse` manually. In CentOS 5.x and CentOS 6.x systems, the ossfs installation package provided by Alibaba Cloud includes `libfuse-2.8.4`. If `libfuse-2.8.3` is present in the runtime environment and ossfs is linked to the older version of FUSE, this error occurs.

You can run the ldd $(which ossfs) | grep fuse command to confirm the FUSE version that ossfs links to at runtime. If the result is /lib64/libfuse.so.2, you can run the ls -l /lib64/libfuse* command to see the FUSE version.

An error occurs when you install the fuse dependency library

Cause: The FUSE version does not meet the requirements of ossfs.

Solution: Manually download and install the latest version of FUSE. Do not use `yum` for the installation. For more information, see fuse.

The "Input/Output error" occurs when you use ls to list files

Cause: This issue mainly occurs in CentOS environments. The log reports an NSS error -8023. An issue occurs when ossfs uses `libcurl` for HTTPS communication. This may be because the version of the Network Security Services (NSS) library that `libcurl` depends on is too low.

Solution: Run the following command to upgrade the NSS library to the latest version.

yum update nss

The "conflicts with file from package fuse-devel" error occurs when you use yum/apt-get to install ossfs

Cause: An old version of FUSE exists in the system, which conflicts with the dependency version in ossfs.

Solution: First, use the relevant package manager to uninstall FUSE, and then reinstall ossfs.

What are the suggestions for stuttering issues after mounting with ossfs when there are many OSS files or in high-concurrency scenarios?

ossfs 1.0 is not recommended for high-concurrency scenarios. If you must use it in such scenarios, consider the following solutions:

• Solution 1: Mount with ossfs 2.0. Compared to ossfs 1.0, ossfs 2.0 provides significant performance improvements for sequential reads and writes, and for reading small files with high concurrency. For more information about the performance of ossfs 2.0, see the Performance improvements section.

• Solution 2: Mount OSS using Cloud Storage Gateway for better performance. The ossfs 1.0 tool is not recommended for high-concurrency uploads and downloads of large files. It is better suited for daily operations on small files.

The Content-Type of files uploaded to OSS using ossfs is always application/octet-stream

Cause: When uploading a file, ossfs sets the `Content-Type` of the file by querying the content of /etc/mime.types. If this file does not exist, the `Content-Type` is set to `application/octet-stream` by default.

Why does ossfs identify a folder as a regular file?

• Scenario 1:

  Cause: When you create a folder object (an object whose name ends with /) and specify the `Content-Type` as `text/plain`, ossfs identifies this object as a regular file.

  Solution: You can add the `-ocomplement_stat` parameter during mounting. If the size of the folder object is 0 or 1, ossfs identifies it as a directory.

• Scenario 2:

  Cause: You can run the ossutil stat folder_object/ command (for example, ossutil stat oss://[bucket]/folder/). After you run the command:

  1. Check the `Content-Length` field of the object, which is the object size. If the object size is not 0, the object is identified as a file.

     Solution: If you no longer need the content of the folder object, you can run the ossutil rm oss://[bucket]/folder/ command to delete the object. This does not affect the files under the directory. Alternatively, you can upload a zero-byte object with the same name to overwrite it.

  2. If the object size is 0, check the `Content-Type` field, which is the object attribute. If the `Content-Type` is not application/x-directory, httpd/unix-directory, binary/octet-stream, or application/octet-stream, the object is also identified as a file.

     Solution: You can run the ossutil rm oss://[bucket]/folder/ command to delete the object. This does not affect the files under the directory.

The mv operation fails in ossfs

Cause: The `mv` operation in ossfs may fail because the source file is an Archive, Cold Archive, or Deep Cold Archive object.

Solution: Before you run the `mv` command on an Archive, Cold Archive, or Deep Cold Archive object, you must restore the object. For more information, see Restore objects.

Does ossfs support mounting buckets in a Windows environment?

No, it does not. You can use Rclone to mount a bucket in Windows. For more information, see Rclone.

Does ossfs support mounting an OSS bucket to multiple Linux ECS servers?

Yes, it does. You can mount a bucket to multiple Linux ECS servers. For more information, see Mount a bucket.

Why is the file information (such as size) seen in ossfs inconsistent with what is seen in other tools?

Cause: By default, ossfs caches file metadata, including size and permissions. This cache speeds up `ls` operations because it avoids sending a request to OSS each time. If a user modifies a file using another program, such as an SDK, the OSS console, or ossutil, ossfs does not immediately update its cache. This causes the file information to be inconsistent with the information displayed in other tools.

Solution: You can add the -omax_stat_cache_size=0 parameter during mounting to disable the metadata cache feature. When the cache is disabled, each time you run `ls`, a request is sent to OSS to retrieve the latest file information.

Why does mounting become slow after versioning is enabled for a bucket?

Cause: By default, ossfs lists files by calling the ListObjects (GetBucket) operation. If versioning is enabled for a bucket and the bucket contains one or more previous versions of objects and many expired delete markers, the response speed of the ListObjects (GetBucket) operation decreases when listing current object versions. This slows down the ossfs mount operation.

Solution: Use the -olistobjectsV2 option to configure ossfs to use the ListObjectsV2 (GetBucketV2) operation to improve file listing performance.

How do I mount a bucket over HTTPS?

ossfs supports mounting over HTTPS. The following command is an example for the China (Hangzhou) region:

ossfs examplebucket /tmp/ossfs -o url=https://oss-cn-hangzhou.aliyuncs.com

Why is the ls command slow when a folder contains a very large number of files?

Cause: If a directory contains N files, the `ls` command for that directory requires at least N HTTP requests to OSS. When a directory contains a very large number of files, this can cause serious performance issues.

Solution: Increase the size of the stat cache using the -omax_stat_cache_size=xxx parameter. The first `ls` operation will be slower, but subsequent `ls` operations will be faster because the object metadata is stored in the local cache. Before version 1.91.1, the default value was 1,000. Starting from version 1.91.1, the default value is 100,000, which uses several tens of megabytes of memory. You can adjust this value based on your machine's memory.

The "fusermount: failed to unmount /mnt/ossfs-bucket: Device or resource busy" error occurs during unmounting

Cause: A process is accessing a file in the mounted directory /mnt/ossfs-bucket, which prevents the directory from being unmounted.

Solution:

1. Run lsof /mnt/ossfs-bucket to find the process that is accessing the directory.

2. Run the `kill` command to force the process to close.

3. Run fusermount -u /mnt/ossfs-bucket to unmount the bucket.

Many 404 logs appear when you use ossfs 1.0

Background: When you use ossfs 1.0, it is common to see 404 Not Found records in the logs. In most cases, this is not a system error. It is the normal interactive behavior that ossfs 1.0 uses to simulate local file system semantics.

Before operating on a file, the operating system checks whether the target object exists. This process can cause many probe requests to be sent to OSS. If the object does not exist, OSS returns a 404 status code.

The complete process that ossfs 1.0 uses to check for an object's existence is as follows:

1. Sends a `HeadObject` request to query whether the specified path, such as object, exists as an actual object.

   If the object exists, OSS returns the object metadata. If the object does not exist, OSS returns a 404 error and ossfs proceeds to the next step.

2. After receiving the 404 error, ossfs sends a `HeadObject` request to query whether the object/ object exists.

   If the object exists, OSS returns the object metadata. If the object does not exist, OSS returns a 404 error and ossfs proceeds to the next step.

3. After receiving the 404 error, ossfs sends a `HeadObject` request to query whether the object_$folder$ object exists.

   If the object exists, OSS returns the object metadata. If the object does not exist, OSS returns a 404 error and ossfs proceeds to the next step.

4. After receiving the 404 error, ossfs sends a `ListObjects` request to determine whether the specified path is a "directory" by querying for objects with the prefix object/.

   If the result is empty, the path is determined not to exist. If the result is not empty, the directory exists, and the system lists the directory's contents.

Analysis:

• When you use commands such as `stat` to access a file that does not exist, OSS returns a 404 error, which is mapped to the local file system's "No such file or directory" error.

• Before creating files or directories in a batch, the operating system first checks whether the target file exists. It sends a create request only if the file does not exist. The 404 error generated during this check is expected behavior, not a system exception.

Solution: Although the 404 error is normal behavior, frequent probe requests can affect performance in high-concurrency or batch operation scenarios. You can optimize the configuration in the following ways:

1. Increase the metadata cache duration and the number of cached entries by increasing the values of -o stat_cache_expire and -o max_stat_cache_size.

   Querying a file or directory before its metadata expires avoids repeated OSS requests.

   • -o stat_cache_expire: The expiration time of the metadata cache in seconds. The default value is 900.

   • -o max_stat_cache_size: The number of entries in the metadata cache. The default value is 100,000.

2. If you are using a version of ossfs earlier than 1.91.6, enable the negative cache by configuring -o enable_noobj_cache. The negative cache is enabled by default in ossfs 1.91.6 and later.

   When a file is queried for the first time and does not exist, the result is cached in memory. Subsequent queries for the file before the cache expires directly query the local negative cache instead of sending an OSS request.

   Note

   The file negative cache of ossfs 1.0 is part of the metadata cache. The expiration time is controlled by the -o stat_cache_expire parameter, and the maximum number of cached entries is controlled by the -o max_stat_cache_size parameter.