Object Storage Service:ossfs 2.0 overview

Performance improvements

ossfs 2.0 offers significant performance improvements over ossfs 1.0 in sequential read and write operations and high-concurrency small file reads. For more information about the performance of ossfs 2.0, see Performance Testing.

• Sequential write performance: For single-threaded sequential writes of large files, ossfs 2.0 provides nearly 18× the bandwidth of ossfs 1.0.

• Sequential read performance

  • For single-threaded sequential reads of large files, ossfs 2.0 provides about 8.5× the bandwidth of ossfs 1.0.

  • For multi-threaded (4 threads) sequential reads of large files, ossfs 2.0 provides more than 5× the bandwidth of ossfs 1.0.

• Concurrent small file read performance: For high-concurrency (128 threads) reads of small files, ossfs 2.0 provides more than 280× the bandwidth of ossfs 1.0.

Scenarios

ossfs 2.0 is ideal for scenarios that demand high storage access performance, such as AI training, inference, big data processing, and autonomous driving. These compute-intensive workloads primarily involve sequential and random reads, sequential writes (append-only), and do not depend on full POSIX semantics.

Operating environment

ossfs 2.0 is developed based on Filesystem in Userspace (FUSE).

Limits

• Permissions: Ensure the AccessKey has full permissions for the resources in the target bucket or prefix. Otherwise, mounting may fail or features may not work correctly.

• Storage class: Mounting buckets of the Archive Storage, Cold Archive, or Deep Cold Archive storage classes is not supported.

• File name length: The maximum file name length is 255 characters, which complies with the Linux limit. Files or directories in OSS that exceed this limit are not visible in the mount target.

• File reads: A newly written file is uploaded to OSS only after it is closed. An error may occur if you read a file that is being written but has not yet been closed.

• File writes: Random writes and concurrent writes to the same file are not supported. The default write shard size is 8,388,608 bytes (8 MiB), which supports a maximum file size of 83,886,080,000 bytes (78.125 GiB). The shard size can be configured with the upload_buffer_size mount option.

• File rename operations (non-atomic)

  • File: Deletes the remote file after a successful copy.

  • Directories: A rename operation first copies all files in the directory, and then deletes the original source files in a batch operation. By default, the number of descendant files is limited to 2 million when a directory is renamed. This limit can be configured with the rename_dir_limit mount option.

• Concurrent write consistency: Data consistency is not guaranteed when multiple clients mount the same bucket and concurrently write to the same file.

• POSIX API compatibility: Partially compatible. For more information, see POSIX API support.

Features

Next steps

• Install ossfs 2.0

• Configure ossfs 2.0

• Mount a bucket to a local file system using ossfs 2.0

• ossfs 2.0 mount options

• ossfs 2.0 performance testing

• File and folder detection mechanism

• Reduce OSS requests and improve mount target performance

• FAQ

• Release notes