Object Storage Service:ossfs 2.0 overview

Performance improvements

Compared with ossfs 1.0, ossfs 2.0 achieves significant performance improvements in sequential reads and writes and concurrent reads of small objects. For more information, see Performance test.

• Sequential writes: In scenarios in which large objects are sequentially written using a single thread, the bandwidth of ossfs 2.0 increases by approximately 18 times compared with that of ossfs 1.0.

• Sequential reads

  • In scenarios in which large objects are sequentially written using a single thread, the bandwidth of ossfs 2.0 increases by approximately 8.5 times compared with that of ossfs 1.0.

  • In scenarios in which large objects are sequentially written using multiple threads (4 threads), the bandwidth of ossfs 2.0 increases by more than 5 times compared with that of ossfs 1.0.

• Concurrent reads of small objects: In scenarios in which small objects are read with high concurrency (128 threads), the bandwidth of ossfs 2.0 increases by more than 280 times compared with that of ossfs 1.0.

Scenarios

ossfs 2.0 is suitable for scenarios with high requirements on storage access, such as AI training, inference, big data processing, autonomous driving, and other next-generation compute-intensive workloads. These workloads involve sequential and random reads and sequential (append-only) writes, and do not depend on complete POSIX semantics.

Runtime environments

ossfs 2.0 is developed based on Filesystem in Userspace (FUSE ). It supports the x86_64 architecture and Alibaba Cloud Linux with kernel version 3.10 or later.

Limits

• Permissions: You must make sure that the AccessKey pair has full permissions on the destination bucket or resources related to prefixes. Otherwise, mounting may fail or features may not work properly.

• Storage class: You cannot use ossfs 2.0 to mount buckets whose storage class is Archive Storage, Cold Archive, or Deep Cold Archive.

• Object name: The name of an object cannot exceed 255 characters in length. If the name of an object or directory in OSS exceeds the limit, the object or directory is invisible at the mount point.

• Object read: A newly written object will be uploaded to OSS only after the object is closed. The reading of an object that is being written and not yet closed may cause errors.

• Object write: Concurrent writes to the same object or random writes are not supported. By default, non-appendable OSS objects are written. The default size of a written part is 8,388,608 bytes (8 MiB), meaning the maximum size of a written object is 83,886,080,000 bytes (78.125 GiB). You can configure the part size using the upload_buffer_size option.

• Object rename operation (non-atomicity)

  • Object: Copy a remote object before you delete it.

  • Directory: Copy all objects in the directory before you batch delete the source objects. By default, the maximum number of subdirectories is 2 million when you rename a directory. You can limit the number of subdirectories using the rename_dir_limit option.

• Concurrent write consistency: When you mount a bucket to multiple clients at the same time and write the same object to them, data consistency cannot be ensured.

• Compatibility with POSIX API operations: ossfs is partially compatible with POSIX API operations. For more information, see Support for POSIX API operations.

Feature introduction

References

• Install ossfs 2.0

• Options supported by ossfs 2.0

• Mount a bucket

• Performance test

• FAQ

• Release notes