ossfs 2.0 is a Filesystem in Userspace (FUSE) client that mounts Alibaba Cloud Object Storage Service (OSS) buckets to a local file system. This allows application containers to access data in OSS using POSIX operations in the same way they access local files. Compared to ossfs 1.0, ossfs 2.0 provides improved performance for sequential reads and writes, and for high-concurrency reads of small files. ossfs 2.0 is suitable for scenarios that require high-performance storage access, such as AI training, inference, big data processing, and autonomous driving.
Performance improvements
ossfs 2.0 delivers significant performance improvements over ossfs 1.0 in sequential reads and writes, and in highly concurrent reads of small objects. For more information, see Performance testing.
Sequential write performance: In single-threaded, large-object sequential write scenarios, ossfs 2.0 provides nearly 18 times the bandwidth of ossfs 1.0.
Sequential read performance
In single-threaded, large-object sequential read scenarios, ossfs 2.0 provides about 8.5 times the bandwidth of ossfs 1.0.
In multi-threaded (4 threads), large-object sequential read scenarios, ossfs 2.0 provides more than 5 times the bandwidth of ossfs 1.0.
Concurrent small-object read performance: In high-concurrency (128 threads) small-object read scenarios, ossfs 2.0 provides more than 280 times the bandwidth of ossfs 1.0.
Use cases
ossfs 2.0 is suitable for scenarios that require high-performance storage access, such as AI training, inference, big data processing, autonomous driving, and other compute-intensive workloads. These workloads primarily involve sequential and random reads and sequential (append-only) writes, and do not require full POSIX semantics.
Limitations
Permission requirements: The AccessKey must have full permissions on the target bucket or prefix; otherwise, the mount operation may fail or behave abnormally.
Storage class limitations: Buckets that use the Archive, Cold Archive, and Deep Cold Archive storage classes cannot be mounted.
Object name limitations: Object names are limited to 255 characters due to a Linux constraint. Objects in OSS with names that exceed this limit are not visible in the mount point.
Object read limitations: ossfs uploads new objects to OSS only after the corresponding file handles are closed. Reading an object before its handle is closed may cause an error.
Object write limitations: Random writes and concurrent writes to the same object are not supported. The default part size is 8,388,608 bytes (8 MiB). Therefore, the maximum supported object size is 83,886,080,000 bytes (78.125 GiB). You can configure the part size by using the upload_buffer_size mount option.
Object rename operations (non-atomic)
Objects: This operation copies the remote object and then deletes the original.
Directories: This operation first copies all objects and then deletes the source objects in a bulk operation. By default, a directory rename operation is limited to 2 million descendant objects. You can configure this limit by using the rename_dir_limit mount option.
Concurrent write consistency: When multiple clients mount the same bucket and write to the same object concurrently, data consistency cannot be guaranteed.
POSIX API compatibility: Partially compatible. To learn more, see Support for POSIX API operations.
Features
Feature highlights
Offers basic POSIX compatibility and focuses on maximizing the server-side read and write performance of OSS.
Delivers efficient sequential read and write performance for large objects through end-to-end optimization of the read and write path.
Enables efficient concurrent loading of small objects through optimized metadata management.
Support for POSIX API operations
The following table compares the POSIX API support in ossfs 1.0 and ossfs 2.0.
Feature category | Operation | ossfs 1.0 | ossfs 2.0 |
Basic object operations |
| ||
| |||
| |||
Object reads and writes |
| ||
| Supports random writes (requires a disk cache) | Supports only sequential writes (no disk cache required) | |
| (can be truncated to any size) | Only supports truncating an object to zero length | |
Object meta operations |
| ||
| |||
| |||
Directory operations |
| ||
| |||
| |||
Permissions and attributes |
| ||
| |||
| |||
| |||
Extended features |
| ||
| |||
|
How it works
The ossfs 2.0 client runs as a container in a pod in the ack-csi-fuse namespace.
References
To mount an OSS bucket as an ossfs 2.0 persistent volume using a static persistent volume (PV) and a persistent volume claim (PVC), see Use a static ossfs 2.0 persistent volume.
To mount an OSS bucket as an ossfs 2.0 persistent volume using a dynamically provisioned volume, see Use a dynamic ossfs 2.0 persistent volume.