New features of ossfs V1.91.2 - Object Storage Service - Alibaba Cloud Documentation Center

ossfs 1.91.2 introduces direct read mode, which bypasses disk I/O to deliver sequential read throughput limited only by network bandwidth.

How it works

Default read mode

When ossfs reads an object from a bucket, it downloads the object and writes it to local disk as a file. Disk writes are asynchronous: ossfs first writes downloaded data to the page cache in memory, then flushes dirty pages to disk in the background.

By default, ossfs retains all prefetched data on the local disk until the available disk space is occupied.

Memory is sufficient to hold the file: all data stays in the page cache. Read performance is limited by network bandwidth only.
Memory is not sufficient: ossfs spills data to disk. Read performance is limited by both disk bandwidth and network bandwidth, with disk bandwidth as the primary bottleneck.

For large files that exceed available memory, disk bandwidth caps read throughput — regardless of network capacity.

Direct read mode

Direct read mode removes the disk from the data path entirely. ossfs downloads data into a memory buffer and serves read requests from the buffer, so read performance scales with network bandwidth instead of disk bandwidth.

ossfs manages buffered data in chunks:

Each prefetch task downloads one chunk (4 MB by default, configurable via direct_read_chunk_size).
In memory, ossfs retains data from the chunk before the current position through the next direct_read_prefetch_chunks chunks ahead.
If the next read offset falls outside this window, ossfs discards the buffered data and starts prefetching from the new offset.

When to use direct read mode

Direct read mode is designed for sequential reads of large files where the working set exceeds available memory.

Use direct read mode when:

Reading large files sequentially (for example, ML training datasets, video files, or big data pipelines)
The file size exceeds available memory, causing disk bandwidth to bottleneck default read mode
Maximizing read throughput is the priority

Do not use direct read mode when:

Random reads: ossfs retains data only within the current offset window. Random access patterns outside that window cause repeated download-and-discard cycles, wasting network bandwidth and degrading performance.
Writes: ossfs always writes data to local disk first. If a write request arrives while a file is in direct read mode, ossfs automatically switches that file to default read mode (downloading it to disk first). Other files remain in direct read mode.

Enable direct read mode

Add -odirect_read to your mount command:

ossfs <bucket-name> <mountpoint> -ourl=<endpoint> -odirect_read

For large file workloads on high-bandwidth networks, tune the chunk size and prefetch depth:

ossfs <bucket-name> <mountpoint> -ourl=<endpoint> -odirect_read -odirect_read_chunk_size=8 -odirect_read_prefetch_chunks=64

Parameters

Parameter	Description	Default
`direct_read`	Enables direct read mode. Set with `-odirect_read` in the mount command.	Disabled
`direct_read_chunk_size`	Size (in MB) of data downloaded by each prefetch task.	4
`direct_read_prefetch_chunks`	Number of chunks to prefetch ahead of the current read position.	32
`direct_read_prefetch_limit`	Maximum total prefetch size (in MB) held in memory.	1024
`direct_read_prefetch_thread`	Number of threads performing prefetch downloads.	64

Performance testing

The following benchmark compares sequential read throughput in default read mode versus direct read mode on the same machine.

Test setup:

Memory: 32 GB
Disk bandwidth: 130 Mbit/s

Mount command (default read mode):

  ossfs <bucket-name> <mountpoint> -ourl=<endpoint> -oparallel_count=32 -omultipart_size=16

Mount command (direct read mode):

  ossfs <bucket-name> <mountpoint> -ourl=<endpoint> -odirect_read -odirect_read_chunk_size=8 -odirect_read_prefetch_chunks=64

Test script:

  dd if=testfile of=/dev/null bs=1M status=progress

Results:

File size	Default read mode	Direct read mode
1 GB	646 MB/s	592 MB/s
5 GB	630 MB/s	611 MB/s
10 GB	260 MB/s	574 MB/s

Use direct read mode when your file sizes regularly exceed available memory.