Object Storage Service:New features of ossfs V1.91.2

Direct read mode

When you use ossfs to read an object in a bucket, ossfs automatically downloads the object from the bucket, writes the object to a local disk as a file, and returns the file to you from the local disk. In ossfs, disk writes are asynchronous, which means that ossfs first writes a downloaded file to the page cache in the memory and then asynchronously flushes the content of the dirty pages to the local disk. When you read a file and experience a cache hit, the file is directly served from the cache.

By default, ossfs retains all prefetched data on the local disk until the available disk space is occupied.

• If the memory size is sufficient, all downloaded data is written to the page cache. Read performance varies based on only the network bandwidth of the machine, not disk bandwidth.

• If the memory size is not sufficient to hold the entire file, the file is written to the local disk and served from the disk to process a read request. As a result, read performance varies based on the disk bandwidth and network bandwidth. Of these, the former is a major performance constraint.

  

In default read mode, the performance of reading large files is limited by disk bandwidth. To address this performance limitation, ossfs supports the direct read mode.

• In direct read mode, ossfs downloads data into a memory buffer and serves read requests from the buffer. In direct read mode, disks are removed from the data reading process, which results in improved sequential read performance that leverages network bandwidth.

• In direct read mode, ossfs manages downloaded data in chunks. The size of each chunk is 4 MB by default and can be changed by using the direct_read_chunk_size parameter. In the memory, ossfs retains the data within the range from the previous chunk before the current chunk to the next direct_read_prefetch_chunks chunks after the current chunk. If the offset of the next read operation exceeds this range, ossfs discards the already downloaded data and starts prefetching data from the new offset.

The direct read mode is suitable only for sequential reads. It is not applicable in the following scenarios:

• Random reads: ossfs retains data only in the current offset window. If the offsets of random reads do not fall within the window, data is downloaded and released multiple times, which consumes unnecessary network resources and degrades read performance. Therefore, the direct read mode is not suitable for scenarios in which a large number of random read operations are performed.

• Writes: ossfs still writes data to the local disk first. If a file write request occurs during direct read operations on the file, ossfs automatically switches to the default read mode, in which the file is first downloaded to the local disk. A write operation on a file during a direct read operation does not cause other files to exit from the direct read mode.

The following table describes parameters that are related to the direct read mode.