Read-only file-like access (Python SDK V2) - Object Storage Service

Usage notes

The sample code uses the China (Hangzhou) region (cn-hangzhou) with the public endpoint. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. OSS regions and endpoints.
Access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
Downloading requires the oss:GetObject permission. Grant custom permissions to a RAM user.

Method

OSS SDK for Python 2.0 provides the File-Like interface for read-only access to bucket objects via the ReadOnlyFile class.

ReadOnlyFile supports single-stream and prefetch modes. Adjust parallel task count to improve read speed.
Built-in reconnection handles connection drops in unstable networks.

class ReadOnlyFile:
    ...


def open_file(self, bucket: str, key: str, version_id: Optional[str] = None, request_payer: Optional[str] = None, **kwargs) -> ReadOnlyFile:
    ...

Request parameters

Parameter	Type	Description
bucket	str	The name of the bucket.
key	str	The name of the object.
version_id	str	The version ID of the object. Valid only when multiple versions exist.
request_payer	str	Set to requester when pay-by-requester is enabled.
**kwargs	Any	Optional keyword arguments (dictionary).

Options of kwargs

Option	Type	Description
enable_prefetch	bool	Enables prefetch mode. Default: disabled.
prefetch_num	int	Number of prefetch chunks. Default: 3. Effective only in prefetch mode.
chunk_size	int	Size of each prefetch chunk in MiB. Default: 6. Effective only in prefetch mode.
prefetch_threshold	int	Sequential read threshold before prefetch activates, in MiB. Default: 20.
block_size	int	The size of a chunk. Default value: None.

Response parameters

Parameter	Type	Description
file	ReadOnlyFile	The ReadOnlyFile instance.

Common methods of ReadOnlyFile

Method	Description
close(self)	Closes the file and releases resources, such as memory and active sockets.
read(self, n=None)	Reads up to n bytes from the object and returns the data read.
seek(self, pos, whence=0)	Sets the read position. whence values: 0 (start), 1 (current position), 2 (end).
Stat() (os.FileInfo, error)	Queries object information, including the object size, last modified time, and metadata.

Important

If multiple out-of-order reads occur in prefetch mode, the SDK falls back to single-stream mode.

Examples

Read the entire object using the single stream mode

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser for parsing arguments from the command line.
parser = argparse.ArgumentParser(description="open file sample")

# (Required) Specify the region parameter, which specifies the region in which the bucket is located. 
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)

# (Required) Specify the --bucket parameter, which specifies the name of the bucket.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)

# (Optional) Specify the --endpoint parameter, which specifies the endpoint that other services can use to access OSS.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

# (Required) Specify the --key parameter, which specifies the name of the object.
parser.add_argument('--key', help='The name of the object.', required=True)


def main():
    # Parse the command-line parameters.
    args = parser.parse_args()

    // Obtain access credentials (AccessKey ID and AccessKey secret) from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK.
    cfg = oss.config.load_default()

    # Specify the credential provider.
    cfg.credentials_provider = credentials_provider

    # Specify the region in which the bucket is located.
    cfg.region = args.region

    # If a custom endpoint is provided, modify the endpoint parameter.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSSClient instance.
    client = oss.Client(cfg)

    # Use the open_file method to open the object in the bucket.
    result = client.open_file(
        bucket=args.bucket,           # The name of the bucket.
        key=args.key,                # The name of the object.
    )

    # Display the object, read the data, and decode it to the string format.
    print(f'content: {result.read().decode()}')

    # Closes the object to release resources.
    result.close()


if __name__ == "__main__":
    main() # Specify the entry points in the main function of the script when the script is directly run.
    main()

Read the entire object using the prefetch mode

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser for parsing arguments from the command line.
parser = argparse.ArgumentParser(description="open file sample")

# (Required) Specify the region parameter, which specifies the region in which the bucket is located. 
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)

# (Required) Specify the --bucket parameter, which specifies the name of the bucket.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)

# (Optional) Specify the --endpoint parameter, which specifies the endpoint that other services can use to access OSS.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

# (Required) Specify the --key parameter, which specifies the name of the object.
parser.add_argument('--key', help='The name of the object.', required=True)


def main():
    # Parse the command-line parameters.
    args = parser.parse_args()

    // Obtain access credentials (AccessKey ID and AccessKey secret) from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK.
    cfg = oss.config.load_default()

    # Specify the credential provider.
    cfg.credentials_provider = credentials_provider

    # Specify the region in which the bucket is located.
    cfg.region = args.region

    # If a custom endpoint is provided, modify the endpoint parameter.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSSClient instance.
    client = oss.Client(cfg)

    # Use the open_file method to open the object in the bucket.
    result = client.open_file(
        bucket=args.bucket,           # The name of the bucket.
        key=args.key,                # The name of the object.
        enable_prefetch=True,        # Specify whether to enable the prefetch mode. Default value: true.
   )

    # Display the object, read the data, and decode it to the string format.
    print(f'content: {result.read().decode()}')

    # Closes the object to release resources.
    result.close()


if __name__ == "__main__":
    main() # Specify the entry points in the main function of the script when the script is directly run.
    main()

Read remaining data from a specific position using the Seek method

import argparse
import os
import io
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser for parsing arguments from the command line.
parser = argparse.ArgumentParser(description="open file sample")

# (Required) Specify the region parameter, which specifies the region in which the bucket is located. 
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)

# (Required) Specify the --bucket parameter, which specifies the name of the bucket.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)

# (Optional) Specify the --endpoint parameter, which specifies the endpoint that other services can use to access OSS.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

# (Required) Specify the --key parameter, which specifies the name of the object.
parser.add_argument('--key', help='The name of the object.', required=True)


def main():
    # Parse the command-line parameters.
    args = parser.parse_args()

    // Obtain access credentials (AccessKey ID and AccessKey secret) from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK.
    cfg = oss.config.load_default()

    # Specify the credential provider.
    cfg.credentials_provider = credentials_provider

    # Specify the region in which the bucket is located.
    cfg.region = args.region

    # If a custom endpoint is provided, modify the endpoint parameter.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSSClient instance.
    client = oss.Client(cfg)

    // Initialize the oss.ReadOnlyFile object.
    rf: oss.ReadOnlyFile = None

    # Use the WITH statement to open the object and ensure that the resources are automatically closed after the object read operation is complete.
    with client.open_file(args.bucket, args.key) as f:
        rf = f # Assign the object to the rf variable.

        # Move the file pointer to the specified position. In this example, the file pointer is 1 byte offset to the beginning of the object.
        f.seek(1, os.SEEK_SET)

        # Read the content of the object into a byte stream (BytesIO) in memory.
        copied_stream = io.BytesIO(rf.read())

        # Display the length of the data written to the byte stream.
        print(f'written: {len(copied_stream.getvalue())}')

        # Display the read content. The byte stream is decoded to the string format.
        print(f'read: {copied_stream.getvalue()}')


if __name__ == "__main__":
    main() # Specify the entry points in the main function of the script when the script is directly run.
    main()

References

Full API design reference: File-Like (GitHub).