How to upload files using Python SDK V2 - Object Storage Service

This topic describes how to use the simple upload method to upload a single file to OSS. This method is simple and is suitable for fast file uploads to cloud storage.

Important

The sample code in this topic uses the region IDcn-hangzhou for China (Hangzhou) as an example. By default, a public endpoint is used. If you want to access OSS from other Alibaba Cloud products in the same region, you must use an internal endpoint. For more information about the mappings between OSS regions and endpoints, see OSS regions and endpoints.

Permissions

By default, an Alibaba Cloud account has full permissions. RAM users or RAM roles under an Alibaba Cloud account do not have any permissions by default. The Alibaba Cloud account or account administrator must grant operation permissions through RAM Policy or Bucket Policy.

API	Action	Definition
PutObject	`oss:PutObject`	Uploads an object.
	`oss:PutObjectTagging`	When uploading an object, if you specify object tags through x-oss-tagging, this permission is required.
	`kms:GenerateDataKey`	When uploading an object, if the object metadata contains X-Oss-Server-Side-Encryption: KMS, these two permissions are required.
	`kms:Decrypt`

Method definition

put_object(request: PutObjectRequest, **kwargs) → PutObjectResult

Request parameters

Parameter	Type	Description
request	PutObjectRequest	Specifies the request parameters, such as the access control list (ACL) of the object, whether to forbid overwriting (ForbidOverwrite), and custom metadata (Metadata). For more information, see PutObjectRequest

Return values

Type	Description
PutObjectResult	The return value. For more information, see PutObjectResult

For the complete definition of the simple upload method, see put_object.

Upload a local file

When you upload an object, if an object with the same name already exists in the bucket and you have the required permissions, the new object overwrites the existing one.

The common parameters for uploading a file are as follows:

Parameter	Description
bucket_name	The bucket name. The naming conventions for buckets are as follows: Can contain only lowercase letters, digits, and hyphens (-). Must start and end with a lowercase letter or a digit. The length must be between 3 and 63 characters.
object_name	The full path of the object. The full path cannot contain the bucket name. The naming conventions for objects are as follows: Must be UTF-8 encoded. The length must be between 1 and 1,023 characters. Cannot start with a forward slash (/) or a backslash (\).

You can use the put_object_from_file method to upload a local file to the destination bucket.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object from file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)

# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)

# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

# Add the --file_path command-line argument, which specifies the path of the local file to upload. This argument is required.
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the software development kit (SDK) and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region in the configuration.
    cfg.region = args.region

    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Execute the request to upload the object directly from a file.
    # Specify the bucket name, object name, and local file path.
    result = client.put_object_from_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # The name of the bucket.
            key=args.key         # The name of the object.
        ),
        args.file_path          # The path of the local file.
    )

    # Print the result information of the request, including the status code, request ID, Content-MD5, ETag, 64-bit cyclic redundancy check (CRC64) hash, version ID, and server response time.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
    )

# The script entry point. The main function is called when the file is run directly.
if __name__ == "__main__":
    main()

When you use the put_object method to upload a local file, you must open the file in 'rb' mode. This ensures that the original byte stream is uploaded instead of text content, which prevents cyclic redundancy check (CRC) failures.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object from file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)

# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)

# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region in the configuration.
    cfg.region = args.region

    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Execute the request to upload the object directly from a local file.
    # Specify the bucket name, object name, and local file path.
    with open('your-test-file.md', 'rb') as f:
    result = client.put_object(
        oss.PutObjectRequest(
            bucket=args.bucket,  # The name of the bucket.
            key=args.key,        # The name of the object.
            body=f.read()        # Read the file content.
        )
    )

    # Print the result information of the request, including the status code, request ID, Content-MD5, ETag, CRC64 hash, version ID, and server response time.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
    )

# The script entry point. The main function is called when the file is run directly.
if __name__ == "__main__":
    main()

Common scenarios

Upload a string

You can use the following code to upload a string to the destination bucket.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object sample")
# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

def main():
    args = parser.parse_args()  # Parse the command-line arguments.

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    # Set the region in the configuration.
    cfg.region = args.region
    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Define the string to upload.
    text_string = "Hello, OSS!"
    data = text_string.encode('utf-8')  # Encode the string into a UTF-8 byte string.

    # Execute the request to upload the object. Specify the bucket name, object name, and data content.
    result = client.put_object(oss.PutObjectRequest(
        bucket=args.bucket,
        key=args.key,
        body=data,
    ))

    # Print the status code, request ID, Content-MD5, ETag, CRC64 hash, and version ID of the request result to check whether the request is successful.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
    )

if __name__ == "__main__":
    main()  # The script entry point. The main function is called when the file is run directly.

Upload a byte array

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object sample")
# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

def main():
    args = parser.parse_args()  # Parse the command-line arguments.

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    # Set the region in the configuration.
    cfg.region = args.region
    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Define the data content to upload.
    data = b'hello world'

    # Execute the request to upload the object. Specify the bucket name, object name, and data content.
    result = client.put_object(oss.PutObjectRequest(
        bucket=args.bucket,
        key=args.key,
        body=data,
    ))

    # Print the status code, request ID, Content-MD5, ETag, CRC64 hash, and version ID of the request result to check whether the request is successful.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
    )

if __name__ == "__main__":
    main()  # The script entry point. The main function is called when the file is run directly.

Upload a network stream

You can use the following code to upload a network stream to the destination bucket.

import argparse
import requests
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object sample")
# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

def main():
    args = parser.parse_args()  # Parse the command-line arguments.

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    # Set the region in the configuration.
    cfg.region = args.region
    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Send an HTTP GET request to obtain the response content.
    response = requests.get('http://www.aliyun.com')

    # Execute the request to upload the object. Specify the bucket name, object name, and data content.
    result = client.put_object(oss.PutObjectRequest(
        bucket=args.bucket,
        key=args.key,
        body=response.content,
    ))

    # Print the status code, request ID, Content-MD5, ETag, CRC64 hash, and version ID of the request result to check whether the request is successful.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
    )

if __name__ == "__main__":
    main()  # The script entry point. The main function is called when the file is run directly.

Upload a file and configure a callback

The following sample code shows how to notify an application server after a file is uploaded.

import base64
import argparse
import alibabacloud_oss_v2 as oss

parser = argparse.ArgumentParser(description="put object sample")

# Add the required parameters.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
parser.add_argument('--key', help='The name of the object.', required=True)
parser.add_argument('--call_back_url', help='Callback server address.', required=True)


def main():

    args = parser.parse_args()

    # Load credentials from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Configure the SDK client.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    cfg.region = args.region
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client.
    client = oss.Client(cfg)

    # The content to upload (a string).
    data = 'hello world'

    # Construct the callback parameter (callback): Specify the webhook address and the callback request body, and use Base64 encoding.
    callback=base64.b64encode(str('{\"callbackUrl\":\"' + args.call_back_url + '\",\"callbackBody\":\"bucket=${bucket}&object=${object}&my_var_1=${x:var1}&my_var_2=${x:var2}\"}').encode()).decode()
    # Construct the custom variable (callback-var) and use Base64 encoding.
    callback_var=base64.b64encode('{\"x:var1\":\"value1\",\"x:var2\":\"value2\"}'.encode()).decode()

    # Initiate an upload request that includes the callback parameters.
    result = client.put_object(oss.PutObjectRequest(
        bucket=args.bucket,
        key=args.key,
        body=data,
        callback=callback,
        callback_var=callback_var,
    ))
    # Print the returned result, including the status code and request ID.
    print(vars(result))


if __name__ == "__main__":
    main()

Upload a file and display a progress bar

The following sample code shows how to use a progress bar to display the upload progress of a local file.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser.
parser = argparse.ArgumentParser(description="put object sample")
# Add the --region command-line argument, which specifies the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the name of the object. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)

def main():
    args = parser.parse_args()  # Parse the command-line arguments.

    # Load credentials from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Load the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    # Set the region in the configuration.
    cfg.region = args.region
    # If the endpoint argument is provided, set the endpoint in the configuration.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client using the specified configurations.
    client = oss.Client(cfg)

    # Define a dictionary variable named progress_state to save the upload progress status. The initial value is 0.
    progress_state = {'saved': 0}
    def _progress_fn(n, written, total):
        # Use a dictionary to store the cumulative number of written bytes to avoid using global variables.
        progress_state['saved'] += n

        # Calculate the current upload percentage. Divide the number of written bytes by the total number of bytes and round down the result.
        rate = int(100 * (float(written) / float(total)))

        # Print the current upload progress. \r indicates returning to the beginning of the line to implement real-time refresh in the command line.
        # end='' indicates that no line break is added, so that the next printout overwrites the current line.
        print(f'\rUpload progress: {rate}% ', end='')

    # Execute the request to upload the object. Specify the bucket name, object name, and data content.
    result = client.put_object_from_file(oss.PutObjectRequest(
            bucket=args.bucket,
            key=args.key,
            progress_fn=_progress_fn,
        ),
        "/local/dir/example", # Specify the path of the local file.
    )

    # Print the status code, request ID, Content-MD5, ETag, CRC64 hash, and version ID of the request result to check whether the request is successful.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.content_md5},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
    )

if __name__ == "__main__":
    main()  # The script entry point. The main function is called when the file is run directly.

References

For the complete sample code for simple upload, see put_object.py.