Object Storage Service:Multipart copy (Python SDK V2)

Prerequisites

Before you begin, ensure that you have:

• Read permissions on the source object

• Read and write permissions on the destination bucket

• Both the source and destination buckets in the same region

Usage notes

• The sample code uses the China (Hangzhou) region (cn-hangzhou) with the public endpoint. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For region-specific endpoints, see OSS regions and endpoints.

• Cross-region copy is not supported. For example, you cannot copy an object from China (Hangzhou) to China (Qingdao).

• If a retention policy is configured on either the source or destination bucket, the copy fails with the error: The object you specified is immutable.

How it works

A multipart copy consists of three steps:

Notes on part copying:

• Uploading a new part with the same part number for the same upload ID overwrites the existing part data.

• OSS returns the MD5 hash of each received part in the ETag header.

• If the MD5 hash does not match the expected value, OSS returns the InvalidDigest error code.

Method definition

upload_part_copy(request: UploadPartCopyRequest, **kwargs) → UploadPartCopyResult

Parameters

Return value

For the complete method definition, see upload_part_copy.

Copy an object using multipart copy

up_result = client.upload_part_copy(oss.UploadPartCopyRequest(
    bucket=destination_bucket,
    key=destination_key,
    upload_id=upload_id,
    part_number=part_number,          # starts from 1
    source_bucket=source_bucket,
    source_key=source_key,
    source_range=f'bytes={offset}-{end}',  # byte range in the source object
))

import argparse
import alibabacloud_oss_v2 as oss

parser = argparse.ArgumentParser(description="upload part copy synchronously sample")
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the destination bucket.', required=True)
parser.add_argument('--endpoint', help='The domain name that other services can use to access OSS.')
parser.add_argument('--key', help='The object key in the destination bucket.', required=True)
parser.add_argument('--source_bucket', help='The name of the source bucket.', required=True)
parser.add_argument('--source_key', help='The object key in the source bucket.', required=True)

def main():
    args = parser.parse_args()

    # Load credentials from environment variables
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    cfg.region = args.region
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    client = oss.Client(cfg)

    # Get the source object size to calculate part ranges
    result_meta = client.get_object_meta(oss.GetObjectMetaRequest(
        bucket=args.source_bucket,
        key=args.source_key,
    ))

    # Initiate multipart upload and get an upload ID
    result = client.initiate_multipart_upload(oss.InitiateMultipartUploadRequest(
        bucket=args.bucket,
        key=args.key,
    ))

    part_size = 1024 * 1024  # 1 MiB per part
    total_size = result_meta.content_length
    part_number = 1
    upload_parts = []
    offset = 0

    # Copy each part using the byte range of the source object
    while offset < total_size:
        num_to_upload = min(part_size, total_size - offset)
        end = offset + num_to_upload - 1

        up_result = client.upload_part_copy(oss.UploadPartCopyRequest(
            bucket=args.bucket,
            key=args.key,
            upload_id=result.upload_id,
            part_number=part_number,
            source_bucket=args.source_bucket,
            source_key=args.source_key,
            source_range=f'bytes={offset}-{end}',
        ))

        print(f'status code: {up_result.status_code},'
              f' request id: {up_result.request_id},'
              f' part number: {part_number},'
              f' last modified: {up_result.last_modified},'
              f' etag: {up_result.etag},'
              f' source version id: {up_result.source_version_id}'
        )

        upload_parts.append(oss.UploadPart(part_number=part_number, etag=up_result.etag))
        offset += num_to_upload
        part_number += 1

    # Merge all parts into the final object
    parts = sorted(upload_parts, key=lambda p: p.part_number)
    result = client.complete_multipart_upload(oss.CompleteMultipartUploadRequest(
        bucket=args.bucket,
        key=args.key,
        upload_id=result.upload_id,
        complete_multipart_upload=oss.CompleteMultipartUpload(parts=parts),
    ))

    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' bucket: {result.bucket},'
          f' key: {result.key},'
          f' location: {result.location},'
          f' etag: {result.etag},'
          f' encoding type: {result.encoding_type},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id}'
    )

if __name__ == "__main__":
    main()

References

• Complete sample code: upload_part_copy.py