Alibaba Cloud Object Storage Service (OSS) provides multipart upload for you to split an object you want to upload into multiple parts and upload the parts separately. After the parts are uploaded, you can call CompleteMultipartUpload to combine these parts together. If the upload fails due to network errors, you can continue the upload from the last uploaded part to implement resumable upload.

Scenarios

  • Accelerated upload of large objects

    When the object that you want to upload is larger than 5 GB, you can use multipart upload to split the object into parts and concurrently upload the parts to accelerate the upload.

  • Poor network environments

    We recommend that you use multipart upload in a poor network environment. When the multipart upload fails, you need only to upload the parts that failed to be uploaded.

  • Stream upload

    You can use stream upload to upload objects of unknown sizes. This scenario is common in industry applications such as video surveillance.

Multipart upload process

The following flowchart shows the basic process of multipart upload.

The preceding process consists of the following steps:

  1. Split the object that you want to upload into parts based on a specific size.
  2. Call the InitiateMultipartUpload operation to initiate a multipart upload task.
  3. Call the UploadPart operation to upload the parts.

    After the object is split into parts, a partNumber is specified for each part to indicate the sequence of the parts. Therefore, you can concurrently upload the parts in sequence. More concurrent uploads do not necessarily result in faster upload speeds. Therefore, we recommend that you specify the number of concurrent uploads based on your network conditions and the workload of your devices.

    If you want to cancel a multipart upload task, you can call the AbortMultipartUpload operation. After a multipart upload task is canceled, parts that are uploaded by the task are also deleted.

  4. Call the CompleteMultipartUpload operation to combine the uploaded parts together.

Limits

Item Limit
Object size Multipart upload supports objects up to 48.8 TB in size.
Number of parts You can set the number of parts to a value that ranges from 1 to 10000
Part size Each part can be 100 KB to 5 GB in size. The size of the last part is not limited.
Maximum number of parts that can be returned for a single ListParts request Up to 1,000 parts can be returned for a single ListParts request.
Maximum number of multipart upload tasks that can be returned for a single ListMultipartUploads request Up to 1,000 multipart upload tasks can be returned for a single ListMultipartUploads request.

Usage notes

  • Optimize performance for object upload

    If you upload a large number of objects whose names have sequential prefixes such as timestamps and letters, multiple object indexes may be stored in a single partition of a bucket. If an excessive number of requests are sent to query these objects, the responsiveness may become slow. We recommend that you do not upload a large number of objects whose names have sequential prefixes. For more information, see OSS performance and scalability best practices.

  • Overwrite objects

    If you upload an object whose name is the same as an existing object in OSS, the existing object is overwritten. You can use the following methods to prevent objects from being unexpectedly overwritten:

    • Enable versioning

      When versioning is enabled for a bucket, overwritten objects are saved as previous versions. You can recover a previous version anytime. For more information, see Overview.

    • Include a specific header in the upload request

      Include the x-oss-forbid-overwrite header in the upload request and set the parameter to true. This way, when you upload an object whose object name is the same as that of an existing object, the object fails to be uploaded. OSS returns the FileAlreadyExists error. For more information, see InitiateMultipartUpload.

  • Delete parts
    When a multipart upload task is interrupted, parts that are uploaded by the task are stored in the specified bucket. To avoid additional storage fees, we recommend that you use the following methods to delete these parts if you no longer use these parts:

Implementation methods for multipart upload

Implementation method Description
ossutil A high-performance command-line tool
Java SDK SDK demos for various programming languages
Python SDK
PHP SDK
Go SDK
C SDK
.NET SDK
Node.js SDK
Android SDK
iOS SDK
Browser.js SDK

Implementation methods for resumable upload

If a multipart upload task fails, you can call the ListMultipartUploads operation to query all multipart upload tasks that are initiated to upload a specific object and call the ListParts operation to list the uploaded parts in each multipart upload task. Then, you can continue the upload from the last uploaded part to implement resumable upload.
Implementation method Description
Java SDK SDK demos for various programming languages
Python SDK
Go SDK
C SDK
.NET SDK
Node.js SDK
Android SDK
iOS SDK