By using the multipart upload feature provided by OSS, you can split a large object into multiple parts and upload them separately. After all parts are uploaded, call the CompleteMultipartUpload operation to combine these parts into a single object to implement resumable upload.

Background information

You can call the MultipartUpload operation to upload a large object. In multipart upload, an object is split into multiple parts to be uploaded individually. If some parts fail to be uploaded, Object Storage Service (OSS) records the upload progress. When you resume the upload, you need only to upload the parts that fail to be uploaded.

If the ConnectionTimeoutError error occurs when you call the MultipartUpload method, you must handle the error. You can reduce the size of each part, extend the timeout period, or resend the request. You can also capture ConnectionTimeoutError to analyze the specific cause and fix the error. For more information, see Network connection timeout handling.

The following table describes the parameters that you can configure when you upload objects by using multipart upload.

Type Parameter Description
Required parameters name {String} The full path of the object that you want to upload. The path cannot contain the bucket name.
file {String|File} The path of the local file or HTML5 file that you want to upload.
[options] {Object} Optional parameters [checkpoint] {Object} The checkpoint file that records the result of the multipart upload task. You must set this parameter if you want to perform resumable upload by using multipart upload. The checkpoint file stores the progress information about a multipart upload task. If a part fails to be uploaded, the task can be continued based on the progress recorded in the checkpoint file. After the local file is uploaded, the checkpoint file is deleted.
[parallel] {Number} The number of parts that can be concurrently uploaded. Default value: 5. Do not modify the value of this parameter unless necessary.
[partSize] {Number} The size of each part that you want to upload. Valid values: 100 KB to 5 GB. Default value: 1 MB (1 x 1024 x 1024). Do not modify the value of this parameter unless necessary.
[progress] {Function} The callback function that is used to query the progress of the upload task. The callback function can be an async function. The callback function includes the following parameters:
  • percentage {Number}: the progress of the upload task in percentage. Valid values: decimals between 0 and 1.
  • checkpoint {Object}: the checkpoint file that records the result of the multipart upload task.
  • res {Object}: the response returned when a part is uploaded.
[meta] {Object} The user metadata header, which is prefixed by x-oss-meta.
[mime] {String} The Content-Type request header.
[headers] {Object} Other headers. For more information, see RFC 2616. Examples:
  • Cache-Control: specifies the caching behavior by using specific commands in HTTP requests or responses. Example: Cache-Control: public, no-cache.
  • Content-Disposition: specifies whether the returned content is displayed as a web page or downloaded as an attachment and stored in your on-premises machine. Example: Content-Disposition: somename.
  • Content-Encoding: specifies the method that is used to compress data of the specified media type. Example: Content-Encoding: gzip.
  • Expires: specifies the validity period of the request. Unit: milliseconds.

Complete sample code

Notice OSS SDK for Node.js does not support MD5 verification in multipart upload tasks. We recommend that you call the CRC64 library to determine whether to perform CRC64 checks after the multipart upload task is completed.

The following code provides an example on how to call the multipartUpload method to perform multipart upload:

const OSS = require('ali-oss');
const path = require("path");

const client = new OSS({
  // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
  region: 'yourRegion',
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  accessKeyId: 'yourAccessKeyId',
  accessKeySecret: 'yourAccessKeySecret',
  // Specify the bucket name. Example: examplebucket. 
  bucket: 'examplebucket',
});


const progress = (p, _checkpoint) => {
  // Record the upload progress of the object. 
  console.log(p); 
  // Record the checkpoint information about the multipart upload task. 
  console.log(_checkpoint); 
};

const headers = {
  // Specify the caching behavior of the web page when the object is downloaded. 
  'Cache-Control': 'no-cache',
  // Specify the name of the object when the object is downloaded. 
  'Content-Disposition': 'example.txt',
  // Specify the content encoding format of the object when the object is downloaded. 
  'Content-Encoding': 'utf-8', 
  // Specify the validity period of the request. Unit: milliseconds. 
  'Expires': '1000',
  // Specify the storage class of the object. 
  'x-oss-storage-class': 'Standard', 
  // Specify tags for the object. You can specify multiple tags for the object at the same time. 
  'x-oss-tagging': 'Tag1=1&Tag2=2', 
  // Specify whether to overwrite existing objects with the same names as the objects to upload when the multipart upload task is initialized. In this example, this parameter is set to true, which indicates that existing objects cannot be overwritten by objects with the same names. 
  'x-oss-forbid-overwrite': 'true'
}

// Start the multipart upload task. 
async function multipartUpload() {
  try {
    // Specify the full path of the object. Example: exampledir/exampleobject.txt. Then, specify the full path of the local file. Example: D:\\localpath\\examplefile.txt. The full path of the object cannot contain the bucket name. 
    // By default, if you set this parameter to the name of a local file such as examplefile.txt without specifying the local path, the local file is uploaded from the local path of the project to which the sample program belongs. 
    const result = await client.multipartUpload('exampledir/exampleobject.txt', path.normalize('D:\\localpath\\examplefile.txt'), {
      progress,
      // headers,
      // Configure the meta parameter to specify metadata for the object. You can call the HeadObject operation to query the object metadata. 
      meta: {
        year: 2020,
        people: 'test',
      },
    });
    console.log(result);
    // Specify the full path of the object. Example: exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. 
    const head = await client.head('exampledir/exampleobject.txt');
    console.log(head);
  } catch (e) {
    // Handle timeout exceptions. 
    if (e.code === 'ConnectionTimeoutError') {
      console.log('TimeoutError');
      // do ConnectionTimeoutError operation
    }
    console.log(e);
  }
}

multipartUpload();

The multipartUpload method that is called in the preceding code contains the .initMultipartUpload, .uploadPart, and .completeMultipartUpload operations. If you want to perform multipart upload step by step, call the preceding operations in sequence.

Cancel a multipart upload task

You can call the client.abortMultipartUpload method to cancel a multipart upload task. If a multipart upload task is cancelled, the upload ID cannot be used to upload parts. The uploaded parts are deleted.

The following code provides an example on how to cancel a multipart upload task:

const OSS = require('ali-oss');

const client = new OSS({
  // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
  region: 'yourRegion',
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  accessKeyId: 'yourAccessKeyId',
  accessKeySecret: 'yourAccessKeySecret',
  // Specify the bucket name. Example: examplebucket. 
  bucket: 'examplebucket',
});

async function abortMultipartUpload() {
  // Specify the full path of the object. Example: exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. 
  const name = 'exampledir/exampleobject.txt';
  // Specify the upload ID. You can obtain the upload ID from the information that is returned by the multipartUpload method. 
  const uploadId = '0004B999EF518A1FE585B0C9360D****'; 
  const result = await client.abortMultipartUpload(name, uploadId);
  console.log(result);
}

abortMultipartUpload();

List multipart upload tasks

You can call the client.listUploads method to list all ongoing multipart upload tasks. Ongoing multipart upload tasks include tasks that have been initiated but are not completed and tasks that have been canceled.

const OSS = require('ali-oss');

const client = new OSS({
  // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
  region: 'yourRegion',
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  accessKeyId: 'yourAccessKeyId',
  accessKeySecret: 'yourAccessKeySecret',
  // Specify the bucket name. Example: examplebucket. 
  bucket: 'examplebucket',
});

async function listUploads(query = {}) {
  // You can configure parameters such as prefix, marker, delimiter, upload-id-marker, and max-uploads for query. 
  const result = await client.listUploads(query);

  result.uploads.forEach(upload => {
    // Specify the upload IDs of the multipart upload tasks. 
    console.log(upload.uploadId);
    // Combine all parts for each multipart upload task into a complete object and specify the full path of the object.  
    console.log(upload.name); 
  });
}

const query = {
  // Specify the maximum number of multipart upload tasks to return for the current list operation. The default value and the maximum value of max-uploads are both 1000. 
  'max-uploads': 1000, 
};
listUploads(query);

List uploaded parts

You can call the client.listParts method to list all parts that are uploaded by using the specified upload ID in the multipart upload task.

  • List uploaded parts by using simple list

    The following code provides an example on how to list uploaded parts by using simple list:

    const OSS = require('ali-oss');
    
    const client = new OSS({
      // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
      region: 'yourRegion',
      // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
      accessKeyId: 'yourAccessKeyId',
      accessKeySecret: 'yourAccessKeySecret',
      // Specify the bucket name. Example: examplebucket. 
      bucket: 'examplebucket',
    });
    
    async function listParts() {
      const query = {
        // Specify the maximum number of parts to return for the current list operation. The default value and the maximum value of max-parts are both 1000. 
        'max-parts': 1000, 
      };
      // Specify the full path of the object such as exampledir/exampleobject.txt and the upload ID. The full path of the object cannot contain the bucket name. 
      const result = await client.listParts('exampledir/exampleobject.txt', '0004B999EF518A1FE585B0C9360D****', query);
    
      result.parts.forEach(part => {
        console.log(part.PartNumber);
        console.log(part.LastModified);
        console.log(part.ETag);
        console.log(part.Size);
      });
    }
    
    listParts();
  • List all uploaded parts

    By default, you can list up to 1,000 parts at the same time by using client.listParts. To list more than 1,000 uploaded parts, use the following code:

    const OSS = require('ali-oss');
    
    const client = new OSS({
      // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to oss-cn-hangzhou. 
      region: 'yourRegion',
      // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
      accessKeyId: 'yourAccessKeyId',
      accessKeySecret: 'yourAccessKeySecret',
      // Specify the bucket name. Example: examplebucket. 
      bucket: 'examplebucket',
    });
    
    async function listParts() {
      const query = {
        // Specify the maximum number of parts to return for the current list operation. The default value and the maximum value of max-parts are both 1000. 
        'max-parts': 1000, 
      };
      let result;
      do {
        // Specify the full path of the object such as exampledir/exampleobject.txt and the upload ID. The full path of the object cannot contain the bucket name. 
        result = await client.listParts('exampledir/exampleobject.txt', '0004B999EF518A1FE585B0C9360D****', query);
        // Specify the position from which the list operation starts. Example: 2. Only the parts whose part numbers are greater than the value of this parameter are listed. 
        query['2'] = result.nextPartNumberMarker;
        result.parts.forEach(part => {
          console.log(part.PartNumber);
          console.log(part.LastModified);
          console.log(part.ETag);
          console.log(part.Size);
        });
      } while (result.isTruncated === "true");
    }
    
    listParts();

References

  • For the complete sample code that is used to perform multipart upload, visit GitHub.
  • The multipartUpload method that is used by OSS SDK for Node.js to perform multipart upload contains the following API operations:
    • The API operation that you can call to initiate a multipart upload task. For more information, see InitiateMultipartUpload.
    • The API operation that you can call to upload data by part. For more information, see UploadPart.
    • The API operation that you can call to complete a multipart upload task. For more information, see CompleteMultipartUpload.
  • For more information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.
  • For more information about the API operation that you can call to list uploaded parts, see ListParts.
  • For more information about the API operation that you can call to list ongoing multipart upload tasks, see ListMultipartUploads. Ongoing multipart upload tasks include tasks that have been initiated but are not completed and tasks that have been canceled.