By using the multipart upload feature provided by OSS, you can split a large object into multiple parts and upload them separately. After all parts are uploaded, call the CompleteMultipartUpload operation to combine these parts into a single object to implement resumable upload.

Process

To implement multipart upload, perform the following steps:

  1. Initiate a multipart upload task.

    Call the InitiateMultipartUpload method to obtain a unique upload ID in Object Storage Service (OSS).

  2. Upload parts.

    Call the UploadPart method to upload the parts.

    Note
    • If parts are uploaded by a multipart upload task that has a specific upload ID, part numbers are used to identify the relative positions of the parts in an object. If you upload a part that has the same part number as an existing part, the existing part is overwritten by the uploaded part.
    • OSS includes the MD5 hash of each uploaded part in the ETag header in the response.
    • OSS calculates the MD5 hash of uploaded data and compares the MD5 hash with the MD5 hash that is calculated by the SDK. If the two hashes are different, OSS returns the InvalidDigest error code.
  3. Complete the multipart upload task.

    After all parts are uploaded, call the CompleteMultipartUpload method to combine the parts into a complete object.

Complete sample code

The following code provides a complete example that describes the process of multipart upload:

#include <alibabacloud/oss/OssClient.h>

int64_t getFileSize(const std::string& file)
{
    std::fstream f(file, std::ios::in | std::ios::binary);
    f.seekg(0, f.end);
    int64_t size = f.tellg();
    f.close();
    return size;
}

using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the information about the account used to access OSS. */
    std::string AccessKeyId = "yourAccessKeyId";
    std::string AccessKeySecret = "yourAccessKeySecret";
    std::string Endpoint = "yourEndpoint";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the name of the bucket. Example: exampledir/exampleobject.txt.  */
    std::string ObjectName = "exampledir/exampleobject.txt";

    /* Initialize resources such as networks. */
    InitializeSdk();

    ClientConfiguration conf;
    OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);
    /* Optional. Specify the storage class. */
    //initUploadRequest.MetaData().addHeader("x-oss-storage-class", "Standard");

    /* Initiate the multipart upload task. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    auto uploadId = uploadIdResult.result().UploadId();
    std::string fileToUpload = "yourLocalFilename";
    int64_t partSize = 100 * 1024;
    PartList partETagList;
    auto fileSize = getFileSize(fileToUpload);
    int partCount = static_cast<int>(fileSize / partSize);
    /* Calculate the number of parts. */
    if (fileSize % partSize != 0) {
        partCount++;
    }

    /* Upload each part. */
    for (int i = 1; i <= partCount; i++) {
        auto skipBytes = partSize * (i - 1);
        auto size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
        std::shared_ptr<std::iostream> content = std::make_shared<std::fstream>(fileToUpload, std::ios::in|std::ios::binary);
        content->seekg(skipBytes, std::ios::beg);

        UploadPartRequest uploadPartRequest(BucketName, ObjectName, content);
        uploadPartRequest.setContentLength(size);
        uploadPartRequest.setUploadId(uploadId);
        uploadPartRequest.setPartNumber(i);
        auto uploadPartOutcome = client.UploadPart(uploadPartRequest);
        if (uploadPartOutcome.isSuccess()) {
            Part part(i, uploadPartOutcome.result().ETag());
            partETagList.push_back(part);
        }
        else {
            std::cout << "uploadPart fail" <<
            ",code:" << uploadPartOutcome.error().Code() <<
            ",message:" << uploadPartOutcome.error().Message() <<
            ",requestId:" << uploadPartOutcome.error().RequestId() << std::endl;
        }

    }

    /* Complete the multipart upload task. */
    CompleteMultipartUploadRequest request(BucketName, ObjectName);
    request.setUploadId(uploadId);
    request.setPartList(partETagList);
    /* Optional. Specify the ACL of the object. */
    //request.setAcl(CannedAccessControlList::Private);

    auto outcome = client.CompleteMultipartUpload(request);

    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "CompleteMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        ShutdownSdk();
        return -1;
    }

    /* Release resources such as networks. */
    ShutdownSdk();
    return 0;
}

To call the CompleteMultipartUpload operation when you obtain uploaded parts, you must provide the ETag value of each part. You can obtain the ETag value of each part by using one of the following methods:

  • When a part is uploaded, the ETag value of the part is included in the response. You can save and use this ETag value. In this example, the first method is used.
  • Call the ListParts method to query the ETag values of the uploaded parts.

List uploaded parts

You can call the ListParts method to list all uploaded parts that use the specified upload ID.
  • List all uploaded parts
    Note By default, ListParts can list up to 1,000 parts at a time. To list more than 1,000 parts, you must list the results by page in the multipart upload operation.
    The following code provides an example on how to list all uploaded parts:
    #include <alibabacloud/oss/OssClient.h>
    using namespace AlibabaCloud::OSS;
    
    int main(void)
    {
        /* Initialize the information about the account used to access OSS. */
        std::string AccessKeyId = "yourAccessKeyId";
        std::string AccessKeySecret = "yourAccessKeySecret";
        std::string Endpoint = "yourEndpoint";
        /* Specify the name of the bucket. Example: examplebucket. */
        std::string BucketName = "examplebucket";
        /* Specify the full path of the object. The full path cannot contain the name of the bucket. Example: exampledir/exampleobject.txt.  */
        std::string ObjectName = "exampledir/exampleobject.txt";
    
        /* Initialize resources such as networks. */
        InitializeSdk();
    
        ClientConfiguration conf;
        OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    
        /* List the uploaded parts. By default, 1,000 parts are listed. */
        ListPartsRequest listuploadrequest(BucketName, ObjectName);
        listuploadrequest.setUploadId(uploadId);
        do {
            auto listuploadresult = client.ListParts(listuploadrequest);
            if (!listUploadResult.isSuccess()) {
                /* Handle exceptions. */
                std::cout << "ListParts fail" <<
                ",code:" << listuploadresult.error().Code() <<
                ",message:" << listuploadresult.error().Message() <<
                ",requestId:" << listuploadresult.error().RequestId() << std::endl;
                break;
            }
            else {
                for (const auto& part : listuploadresult.result().PartList()) {
                    std::cout << "part"<<
                    ",name:" << part.PartNumber() <<
                    ",size:" << part.Size() <<
                    ",etag:" << part.ETag() <<
                    ",lastmodify time:" << part.LastModified() << std::endl;
                }
            }
            listuploadrequest.setPartNumberMarker(listuploadresult.result().NextPartNumberMarker());
        } while (listuploadresult.result().IsTruncated());
    
        /* Release resources such as networks. */
        ShutdownSdk();
        return 0;
    }
  • List all uploaded parts by page
    The following code provides an example on how to specify the maximum number of parts to list on each page and list all parts by page:
    #include <alibabacloud/oss/OssClient.h>
    using namespace AlibabaCloud::OSS;
    
    int main(void)
    {
        /* Initialize the information about the account used to access OSS. */
        std::string AccessKeyId = "yourAccessKeyId";
        std::string AccessKeySecret = "yourAccessKeySecret";
        std::string Endpoint = "yourEndpoint";
        /* Specify the name of the bucket. Example: examplebucket. */
        std::string BucketName = "examplebucket";
        /* Specify the full path of the object. The full path cannot contain the name of the bucket. Example: exampledir/exampleobject.txt.  */
        std::string ObjectName = "exampledir/exampleobject.txt";
    
        /* Initialize resources such as networks. */
        InitializeSdk();
    
        ClientConfiguration conf;
        OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    
        /* List all uploaded parts by page. */
        /* Set the maximum number of parts to list on each page. */
        ListPartsRequest listuploadrequest(BucketName, ObjectName);
        listuploadrequest.setMaxParts(50);
        listuploadrequest.setUploadId(uploadId);
        do {
            listuploadresult = client.ListParts(listuploadrequest);
            if (!listUploadResult.isSuccess()) {
                /* Handle exceptions. */
                std::cout << "ListParts fail" <<
                ",code:" << listuploadresult.error().Code() <<
                ",message:" << listuploadresult.error().Message() <<
                ",requestId:" << listuploadresult.error().RequestId() << std::endl;
                break;
            }
            else {
                for (const auto& part : listuploadresult.result().PartList()) {
                    std::cout << "part"<<
                    ",name:" << part.PartNumber() <<
                    ",size:" << part.Size() <<
                    ",etag:" << part.ETag() <<
                    ",lastmodify time:" << part.LastModified() << std::endl;
                }
            }  
            listuploadrequest.setPartNumberMarker(listuploadresult.result().NextPartNumberMarker());     
        } while (listuploadresult.result().IsTruncated());
    
        /* Release resources such as networks. */
        ShutdownSdk();
        return 0;
    }

List multipart upload tasks

You can call the ListMultipartUploads method to list all ongoing multipart upload tasks. Ongoing multipart upload tasks are tasks that have been initiated but not completed or tasks that have been canceled.

  • List all multipart upload tasks
    Note By default, the ListMultipartUploads method can list up to 1,000 multipart upload tasks at a time. To list more than 1,000 multipart upload tasks, you must list the results by page in the multipart upload operation..

    The following code provides an example on how to list all multipart upload tasks:

    #include <alibabacloud/oss/OssClient.h>
    using namespace AlibabaCloud::OSS;
    
    int main(void)
    {
        /* Initialize the information about the account used to access OSS. */
        std::string AccessKeyId = "yourAccessKeyId";
        std::string AccessKeySecret = "yourAccessKeySecret";
        std::string Endpoint = "yourEndpoint";
        /* Specify the name of the bucket. Example: examplebucket. */
        std::string BucketName = "examplebucket";    
    
        /* Initialize resources such as networks. */
        InitializeSdk();
    
        ClientConfiguration conf;
        OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    
        /* List the completed multipart upload tasks. By default, 1,000 parts are listed. */
        ListMultipartUploadsRequest listmultiuploadrequest(BucketName);
        do {
            auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
            if (!listresult.isSuccess()) {
                /* Handle exceptions. */
                std::cout << "ListMultipartUploads fail" <<
                ",code:" << listresult.error().Code() <<
                ",message:" << listresult.error().Message() <<
                ",requestId:" << listresult.error().RequestId() << std::endl;
                break;
            }
            else {
                for (const auto& part : listresult.result().MultipartUploadList()) {
                    std::cout << "part"<<
                    ",name:" << part.Key <<
                    ",uploadid:" << part.UploadId <<
                    ",initiated time:" << part.Initiated << std::endl;
                }
            }
            listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
            listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker()); 
        } while (listresult.result().IsTruncated());
    
        /* Release resources such as networks. */
        ShutdownSdk();
        return 0;
    }
  • List all multipart upload tasks by page
    The following code provides an example on how to list all multipart upload tasks by page:
    #include <alibabacloud/oss/OssClient.h>
    using namespace AlibabaCloud::OSS;
    
    int main(void)
    {
        /* Initialize the information about the account used to access OSS. */
        std::string AccessKeyId = "yourAccessKeyId";
        std::string AccessKeySecret = "yourAccessKeySecret";
        std::string Endpoint = "yourEndpoint";
        /* Specify the name of the bucket. Example: examplebucket. */
        std::string BucketName = "examplebucket"; 
    
        /* Initialize resources such as networks. */
        InitializeSdk();
    
        ClientConfiguration conf;
        OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    
        /* List all multipart upload tasks by page. */
        /* Set the maximum number of multipart upload tasks to list on each page. */
        ListMultipartUploadsRequest  listmultiuploadrequest(BucketName);
        listmultiuploadrequest.setMaxUploads(50);
        do {
            listresult = client.ListMultipartUploads(listmultiuploadrequest);
            if (!listresult.isSuccess()) {
                /* Handle exceptions. */
                std::cout << "ListMultipartUploads fail" <<
                ",code:" << listresult.error().Code() <<
                ",message:" << listresult.error().Message() <<
                ",requestId:" << listresult.error().RequestId() << std::endl;
                break;
            }
            else {
                for (const auto& part : listresult.result().MultipartUploadList()) {
                    std::cout << "part"<<
                    ",name:" << part.Key <<
                    ",uploadid:" << part.UploadId <<
                    ",initiated time:" << part.Initiated << std::endl;
                }
            }  
            listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
            listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker()); 
        } while (listresult.result().IsTruncated());
    
        /* Release resources such as networks. */
        ShutdownSdk();
        return 0;
    }

Cancel a multipart upload task

You can call the client.AbortMultipartUpload method to cancel a multipart upload task. If a multipart upload task is canceled, the upload ID cannot be used to upload parts. In addition, the uploaded parts are deleted.

The following code provides an example on how to cancel a multipart upload task:
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the information about the account used to access OSS. */
    std::string AccessKeyId = "yourAccessKeyId";
    std::string AccessKeySecret = "yourAccessKeySecret";
    std::string Endpoint = "yourEndpoint";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the name of the bucket. Example: exampledir/exampleobject.txt.  */
    std::string ObjectName = "exampledir/exampleobject.txt";

    /* Initialize resources such as networks. */
    InitializeSdk();

    ClientConfiguration conf;
    OssClient client(Endpoint, AccessKeyId, AccessKeySecret, conf);
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);

    /* Initiate the multipart upload task. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    auto uploadId = uploadIdResult.result().UploadId();

    /* Cancel the multipart upload task. */
    AbortMultipartUploadRequest  abortUploadRequest(BucketName, ObjectName, uploadId);
    auto abortUploadIdResult = client.AbortMultipartUpload(abortUploadRequest);

    if (!abortUploadIdResult.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "AbortMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        ShutdownSdk();
        return -1;
    }

    /* Release resources such as networks. */
    ShutdownSdk();
    return 0;
}

References

  • For more information about the complete sample code for multipart upload, visit GitHub.
  • The following API operations are required to perform multipart upload:
    • The API operation that you can call to initiate a multipart upload task. For more information, see InitiateMultipartUpload.
    • The API operation that you can call to upload data by part. For more information, see UploadPart.
    • The API operation that you can call to complete a multipart upload task. For more information, see CompleteMultipartUpload.
  • For more information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.
  • For more information about the API operation that you can call to list the uploaded parts, see ListParts.
  • For more information about the API operation that you can call to list all ongoing multipart upload tasks, see ListMultipartUploads. Ongoing multipart upload tasks include tasks that have been initiated but are not completed and tasks that are canceled.