Object Storage Service:Multipart upload

Usage notes

• In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS by using other Alibaba Cloud services in the same region as OSS, use an internal endpoint For more information about the regions and endpoints supported by OSS, see Regions and endpoints.
• In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or STS, see Initialization.
• The oss:PutObject permission is required to perform multipart upload. For more information, see Attach a custom policy to a RAM user.

Process

To upload an object by using multipart upload, perform the following steps:

1. Initiate a multipart upload task.

   Call the InitiateMultipartUploadRequest method to obtain a unique upload ID in OSS.

2. Upload parts.

   Call the UploadPartRequest method to upload parts.

   Note

   • If parts are uploaded by a multipart upload task that has a specific upload ID, part numbers are used to identify the relative positions of the parts in an object. If you upload a part that has the same part number as an existing part, the existing part is overwritten by the uploaded part.
   • OSS includes the MD5 hash of each uploaded part in the ETag header in the response.
   • OSS calculates the MD5 hash of uploaded data and compares the MD5 hash with the MD5 hash that is calculated by OSS SDK for Java. If the two hashes are different, OSS returns the InvalidDigest error code.
3. Complete the multipart upload task.

   After all parts are uploaded, call the CompleteMultipartUploadRequest method to combine the parts into a complete object.

Examples

The following code provides an example on how to perform multipart upload:

using Aliyun.OSS;

// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
var accessKeyId = "yourAccessKeyId";
var accessKeySecret = "yourAccessKeySecret";
// Specify the name of the bucket. 
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path of the object. 
var objectName = "exampleobject.txt";
// Specify the full path of the local file that you want to upload. By default, if you do not specify the full path of the local object, the local object is uploaded from the path of the project to which the sample program belongs. 
var localFilename = "D:\\localpath\\examplefile.txt";
// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);

// Initiate the multipart upload task and obtain the upload ID in the response. 
var uploadId = "";
try
{
    // Specify the name of the object that you want to upload and the bucket to which you want to upload the object. You can configure object metadata in InitiateMultipartUploadRequest. However, you do not need to specify ContentLength. 
    var request = new InitiateMultipartUploadRequest(bucketName, objectName);
    var result = client.InitiateMultipartUpload(request);
    uploadId = result.UploadId;
    // Display the upload ID. 
    Console.WriteLine("Init multi part upload succeeded");
    Console.WriteLine("Upload Id:{0}", result.UploadId);
}
catch (Exception ex)
{
    Console.WriteLine("Init multi part upload failed, {0}", ex.Message);
}
// Calculate the total number of parts. 
var partSize = 100 * 1024;
var fi = new FileInfo(localFilename);
var fileSize = fi.Length;
var partCount = fileSize / partSize;
if (fileSize % partSize != 0)
{
    partCount++;
}
// Start the multipart upload task. partETags is a list of part ETags. OSS verifies the validity of each part after it receives the part list. After all parts are verified, OSS combines these parts into a complete object. 
var partETags = new List<PartETag>();
try
{
    using (var fs = File.Open(localFilename, FileMode.Open))
    {
        for (var i = 0; i < partCount; i++)
        {
            var skipBytes = (long)partSize * i;
            // Find the start position of the multipart upload task. 
            fs.Seek(skipBytes, 0);
            // Calculate the part size of this upload. The size of the last part is the size of the remainder after the object is split by the calculated part size. 
            var size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
            var request = new UploadPartRequest(bucketName, objectName, uploadId)
            {
                InputStream = fs,
                PartSize = size,
                PartNumber = i + 1
            };
            // Call UploadPart to upload parts. The returned results contain the ETag values of parts. 
            var result = client.UploadPart(request);
            partETags.Add(result.PartETag);
            Console.WriteLine("finish {0}/{1}", partETags.Count, partCount);
        }
        Console.WriteLine("Put multi part upload succeeded");
    }
}
catch (Exception ex)
{
    Console.WriteLine("Put multi part upload failed, {0}", ex.Message);
}
// Complete the multipart upload task. 
try
{
    var completeMultipartUploadRequest = new CompleteMultipartUploadRequest(bucketName, objectName, uploadId);
    foreach (var partETag in partETags)
    {
        completeMultipartUploadRequest.PartETags.Add(partETag);
    }
    var result = client.CompleteMultipartUpload(completeMultipartUploadRequest);
    Console.WriteLine("complete multi part succeeded");
}
catch (Exception ex)
{
    Console.WriteLine("complete multi part failed, {0}", ex.Message);
}

Cancel a multipart upload task

You can call the client.AbortMultipartUpload method to cancel a multipart upload task. If you cancel a multipart upload task, you cannot use the upload ID to upload any parts. The uploaded parts are deleted.

The following sample code provides an example on how to cancel a multipart upload task:

using Aliyun.OSS;

// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
var accessKeyId = "yourAccessKeyId";
var accessKeySecret = "yourAccessKeySecret";
// Specify the name of the bucket. 
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path of the object. 
var objectName = "exampleobject.txt";
// Specify the upload ID. 
var uploadId = "yourUploadId";

// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);
// Initiate the multipart upload task. 
try
{
    var request = new InitiateMultipartUploadRequest(bucketName, objectName);
    var result = client.InitiateMultipartUpload(request);
    uploadId = result.UploadId;
    // Display the upload ID. 
    Console.WriteLine("Init multi part upload succeeded");
    Console.WriteLine("Upload Id:{0}", result.UploadId);
}
catch (Exception ex)
{
    Console.WriteLine("Init multi part upload failed, {0}", ex.Message);
}
// Cancel the multipart upload task. 
try
{
    var request = new AbortMultipartUploadRequest(bucketName, objectName, uploadId);
    client.AbortMultipartUpload(request);
    Console.WriteLine("Abort multi part succeeded, {0}", uploadId);
}
catch (Exception ex)
{
    Console.WriteLine("Abort multi part failed, {0}", ex.Message);
}

List uploaded parts

The following code provides an example on how to list uploaded parts:

using Aliyun.OSS;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
var accessKeyId = "yourAccessKeyId";
var accessKeySecret = "yourAccessKeySecret";
// Specify the name of the bucket. 
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path of the object. 
var objectName = "exampleobject.txt";
// Specify the upload ID. 
var uploadId = "yourUploadId";

// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);
try
{
    PartListing listPartsResult = null;
    var nextMarker = 0;
    do 
    {
        var listPartsRequest = new ListPartsRequest(bucketName, objectName, uploadId) 
        {
            PartNumberMarker = nextMarker,
        };
        // List uploaded parts. 
        listPartsResult = client.ListParts(listPartsRequest);
        Console.WriteLine("List parts succeeded");
        // Traverse all the uploaded parts. 
        var parts = listPartsResult.Parts;
        foreach (var part in parts)
        {
            Console.WriteLine("partNumber: {0}, ETag: {1}, Size: {2}", part.PartNumber, part.ETag, part.Size);
        }
        nextMarker = listPartsResult.NextPartNumberMarker;
    } while (listPartsResult.IsTruncated);
}
catch (Exception ex)
{
    Console.WriteLine("List parts failed, {0}", ex.Message);
}

List multipart upload tasks

You can call the ossClient.listMultipartUploads method to list all ongoing multipart upload tasks, which include tasks that have been initiated but are not completed or canceled. The following table describes the parameters that you can configure when you list all ongoing multipart upload tasks.

The following sample code provides an example on how to use simple list to list multipart upload tasks:

using Aliyun.OSS;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
var accessKeyId = "yourAccessKeyId";
var accessKeySecret = "yourAccessKeySecret";
// Specify the name of the bucket. 
var bucketName = "examplebucket";

// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);
try
{
    MultipartUploadListing multipartUploadListing = null;
    var nextMarker = string.Empty;
    do
    {
        // List multipart upload tasks. 
        var request = new ListMultipartUploadsRequest(bucketName)
        {
            KeyMarker = nextMarker,
        };
        multipartUploadListing = client.ListMultipartUploads(request);
        Console.WriteLine("List multi part succeeded");
        // List information about each multipart upload task. 
        foreach (var mu in multipartUploadListing.MultipartUploads)
        {
            Console.WriteLine("Key: {0},  UploadId: {1}", mu.Key, mu.UploadId);
        }
        // If the value of the isTruncated field in the returned result is false, values of nextKeyMarker and nextUploadIdMarker are returned and used as the start point for the next read. 
        nextMarker = multipartUploadListing.NextKeyMarker;
    } while (multipartUploadListing.IsTruncated);
}
catch (Exception ex)
{
    Console.WriteLine("List multi part uploads failed, {0}", ex.Message);
}

List multipart upload tasks by specifying a prefix and the maximum number of records to return

The following code provides an example on how to list multipart upload tasks by specifying a prefix and the maximum number of records to return:

using Aliyun.OSS;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
var endpoint = "yourEndpoint";
// The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
var accessKeyId = "yourAccessKeyId";
var accessKeySecret = "yourAccessKeySecret";
// Specify the name of the bucket. 
var bucketName = "examplebucket";

// Specify the prefix. 
var prefix = "yourObjectPrefix";
// Specify the maximum number of records to return as 100. 
var maxUploads = 100;
// Create an OSSClient instance. 
var client = new OssClient(endpoint, accessKeyId, accessKeySecret);
try
{
    MultipartUploadListing multipartUploadListing = null;
    var nextMarker = string.Empty;
    do
    {
        // List multipart upload tasks. By default, 1,000 multipart upload tasks are listed. 
        var request = new ListMultipartUploadsRequest(bucketName)
        {
            KeyMarker = nextMarker,
            // Specify the prefix that is contained in the names of the objects that you want to list. 
            Prefix = prefix,
            // Specify the maximum number of records to return. 
            MaxUploads = maxUploads,
        };
        multipartUploadListing = client.ListMultipartUploads(request);
        Console.WriteLine("List multi part succeeded");
        // List information about each multipart upload task. 
        foreach (var mu in multipartUploadListing.MultipartUploads)
        {
            Console.WriteLine("Key: {0},  UploadId: {1}", mu.Key, mu.UploadId);
        }
        nextMarker = multipartUploadListing.NextKeyMarker;
    } while (multipartUploadListing.IsTruncated);
}
catch (Exception ex)
{
    Console.WriteLine("List multi part uploads failed, {0}", ex.Message);
}

References

• For the complete sample code that is used to perform multipart upload, visit GitHub.
• The following API operations are required to perform multipart upload:
  • The API operation that you can call to initiate a multipart upload task. For more information, see InitiateMultipartUpload.
  • The API operation that you can call to upload data by part. For more information, see UploadPart.
  • The API operation that you can call to complete a multipart upload task. For more information, see CompleteMultipartUpload.
• For more information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.
• For more information about the API operation that you can call to list the uploaded parts, see ListParts.
• For more information about the API operation that you can call to list all ongoing multipart upload tasks, see ListMultipartUploads. Ongoing multipart upload tasks include tasks that have been initiated but are not completed and tasks that are canceled.