All Products
Search
Document Center

Object Storage Service:Multipart upload

Last Updated:Nov 01, 2023

Object Storage Service (OSS) provides the multipart upload feature. This feature allows you to split a large object into multiple parts and upload the parts separately. After the upload is complete, you can call the CompleteMultipartUpload operation to combine these parts into a complete object.

Procedure

To implement multipart upload, perform the following steps:

  1. Split the object that you want to upload into multiple parts.

  2. Call the InitiateMultipartUpload operation to initiate a multipart upload task.

  3. Call the UploadPart operation to upload the parts one by one or concurrently.

  4. Call the CompleteMultipartUpload operation to complete the upload task.

Before you start a multipart upload task, take note of the following items:

  • Each part except the last part must be at least 100 KB in size. Otherwise, you may fail to call the CompleteMultipartUpload operation.

  • The parts of the object that you want to upload are sorted based on the part numbers that you specify during the upload. However, parts are not uploaded in order, but are concurrently uploaded.

    If you upload a large number of parts at the same time, the upload speed may vary based on network conditions and device loads. We recommend that you increase the size of each part when network conditions are stable. Otherwise, decrease the size of each part.

  • By default, if you upload the parts of an object but do not call the CompleteMultipartUpload operation to combine these parts, the uploaded parts are not automatically deleted. You can call the AbortMultipartUpload operation to terminate the upload task and delete the parts. For more information about how to automatically delete the uploaded parts, see Lifecycle rules based on the last modified time.

Sample code of multipart upload

The following code provides an example on how to implement a multipart upload task by following the multipart upload process:

__block NSString * uploadId = nil;
__block NSMutableArray * partInfos = [NSMutableArray new];
// Specify the name of the bucket. Example: examplebucket. 
NSString * uploadToBucket = @"examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path. 
NSString * uploadObjectkey = @"exampledir/exampleobject.txt";
// You can use OSSInitMultipartUploadRequest to specify the name of the uploaded object and the name of the bucket in which the object is stored. 
OSSInitMultipartUploadRequest * init = [OSSInitMultipartUploadRequest new];
init.bucketName = uploadToBucket;
init.objectKey = uploadObjectkey;
// init.contentType = @"application/octet-stream";
// The response to multipartUploadInit contains the upload ID. The upload ID is the unique ID of the multipart upload task. 
OSSTask * initTask = [client multipartUploadInit:init];
[initTask waitUntilFinished];
if (!initTask.error) {
    OSSInitMultipartUploadResult * result = initTask.result;
    uploadId = result.uploadId;
    // Cancel the multipart upload task or list uploaded parts based on the upload ID. 
    // If you want to cancel a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task. 
    // If you want to list the uploaded parts in a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task but before you call the CompleteMultipartUpload operation to complete the multipart upload task. 
    //NSLog(@"UploadId": %@, uploadId);
} else {
    NSLog(@"multipart upload failed, error: %@", initTask.error);
    return;
}

// Specify the object that you want to upload. 
NSString * filePath = @"<filepath>";
// Query the size of the object. 
uint64_t fileSize = [[[NSFileManager defaultManager] attributesOfItemAtPath:filePath error:nil] fileSize];
// Specify the number of parts. 
int chuckCount = 10;
// Specify the size of each part. 
uint64_t offset = fileSize/chuckCount;
for (int i = 1; i <= chuckCount; i++) {
    OSSUploadPartRequest * uploadPart = [OSSUploadPartRequest new];
    uploadPart.bucketName = uploadToBucket;
    uploadPart.objectkey = uploadObjectkey;
    uploadPart.uploadId = uploadId;
    uploadPart.partNumber = i; // part number start from 1

    NSFileHandle* readHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
    [readHandle seekToFileOffset:offset * (i -1)];

    NSData* data = [readHandle readDataOfLength:offset];
    uploadPart.uploadPartData = data;

    OSSTask * uploadPartTask = [client uploadPart:uploadPart];

    [uploadPartTask waitUntilFinished];

    if (!uploadPartTask.error) {
        OSSUploadPartResult * result = uploadPartTask.result;
        uint64_t fileSize = [[[NSFileManager defaultManager] attributesOfItemAtPath:uploadPart.uploadPartFileURL.absoluteString error:nil] fileSize];
        [partInfos addObject:[OSSPartInfo partInfoWithPartNum:i eTag:result.eTag size:fileSize]];
    } else {
        NSLog(@"upload part error: %@", uploadPartTask.error);
        return;
    }
}
OSSCompleteMultipartUploadRequest * complete = [OSSCompleteMultipartUploadRequest new];
complete.bucketName = uploadToBucket;
complete.objectKey = uploadObjectkey;
complete.uploadId = uploadId;
complete.partInfos = partInfos;

OSSTask * completeTask = [client completeMultipartUpload:complete];

[[completeTask continueWithBlock:^id(OSSTask *task) {
    if (!task.error) {
        OSSCompleteMultipartUploadResult * result = task.result;
        // ...
    } else {
        // ...
    }
    return nil;
}] waitUntilFinished];

Upload a local file by using multipart upload

Note

The preceding complete sample code is used to implement a multipart upload task by following the multipart upload process. The complete sample code is encapsulated to upload a local file by using multipart upload. This way, you need to only use MultipartUploadRequest to implement a multipart upload task for a local file.

The following code provides an example on how to upload a local file by using multipart upload:

// Specify the name of the bucket. Example: examplebucket. 
NSString *bucketName = @"examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path. 
NSString *objectKey = @"exampledir/exampleobject.txt";

OSSMultipartUploadRequest * multipartUploadRequest = [OSSMultipartUploadRequest new];
multipartUploadRequest.bucketName = bucketName;
multipartUploadRequest.objectKey = objectKey;
// Specify the size of parts. The default size is 256 KB. 
multipartUploadRequest.partSize = 1024 * 1024;
multipartUploadRequest.uploadProgress = ^(int64_t bytesSent, int64_t totalByteSent, int64_t totalBytesExpectedToSend) {
    NSLog(@"progress: %lld, %lld, %lld", bytesSent, totalByteSent, totalBytesExpectedToSend);
};

multipartUploadRequest.uploadingFileURL = [[NSBundle mainBundle] URLForResource:@"wangwang" withExtension:@"zip"];
OSSTask * multipartTask = [client multipartUpload:multipartUploadRequest];
[[multipartTask continueWithBlock:^id(OSSTask *task) {
    if (task.error) {
        NSLog(@"error: %@", task.error);
    } else {
        NSLog(@"Upload file success");
    }
    return nil;
}] waitUntilFinished];

List uploaded parts

The following code provides an example on how to call the listParts operation to obtain all uploaded parts of a multipart upload task:

OSSListPartsRequest * listParts = [OSSListPartsRequest new];
// Specify the name of the bucket. Example: examplebucket. 
listParts.bucketName = @"examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path. 
listParts.objectKey = @"exampledir/exampleobject.txt";
// Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task. 
listParts.uploadId = @"0004B999EF518A1FE585B0C9****";

OSSTask * listPartTask = [client listParts:listParts];

[listPartTask continueWithBlock:^id(OSSTask *task) {
    if (!task.error) {
        NSLog(@"list part result success!");
        OSSListPartsResult * listPartResult = task.result;
        for (NSDictionary * partInfo in listPartResult.parts) {
            NSLog(@"each part: %@", partInfo);
        }
    } else {
        NSLog(@"list part result error: %@", task.error);
    }
    return nil;
}];            
Important

By default, if a bucket contains more than 1,000 parts that are uploaded by using multipart upload, OSS returns the first 1,000 parts. In the response, the IsTruncated field is set to false and NextPartNumberMarker is returned to indicate the next starting position at which data is read.

Cancel a multipart upload task

The following code provides an example on how to cancel a multipart upload task that has a specific upload ID:

OSSAbortMultipartUploadRequest * abort = [OSSAbortMultipartUploadRequest new];
// Specify the name of the bucket. Example: examplebucket. 
abort.bucketName = @"examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path. 
abort.objectKey = @"exampledir/exampleobject.txt";
// Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. 
abort.uploadId = @"0004B999EF518A1FE585B0C9****";

OSSTask * abortTask = [client abortMultipartUpload:abort];

[abortTask waitUntilFinished];

if (!abortTask.error) {
    OSSAbortMultipartUploadResult * result = abortTask.result;
    uploadId = result.uploadId;
} else {
    NSLog(@"multipart upload failed, error: %@", abortTask.error);
    return;
}           

References

  • For the complete sample code that is used to perform multipart upload, visit GitHub.

  • A multipart upload involves three API operations. For more information about the operations, see the following topics:

  • For more information about the API operation that you can call to list uploaded parts, see ListParts.

  • For more information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.