Object Storage Service:Multipart upload

Multipart upload process

To implement multipart upload, perform the following operations:

1. Split the object to upload into multiple parts.
2. Call the InitiateMultipartUpload operation to initiate a multipart upload task.
3. Call the UploadPart operation to upload the parts one by one or concurrently.
4. Call the CompleteMultipartUpload operation to complete the upload.

Before you start a multipart upload task, take note of the following items:

• Each part except the last part must be at least 100 KB in size. Otherwise, you may fail to call the CompleteMultipartUpload operation.
• The parts of the object that you want to upload are sorted based on the part numbers that you specify during the upload. However, parts are not uploaded in order, but are concurrently uploaded.

  If you upload a large number of parts at the same time, the upload speed may vary based on network conditions and device loads. We recommend that you increase the size of each part when network conditions are good, and decrease the size of each part when network conditions are poor.

• By default, if you upload the parts of an object but you do not call the CompleteMultipartUpload operation to combine these parts, the uploaded parts are not automatically deleted. You can call the AbortMultipartUpload operation to terminate the upload task and delete the parts. For more information about how to automatically delete the uploaded parts, see Manage lifecycle rules.

Initialize a multipart upload task

The following code provides an example on how to initialize a multipart upload task:

__block NSString * uploadId = nil;
__block NSMutableArray * partInfos = [NSMutableArray new];
NSString * uploadToBucket = @"<bucketName>";
// objectKey is equivalent to objectName and specifies the full path of the object that you want to upload to OSS. The path must include the file extension of the object. For example, you can set objectKey to abc/efg/123.jpg.
NSString * uploadObjectkey = @"<objectKey>";
// You can use OSSInitMultipartUploadRequest to specify the name of the object you want to upload and specify the name of the bucket to which the object belongs. 
OSSInitMultipartUploadRequest * init = [OSSInitMultipartUploadRequest new];
init.bucketName = uploadToBucket;
init.objectKey = uploadObjectkey;
// init.contentType = @"application/octet-stream";
// The response to multipartUploadInit contains the upload ID. The upload ID is the unique ID of the multipart upload task. 
OSSTask * initTask = [client multipartUploadInit:init];
[initTask waitUntilFinished];
if (!initTask.error) {
    OSSInitMultipartUploadResult * result = initTask.result;
    uploadId = result.uploadId;
} else {
    NSLog(@"multipart upload failed, error: %@", initTask.error);
    return;
}

Upload parts

The following code provides an example on how to upload parts:

// Specify the object you want to upload. 
NSString * filePath = [docDir stringByAppendingPathComponent:@"***"];
// Query the access control list (ACL) of the object. 
uint64_t fileSize = [[[NSFileManager defaultManager] attributesOfItemAtPath:filePath error:nil] fileSize];
// Specify the number of parts. 
int chuckCount = *;
// Specify the size of each part. 
uint64_t offset = fileSize/chuckCount;
for (int i = 1; i <= chuckCount; i++) {
    OSSUploadPartRequest * uploadPart = [OSSUploadPartRequest new];
    uploadPart.bucketName = uploadToBucket;
    uploadPart.objectkey = uploadObjectkey;
    uploadPart.uploadId = uploadId;
    uploadPart.partNumber = i; // part number start from 1

    NSFileHandle* readHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
    [readHandle seekToFileOffset:offset * (i -1)];

    NSData* data = [readHandle readDataOfLength:offset];
    uploadPart.uploadPartData = data;

    OSSTask * uploadPartTask = [client uploadPart:uploadPart];

    [uploadPartTask waitUntilFinished];

    if (!uploadPartTask.error) {
        OSSUploadPartResult * result = uploadPartTask.result;
        uint64_t fileSize = [[[NSFileManager defaultManager] attributesOfItemAtPath:uploadPart.uploadPartFileURL.absoluteString error:nil] fileSize];
        [partInfos addObject:[OSSPartInfo partInfoWithPartNum:i eTag:result.eTag size:fileSize]];
    } else {
        NSLog(@"upload part error: %@", uploadPartTask.error);
        return;
    }
}            

The preceding code uses uploadPart to upload each part.

• An upload ID and a part number must be specified for each multipart upload request.
• After a part is uploaded, the uploadPart operation does not immediately check the size of a part. This API operation checks the part size only if the multipart upload task is completed.
• The part number ranges from 1 to 10000. If a part number is not within this range, the InvalidArgument error code is returned.
• Each time you upload a part, make sure that the stream is directed to the start position of the part to upload.
• Each time a part is uploaded, OSS returns a response that contains the ETag value of the part. If the ETag value is the same as the MD5 hash, combine the ETag value with the part number into a PartETag. Then, save the PartETag for subsequent uploads of parts.

Complete the multipart upload task

In the following code, the partInfos field value specifies the list of PartETags saved during the multipart upload. After OSS receives the partInfos field value, OSS verifies the validity of each part. After OSS validates all parts of the object, OSS combines the parts into a complete object.

The following code provides an example on how to upload an object by using multipart upload:

OSSCompleteMultipartUploadRequest * complete = [OSSCompleteMultipartUploadRequest new];
complete.bucketName = uploadToBucket;
complete.objectKey = uploadObjectkey;
complete.uploadId = uploadId;
complete.partInfos = partInfos;

OSSTask * completeTask = [client completeMultipartUpload:complete];

[[completeTask continueWithBlock:^id(OSSTask *task) {
    if (!task.error) {
        OSSCompleteMultipartUploadResult * result = task.result;
        // ...
    } else {
        // ...
    }
    return nil;
}] waitUntilFinished];           

You can set the servercallback parameter to complete the multipart upload request. After the multipart upload task is completed, a callback request is sent to the server address that you specify. You can view the servercallback result in result.serverReturnJsonString of the response.

OSSCompleteMultipartUploadRequest * complete = [OSSCompleteMultipartUploadRequest new];
complete.bucketName = @"<bucketName>";
complete.objectKey = @"<objectKey>";
complete.uploadId = uploadId;
complete.partInfos = partInfos;
complete.callbackParam = @{
                          @"callbackUrl": @"<server address>",
                          @"callbackBody": @"<test>"
                          };
complete.callbackVar = @{
                        @"var1": @"value1",
                        @"var2": @"value2"
                        };
OSSTask * completeTask = [client completeMultipartUpload:complete];
[[completeTask continueWithBlock:^id(OSSTask *task) {
    if (!task.error) {
        OSSCompleteMultipartUploadResult * result = task.result;
        NSLog(@"server call back return : %@", result.serverReturnJsonString);
    } else {
        // ...
    }
    return nil;
}] waitUntilFinished];

You can include the Content-MD5 field value in the upload request to check whether the object uploaded to OSS is the same as the local file. The OSS server helps you with the MD5 verification. The upload is successful only if the MD5 hash of the object received by the OSS server is the same as the Content-MD5 field value. This method can ensure the consistency of the uploaded object.

The following code provides an example on how to configure MD5 verification:

OSSUploadPartRequest * uploadPart = [OSSUploadPartRequest new];
uploadPart.bucketName = TEST_BUCKET;
uploadPart.uploadId = uploadId;
....
uploadPart.contentMd5 = [OSSUtil fileMD5String:filepath];            

List parts

The following code provides an example on how to call the listParts method to obtain all uploaded parts of a multipart upload task:

OSSListPartsRequest * listParts = [OSSListPartsRequest new];
listParts.bucketName = @"<bucketName>";
listParts.objectKey = @"<objectkey>";
listParts.uploadId = @"<uploadid>";

OSSTask * listPartTask = [client listParts:listParts];

[listPartTask continueWithBlock:^id(OSSTask *task) {
    if (!task.error) {
        NSLog(@"list part result success!");
        OSSListPartsResult * listPartResult = task.result;
        for (NSDictionary * partInfo in listPartResult.parts) {
            NSLog(@"each part: %@", partInfo);
        }
    } else {
        NSLog(@"list part result error: %@", task.error);
    }
    return nil;
}];            

Cancel a multipart upload task

The following code provides an example on how to cancel a multipart upload request that is mapped to the specified upload ID:

OSSAbortMultipartUploadRequest * abort = [OSSAbortMultipartUploadRequest new];
abort.bucketName = @"<bucketName>";
abort.objectKey = @"<objectKey>";
abort.uploadId = uploadId;

OSSTask * abortTask = [client abortMultipartUpload:abort];

[abortTask waitUntilFinished];

if (!abortTask.error) {
    OSSAbortMultipartUploadResult * result = abortTask.result;
    uploadId = result.uploadId;
} else {
    NSLog(@"multipart upload failed, error: %@", abortTask.error);
    return;
}