Object Storage Service (OSS) provides the multipart upload feature. Multipart upload allows you to split a large object into multiple parts to upload. After these parts are uploaded, you can call the CompleteMultipartUpload operation to combine the parts into a complete object.
Usage notes
Before you run the sample code in this topic, you must create an OSSClient instance by using methods such as using a custom domain name or Security Token Service (STS). For more information, see Initialization.
Multipart upload process
A multipart upload consists of the following three steps:
Initiate a multipart upload event.
Call the oss.initMultipartUpload method. OSS then returns a globally unique uploadId.
Upload parts.
Call the oss.uploadPart method to upload each part.
NoteIn a multipart upload task, part numbers are used to identify the relative positions of the parts in an object. If you upload a part that has the same part number as an existing part, the existing part is overwritten by the uploaded part.
OSS includes the MD5 hash of each uploaded part in the ETag header in the response.
OSS calculates the MD5 hash of the uploaded parts and compares the MD5 hash with the MD5 hash that is calculated by OSS SDK for Java. If the two hashes are different, OSS returns the InvalidDigest error code.
Complete the multipart upload.
After all parts are uploaded, call the oss.CompleteMultipartUpload method to combine all parts into a complete object.
Complete example of a multipart upload
The following sample code provides an example on how to implement a multipart upload task by following the multipart upload process:
// Specify the bucket name, for example, examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name.
String objectName = "exampledir/exampleobject.txt";
// Specify the full path of the local file, for example, /storage/emulated/0/oss/examplefile.txt.
String localFilepath = "/storage/emulated/0/oss/examplefile.txt";
// Initialize the multipart upload.
InitiateMultipartUploadRequest init = new InitiateMultipartUploadRequest(bucketName, objectName);
InitiateMultipartUploadResult initResult = oss.initMultipartUpload(init);
// The uploadId is returned.
String uploadId = initResult.getUploadId();
// Use the uploadId to cancel the multipart upload event or list the uploaded parts.
// To cancel the multipart upload event based on the uploadId, obtain the uploadId after you call InitiateMultipartUpload to initialize the multipart upload.
// To list the uploaded parts based on the uploadId, obtain the uploadId after you call InitiateMultipartUpload to initialize the multipart upload and before you call CompleteMultipartUpload to complete the multipart upload.
// Log.d("uploadId", uploadId);
// Set the size of a single part in bytes. The minimum size of a part is 100 KB, and the maximum size is 5 GB. The size of the last part can be less than 100 KB.
int partCount = 100 * 1024;
// Upload parts.
List<PartETag> partETags = new ArrayList<>();
for (int i = 1; i < 5; i++) {
byte[] data = new byte[partCount];
RandomAccessFile raf = new RandomAccessFile(localFilepath, "r");
long skip = (i-1) * partCount;
raf.seek(skip);
raf.readFully(data, 0, partCount);
UploadPartRequest uploadPart = new UploadPartRequest();
uploadPart.setBucketName(bucketName);
uploadPart.setObjectKey(objectName);
uploadPart.setUploadId(uploadId);
// Set the part number, which starts from 1. Each uploaded part has a part number that ranges from 1 to 10,000.
uploadPart.setPartNumber(i);
uploadPart.setPartContent(data);
try {
UploadPartResult result = oss.uploadPart(uploadPart);
PartETag partETag = new PartETag(uploadPart.getPartNumber(), result.getETag());
partETags.add(partETag);
} catch (ServiceException serviceException) {
OSSLog.logError(serviceException.getErrorCode());
}
}
Collections.sort(partETags, new Comparator<PartETag>() {
@Override
public int compare(PartETag lhs, PartETag rhs) {
if (lhs.getPartNumber() < rhs.getPartNumber()) {
return -1;
} else if (lhs.getPartNumber() > rhs.getPartNumber()) {
return 1;
} else {
return 0;
}
}
});
// Complete the multipart upload.
CompleteMultipartUploadRequest complete = new CompleteMultipartUploadRequest(bucketName, objectName, uploadId, partETags);
// Upload callback. When you complete the multipart upload request, you can set the CALLBACK_SERVER parameter. After the request is complete, a callback request is sent to the specified server address. You can view the server callback result from completeResult.getServerCallbackReturnBody() in the returned result.
complete.setCallbackParam(new HashMap<String, String>() {
{
put("callbackUrl", CALLBACK_SERVER); // Change the value to your server address.
put("callbackBody", "test");
}
});
CompleteMultipartUploadResult completeResult = oss.completeMultipartUpload(complete);
OSSLog.logError("-------------- serverCallback: " + completeResult.getServerCallbackReturnBody());The preceding code calls uploadPart to upload each part.
Each part upload request must specify an uploadId and a partNumber. The partNumber value must be in the range of 1 to 10,000. If the partNumber is outside this range, OSS returns the InvalidArgument error code.
When you call uploadPart, all parts except for the last one must be larger than 100 KB. OSS checks the part size only when you complete the multipart upload.
Before you upload each part, you must position the stream to the beginning of the data for that part.
After each part is uploaded, the OSS response contains an ETag value for that part. The ETag value is the MD5 hash of the part data. You must combine the ETag value and the part number into a PartETag and save it to complete the multipart upload.
Multipart upload of a local file
You can upload a local file to OSS using a synchronous or asynchronous multipart upload.
The previous section provides a complete example that shows the multipart upload process step-by-step. The code in this section for uploading a local file encapsulates that process. You only need to use MultipartUploadRequest to perform a multipart upload.
Upload a local file using the synchronous multipart upload operation
The following code shows how to synchronously upload the examplefile.txt file to the exampleobject.txt object in the exampledir folder of the examplebucket bucket.
// Specify the bucket name, for example, examplebucket. String bucketName = "examplebucket"; // Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. String objectName = "exampledir/exampleobject.txt"; // Specify the full path of the local file, for example, /storage/emulated/0/oss/examplefile.txt. String localFilepath = "/storage/emulated/0/oss/examplefile.txt"; ObjectMetadata meta = new ObjectMetadata(); // Set file metadata. meta.setHeader("x-oss-object-acl", "public-read-write"); MultipartUploadRequest rq = new MultipartUploadRequest(bucketName, objectName, localFilepath, meta); // Set the part size. The default part size is 256 KB. The minimum part size is 100 KB. rq.setPartSize(1024 * 1024); rq.setProgressCallback(new OSSProgressCallback<MultipartUploadRequest>() { @Override public void onProgress(MultipartUploadRequest request, long currentSize, long totalSize) { OSSLog.logDebug("[testMultipartUpload] - " + currentSize + " " + totalSize, false); } }); CompleteMultipartUploadResult result = oss.multipartUpload(rq);For scoped storage on Android 10 or later, you can use the URI of a file to upload the file to OSS.
// Specify the bucket name, for example, examplebucket. String bucketName = "examplebucket"; // Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. String objectName = "exampledir/exampleobject.txt"; ObjectMetadata meta = new ObjectMetadata(); // Set file metadata. meta.setHeader("x-oss-object-acl", "public-read-write"); MultipartUploadRequest rq = new MultipartUploadRequest(bucketName, objectName, fileUri, meta); // Set the part size. The default part size is 256 KB. The minimum part size is 100 KB. rq.setPartSize(1024 * 1024); rq.setProgressCallback(new OSSProgressCallback<MultipartUploadRequest>() { @Override public void onProgress(MultipartUploadRequest request, long currentSize, long totalSize) { OSSLog.logDebug("[testMultipartUpload] - " + currentSize + " " + totalSize, false); } }); CompleteMultipartUploadResult result = oss.multipartUpload(rq);Call an asynchronous API to perform a multipart upload of a local file
The following code shows how to asynchronously upload the examplefile.txt file to the exampledir folder in the examplebucket bucket, saving it as the exampleobject.txt object.
// Specify the bucket name, for example, examplebucket. String bucketName = "examplebucket"; // Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. String objectName = "exampledir/exampleobject.txt"; // Specify the full path of the local file, for example, /storage/emulated/0/oss/examplefile.txt. String localFilepath = "/storage/emulated/0/oss/examplefile.txt"; MultipartUploadRequest request = new MultipartUploadRequest(bucketName, objectName, localFilepath); request.setProgressCallback(new OSSProgressCallback<MultipartUploadRequest>() { @Override public void onProgress(MultipartUploadRequest request, long currentSize, long totalSize) { OSSLog.logDebug("[testMultipartUpload] - " + currentSize + " " + totalSize, false); } }); OSSAsyncTask task = oss.asyncMultipartUpload(request, new OSSCompletedCallback<MultipartUploadRequest, CompleteMultipartUploadResult>() { @Override public void onSuccess(MultipartUploadRequest request, CompleteMultipartUploadResult result) { OSSLog.logInfo(result.getServerCallbackReturnBody()); } @Override public void onFailure(MultipartUploadRequest request, ClientException clientException, ServiceException serviceException) { OSSLog.logError(serviceException.getRawMessage()); } }); //Thread.sleep(100); // Cancel the multipart upload. //task.cancel(); task.waitUntilFinished();For scoped storage on Android 10 or later, you can use the URI of a file to upload the file to OSS.
// Specify the bucket name, for example, examplebucket. String bucketName = "examplebucket"; // Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name. String objectName = "exampledir/exampleobject.txt"; MultipartUploadRequest request = new MultipartUploadRequest(bucketName, objectName, fileUri); request.setProgressCallback(new OSSProgressCallback<MultipartUploadRequest>() { @Override public void onProgress(MultipartUploadRequest request, long currentSize, long totalSize) { OSSLog.logDebug("[testMultipartUpload] - " + currentSize + " " + totalSize, false); } }); OSSAsyncTask task = oss.asyncMultipartUpload(request, new OSSCompletedCallback<MultipartUploadRequest, CompleteMultipartUploadResult>() { @Override public void onSuccess(MultipartUploadRequest request, CompleteMultipartUploadResult result) { OSSLog.logInfo(result.getServerCallbackReturnBody()); } @Override public void onFailure(MultipartUploadRequest request, ClientException clientException, ServiceException serviceException) { OSSLog.logError(serviceException.getRawMessage()); } }); //Thread.sleep(100); // Cancel the multipart upload. //task.cancel(); task.waitUntilFinished();
List uploaded parts
Call the oss.listParts method to obtain all uploaded parts for a specific multipart upload event.
The following code shows how to list uploaded parts.
// Specify the bucket name, for example, examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name.
String objectName = "exampledir/exampleobject.txt";
// Specify the uploadId. The uploadId is obtained from the response after you call InitiateMultipartUpload to initialize the multipart upload and before you call CompleteMultipartUpload to complete the multipart upload.
String uploadId = "0004B999EF518A1FE585B0C9****";
// List parts.
ListPartsRequest listParts = new ListPartsRequest(bucketName, objectName, uploadId);
ListPartsResult result = oss.listParts(listParts);
List<PartETag> partETagList = new ArrayList<PartETag>();
for (PartSummary part : result.getParts()) {
partETagList.add(new PartETag(part.getPartNumber(), part.getETag()));
}By default, if a bucket contains more than 1,000 parts that are uploaded by using multipart upload, OSS returns the first 1,000 parts. In the response, the value of the IsTruncated field is false and NextPartNumberMarker is returned to indicate the start position of the next list operation.
Cancel a multipart upload event
Call the oss.abortMultipartUpload method to cancel a multipart upload event. After a multipart upload event is canceled, you can no longer use its uploadId for any operations. The uploaded parts are deleted.
The following code shows how to cancel a multipart upload event.
// Specify the bucket name, for example, examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path of the object cannot contain the bucket name.
String objectName = "exampledir/exampleobject.txt";
// Specify the uploadId. The uploadId is obtained from the response after you call InitiateMultipartUpload to initialize the multipart upload.
String uploadId = "0004B999EF518A1FE585B0C9****";
// Cancel the multipart upload.
AbortMultipartUploadRequest abort = new AbortMultipartUploadRequest(bucketName, objectName, uploadId);
AbortMultipartUploadResult abortResult = oss.abortMultipartUpload(abort);References
For the complete sample code for multipart upload, see GitHub.
A multipart upload involves three API operations. For more information about the operations, see the following topics:
For more information about the API operation used to cancel a multipart upload event, see AbortMultipartUpload.
For more information about the API operation used to list uploaded parts, see ListParts.
For more information about the API operation used to list all in-progress multipart upload events (events that are initiated but not yet completed or canceled), see ListMultipartUploads.
For more information about how to initialize an OSSClient instance, see Initialize an OSSClient instance for Android.