Multipart upload

Last Updated: Sep 27, 2017

The following describes how to upload a file through Multipart Upload.

Initialize multipart upload

  1. String uploadId;
  2. InitiateMultipartUploadRequest init = new InitiateMultipartUploadRequest("<bucketName>", "<objectKey>");
  3. InitiateMultipartUploadResult initResult = oss.initMultipartUpload(init);
  4. uploadId = initResult.getUploadId();
  • InitiateMultipartUploadRequest is used to specify the name and bucket of the file to be uploaded.
  • In InitiateMultipartUploadRequest, you can set ObjectMeta, but do not need to specify the ContentLength.
  • The result returned by InitiateMultipartUpload includes the UploadID which uniquely identifies a multipart upload task and will be used in subsequent operations.

Upload parts

  1. long partSize = 128 * 1024; // Set the part size
  2. int currentIndex = 1; // The upload part number starting from 1
  3. File uploadFile = new File("<uploadFilePath>"); // The file to be uploaded in parts
  4. InputStream input = new FileInputStream(uploadFile);
  5. long fileLength = uploadFile.length();
  6. long uploadedLength = 0;
  7. List<PartETag> partETags = new ArrayList<PartETag>(); // Save the multipart upload result
  8. while (uploadedLength < fileLength) {
  9. int partLength = (int)Math.min(partSize, fileLength - uploadedLength);
  10. byte[] partData = IOUtils.readStreamAsBytesArray(input, partLength); // Read a portion of the file according to the part size
  11. UploadPartRequest uploadPart = new UploadPartRequest("<bucketName>", "<objectKey>", uploadId, currentIndex);
  12. uploadPart.setPartContent(partData); // Set the part content
  13. UploadPartResult uploadPartResult = oss.uploadPart(uploadPart);
  14. partETags.add(new PartETag(currentIndex, uploadPartResult.getETag())); // Save the results returned after successful multipart upload
  15. uploadedLength += partLength;
  16. currentIndex++;
  17. }

When you use the above code to upload each part through UploadPart, note the following points:

  • The UploadPart method requires that all parts except the last one be larger than 100 KB. However, the Upload Part interface does not check the uploaded part size immediately, because it does not know whether the part is the last one. It checks the size only when multipart upload is completed.
  • OSS will insert the MD5 value of the part data that the server receives to the ETag header and return it to the user.
  • The part number ranges from 1 to 10,000. If the part number exceeds this range, the OSS will return the InvalidArgument error code.
  • When each part is uploaded, the stream is directed to the start point of the part.
  • After each part is uploaded, the result returned by OSS will contain an ETag of the part. You need to combine the ETag with the part number to form a partEtag and save it to the list, which will be used when multipart upload is completed.

Complete multipart upload

  1. CompleteMultipartUploadRequest complete = new CompleteMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId, partETags);
  2. CompleteMultipartUploadResult completeResult = oss.completeMultipartUpload(complete);
  3. Log.d("multipartUpload", "multipart upload success! Location: " + completeResult.getLocation());

In the above code, the PartETags are saved to the PartETags list during multipart upload. After OSS receives the PartETags list submitted by the user, it will check the validity of each data part. After all data parts are verified, OSS will combine these parts into a complete object.

Complete multipart upload (set “Server Callback”)

  1. CompleteMultipartUploadRequest complete = new CompleteMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId, partETags);
  2. CompleteMultipartUploadResult completeResult = oss.completeMultipartUpload(complete);
  3. complete.setCallbackParam(new HashMap<String, String>() {
  4. {
  5. put("callbackUrl", "<server address>");
  6. put("callbackBody", "<test>");
  7. }
  8. });
  9. Log.d("multipartUploadWithServerCallback", completeResult.getServerCallbackReturnBody());

You can set the Server Callback parameter to complete the multipart upload request. Upon completion, a callback request will be sent to the specified server address.

Delete a multipart upload event

The following code uses the abortMultipartUpload method to cancel a multipart upload task.

  1. AbortMultipartUploadRequest abort = new AbortMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId);
  2. oss.abortMultipartUpload(abort); // If no exception is thrown, it indicates the multipart upload task is successfully deleted.

List parts

The following code uses the listParts method to get all the uploaded parts of an uploading task.

  1. ListPartsRequest listParts = new ListPartsRequest("<bucketName>", "<objectKey>", uploadId);
  2. ListPartsResult result = oss.listParts(listParts);
  3. for (int i = 0; i < result.getParts().size(); i++) {
  4. Log.d("listParts", "partNum: " + result.getParts().get(i).getPartNumber());
  5. Log.d("listParts", "partEtag: " + result.getParts().get(i).getETag());
  6. Log.d("listParts", "lastModified: " + result.getParts().get(i).getLastModified());
  7. Log.d("listParts", "partSize: " + result.getParts().get(i).getSize());
  8. }
  • By default, if a bucket contains more than 1,000 multipart upload tasks, only the information of the first 1,000 multipart upload tasks will be returned and the IsTruncated parameter in the returned results will be false. The NextPartNumberMarker will be returned to serve as the start point of next read.
  • To increase the quantity of returned multipart upload tasks, you can modify the MaxParts parameter or use the PartNumberMarker parameter for multiple accesses.
Thank you! We've received your feedback.