Multipart upload

Last Updated: Oct 25, 2017

This document describes how to upload a file through Multipart Upload.

Initialize multipart upload

  1. String uploadId;
  2. InitiateMultipartUploadRequest init = new InitiateMultipartUploadRequest("<bucketName>", "<objectKey>");
  3. InitiateMultipartUploadResult initResult = oss.initMultipartUpload(init);
  4. uploadId = initResult.getUploadId();
  • InitiateMultipartUploadRequest specifies the name and bucket of the file to be uploaded.

  • In InitiateMultipartUploadRequest, you can set ObjectMeta, but do not need to specify the ContentLength.

  • The result returned by InitiateMultipartUpload includes the UploadID, which uniquely identifies a multipart upload task and is used in subsequent operations.

Upload parts

  1. long partSize = 128 * 1024; // Set the part size
  2. int currentIndex = 1; // The upload part number starting from 1
  3. File uploadFile = new File("<uploadFilePath>"); // The file to be uploaded in parts
  4. InputStream input = new FileInputStream(uploadFile);
  5. long fileLength = uploadFile.length();
  6. long uploadedLength = 0;
  7. List<PartETag> partETags = new ArrayList<PartETag>(); // Save the multipart upload result
  8. while (uploadedLength < fileLength) {
  9. int partLength = (int)Math.min(partSize, fileLength - uploadedLength);
  10. byte[] partData = IOUtils.readStreamAsBytesArray(input, partLength); // Read a portion of the file according to the part size
  11. UploadPartRequest uploadPart = new UploadPartRequest("<bucketName>", "<objectKey>", uploadId, currentIndex);
  12. uploadPart.setPartContent(partData); // Set the part content
  13. UploadPartResult uploadPartResult = oss.uploadPart(uploadPart);
  14. partETags.add(new PartETag(currentIndex, uploadPartResult.getETag())); // Save the results returned after successful multipart upload
  15. uploadedLength += partLength;
  16. currentIndex++;
  17. }

When you use the preceding code to upload each part through UploadPart, consider the following points:

  • The UploadPart method requires that all parts except the last one be larger than 100 KB. However, the Upload Part interface does not check the uploaded part size immediately, because it does not know whether the part is the last one. It checks the size only when multipart upload is completed.

  • OSS inserts the MD5 value of the part data that the server receives to the ETag header and return it to the user.

  • The part number ranges from 1 to 10,000. If the part number exceeds this range, the OSS returns the InvalidArgument error code.

  • When each part is uploaded, the stream is directed to the start point of the part.

  • After each part is uploaded, the result returned by OSS contains an ETag of the part. You must combine the ETag with the part number to form a partEtag and save it to the list, which is used when multipart upload is completed.

Complete multipart upload

  1. CompleteMultipartUploadRequest complete = new CompleteMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId, partETags);
  2. CompleteMultipartUploadResult completeResult = oss.completeMultipartUpload(complete);
  3. Log.d("multipartUpload", "multipart upload success! Location: " + completeResult.getLocation());

In the preceding code, the PartETags are saved to the PartETags list during multipart upload. Once OSS receives the PartETags list submitted by the user, it checks the validity of each data part. When all the data parts are verified, OSS combines these parts into a complete object.

Complete multipart upload (set “Server Callback”)

  1. CompleteMultipartUploadRequest complete = new CompleteMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId, partETags);
  2. CompleteMultipartUploadResult completeResult = oss.completeMultipartUpload(complete);
  3. complete.setCallbackParam(new HashMap<String, String>() {
  4. {
  5. put("callbackUrl", "<server address>");
  6. put("callbackBody", "<test>");
  7. }
  8. });
  9. Log.d("multipartUploadWithServerCallback", completeResult.getServerCallbackReturnBody());

You can set the Server Callback parameter to complete the multipart upload request. Upon completion, a callback request is sent to the specified server address.

Delete a multipart upload event

The following code uses the abortMultipartUpload method to cancel a multipart upload task.

  1. AbortMultipartUploadRequest abort = new AbortMultipartUploadRequest("<bucketName>", "<objectKey>", uploadId);
  2. oss.abortMultipartUpload(abort); // If no exception is thrown, it indicates the multipart upload task is successfully deleted.

List parts

The following code uses the listParts method to get all the uploaded parts of an uploading task.

  1. ListPartsRequest listParts = new ListPartsRequest("<bucketName>", "<objectKey>", uploadId);
  2. ListPartsResult result = oss.listParts(listParts);
  3. for (int i = 0; i < result.getParts().size(); i++) {
  4. Log.d("listParts", "partNum: " + result.getParts().get(i).getPartNumber());
  5. Log.d("listParts", "partEtag: " + result.getParts().get(i).getETag());
  6. Log.d("listParts", "lastModified: " + result.getParts().get(i).getLastModified());
  7. Log.d("listParts", "partSize: " + result.getParts().get(i).getSize());
  8. }
  • By default, if a bucket contains more than 1,000 multipart upload tasks, only the information of the first 1,000 multipart upload tasks is returned and the IsTruncated parameter in the returned results be false. The NextPartNumberMarker is returned to serve as the start point of next read.

  • To increase the quantity of returned multipart upload tasks, you can modify the MaxParts parameter or use the PartNumberMarker parameter for multiple accesses.

Thank you! We've received your feedback.