Upload objects

Last Updated: Oct 20, 2017

In OSS, objects are the basic data units for user operations. OSS Java SDK provides a variety of object uploading interfaces. You can upload an object by any of the following methods:

  • Stream upload
  • Object upload
  • Append upload
  • Multipart upload
  • Resumable upload

The size of objects for stream upload, object upload and append upload operations may not exceed 5 GB. When the object size is large, use the multipart upload method. The size of parts for multipart upload may not exceed 48.8 TB.

Resumable upload supports concurrent resumable uploads and customized sizes of parts. It is the encapsulated and enhanced version of the multipart upload, and is recommended for uploading objects.

Simple upload

Stream upload and object upload are called simple upload. Stream upload uses the InputStream as the object data source. Object upload uses the local files as the object data source.

Note: Complete simple upload code can be found at GitHub.

Stream upload

You can use OSSClient.putObject to upload your data streams to OSS.

Upload strings

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Upload the string
  9. String content = "Hello OSS";
  10. ossClient.putObject("<yourBucketName>", "<yourKey>", new ByteArrayInputStream(content.getBytes()));
  11. // Disable the client
  12. ossClient.shutdown();

Upload byte arrays

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId,accessKeySecret);
  8. // Upload
  9. byte[] content = "Hello OSS".getBytes();
  10. ossClient.putObject("<yourBucketName>", "<yourKey>", new ByteArrayInputStream(content));
  11. // Disable the client
  12. ossClient.shutdown();

Upload network streams

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Upload
  9. InputStream inputStream = new URL("https://www.aliyun.com/").openStream();
  10. ossClient.putObject("<yourBucketName>", "<yourKey>", inputStream);
  11. // Disable the client
  12. ossClient.shutdown();

Upload object streams

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Upload the object stream
  9. InputStream inputStream = new FileInputStream("localFile");
  10. ossClient.putObject("<yourBucketName>", "<yourKey>", inputStream);
  11. // Disable the client
  12. ossClient.shutdown();

Upload local files

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Upload a file
  9. ossClient.putObject("<yourBucketName>", "<yourKey>", new File("localFile"));
  10. // Disable the client
  11. ossClient.shutdown();

Set the metadata

Object Meta describes the attributes of objects uploaded to OSS. These attributes come in two types: HTTP standard attributes (HTTP Headers) and user-defined metadata (User Meta). The object metadata can be set during various upload modes (stream upload, object upload, append upload, multipart upload and resumable upload) or object copy operations. The names of metadata are case-insensitive. For more information on object metadata, see Object Meta.

Set the HTTP Header

The OSS allows you to customize HTTP headers. For more information on HTTP headers, see RFC2616. Several frequently-used HTTP headers are described as follows:

Name Description Default value
Content-MD5 Object data verification. Once this value is set, the OSS enables the MD5 verification on the object content and compare the MD5 you provide with the MD5 of the object. An error is thrown if any inconsistency is found. None
Content-Type The object MIME. It defines the object type and webpage encoding and decides the form and encoding used by the browser to read objects. If not specified, the object MIME is generated based on the key or object name extension. If no extension is available, the default value applies. application/octet-stream
Content-Disposition It instructs the MIME user proxy how to display, open, or download the attached object and the object name. None
Content-Length The length of the uploaded object. The excessive length of the stream/object is truncated. If the data falls short of the length, the actual data length applies. The time length of the stream/object
Expires The expiration time of cache. It is not used in the OSS. The time format is in GMT. None
Cache-Control It specifies the cache action of the webpage when the object is downloaded. None
  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String content = "Hello OSS";
  7. // Create the metadata for the uploaded object
  8. ObjectMetadata meta = new ObjectMetadata();
  9. // Set the length of the uploaded object
  10. meta.setContentLength(content.length());
  11. // Set the MD5 verification for uploaded objects
  12. String md5 = BinaryUtil.toBase64String(BinaryUtil.calculateMd5(content.getBytes()));
  13. meta.setContentMD5(md5);
  14. // Set the type of the uploaded content
  15. meta.setContentType("text/plain");
  16. // Create an OSSClient instance
  17. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  18. // Upload a file
  19. ossClient.putObject("<yourBucketName>", "<yourKey>", new ByteArrayInputStream(content.getBytes()), meta);
  20. // Disable the client
  21. ossClient.shutdown();

Note:

  • ObjectMetadata provides frequently-used HTTP Header settings, such as Content-MD5, Content-Type, Content-Length, Content-Disposition, Content-Encoding, Expires, and x-oss-server-side-encryption;

  • Use ObjectMetadata.setHeader(String key, Object value) to set the header.

User-defined metadata

OSS allows you to customize object metadata to describe objects.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String content = "Hello OSS";
  7. // Create an OSSClient instance
  8. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  9. // Create the metadata for the uploaded object
  10. ObjectMetadata meta = new ObjectMetadata();
  11. // Set the name value of UserMetadata to "my-data"
  12. meta.addUserMetadata("property", "property-value");
  13. // Upload a file
  14. ossClient.putObject("<yourBucketName>", "<yourKey>", new ByteArrayInputStream(content.getBytes()), meta);
  15. // Disable the client
  16. ossClient.shutdown();

Note:

  • The preceding code defines a UserMetadata item with its name being “property” and its value being “property-value”.

  • You can get the object metadata through OSSClient.getObjectMetadata.

  • The object metadata is downloaded along with the object.

  • An object may contain multiple metadata items, but the total size of metadata must not exceed 8 KB.

Create a simulated folder

The OSS does not use folders. All elements are stored as objects. Creating a simulated folder is in fact creating an object with zero size. This object can be uploaded and downloaded, but the OSS console displays the object ended with a slash as a folder.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. final String keySuffixWithSlash = "parent_directory/";
  9. ossClient.putObject("<bucketName>", keySuffixWithSlash, new ByteArrayInputStream(new byte[0]));
  10. // Disable the client
  11. ossClient.shutdown();

Note:

  • Creating a simulated folder is equivalent to creating an object whose name ends with a slash (/).

  • This object can be uploaded and downloaded as a normal file, but the OSS console displays the object ended with a slash as a folder.

  • For a multi-level directory, you only need to create the last level. For example, for “dir1/dir2/dir3/“, you only need to create “dir1/dir2/dir3/“, and do not need to create “dir1/“ and “dir1/dir2/“ directories.

  • For more information, see Object Management.

Append upload

Objects created by simple upload, multipart upload, and resumable upload operations are all Normal objects, which have fixed content after the upload. They can only be read, but cannot be modified. If the object content changes, you can only upload an object of the same name to overwrite the old content. This is a major difference between OSS and general file systems.

This feature leads to inconvenience in many scenarios, typical ones being video monitoring and live video fields, since video data is constantly produced in real time.

OSS allows you to directly append content to the end of an object using the Append Object method. Objects uploaded by this method are Appendable objects, while objects uploaded via other methods are Normal objects. The data appended is instantly readable.

You can use OSSClient.appendObject to upload an object in the append upload mode.

  1. AppendObjectResult appendObject(AppendObjectRequest appendObjectRequest)

Configurable parameters of the AppendObjectRequest interface include the following:

Parameter Role Method
BucketName Bucket name setBucketName(String bucketName)
Key Object name setKey(String key)
InputStream Content to be appended, InputStream or File setInputStream(InputStream inputStream)
File Content to be appended, InputStream or File setFile(File file)
ObjectMetadata Specify the object metadata. Valid at the first append setMetadata(ObjectMetadata metadata)
Position The position to append the object setPosition(Long position)

Parameters of AppendObjectResult are as follows:

Parameter Meaning Method
nextPosition Specify the position that is provided in the next request. It is in fact the current object length. Long getNextPosition()
objectCRC64 The 64-bit CRC value of the object. This 64-bit CRC is computed according to ECMA-182 String getObjectCRC64()

Note: Complete append upload code can be found at GitHub.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String content = "Hello OSS";
  7. // Create an OSSClient instance
  8. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  9. AppendObjectRequest appendObjectRequest = new AppendObjectRequest("<yourBucketName>",
  10. "<yourKey>", new ByteArrayInputStream(content.getBytes()));
  11. // The first append
  12. appendObjectRequest.setPosition(0L);
  13. AppendObjectResult appendObjectResult = ossClient.appendObject(appendObjectRequest);
  14. // The second append
  15. appendObjectRequest.setPosition(appendObjectResult.getNextPosition());
  16. appendObjectResult = ossClient.appendObject(appendObjectRequest);
  17. // The third append
  18. appendObjectRequest.setPosition(appendObjectResult.getNextPosition());
  19. appendObjectResult = ossClient.appendObject(appendObjectRequest);
  20. // Disable the client
  21. ossClient.shutdown();

Note:

  • There is no ceiling on the number of append uploads. The maximum object size is 5 GB. You can use multipart upload to upload larger files.

  • Objects uploaded by Append Object do not support copyObject operations for the moment.

Resumable upload

If the network jitters or the program crashes when a large object is being uploaded, the whole upload operation fails. You have to re-upload the objects by wasting resources. When the network is unstable, you may have to try multiple times.

You can use the OSSClient.uploadFile interface to implement resumable upload. The parameter is UploadFileRequest and the request has the following parameters:

  • bucket: Name of the bucket. A required parameter. It is set through the constructor.

  • Key: Name of the object uploaded to OSS. A required parameter. It is set through the constructor.

  • uploadFile: Name of the local file to be uploaded. A required parameter. It is set through the constructor or setUploadFile.

  • partSize: Part size from 100 KB to 5 GB. Unit: bytes. An optional parameter. The default value is 100K. It is set through setPartSize.

  • taskNum: The number of concurrent parts for uploads. An optional parameter. The default value is 1. It is set through setTaskNum.

  • enableCheckpoint: Whether to enable resumable upload. An optional parameter. The resumable download feature is disabled by default. It is set through setEnableCheckpoint.

  • checkpointFile: When resumable upload is enabled, you must record the multipart upload result in the local file. If the upload fails, uploaded parts are not uploaded again. An optional parameter. By default, it is in the same directory as the local file to be uploaded, that is, uploadFile.ucp. You can set it through setCheckpointFile.

  • objectMetadata: Object metadata. An optional parameter. You can set it through setObjectMetadata.

  • callback: Callback after successful uploads. An optional parameter. You can set it through setCallback.

The principle is to divide the object to be uploaded into multiple parts and upload them separately. When all the parts are uploaded, the upload of the entire object is completed.

The progress of the current upload is recorded during the uploading process (in the checkpoint object). If the upload of any part fails during the process, the next upload attempt starts from the recorded position in the checkpoint object. This requires that the same checkpoint object with the previous upload should be used in the next call. When the upload is completed, the checkpoint object gets deleted.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Set the resumable upload request
  9. UploadFileRequest uploadFileRequest = new UploadFileRequest("<yourBucketName>", "<yourKey>");
  10. // Specify the local file to be uploaded
  11. uploadFileRequest.setUploadFile("<yourLocalFile>");
  12. // Specify the number of concurrent threads of the upload
  13. uploadFileRequest.setTaskNum(5);
  14. // Specify the part size of the upload
  15. uploadFileRequest.setPartSize(1 * 1024 * 1024);
  16. // Enable resumable upload
  17. uploadFileRequest.setEnableCheckpoint(true);
  18. // Resumable upload
  19. ossClient.uploadFile(uploadFileRequest);
  20. // Disable the client
  21. ossClient.shutdown();

Note:

  • Resumable upload is the encapsulated and enhanced version of multipart upload. It is implemented through multipart upload.

  • Multipart upload is recommended when the object is large or the network environment is poor.

  • Resumable upload supports specifying Object Metadata and callback after the upload.

Multipart upload

You can split a large file into multiple parts for upload. You can apply the multipart upload mode in the following scenarios (but not limited to the following):

  • Resumable upload is required.

  • Upload an object larger than 100 MB.

  • The network conditions are poor and the OSS server is frequently disconnected.

  • The size of the object to be uploaded is unknown.

Multipart upload can be divided into three steps as follows:

  • Initialize a multipart upload task (InitiateMultipartUpload).

  • Upload the parts one by one in a sequence or in parallel (UploadPart).

  • Complete the multipart upload (CompleteMultipartUpload) or cancel the multipart upload (AbortMultipartUpload).

Complete multipart upload in steps

Note: Complete multipart upload code can be found at GitHub.

Initialize multipart upload

Before transmitting data in multipart upload mode, you must notify the OSS to initialize a multipart upload event. The operation returns a globally unique Upload ID created by the OSS server to identify this multipart upload event. You can initiate related operations based on this ID, such as aborting multipart upload and querying multipart upload.

The following code calls OSSClient.initiateMultipartUpload to initialize a multipart upload task:

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String bucketName = "<yourBucketName>";
  7. String key = "yourKey";
  8. // Create an OSSClient instance
  9. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  10. InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, key);
  11. InitiateMultipartUploadResult result = ossClient.initiateMultipartUpload(request);
  12. String uploadId = result.getUploadId();

Note:

  • InitiateMultipartUploadRequest is used to specify the name and bucket of the object to be uploaded.

  • In InitiateMultipartUploadRequest, you can set ObjectMeta.

  • The results returned by InitiateMultipartUpload include the UploadID, which uniquely identifies a multipart upload task and is used in subsequent operations.

Multipart upload

After initiating a multipart upload task, you can upload data in parts based on the specified object name and Upload ID. Each uploaded part has a part number ranging from 1 to 10,000. For the same Upload ID, this part number identifies not only this part of the data but also the relative location of this part in the entire object. If you upload new data using the same part number, OSS overwrites the existing data identified by this part number.

Except the last part, the minimum size of other parts is 100 KB. There are no restrictions on the size of the last part. Parts are not necessarily uploaded in order. They can be uploaded in different processes from different servers. The OSS composes the large object by sorting the parts based on their part numbers.

The following code calls OSSClient.uploadPart to upload parts:

  1. List<PartETag> partETags = new ArrayList<PartETag>();
  2. InputStream instream = new FileInputStream(new File("<localFile>"));
  3. UploadPartRequest uploadPartRequest = new UploadPartRequest();
  4. uploadPartRequest.setBucketName(bucketName);
  5. uploadPartRequest.setKey(key);
  6. uploadPartRequest.setUploadId(uploadId);
  7. uploadPartRequest.setInputStream(instream);
  8. // Set the part size. Except the last part, the size of all other parts must be larger than 100 KB.
  9. uploadPartRequest.setPartSize(100 * 1024);
  10. // Set the part number. The range is 1 - 10,000.
  11. uploadPartRequest.setPartNumber(1);
  12. UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
  13. partETags.add(uploadPartResult.getPartETag());

Note:

  • The UploadPart method requires all parts except the last one be larger than 100 KB. However, the Upload Part interface does not check the uploaded part size immediately because it does not know whether the part is the last one. It checks the size only when multipart upload is completed.

  • The OSS inserts the MD5 value of the part data that the server receives to the ETag header and return it to the user.

  • Make sure that the data transmitted over the network is free of errors, the SDK automatically sets Content-MD5. OSS calculates the MD5 value of the uploaded data and compares it with the MD5 value calculated by the SDK. If they are inconsistent, the system returns the InvalidDigest error code.

  • The part number ranges from 1 to 10,000. If the part number exceeds this range, OSS returns the InvalidArgument error code.

  • When each part is uploaded, the stream is directed to the start point of the uploaded part.

  • After each part is uploaded, the returned result of OSS contains a PartETag which combines the ETag and PartNumber of the uploaded part.

  • You must save the PartETag, which is used for completing the multipart upload task. Generally, we save these PartETag objects in the List.

Complete multipart upload

After uploading all data parts, you can call the Complete Multipart Upload API to complete the multipart upload operation for the entire object.

During execution of this operation, you must provide a list of all the valid parts (including the part number and part ETag). After the OSS receives the submitted part list, it verifies the validity of each part one by one. After all the data parts have been verified, OSS combines these parts into a complete object.

The following code calls OSSClient.completeMultipartUpload to complete a multipart upload task:

  1. Collections.sort(partETags, new Comparator<PartETag>() {
  2. @Override
  3. public int compare(PartETag p1, PartETag p2) {
  4. return p1.getPartNumber() - p2.getPartNumber();
  5. }
  6. });
  7. CompleteMultipartUploadRequest completeMultipartUploadRequest =
  8. new CompleteMultipartUploadRequest(bucketName, key, uploadId, partETags);
  9. ossClient.completeMultipartUpload(completeMultipartUploadRequest);

Note:

  • The partETags in the preceding code are the list of partETags saved during multipart upload. It must be in ascending sort by part number.

  • Parts can be discontinuous.

Cancel a multipart upload task

This interface can abort a multipart upload event based on the Upload ID. When a multipart upload event is aborted, you cannot use this Upload ID to perform any operations and data of the uploaded parts is deleted.

The following code calls OSSClient.abortMultipartUpload to cancel a multipart upload task:

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // Upload the string
  9. String content = "Hello OSS";
  10. ossClient.putObject("<yourBucketName>", "<yourKey>", new ByteArrayInputStream(content.getBytes()));
  11. // Cancel the multipart upload. The uploadId is from initiateMultipartUpload
  12. AbortMultipartUploadRequest abortMultipartUploadRequest =
  13. new AbortMultipartUploadRequest("<yourBucketName>", "<yourKey>", "<uploadId>");
  14. ossClient.abortMultipartUpload(abortMultipartUploadRequest);
  15. // Disable the client
  16. ossClient.shutdown();

Get uploaded parts

The get uploaded parts operation can list all successfully uploaded parts mapped to a specified Upload ID.You can call OSSClient.listParts to retrieve all uploaded parts of a multipart upload task. Configurable parameters of the listParts interface include the following:

Parameter Role Method
UploadId Upload ID, obtained from the returned result of initiateMultipartUpload. ListPartsRequest.setUploadId(String uploadId)
MaxParts The maximum number of parts in the OSS response, that is, the number of parts in each page. ListPartsRequest.setMaxParts(int maxParts)
PartNumberMarker Starting position in a specific list. Only the parts with a part number greater than the value of this parameter is listed. ListPartsRequest.setPartNumberMarker(Integer partNumberMarker)

Simple list

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // List uploaded parts. The uploadId is from initiateMultipartUpload
  9. ListPartsRequest listPartsRequest = new ListPartsRequest("<yourBucketName>", "<yourKey>", "<uploadId>");
  10. PartListing partListing = ossClient.listParts(listPartsRequest);
  11. for (PartSummary part : partListing.getParts()) {
  12. // Part number. It is specified during the upload
  13. part.getPartNumber();
  14. // Size of the part data
  15. part.getSize();
  16. // Part ETag
  17. part.getETag();
  18. // The last modification time to the part
  19. part.getLastModified();
  20. }
  21. // Disable the client
  22. ossClient.shutdown();

Note:

  • By default, if a bucket contains more than 1,000 multipart upload tasks, only the information of the first 1,000 multipart upload tasks is returned and the IsTruncated parameter in the returned results is false. The NextPartNumberMarker is returned to serve as the start point of next read.

  • Data parts can be retrieved by page if they cannot be retrieved all at once.

Obtain all uploaded parts

By default, listParts only lists 1,000 parts. If the number of parts exceeds 1,000, you can see the example that follows, to list all the parts.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // List all uploaded parts
  9. PartListing partListing;
  10. ListPartsRequest listPartsRequest = new ListPartsRequest("<yourBucketName>", "<yourKey>", "<uploadId>");
  11. do {
  12. partListing = ossClient.listParts(listPartsRequest);
  13. for (PartSummary part : partListing.getParts()) {
  14. // Part number. It is specified during the upload
  15. part.getPartNumber();
  16. // Size of the part data
  17. part.getSize();
  18. // Part ETag
  19. part.getETag();
  20. // Upload the last modification to the part
  21. part.getLastModified();
  22. }
  23. listPartsRequest.setPartNumberMarker(partListing.getNextPartNumberMarker());
  24. } while (partListing.isTruncated());
  25. // Disable the client
  26. ossClient.shutdown()

Obtain all parts by page

By default, listParts can list 1,000 parts at a time. The preceding “obtain all uploaded parts” operation is a special case of listing parts by page. Each page contains 1,000 parts. If you want to specify the number of parts for each page, that is, the number of parts in one list, see the following code:

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. // Create an OSSClient instance
  7. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  8. // List uploaded parts by page
  9. PartListing partListing;
  10. ListPartsRequest listPartsRequest = new ListPartsRequest("<yourBucketName>", "<yourKey>", "<uploadId>");
  11. // 100 parts for each page
  12. listPartsRequest.setMaxParts(100);
  13. do {
  14. partListing = ossClient.listParts(listPartsRequest);
  15. for (PartSummary part : partListing.getParts()) {
  16. // Part number. It is specified during the upload
  17. part.getPartNumber();
  18. // Size of the part data
  19. part.getSize();
  20. // Part ETag
  21. part.getETag();
  22. // Upload the last modification to the part
  23. part.getLastModified();
  24. }
  25. listPartsRequest.setPartNumberMarker(partListing.getNextPartNumberMarker());
  26. } while (partListing.isTruncated());
  27. // Disable the client
  28. ossClient.shutdown();

Obtain all multipart upload tasks in a bucket

The List Multipart Uploads interface can be used to list all multipart upload events in execution, that is, multipart upload events that have been initiated but not completed or aborted. Configurable parameters of the List Multipart Uploads interface include the following:

Parameter Role Method
Prefix Specify the prefix that the names of returned objects must contain. Note that when you make a query using the prefix, the returned object name still contains the prefix. ListMultipartUploadsRequest.setPrefix(String prefix)
Delimiter A character to group object names. All those objects whose name contain the specified prefix and are between the delimiter for the first time form a group of elements. ListMultipartUploadsRequest.setDelimiter(String delimiter)
MaxUploads The maximum number of multipart upload events returned for one request. The default value is 1,000. The max-uploads value cannot exceed 1,000. ListMultipartUploadsRequest.setMaxUploads(Integer maxUploads)
KeyMarker The multipart events in which the lexicographic orders of all object names are greater than the value of the KeyMarker parameter. KeyMarker can be used together with UploadIdMarker parameter to specify the start point of the returned result. ListMultipartUploadsRequest.setKeyMarker(String keyMarker)
UploadIdMarker UploadIdMarker is used together with KeyMarker parameter to specify the start point of the returned result. If the KeyMarker parameter is not specified, OSS ignores the UploadIdMarker parameter. If the KeyMarker parameter is specified, the query result includes all the multipart upload events in which the lexicographic orders of object names are greater than the value of the KeyMarker parameter and all the multipart upload events in which the object names are the same as the value of the KeyMarker parameter and the Upload IDs are greater than the value of the UploadIdMarker parameter. ListMultipartUploadsRequest.setUploadIdMarker(String uploadIdMarker)

The following code calls OSSClient.listMultipartUploads to retrieve all multipart upload events in a bucket.

Simple list of multipart upload events

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String bucketName = "<yourBucketName>";
  7. // Create an OSSClient instance
  8. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  9. // List the multipart upload event
  10. ListMultipartUploadsRequest listMultipartUploadsRequest = new ListMultipartUploadsRequest(bucketName);
  11. MultipartUploadListing multipartUploadListing = ossClient.listMultipartUploads(listMultipartUploadsRequest);
  12. for (MultipartUpload multipartUpload : multipartUploadListing.getMultipartUploads()) {
  13. // Upload Id
  14. multipartUpload.getUploadId();
  15. // Key
  16. multipartUpload.getKey();
  17. // Date of initiate multipart upload
  18. multipartUpload.getInitiated();
  19. }
  20. // Disable the client
  21. ossClient.shutdown();

Note:

  • By default, if a bucket contains more than 1,000 multipart upload events, only the first 1,000 objects are returned and the IsTruncated parameter in the returned results is false. The returned NextKeyMarker and NextUploadIdMarker can be used as the start point of the next read.

  • You can adopt the list by page mode if the upload events are not obtained all at once.

List all upload events

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String bucketName = "<yourBucketName>";
  7. // Create an OSSClient instance
  8. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  9. // List the multipart upload event
  10. MultipartUploadListing multipartUploadListing;
  11. ListMultipartUploadsRequest listMultipartUploadsRequest = new ListMultipartUploadsRequest(bucketName);
  12. do {
  13. multipartUploadListing = ossClient.listMultipartUploads(listMultipartUploadsRequest);
  14. for (MultipartUpload multipartUpload : multipartUploadListing.getMultipartUploads()) {
  15. // Upload Id
  16. multipartUpload.getUploadId();
  17. // Key
  18. multipartUpload.getKey();
  19. // Date of initiate multipart upload
  20. multipartUpload.getInitiated();
  21. }
  22. listMultipartUploadsRequest.setKeyMarker(multipartUploadListing.getNextKeyMarker());
  23. listMultipartUploadsRequest.setUploadIdMarker(multipartUploadListing.getNextUploadIdMarker());
  24. } while (multipartUploadListing.isTruncated());
  25. // Disable the client
  26. ossClient.shutdown();

List all upload events by page

By default, listMultipartUploads lists 1,000 upload events at a time. The preceding “list all upload events” operation is a special case of listing upload events by page. Each page contains 1,000 events. If you want to specify the number of events for each page, that is, the number of events in one list, see the following code.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String bucketName = "<yourBucketName>";
  7. // Create an OSSClient instance
  8. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  9. // List the multipart upload event
  10. MultipartUploadListing multipartUploadListing;
  11. ListMultipartUploadsRequest listMultipartUploadsRequest = new ListMultipartUploadsRequest(bucketName);
  12. // Number of events in each page
  13. listMultipartUploadsRequest.setMaxUploads(50);
  14. do {
  15. multipartUploadListing = ossClient.listMultipartUploads(listMultipartUploadsRequest);
  16. for (MultipartUpload multipartUpload : multipartUploadListing.getMultipartUploads()) {
  17. // Upload Id
  18. multipartUpload.getUploadId();
  19. // Key
  20. multipartUpload.getKey();
  21. // Date of initiate multipart upload
  22. multipartUpload.getInitiated();
  23. }
  24. listMultipartUploadsRequest.setKeyMarker(multipartUploadListing.getNextKeyMarker());
  25. listMultipartUploadsRequest.setUploadIdMarker(multipartUploadListing.getNextUploadIdMarker());
  26. } while (multipartUploadListing.isTruncated());
  27. // Disable the client
  28. ossClient.shutdown();

Post upload

OSS Java SDK currently does not support post upload of objects. The example provides the PostObject example project for your direct use or use after revisions. See PostObjectSample.

If you have any questions regarding Put Object, see OSS Post Object API and PostObject.

Upload progress bar

The OSS Java SDK supports the progress bar feature to indicate the uploading/downloading progresses. The following code takes OSSClient.putObject as an example to describe the usage of the progress bar feature.

Note: Complete upload progress bar code can be found at GitHub.

  1. static class PutObjectProgressListener implements ProgressListener {
  2. private long bytesWritten = 0;
  3. private long totalBytes = -1;
  4. private boolean succeed = false;
  5. @Override
  6. public void progressChanged(ProgressEvent progressEvent) {
  7. long bytes = progressEvent.getBytes();
  8. ProgressEventType eventType = progressEvent.getEventType();
  9. switch (eventType) {
  10. case TRANSFER_STARTED_EVENT:
  11. System.out.println("Start to upload......");
  12. break;
  13. case REQUEST_CONTENT_LENGTH_EVENT:
  14. this.totalBytes = bytes;
  15. System.out.println(this.totalBytes + " bytes in total will be uploaded to OSS");
  16. break;
  17. case REQUEST_BYTE_TRANSFER_EVENT:
  18. this.bytesWritten += bytes;
  19. if (this.totalBytes != -1) {
  20. int percent = (int)(this.bytesWritten * 100.0 / this.totalBytes);
  21. System.out.println(bytes + " bytes have been written at this time, upload progress: " + percent + "%(" + this.bytesWritten + "/" + this.totalBytes + ")");
  22. } else {
  23. System.out.println(bytes + " bytes have been written at this time, upload ratio: unknown" + "(" + this.bytesWritten + "/...)");
  24. }
  25. break;
  26. case TRANSFER_COMPLETED_EVENT:
  27. this.succeed = true;
  28. System.out.println("Succeed to upload, " + this.bytesWritten + " bytes have been transferred in total");
  29. break;
  30. case TRANSFER_FAILED_EVENT:
  31. System.out.println("Failed to upload, " + this.bytesWritten + " bytes have been transferred");
  32. break;
  33. default:
  34. break;
  35. }
  36. }
  37. public boolean isSucceed() {
  38. return succeed;
  39. }
  40. }
  41. public static void main(String[] args) {
  42. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  43. String accessKeyId = "<accessKeyId>";
  44. String accessKeySecret = "<accessKeySecret>";
  45. String bucketName = "<bucketName>";
  46. String key = "object-get-progress-sample";
  47. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  48. try {
  49. // Upload with a progress bar
  50. ossClient.putObject(new PutObjectRequest(bucketName, key, new FileInputStream("<yourLocalFile>")).
  51. <PutObjectRequest>withProgressListener(new PutObjectProgressListener()));
  52. } catch (Exception e) {
  53. e.printStackTrace();
  54. }
  55. ossClient.shutdown();
  56. }

Note:

  • The putObject/getObject/uploadPart support the progress bar feature.

  • The uploadFile/downloadFile do not support the progress bar feature.

Upload callback

When an object upload is completed, the OSS can provide a callback to the app server. To implement the callback, you must attach the relevant Callback parameter to the request sent to the OSS. APIs that currently support CallBack include PutObject, PostObject, and CompleteMultipartUpload.

A typical upload callback scenario is when authorized third-party uploads occur at the same time, the client specifies the callback of the server when it uploads objects to OSS. After the upload task of the client is completed in the OSS, the OSS automatically initiates an HTTP request for the callback to the app server. This promptly notifies the server that the upload is completed, so it can complete operations such as database modification. When the callback request receives a response from the server, the OSS returns the status to the client.

For more information on upload callback, see Upload Callback. Next example is of PutObject interface used to explain the upload callback usage.

Note: Complete upload callback code can be found at GitHub.

  1. // Take the Hangzhou endpoint for example. Other regions can be filled in based on actual conditions
  2. String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
  3. // Cloud account AccessKey has an access to all APIs, it is suggested to create a RAM sub account and use it following Alibaba Cloud Best Security Practices for API access and daily O&M, log on to https://ram.console.aliyun.com to create an account
  4. String accessKeyId = "<yourAccessKeyId>";
  5. String accessKeySecret = "<yourAccessKeySecret>";
  6. String bucketName = "<yourBucketName>";
  7. // Your callback server address, such as http://oss-demo.aliyuncs.com or http://127.0.0.1:9090
  8. String callbackUrl = "<yourCallbackServerUrl>";
  9. // Create an OSSClient instance
  10. OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);
  11. String content = "Hello OSS";
  12. PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, "key",
  13. new ByteArrayInputStream(content.getBytes()));
  14. // Upload callback parameter
  15. Callback callback = new Callback();
  16. callback.setCallbackUrl(callbackUrl);
  17. callback.setCallbackHost("oss-cn-hangzhou.aliyuncs.com");
  18. callback.setCallbackBody("{\\\"mimeType\\\":${mimeType},\\\"size\\\":${size}}");
  19. callback.setCallbackBodyType(CallbackBodyType.JSON);
  20. callback.addCallbackVar("x:var1", "value1");
  21. callback.addCallbackVar("x:var2", "value2");
  22. putObjectRequest.setCallback(callback);
  23. PutObjectResult putObjectResult = ossClient.putObject(putObjectRequest);
  24. // Read the content of the message returned of the upload callback
  25. byte[] buffer = new byte[1024];
  26. putObjectResult.getCallbackResponseBody().read(buffer);
  27. // Make sure to close the connection. Otherwise the connection resource may leak
  28. putObjectResult.getCallbackResponseBody().close();
  29. // Disable the client
  30. ossClient.shutdown();

Note:

  • For parameters in the upload callback, see instructions in Upload Callback.

  • Make sure to close the message body returned of the upload callback. Otherwise, connection resource leakage may occur.

Thank you! We've received your feedback.