Object Storage Service (OSS) provides the multipart upload feature. Multipart upload allows you to split a large object into multiple parts to upload. After these parts are uploaded, you can call the CompleteMultipartUpload operation to combine the parts into a complete object.
Usage notes
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configuration examples for common scenarios.
To implement multipart upload, you must call the InitiateMultipartUpload, UploadPart, and CompleteMultipartUpload operations. Therefore, you must have the
oss:PutObjectpermission. For more information, see Attach a custom policy to a RAM user.
Procedure
To implement multipart upload, perform the following steps:
Initiate a multipart upload task.
Call the ossClient.initiateMultipartUpload method to obtain an upload ID that is unique in OSS.
Upload parts.
Call the ossClient.uploadPart method to upload the parts.
NoteIn a multipart upload task, part numbers are used to identify the relative positions of the parts in an object. If you upload a part that has the same part number as an existing part, the existing part is overwritten by the uploaded part.
OSS includes the MD5 hash of each uploaded part in the ETag header in the response.
OSS calculates the MD5 hash of the uploaded parts and compares the MD5 hash with the MD5 hash that is calculated by OSS SDK for Java. If the two hashes are different, OSS returns the InvalidDigest error code.
Complete the multipart upload task.
After all parts are uploaded, call the ossClient.completeMultipartUpload method to combine these parts into a complete object.
Sample code
The following sample code provides an example on how to implement a multipart upload task by following the multipart upload process:
import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.internal.Mimetypes;
import com.aliyun.oss.model.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
public class Demo {
public static void main(String[] args) throws Exception {
// In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint.
String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
// Specify the name of the bucket. Example: examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path.
String objectName = "exampledir/exampleobject.txt";
// Specify the full path of the local file that you want to upload.
String filePath = "D:\\localpath\\examplefile.txt";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
String region = "cn-hangzhou";
// Create an OSSClient instance.
ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(credentialsProvider)
.clientConfiguration(clientBuilderConfiguration)
.region(region)
.build();
try {
// Create an InitiateMultipartUploadRequest object.
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, objectName);
// Create an ObjectMetadata object and specify the Content-Type parameter.
ObjectMetadata metadata = new ObjectMetadata();
if (metadata.getContentType() == null) {
metadata.setContentType(Mimetypes.getInstance().getMimetype(new File(filePath), objectName));
}
System.out.println("Content-Type: " + metadata.getContentType());
// Bind the metadata to the upload request.
request.setObjectMetadata(metadata);
// Initialize the multipart upload task.
InitiateMultipartUploadResult upresult = ossClient.initiateMultipartUpload(request);
// Obtain the upload ID.
String uploadId = upresult.getUploadId();
// partETags is the set of PartETags. A PartETag consists of the part number and ETag of an uploaded part.
List<PartETag> partETags = new ArrayList<PartETag>();
// Specify the size of each part, which is used to calculate the number of parts of the object. Unit: bytes.
final long partSize = 1 * 1024 * 1024L; // Set the part size to 1 MB.
// Calculate the number of parts based on the size of the uploaded data. In the following sample code, a local file is used as an example to describe how to use the File.length() method to obtain the size of the uploaded data.
final File sampleFile = new File(filePath);
long fileLength = sampleFile.length();
int partCount = (int) (fileLength / partSize);
if (fileLength % partSize != 0) {
partCount++;
}
// Upload each part until all parts are uploaded.
for (int i = 0; i < partCount; i++) {
long startPos = i * partSize;
long curPartSize = (i + 1 == partCount) ? (fileLength - startPos) : partSize;
UploadPartRequest uploadPartRequest = new UploadPartRequest();
uploadPartRequest.setBucketName(bucketName);
uploadPartRequest.setKey(objectName);
uploadPartRequest.setUploadId(uploadId);
// Specify the input stream of the multipart upload task.
// In the following sample code, a local file is used as an example to describe how to create a FileInputStream object and use the InputStream.skip() method to skip the specified data.
InputStream instream = new FileInputStream(sampleFile);
instream.skip(startPos);
uploadPartRequest.setInputStream(instream);
// Specify the size available for each part. The size of each part except for the last part must be greater than or equal to 100 KB.
uploadPartRequest.setPartSize(curPartSize);
// Specify part numbers. Each part has a part number that ranges from 1 to 10,000. If the part number that you specify does not fall within the specified range, OSS returns the InvalidArgument error code.
uploadPartRequest.setPartNumber(i + 1);
// Parts are not necessarily uploaded in order and can be uploaded from different OSS clients. OSS sorts the parts based on the part numbers and combines the parts into a complete object.
UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
// After a part is uploaded, OSS returns a result that contains a PartETag. The PartETag is stored in partETags.
partETags.add(uploadPartResult.getPartETag());
// Disable the input stream.
instream.close();
}
// Create a CompleteMultipartUploadRequest object.
// When you call the CompleteMultipartUpload operation, you must provide all valid partETags. After OSS receives the PartETags, OSS verifies all parts one by one. After all parts are verified, OSS combines these parts into a complete object.
CompleteMultipartUploadRequest completeMultipartUploadRequest =
new CompleteMultipartUploadRequest(bucketName, objectName, uploadId, partETags);
// Complete the multipart upload task.
CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest);
System.out.println("Upload successful,ETag:" + completeMultipartUploadResult.getETag());
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} catch (ClientException ce) {
System.out.println("Caught a ClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with OSS, "
+ "such as not being able to access the network.");
System.out.println("Error Message:" + ce.getMessage());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
}
}Common scenarios
Configure metadata when you initialize a multipart upload task
Configure the object access control list (ACL) when you complete a multipart upload task
Automatically process the ETags of parts when you complete a multipart upload task
Cancel a multipart upload task
List uploaded parts
List ongoing multipart upload tasks
Perform multipart upload to upload a network or data stream
References
For information about the complete sample code used to perform multipart upload, see GitHub.
A multipart upload involves three API operations. For more information about the operations, see the following topics:
For information about the API operation that you can call to cancel a multipart upload task, see AbortMultipartUpload.
For information about the API operation that you can call to list uploaded parts, see ListParts.
For information about the API operation that you can call to list all ongoing multipart upload tasks, see ListMultipartUploads.