Object Storage Service (OSS) provides the multipart upload feature. Multipart upload allows you to split a large object into multiple parts to upload. After these parts are uploaded, you can call the CompleteMultipartUpload operation to combine the parts into a complete object.
Notes
In this topic, the public endpoint of the China (Hangzhou) region is used. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For details about supported regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configuration examples for common scenarios.
To complete the multipart upload process, which includes the InitiateMultipartUpload, UploadPart, and CompleteMultipartUpload operations, you must have the
oss:PutObjectpermission. For more information, see Grant custom access policies to a RAM user.
Multipart upload process
A multipart upload consists of the following three steps:
Initialize a multipart upload event.
Call the ossClient.initiateMultipartUpload method. OSS returns a globally unique upload ID in the response.
Upload parts.
Call the ossClient.uploadPart method to upload part data.
NoteFor the same upload ID, the part number identifies the position of the part in the entire file. If you upload new data with the same part number, the existing part data in OSS is overwritten.
OSS returns the MD5 hash of the received part data in the ETag header of the response.
OSS calculates the MD5 hash of the uploaded data and compares it with the MD5 hash calculated by the SDK. If the two MD5 hashes do not match, the InvalidDigest error code is returned.
Complete the multipart upload.
After all parts are uploaded, call the ossClient.completeMultipartUpload method to merge the parts into a complete file.
Example code
The following sample code provides an example on how to implement a multipart upload task by following the multipart upload process:
import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.internal.Mimetypes;
import com.aliyun.oss.model.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
public class Demo {
public static void main(String[] args) throws Exception {
// The Endpoint of the China (Hangzhou) region is used as an example. Specify the actual Endpoint.
String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
// Specify the bucket name, for example, examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object, for example, exampledir/exampleobject.txt. The full path cannot contain the bucket name.
String objectName = "exampledir/exampleobject.txt";
// The path of the local file to upload.
String filePath = "D:\\localpath\\examplefile.txt";
// Specify the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the region to cn-hangzhou.
String region = "cn-hangzhou";
// Create an OSSClient instance.
// When the OSSClient instance is no longer used, call the shutdown method to release resources.
ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(credentialsProvider)
.clientConfiguration(clientBuilderConfiguration)
.region(region)
.build();
try {
// Create an InitiateMultipartUploadRequest object.
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, objectName);
// Create an ObjectMetadata object and set the Content-Type.
ObjectMetadata metadata = new ObjectMetadata();
if (metadata.getContentType() == null) {
metadata.setContentType(Mimetypes.getInstance().getMimetype(new File(filePath), objectName));
}
System.out.println("Content-Type: " + metadata.getContentType());
// Bind the metadata to the upload request.
request.setObjectMetadata(metadata);
// Initialize the multipart upload.
InitiateMultipartUploadResult upresult = ossClient.initiateMultipartUpload(request);
// Return the upload ID.
String uploadId = upresult.getUploadId();
// partETags is a collection of PartETag objects. A PartETag object consists of the ETag and part number of a part.
List<PartETag> partETags = new ArrayList<PartETag>();
// The size of each part. This is used to calculate the number of parts. Unit: bytes.
// The minimum part size is 100 KB, and the maximum part size is 5 GB. The size of the last part can be smaller than 100 KB.
// Set the part size to 1 MB.
final long partSize = 1 * 1024 * 1024L;
// Calculate the number of parts based on the size of the data to upload. The following code provides an example of how to obtain the size of the data to upload from a local file using File.length().
final File sampleFile = new File(filePath);
long fileLength = sampleFile.length();
int partCount = (int) (fileLength / partSize);
if (fileLength % partSize != 0) {
partCount++;
}
// Traverse the parts and upload them.
for (int i = 0; i < partCount; i++) {
long startPos = i * partSize;
long curPartSize = (i + 1 == partCount) ? (fileLength - startPos) : partSize;
UploadPartRequest uploadPartRequest = new UploadPartRequest();
uploadPartRequest.setBucketName(bucketName);
uploadPartRequest.setKey(objectName);
uploadPartRequest.setUploadId(uploadId);
// Set the stream of the part to upload.
// The following code provides an example of how to create a FileInputStream object from a local file and skip the specified data using the InputStream.skip() method.
InputStream instream = new FileInputStream(sampleFile);
instream.skip(startPos);
uploadPartRequest.setInputStream(instream);
// Set the part size.
uploadPartRequest.setPartSize(curPartSize);
// Set the part number. Each uploaded part has a part number that ranges from 1 to 10,000. If the part number is not in the range, OSS returns the InvalidArgument error code.
uploadPartRequest.setPartNumber(i + 1);
// Parts do not need to be uploaded in sequence. They can even be uploaded from different clients. OSS sorts the parts by part number to create a complete file.
UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
// After each part is uploaded, the OSS response includes a PartETag. The PartETag is saved in partETags.
partETags.add(uploadPartResult.getPartETag());
// Close the stream.
instream.close();
}
// Create a CompleteMultipartUploadRequest object.
// When you complete the multipart upload, you must provide all valid partETags. After OSS receives the submitted partETags, it verifies the validity of each part. After all parts are verified, OSS combines these parts into a complete file.
CompleteMultipartUploadRequest completeMultipartUploadRequest =
new CompleteMultipartUploadRequest(bucketName, objectName, uploadId, partETags);
// Complete the multipart upload.
CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest);
System.out.println("Upload successful, ETag: " + completeMultipartUploadResult.getETag());
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} catch (ClientException ce) {
System.out.println("Caught a ClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with OSS, "
+ "such as not being able to access the network.");
System.out.println("Error Message:" + ce.getMessage());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
}
}Common scenarios
Set metadata when you initialize a multipart upload
Set file access permissions when you complete a multipart upload
Automatically process part ETags when you complete a multipart upload
Cancel a multipart upload event
List uploaded parts
List multipart upload events
Perform multipart upload for network streams or data streams
References
For the complete sample code for multipart upload, see GitHub example.
A multipart upload involves three API operations. For more information about the operations, see the following topics:
For more information about the API operation used to cancel a multipart upload event, see AbortMultipartUpload.
For more information about the API operation used to list uploaded parts, see ListParts.
For more information about the API operation used to list all in-progress multipart upload events, see ListMultipartUploads.