:"The part number should be in the range of [1-10000]" error when uploading large files using ossimport tool

Challenge

Use the data migration tool ossimport to upload files to Alibaba Cloud Object Storage Service (OSS). The following error is reported when uploading large files:

java.lang.IllegalArgumentException: The part number should be in the range of [1-10000].

Cause

The ossimport tool automatically uses the multipart upload method when uploading large files. The size of objects in the multipart upload mode is determined by parts, and the maximum number of supported parts is 10000 blocks. The minimum range of parts is 100KB and the maximum is 5GB. Therefore, the size of objects cannot exceed 48.8TB. The default part limit of the ossimport tool is 50MB. This setting supports uploading a maximum of 500GB files.

Note: If you want to allocate the size of the uploaded time slice, you must set the size larger than 100KB. Otherwise, this parameter is invalid. The last piece can be smaller than 100KB.

Solution

Set the following parameters in the local_job.cfg configuration file of the ossimport tool. This setting supports uploading a maximum of 500GB files. You can modify the value of this parameter to support uploading of larger files.

multipartUploadPartSize=50m

References

• Multipart upload and resumable upload
• Description and configuration of the ossimport tool
• FAQ about the data migration tool ossimport and their solutions

Applicable scope

• Object Storage Service (OSS)