When using ossimport to upload large files to Object Storage Service (OSS), you may see:
java.lang.IllegalArgumentException: The part number should be in the range of [1-10000].This error means the file is too large for the current part size configuration.
Why this happens
ossimport uses multipart upload for large files. It splits the file into parts and uploads them in sequence. OSS supports a maximum of 10,000 parts per upload, with each part between 100 KB and 5 GB.
The default part size is 50 MB. At this size, the maximum supported file is:
50 MB × 10,000 parts = 500 GBIf your file exceeds 500 GB, the number of required parts exceeds 10,000, and the upload fails with the error above.
Solution
Increase multipartUploadPartSize in the local_job.cfg configuration file.
Calculate the minimum part size for your file:
Minimum part size = File size ÷ 10,000For example, to upload a 1 TB file, set the part size to at least 100 MB (1,000 GB ÷ 10,000 = 100 MB).
The following table shows recommended part sizes for common file sizes:
| File size | Minimum part size | Recommended setting |
|---|---|---|
| Up to 500 GB | 50 MB | multipartUploadPartSize=50m (default) |
| Up to 1 TB | 100 MB | multipartUploadPartSize=100m |
| Up to 5 TB | 500 MB | multipartUploadPartSize=500m |
| Up to 48.8 TB | 5 GB | multipartUploadPartSize=5120m |
Set the parameter in local_job.cfg:
multipartUploadPartSize=<part-size-in-mb>mReplace <part-size-in-mb> with your target value. For example:
multipartUploadPartSize=100mEach part must be at least 100 KB. The last part can be smaller than 100 KB.