Run the cp command to upload local files to OSS, Run the cp command to upload local files to OSS - Object Storage Service

You can run the cp command to upload resources, such as local files, images, and videos, to Object Storage Service (OSS). You can also include the --include and --exclude options in the command to batch upload files that meet the specified conditions.

Usage notes

To upload a local file to OSS, you must have the oss:PutObject permission. For more information, see Attach a custom policy to a RAM user.
For ossutil V1.6.16 and later, you can directly use ossutil as the binary name in the command line. You do not need to update the binary name based on the system. For ossutil earlier than V1.6.16, you must specify a binary name that corresponds to the system. For more information, see ossutil command reference.

When you run the cp command to upload a local file, ossutil automatically selects an upload method based on the file size:
- If the size of the local file that you want to upload is smaller than the threshold for performing a resumable upload task, ossutil uses simple upload. The threshold is specified by the --bigfile-threshold option and the default size threshold is 100 MB.
- If the size of the local file that you want to upload is greater than or equal to the threshold for performing a resumable upload task, ossutil uses resumable upload.
  If the upload task is interrupted, the uploaded data is stored as parts in the OSS bucket. If you no longer need the uploaded parts, we recommend that you delete them by using one of the following methods to avoid additional storage costs:
  - Manually delete parts. For more information, see rm.
  - Configure lifecycle rules to automatically delete parts. For more information, see Configure lifecycle rules.

Command syntax

ossutil cp file_url cloud_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxupspeed <value>]
[--enable-symlink-dir]
[--disable-all-symlink]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--snapshot-path <value>]
[--disable-crc64]
[--disable-dir-object]
[--payer <value>]
[--tagging <value>]
[-j, --job <value>]
[--parallel <value>]
[--start-time <value>]
[--end-time <value>]

The following table describes the options in the command syntax.

Parameter/Option	Description
file_url	The path of the local file. Examples: `/localfolder/examplefile.txt` in Linux and `D:\localfolder\examplefile.txt` in Windows.
cloud_url	The path in which you want to store the uploaded local file. Specify the path in the `oss://bucketname/objectname` format. Example: `oss://examplebucket/examplefile.txt`.
-r, --recursive	Specifies recursive operations. If you specify this option, ossutil performs the operation on all objects in the bucket that meet the specified condition. If you do not specify this option, ossutil performs the operation only on the specified object.
-f --force	Forces the operation without requiring user confirmation.
-u, --update	Specifies that ossutil uploads files only when the destination objects do not exist or when the last modified time of the files is later than that of the destination objects.
--maxupspeed	The maximum upload speed. Unit: KB/s. The default value is 0, which indicates that the upload speed is not limited.
--enable-symlink-dir	Specifies that ossutil uploads subdirectories in directories to which symbolic links point. By default, ossutil does not upload subdirectories in directories to which symbolic links point.
--disable-all-symlink	Specifies that ossutil ignores subdirectories and objects in directories to which symbolic links point.
--disable-ignore-error	Specifies that errors are not ignored during batch operations.
--only-current-dir	Specifies that ossutil uploads only objects in the current directory. Subdirectories in this directory and objects in the subdirectories are ignored.
--bigfile-threshold	The threshold for performing a resumable upload task. If the object size exceeds the threshold, resumable upload is used. Unit: bytes. Default value: 100 MB. Valid values: 0 to 9223372036854775807.
--part-size	The part size. Unit: bytes. By default, ossutil determines the part size based on the object size. Valid values: 1 to 9223372036854775807.
--checkpoint-dir	The directory in which information about the resumable upload task is stored. If the task fails, ossutil automatically creates a directory named `.ossutil_checkpoint` and saves the checkpoint information in the directory. After the task is successful, ossutil deletes the directory. If you specify a checkpoint directory, make sure that the directory can be deleted.
--encoding-type	The encoding of the object names. Set the value to url. If you do not specify this option, the names of objects are not encoded.
--include	Includes all objects that meet the specified conditions. For more information about the syntax and examples, see Batch upload objects that meet specified conditions.
--exclude	Excludes all objects that meet the specified conditions. For more information about the syntax and examples, see Batch upload objects that meet specified conditions.
--meta	The object metadata. Object metadata includes standard HTTP headers and user metadata. User metadata starts with `x-oss-meta-`. Format: `header:value#header:value` format. Example: `Cache-Control:no-cache#Content-Encoding:gzip`. For more information, see Manage object metadata.
--acl	The access control list (ACL) of the objects. Valid values: default: The ACL of the objects is the same as the ACL of the bucket in which the objects are stored. This is the default ACL setting for objects. private: Only the bucket owner can perform read and write operations on the objects in the bucket. Other users cannot access the objects in the bucket. public-read: Only the bucket owner can perform write operations on the objects in the bucket. All users, including anonymous users, can perform read operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpected high fees. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. We recommend that you do not set the object ACL to this value except in special cases. public-read-write: All users, including anonymous users, can perform read and write operations on objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpectedly high fees. Exercise caution when you set the object ACL to public-read-write.
--snapshot-path	The directory in which the snapshots of uploaded objects are stored. In the subsequent upload task, ossutil reads the snapshot information from the specified directory and performs incremental upload.
--disable-crc64	Specifies that CRC-64 is disabled. By default, ossutil enables CRC-64 during data transmission.
--disable-dir-object	Specifies that no OSS object is generated for the directory during object upload.
--payer	The payer of the fees that are generated by the operation. If you want the requester who wants to access the resources in the specified path to pay the fees that are generated by the operation, set this option to requester.
--tagging	The tags of objects. Format: `TagkeyA=TagvalueA&TagkeyB=TagvalueB....`.
-j, --jobs	The number of concurrent tasks performed across multiple objects. Valid values: 1 to 10000. Default value: 3.
--parallel	The number of concurrent tasks that can be performed on a single object. Valid values: 1 to 10000. By default, if you do not specify this option, ossutil sets the value of this option based on the operation type and the object size.
--start-time	The beginning of the time range to filter objects by the last modified time. The value is a UNIX timestamp. Objects whose last modified time is earlier than the UNIX timestamp are ignored. Note Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.
--end-time	The end of the time range to filter objects by the last modified time. The value is a UNIX timestamp. Objects whose last modified time is later than the UNIX timestamp are ignored. Note If both --start-time and --end-time are specified, the command takes effect only on objects whose last modified time is within the range specified by --start-time and --end-time. Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.

If the default number of concurrent tasks does not meet your performance requirements, you can use the -j, --jobs and --parallel options in the preceding command syntax can be used to adjust upload performance. By default, ossutil calculates the value of --parallel based on the object size. When you batch upload large objects, the actual concurrency value is calculated by multiplying the --jobs value and the --parallel value.

We recommend that you adjust the number of concurrent tasks to a value less than 100 if the resources such as the network bandwidth, memory, and CPU of your ECS instance or server are limited. If resources such as network bandwidth, memory, and CPU are left idle, you can increase the concurrency value.
If the number of concurrent tasks is excessively large, the upload performance of ossutil may decrease because threads switch and compete for resources. In addition, a large number of concurrent tasks may cause an end-of-file (EOF) error. To prevent the issues, you can adjust the values of the -j, --jobs and --parallel options based on your available resources. To perform stress testing, we recommend that you initially specify small values for the two options and gradually increase the values to find the optimal values.

Sample environment

In this topic, local files or directories are uploaded from a Linux OS to OSS. You can modify the options in the examples based on your OS and environment. The following resources are used in examples:

Local file: examplefile.txt (a file in the root directory)
Local directory: localfolder (a subdirectory in the root directory)
Destination bucket: examplebucket
Directory in the destination bucket: desfolder

Simple upload

If you want to upload resources, such as local files, images, and videos, to OSS, you can use ossutil. The following sample commands provide examples on how to use ossutil to upload data to OSS.

Upload a single file
If you do not specify the name for the uploaded object, the object in OSS uses the same name as the local file. If you specify the name for the uploaded object, the object in OSS uses the specified name.
```
ossutil cp examplefile.txt oss://examplebucket/desfolder/
```
Upload files from a local directory
Add the -r option to the cp command to upload files from a local directory to the specified path.
```
ossutil cp -r localfolder/ oss://examplebucket/desfolder/
```
Upload files whose last modified time is within the specified time range from a local directory
Run the following command to upload files whose last modified time is within the range of 10:09:18 (UTC+8) October 31, 2023 to 12:55:58 (UTC+8) October 31, 2023 from a local directory to the specified OSS path:
```
ossutil cp -r localfolder/ oss://examplebucket/desfolder/ --start-time 1698718158 --end-time 1698728158
```
Upload a local directory and the files in the directory
Add the -r option to the cp command and include the name of a local directory in the OSS path to upload the local directory and the files in the directory to the specified OSS path. For the specified directory, OSS does not create a directory object. For a subdirectory in the specified directory, OSS creates a zero-byte object whose name ends with a forward slash (/). If you want to create a directory object for the local directory, run the mkdir command.
```
ossutil cp -r localfolder/ oss://examplebucket/desfolder/localfolder/
```
Upload a single file and specify --meta
Specify object metadata in the header:value#header:value... format by using the --meta option when you run the cp command to upload a file.
```
ossutil cp examplefile.txt oss://examplebucket/desfolder/examplefile.txt --meta=Cache-Control:no-cache#Content-Encoding:gzip
```
Upload a directory and skip existing objects
When you resume a failed batch upload task, you can specify the --update (abbreviated as -u) option to skip uploaded objects and perform an incremental upload. Example:
```
ossutil cp -r localfolder/ oss://examplebucket/desfolder/ -u
```

Upload a file to a bucket for which pay-by-requester is enabled

ossutil cp localfolder/examplefile.txt oss://examplebucket/ --payer=requester

Upload only files in the current directory and ignore subdirectories

ossutil cp localfolder/ oss://examplebucket/desfolder/ --only-current-dir -r

Upload a directory without generating an object for the uploaded directory
In OSS, a directory is a zero-byte object whose name ends with a forward slash (/). If you specify the --disable-dir-object option in the cp command to upload a directory, OSS does not generate an object for the directory. However, you can view the directory in the OSS console. If you delete all objects in the directory, the directory is also deleted.
```
ossutil cp localfolder/ oss://examplebucket/desfolder/ --disable-dir-object -r
```

Upload objects in the subdirectory to which a symbolic link points

ossutil cp localfolder/ oss://examplebucket/desfolder/ --enable-symlink-dir -r

Ignore the subdirectory to which a symbolic link points and all objects in this subdirectory during upload
```
ossutil cp localfolder/ oss://examplebucket/desfolder/ -r --disable-all-symlink
```

Upload an object with the maximum upload speed limited

When you upload an object, you can set --maxupspeed to specify the maximum upload speed. Unit: KB/s. Examples

Upload an object to OSS and set the maximum upload speed to 1 MB/s

ossutil cp examplefile.txt oss://examplebucket/desfolder/ --maxupspeed 1024

Upload a directory to OSS and set the maximum upload speed to 1 MB/s

ossutil cp -r localfolder/ oss://examplebucket/desfolder/ --maxupspeed 1024

Upload an object and configure tags for the object

When you run the cp command to upload an object, you can specify the --tagging option to configure tags for the object. If you configure multiple tags for the object, separate them with ampersands (&). Example:

ossutil cp examplefile.txt oss://examplebucket/desfolder/ --tagging "abc=1&bcd=2&……"

For more information, see object-tagging.

Upload an object and specify the storage class of the object

When you run the cp command to upload a local file, you can specify the --meta option to specify the storage class of the object. ossutil supports the following storage classes:

Standard
IA
Archive
ColdArchive
DeepColdArchive

If you do not specify the storage class of the object in the command, OSS applies the storage class of the bucket to the object. For more information, see Overview.

Examples:

Upload an object and set the storage class of the object to IA

ossutil cp examplefile.txt oss://examplebucket/desfolder/ --meta X-oss-Storage-Class:IA

Upload a directory and set the storage class of all objects in the directory to Standard

ossutil cp localfolder/ oss://examplebucket/desfolder/ --meta X-oss-Storage-Class:Standard -r

Upload an object and specify the ACL of the object

When you run the cp command to upload an object, you can set the --acl option to specify the ACL of the object. ossutil supports the following ACLs:

default
private
public-read
public-read-write

Examples:

Upload an object and set the ACL of the object to private

ossutil cp examplefile.txt oss://examplebucket/desfolder/ --acl private

Upload an object and set the ACL of the object to public-read

ossutil cp localfolder/ oss://examplebucket/desfolder/ --acl public-read  -r

Upload an object and specify the encryption method of the object

When you upload an object, you can specify the server-side encryption method for the object. The object is stored in the specified bucket after the object is encrypted. Examples:

Upload an object and set the encryption method of the object to SSE-OSS and the encryption algorithm to AES-256
```
ossutil cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:AES256
```
Upload an object and set the encryption method of the object to SSE-KMS without specifying a customer master key (CMK) ID
```
ossutil cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:KMS
```
When you set the encryption method to SSE-KMS, you are charged for the usage of CMK IDs. For more information, see Billing of KMS.

Upload an object and set the encryption method of the object to SSE-KMS with a specified CMK ID

ossutil cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:KMS#x-oss-server-side-encryption-key-id:7bd6e2fe-cd0e-483e-acb0-f4b9e1******

For more information about server-side encryption, see Server-side encryption.

Generate snapshots for objects when you upload the objects

When you run the cp command to upload multiple objects at a time and specify --snapshot-path in the command, ossutil generates the snapshots for the uploaded objects in the specified directory to record the last modified time of the objects. In subsequent upload tasks, ossutil determines whether to skip existing objects based on their last modified time. If you specify --snapshot-path in the command, make sure that the objects in OSS are not modified by other users from the time when the objects were last uploaded. The --snapshot-path option is used to accelerate incremental uploads. Example:

ossutil cp -r localfolder/ oss://examplebucket/desfolder/ --snapshot-path=path

Important

ossutil does not automatically delete snapshots stored in the directory specified by snapshot-path. To prevent snapshots from consuming excessive storage space, delete the snapshots that you no longer need from the directory specified by snapshot-path.
Additional overheads are required to read and write snapshot information. We recommend that you do not use this option in scenarios in which you have only a small number of objects to upload, the network condition is good, or other users need to perform operations on the objects. Instead, you can use --update to perform incremental uploads.
You can configure both --update and --snapshot-path in a cp command. ossutil determines whether to skip an object based on the snapshots stored in the directory specified by --snapshot-path. If no snapshots are generated for the object, ossutil determines whether to skip the object based on the --update option.

Batch upload objects that meet specified conditions

When you run the cp command to upload multiple objects at a time and specify the --include and --exclude options in the command, ossutil uploads objects that meet the specified conditions.

The --include and --exclude options support the following formats:

Asterisk (*): matches any number of characters. For example, *.txt matches all TXT files.
Question mark (?): matches a single character. For example, abc?.jpg matches all JPG objects whose names are abc followed by a single character, such as abc1.jpg.
[sequence]: matches characters in a sequence. For example, abc[1-5].jpg matches JPG objects whose names are abc followed by a number in sequence [1-5]. The object names are abc1.jpg, abc2.jpg, abc3.jpg, abc4.jpg, and abc5.jpg.
[!sequence]: matches characters outside a sequence. For example, abc[!0-7].jpg matches objects other than abc0.jpg, abc1.jpg, abc2.jpg, abc3.jpg, abc4.jpg, abc5.jpg, abc6.jpg, and abc7.jpg.

A rule can contain multiple conditions specified by --include and --exclude. After these conditions are configured, ossutil evaluates the conditions from left to right to obtain the final results. If the test.txt object exists in a directory for which conditions are specified, ossutil generates match results based on different matching rules.

Rule 1: --include "*test*" --exclude "*.txt". ossutil first evaluates the --include "*test*" condition and includes the test.txt object in the results because the object is a match. Then, ossutil evaluates the --exclude "*.txt" condition, finds that the test.txt object is a match because the object name contains ".txt", and then excludes the object from the results. The test.txt object is excluded from the final results.
Rule 2: --exclude "*.txt" --include "*test*". ossutil first evaluates the --exclude "*.txt" condition and excludes the test.txt object from the results because the object is a match. Then, ossutil evaluates the--include "*test*" condition, finds that the test.txt object is a match because the object name contains "test", and then includes the object in the results. The test.txt object is included in the final results.
Rule 3: --include "*test*" --exclude "*.txt" --include "te?t.txt". ossutil first evaluates the --include "*test*" condition, finds that the test.txt object is a match, and then includes the object in the results. Then, ossutil evaluates the --exclude "*.txt" condition, finds that the test.txt object is a match because the object name contains "txt", and excludes the object from the results. Finally, ossutil evaluates the --include "te?t.txt" condition, finds that the test.txt object is a match, and then includes the object in the results. The test.txt object is included in the final results.

Important

You cannot specify directories in --include and --exclude conditions. For example, --include "/usr/test/.jpg" is invalid.

Examples:

Upload all TXT objects

ossutil cp localfolder/ oss://examplebucket/desfolder/ --include "*.txt" -r

Upload all objects whose names contain abc and whose formats are not JPG or TXT from the specified directory

ossutil cp localfolder/ oss://examplebucket/desfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r

Common options

If you use ossutil to switch to a bucket that is located in another region, add the -e option to specify the endpoint of the region in which the bucket is located. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, add the -i option to specify the AccessKey ID of the specified account, and add the -k option to specify the AccessKey secret of the specified account.

For example, you can run the following command to upload a local file named exampleobject.txt to the destfolder directory of the examplebucket bucket that is located in the China (Shanghai) region and owned by another Alibaba Cloud account:

ossutil cp exampleobject.txt oss://examplebucket/desfolder/ -e oss-cn-shanghai.aliyuncs.com -i yourAccessKeyID -k yourAccessKeySecret

For more information about common options, see Common options.