This topic describes how to run the cp command to upload local files or directories to Object Storage Service (OSS).

Usage notes

  • Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.
  • By default, when you use the cp command to upload an object, multipart upload and resumable upload are used. If the upload is interrupted before the object is completely uploaded, the uploaded data is stored as parts in an OSS bucket. To avoid additional storage fees, we recommend that you use the following methods to delete these parts if you no longer need them:

Command syntax

./ossutil64 cp file_url cloud_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxupspeed <value>]
[--enable-symlink-dir]
[--disable-all-symlink]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--snapshot-path <value>]
[--disable-crc64]
[--disable-dir-object]
[--payer <value>]
[--tagging <value>]
[-j, --job <value>]
[--parallel <value>]

The following table describes the parameters that you can configure to run the cp command to upload objects to OSS.

Parameter Description
file_url Specifies the full path of the local file you want to upload. Examples: The path of the local file in Linux is /localfolder/examplefile.txt. The path of the local file in Windows is D:\localfolder\examplefile.txt.
cloud_url Specifies the path of the OSS object. Format: oss://bucketname/objectname. Example: oss://examplebucket/examplefile.txt.
-r, --recursive Specifies recursive operations. If you specify this option in a command, the command performs operations on all objects in a bucket that meet the specified condition. If you do not specify this option in a command, the command performs operations only on the specified object.
-f --force Forces an operation without prompting the user for confirmation.
-u, --update Specifies that ossutil uploads files from the source only when the objects do not exist in the destination or when the last modified time of the files is later than that of the objects in the destination bucket.
--maxupspeed Specifies the maximum upload speed. Unit: KB/s. Default value: 0. The value of 0 indicates that the upload speed is not limited.
--enable-symlink-dir Specifies that the subdirectory to which the symbolic link points is uploaded. By default, subdirectories are not uploaded.
--disable-all-symlink Specifies that all objects in the subdirectory to which the symbolic link points and the subdirectory to which the symbolic link points are ignored during object upload.
--disable-ignore-error Specifies that errors are not ignored during batch operations.
--only-current-dir Specifies that only objects in the current directory are uploaded. Subdirectories in the current directory and objects in these subdirectories are not uploaded.
--bigfile-threshold Specifies the maximum size of files that can be uploaded by using resumable upload. Unit: bytes.

Default value: 104857600 (100 MB).

Valid values: 0 to 9223372036854775807.

--part-size Specifies the part size. Unit: bytes. By default, ossutil determines the part size based on the object size.

Valid values: 1 to 9223372036854775807.

--checkpoint-dir Specifies the directory in which the log information of resumable upload tasks is stored. Default value: .ossutil_checkpoint. When a resumable upload task fails, ossutil creates this directory and stores the checkpoint information about the task in this directory. ossutil deletes the directory after the task is completed. Therefore, if you specify a checkpoint directory, make sure that the directory can be deleted.
--encoding-type Specifies the method used to encode the names of objects. Valid values: url. If you do not specify this parameter, the names of objects are not encoded.
--include Specifies that the command applies to all objects that meet the specified conditions.
--exclude Specifies that the command applies to all objects that do not meet the specified conditions.
--meta Specifies the object metadata. The object metadata includes part of HTTP headers and user metadata that starts with x-oss-meta-. Format: header:value#header:value. Example: Cache-Control:no-cache#Content-Encoding:gzip. For more information about object metadata, see Manage object metadata.
--acl Specifies the access control list (ACL) of an object. Default value: private. Valid values:
  • default: The ACL of the objects is the same as the ACL of the bucket in which the objects are stored.
  • private: Only the bucket owner can perform read and write operations on objects in the bucket. Other users cannot access the objects in the bucket.
  • public-read: Only the bucket owner can perform write operations on objects in the bucket. Other users, including anonymous users, can perform only read operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpected high fees. Exercise caution when you set this parameter to this value.
  • public-read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpected high fees. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. Therefore, we recommend that you do not set the object ACL to this value except in special cases.
--snapshot-path Specifies the directory in which the snapshots of uploaded objects are stored. Next time the objects are uploaded while this option is specified, ossutil reads the snapshot information from the specified directory and performs an incremental upload.
--disable-crc64 Specifies that CRC-64 is disabled. By default, ossutil enables CRC-64 during data transmission.
--disable-dir-object Specifies that no OSS objects are generated for the directory during object upload.
--payer Specifies the payer of the traffic and request fees charged when the command is run. If you want the requester who accesses the resources in the specified path to pay for the traffic and request fees, set this parameter to requester.
--tagging Specifies the tags of objects in the following format: TagkeyA=TagvalueA&TagkeyB=TagvalueB.....
-j, --jobs Specifies the number of files to upload concurrently. Valid values: 1 to 10000. Default value: 3.
--parallel Specifies the number of concurrent tasks run to upload a single object. Valid values: 1 to 10000. By default, if you do not set this parameter, ossutil sets the value of this parameter based on the operation type and the object size.

The -j, --jobs and --parallel options in the preceding command syntax can be used to adjust performance if the default number of concurrent tasks does not meet your performance requirements. By default, ossutil calculates the number of concurrent operations based on the object size. When you upload multiple large objects, the actual number of concurrent tasks is calculated by multiplying the number of jobs by the number of concurrent operations.

  • We recommend that you adjust the number of concurrent tasks to a value smaller than 100 if the resources such as network bandwidth, memory, and CPU of your ECS instance or server are limited. If resources such as the network bandwidth, memory, and CPU are available, you can increase the number of concurrent operations.
  • If the number of concurrent tasks is too large, the upload performance of ossutil may decrease or an end-of-file (EOF) error may occur because thread resources are switched and threads compete for resources. To resolve this issue, you must adjust the values of the -j, --jobs and --parallel options based on the actual conditions of machines. To perform stress testing, set a small value for the two options before you incrementally increase them to the optimal values.

Sample environment

In this topic, local files or directories are uploaded from a Linux system to OSS. You can modify the parameters in the examples based on your operating system and environment. This topic uses the following common examples:

  • Local file: examplefile.txt (a file in the root directory)
  • Local directory: localfolder (a directory in the root directory)
  • OSS bucket: examplebucket
  • Specified directory in the OSS bucket: desfolder

Simple upload

You can use ossutil to upload local files to OSS. The following examples show how to run the cp command to upload objects to OSS by using simple upload:

  • Upload a single local file

    If you do not specify the name of the uploaded object, the name of the local file is used as the object name. If you specify the name of the uploaded object, the object is stored in OSS based on the specified name.

    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/
  • Upload the files in a local directory
    You can add the -r parameter to the cp command to upload files from a local directory to a specified path of OSS.
    ./ossutil64 cp -r localfolder/ oss://examplebucket/desfolder/
  • Upload a local directory and the files inside

    You can add the -r option to the cp command and add the name of a local directory to upload the local directory and the files inside to a specified path of OSS.

    ./ossutil64 cp -r localfolder/ oss://examplebucket/desfolder/localfolder/
  • Upload a single file and specify the --meta parameter
    When you run the cp command to upload a local file, you can add the --meta option to the command and configure the metadata of the file in the following format: header:value#header:value....
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/examplefile.txt --meta=Cache-Control:no-cache#Content-Encoding:gzip
  • Upload a directory without uploading existing objects

    When you resume a failed batch upload task, you can specify the --update (abbreviated as the -u) option to skip uploaded files. This way, you can upload only incremental data to OSS.

    ./ossutil64 cp -r localfolder/ oss://examplebucket/desfolder/ -u
  • Upload objects to a bucket for which pay-by-requester is enabled
    ./ossutil64 cp localfolder/examplefile.txt oss://examplebucket/ --payer=requester
  • Upload objects only in the current directory
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --only-current-dir -r
  • Upload a directory without generating an object for the uploaded directory

    In OSS, a directory is an object that is 0 KB in size and has a name that ends with a forward slash (/). If you specify the --disable-dir-object parameter in the cp command to upload a directory, OSS does not generate an object for the uploaded directory. However, you can view the directory in the OSS console. If you delete all objects from the directory, the directory is also deleted.

    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --disable-dir-object -r
  • Upload objects including those in subdirectories to which symbolic links point
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --enable-symlink-dir -r
  • Upload objects and ignore the objects in subdirectories and subdirectories to which symbolic links point
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ -r --disable-all-symlink

Configure the maximum upload speed

When you upload an object, you can set --maxupspeed to limit the maximum upload speed. Unit: KB/s. Examples:
  • Upload an object to OSS and set the maximum upload speed to 1 MB/s
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --maxupspeed 1024
  • Upload a directory to OSS and set the maximum upload speed to 1 MB/s
    ./ossutil64 cp -r localfolder/ oss://examplebucket/desfolder/ --maxupspeed 1024

Configure tagging for an object when you upload the object

When you run the cp command to upload an object, you can specify the --tagging option to configure tags for the object. Separate multiple tags with ampersands (&). Example:
./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --tagging "abc=1&bcd=2&……"

For more information about object tagging, see object-tagging (add, modify, query, and delete object tags).

Specify the storage class of an object when you upload the object

When you run the cp command to upload a local file, you can specify the --meta option to specify the storage class of the object. OSS supports the following storage classes:
  • Standard
  • Infrequent Access (IA)
  • Archive
If you do not specify the storage class in the command, the storage class of the uploaded object is the same as that of the bucket in which the object is stored. For more information, see Overview. Examples:
  • Upload an object and set the storage class of the object to IA
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --meta X-oss-Storage-Class:IA
  • Upload a directory and set the storage class of all objects in the specified directory to Standard
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --meta X-oss-Storage-Class:Standard -r

Specify the ACL of an object when you upload the object

When you run the cp command to upload objects, you can specify the --meta option to set the access control list (ACL) of the objects. OSS supports the following object ACLs:
  • default: The ACL of an object is the same as that of the bucket in which the object is stored.
  • private
  • public-read
  • public-read-write
The following examples show how to specify the ACL of objects when you run the cp command to upload objects:
  • Upload an object and set the ACL of the object to private
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --meta x-oss-object-acl:private
  • Upload an object and set the ACL of the object to public read
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --meta x-oss-object-acl:public-read  -r

Specify an encryption method for an object when you upload the object

When you upload an object, you can specify the server-side encryption method for the object. The object is stored in the specified bucket after the object is encrypted. Examples:
  • Upload an object and set the encryption method of the object to SSE-OSS and the encryption algorithm to AES-256
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:AES256
  • Upload an object and set the encryption method of the object to SSE-KMS without specifying a customer master key (CMK) ID
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:KMS

    When you set the encryption method to SSE-KMS, you are charged for the usage of CMK IDs. For more information, see KMS billing methods.

  • Upload an object, set the encryption method of the object to SSE-KMS, and then specify a CMK ID
    ./ossutil64 cp examplefile.txt oss://examplebucket/desfolder/ --meta=x-oss-server-side-encryption:KMS#x-oss-server-side-encryption-key-id:7bd6e2fe-cd0e-483e-acb0-f4b9e1******

For more information about server-side encryption, see Server-side encryption.

Generate snapshots for objects when you upload the objects

When you run the cp command to batch upload objects and specify the --snapshot-path option in the command, ossutil creates the snapshots of the uploaded objects in the specified directory to record the last modified time of the objects. In subsequent upload tasks, ossutil determines whether to skip existing objects based on their last modified time. If you want to specify the --snapshot-path option in the command, make sure that the objects in OSS are not modified by other users since the last time when the objects are uploaded. The --snapshot-path option is used to accelerate incremental batch uploads. The following example shows how to generate snapshots for objects when you run the cp command to upload the objects:
./ossutil64 cp -r localfolder/ oss://examplebucket/desfolder/ --snapshot-path=path                                
Notice
  • ossutil does not delete snapshots stored in the directory specified by the snapshot-path option. To prevent snapshots from consuming too much storage space, delete snapshots that you no longer need from the directory specified by the snapshot-path option.
  • Additional overheads are required to read and write snapshot information. We recommend that you do not use this option in the following scenarios: The number of objects to upload is small. Network conditions are good. Other users need to perform operations on those objects. To implement incremental upload, we recommend that you use the --update option.
  • You can specify both the --update and --snapshot-path options in a command. ossutil determines whether to skip an object first based on the snapshots stored in the directory specified by --snapshot-path. If no snapshots are generated for the object, ossutil determines whether to skip the file based on the --update option.

Batch upload objects that meet specified conditions

When you run the cp command to batch upload objects and specify the --include and --exclude options in the command, ossutil batch uploads objects that meet the specified conditions.

The --include and --exclude options support the following formats:
  • Asterisk (*): matches all characters. For example, *.txt indicates all TXT files.
  • Question mark (?): matches a single character. For example, abc?.jpg specifies all JPG objects whose names start with abc and are followed by a single character such as abc1.jpg.
  • [sequence]: matches characters in a sequence. For example, abc[1-5].jpg specifies the objects whose names start with abc and are followed by a number contained in sequence [1-5]. The objects include abc1.jpg, abc2.jpg, abc3.jpg, abc4.jpg, and abc5.jpg.
  • [sequence]: matches characters in a sequence. For example, abc[!0-7].jpg specifies that objects abc0.jpg, abc1.jpg, abc2.jpg, abc3.jpg, abc4.jpg, and abc5.jpg, abc6.jpg, and abc7.jpg are not matched.
A rule can contain multiple conditions specified by --include and --exclude. After these conditions are configured, ossutil reads each rule from left to right to obtain the final matching results. If the test.txt object exists in a directory for which conditions are specified, results are generated based on different matching rules.
  • Rule 1: --include "*test*" --exclude "*.txt". When ossutil reads the --include "*test*" condition, the test.txt object matches the condition. When ossutil reads the --exclude "*.txt" condition, the test.txt object is excluded because the object name is in the TXT format. The final matching results exclude the test.txt object.
  • Rule 2: --exclude "*.txt" --include "*test*". When ossutil reads the --exclude "*.txt" condition, the test.txt object is excluded. When ossutil reads the --include "*test*" condition, the test.txt object matches the condition because its name contains test. The final matching results include the test.txt object.
  • Rule 3: --include "*test*" --exclude "*.txt" --include "te?t.txt" . When ossutil reads the --include "*test*" condition, the test.txt object matches the condition. When ossutil reads the --exclude "*.txt" condition, the test.txt object is excluded because the object name is in the TXT format. When ossutil reads the --include "te?t.txt" condition, the test.txt object matches the condition. The final matching results include the test.txt object.
Notice Conditions that include directory names such as --include "/usr/test/.jpg" are not supported.
The following examples show how to run the cp command to specify conditions to upload only objects that match the conditions:
  • Upload all objects that are in the TXT format
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --include "*.txt" -r
  • Upload all objects that contain abc in their names and are not in the JPG or TXT format
    ./ossutil64 cp localfolder/ oss://examplebucket/desfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r

Common options

If you use ossutil to switch to a bucket that is located in a different region, you can use the -e option to specify the endpoint of the region in which the specified bucket resides. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, you can use the -i option to specify the AccessKey ID of the specified account, and use the -k option to specify the AccessKey secret of the specified account.

For example, you can run the following command to upload the exampleobject.txt local file to the destfolder directory of a bucket named examplebucket. The bucket is located in the China (Shanghai) region and owned by another Alibaba Cloud account.
./ossutil64 cp exampleobject.txt oss://examplebucket/desfolder/ -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

For more information about other common options that you can use for the cp command, see Common options.