The cp command is used to upload, download, or copy an object.

Command syntax

  • Upload objects
    ./ossutil cp file_url cloud_url  [-r] [-f] [-u] [--output-dir=odir] [--bigfile-threshold=size] [--checkpoint-dir=cdir] [--snapshot-path=sdir] [--payer requester]
  • Download objects
    ./ossutil cp cloud_url file_url  [-r] [-f] [-u] [--output-dir=odir] [--bigfile-threshold=size] [--checkpoint-dir=cdir] [--range=x-y] [--payer requester]
  • Copy objects
    ./ossutil cp cloud_url cloud_url [-r] [-f] [-u] [--output-dir=odir] [--bigfile-threshold=size] [--checkpoint-dir=cdir] [--payer requester]

Examples

  • Upload objects
    • Upload a single object
      ./ossutil cp a.txt oss://bucket/path
    • Upload a single object and specify the --meta option
      When uploading an object, you can use the --meta option to configure object metadata in the header:value#header:value... format. Run the following command to upload the a.txt file and configure its metadata:
      ./ossutil cp a.txt oss://bucket/path --meta=Cache-Control:no-cache#Content-Encoding:gzip
      Note For more information about meta settings, see set-meta.
    • Upload a folder
      You can add the -r option to the cp command to upload a folder to OSS. The command is as follows:
      ./ossutil cp -r dir oss://bucket/path
      Note If the object to be uploaded is a symbolic link that maps to a local folder, add a forward slash (/) to the symbolic link when you use the cp command to upload the object.
      ./ossutil cp -r symbolic_link/ oss://bucket/path
    • Upload an object or folder and configure the maximum upload speed
      When uploading an object, use the --maxupspeed option to configure the maximum upload speed measured in KByte/s. The default value is 0, which indicates that there is no maximum upload speed.
      • Upload an object and set the maximum upload speed to 1 MByte/s
        ./ossutil cp a.jpg oss://bucket/path --maxupspeed 1024
      • Upload a folder and set the maximum upload speed to 1 MByte/s
        ./ossutil cp -r dir oss://bucket/path --maxupspeed 1024
    • Upload multiple objects that meet specified conditions

      When copying multiple objects, you can add the --include/--exclude option to the cp command to specify conditions for which to select objects.

      The --include/--exclude option supports the following formats:
      • *: matches all characters. For example, *.txt represents any txt objects.
      • ? : matches any single character. For example, abc?.jpg represents all JPG objects whose names contain abc followed by any single character, such as abc1.jpg.
      • [sequence]: matches any characters in a sequence. For example, abc[1-5].jpg specifies the objects whose names begin with abc and are followed by a number contained in the sequence [1-5], such as abc1.jpg~abc5.jpg.
      • [! sequence]: matches any characters that are not in a sequence. For example, abc[! 0-7].jpg specifies the objects whose names begin with abc and are not followed by a number contained in the sequence [0-7], such as abc0.jpg~abc7.jpg.
      Note
      • Names that include directories, such as --include "/usr/test/.jpg", are not supported.
      • To specify the --include/--exclude option, you must also specify the --recursive(-r) option.
      • One rule can contain multiple conditions specified by include and exclude. If these conditions are set, ossutil reads each rule sequentially from left to right to obtain the final matching results.
        If the test.txt object exists in a valid folder, results are generated based on different matching rules.
        • Rule 1: --include "*test*" --exclude "*.txt" . When ossutil reads the --include "*test*" condition, the test.txt object matches the condition. When ossutil reads the --exclude "*.txt" condition, the test.txt object is in the TXT format and is excluded. The final matching results exclude the test.txt object.
        • Rule 2: --exclude "*.txt" --include "*test*". When ossutil reads the --exclude "*.txt" condition, the matching results exclude the test.txt object. When ossutil reads the --include "*test*" condition, the test.txt object name contains test and matches the condition. The final matching results include the test.txt object.
        • Rule 3: --include "*test*" --exclude "*.txt" --include "te? t.txt" . When ossutil reads the --include "*test*" condition, the test.txt object matches the condition. When ossutil reads the --exclude "*.txt" condition, the test.txt object is in the TXT format and is excluded. When ossutil reads the --include "te? t.txt" condition, the test.txt object matches the condition. The final matching results include the test.txt object.
      Examples:
      • Upload all objects that are in the txt format.
        ./ossutil cp dir/ oss://my-bucket/path --include "*.txt" -r
      • Upload all objects that contain abc in their names and are not in the jpg or txt format.
        ./ossutil cp dir/ oss://my-bucket/path --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r
    • Upload an object and modify its storage class
      When uploading an object, you can use the --meta option to modify the storage class of the object.
      • Upload an object and set its storage class to IA.
        ./ossutil cp dir/sys.log  oss://my-bucket/path --meta X-oss-Storage-Class:IA
      • Upload a folder and set the storage class of all objects in the folder to IA.
        ./ossutil cp dir/ oss://my-bucket/path --meta X-oss-Storage-Class:IA -r
    • Upload an object and specify the server-side encryption method
      You can specify the server-side encryption method when you upload an object. After you encrypt the object, you can store the object in a bucket. For more information about server-side encryption, see Server-side encryption.
      • Upload an object and set the server-side encryption method to AES256
        ./ossutil cp a.txt oss://my-bucket/path --meta=x-oss-server-side-encryption:AES256
      • Upload an object and set the server-side encryption method to KMS
        ./ossutil cp a.txt oss://my-bucket/path --meta=x-oss-server-side-encryption:KMS

        When you use KMS to encrypt an object, OSS creates a master key in the KMS console for the object, which will incur a small fee when the KMS API operation is called.

    • Upload a folder without uploading the existing objects

      If you specify the --update (abbreviated as -u) option when uploading multiple objects, ossutil only uploads the objects that do not exist at the destination or that have a later last modification time than that of the destination objects. The command is as follows:

      ./ossutil cp -r dir oss://bucket1/path -u

      This option can be used to perform incremental upload on objects that have not been successfully uploaded.

    • Upload a folder and generate snapshot information

      If you specify the --snapshot-path option when uploading multiple objects, ossutil takes a snapshot of the upload and stores the snapshot information in the specified directory. When this option is specified the next time objects are uploaded, ossutil will read the snapshot information from the specified directory to perform incremental upload. The command is as follows:

      ./ossutil cp -r dir oss://bucket1/path --snapshot-path=path                                
      Notice
      • The --snapshot-path option is used in certain scenarios to accelerate the incremental upload or download of multiple objects. This option can be used when the number of objects is large and no other users modify the corresponding objects in OSS during two uploads. This option cannot be used to copy objects.
      • The --snapshot-path option records the local lastModifiedTime of uploaded or downloaded objects, and then compares that recorded lastModifiedTime with that of objects to be uploaded or downloaded next time to determine which objects can be skipped. When using this option, make sure that the corresponding objects in OSS are not modified during the two uploads or downloads. In other scenarios where objects are updated in OSS during the two uploads or downloads, use the --update option to perform incremental upload or download on objects.
      • ossutil does not automatically delete snapshot information from the directory specified by snapshot-path. If the snapshot information is not necessary to retain, remove the directory.
      • Additional overheads are required to read and write snapshot information. We recommend that you do not use this option in any of the following scenarios: The number of objects to be uploaded or downloaded is small. The network is properly connected. Other users need to perform operations on those objects. In this case, you can use the --update option to perform incremental upload or download.
      • The --update and --snapshot-path options can be used together. ossutil first uses the --snapshot-path option to determine whether to skip the upload or download of an object. If the upload or download is not skipped by --snapshot-path, ossutil then uses the --update option to determine whether to skip the upload or download of the object.
    • Upload an object to a bucket that has the pay-by-requester mode enabled
      ./ossutil cp dir/test.mp4 oss://payer/ --payer=requester
  • Download objects
    • Download a single object
      ./ossutil cp oss://my-bucket/path/test1.txt /dir
    • Download parts of an object within a specified range

      You can use the --range option to download parts of an object within a specified range.

      For example, download the 10th to 20th characters from the test1.txt object to your local device as a file.
      ./ossutil cp oss://my-bucket/path/test1.txt /dir  --range=10-20
      Succeed: Total num: 1, size: 11. OK num: 1(download 1 objects).
      0.290769(s) elapsed
    • Download an object from a bucket that has the pay-by-requester mode enabled
      ./ossutil cp oss://payer/test.mp4 dir/ --payer=requester
    • Download a folder
      ./ossutil cp -r oss://my-bucket/path /dir 
    • Download a folder and specify the --update option
      If you specify the --update option when downloading multiple objects, ossutil only downloads the objects that do not exist at the destination, or that have a later last modification time than that of the destination objects. The command is as follows:
      ./ossutil cp -r oss://bucket/path  /dir  --update                           
      This option can be used to perform incremental download on objects that have not been successfully downloaded.
    • Download a folder and generate snapshot information

      If you specify the --snapshot-path option when downloading multiple objects, ossutil takes a snapshot of the download and stores the snapshot information in the specified directory. When this option is specified the next time objects are downloaded, ossutil will read the snapshot information from the specified directory to perform incremental download. For more information, see --snapshot-path. The command is as follows:

      ./ossutil cp -r oss://bucket1/path dir --snapshot-path=path                                
    • Download multiple objects that meet specified conditions
      When downloading multiple objects, you can use --include/--exclude to specify conditions for which to select objects. For more information, see cp.
      • Download all objects that are not in the jpg format
        ./ossutil cp oss://my-bucket/path dir/ --exclude "*.jpg" -r
      • Download all objects that contain abc in their names and are not in the jpg or txt format
        ./ossutil cp oss://my-bucket1/path dir/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r
  • Copy objects

    Only objects can be copied. Parts cannot be copied. You cannot copy objects across regions.

    • Copy a single object
      ./ossutil cp oss://bucket/path1/a oss://bucket/path2/                                 
    • Copy and rename a single object
      ./ossutil cp oss://bucket/path1/a oss://bucket/path2/b                           
    • Copy multiple objects that meet specified conditions
      When copying multiple objects, you can use the --include/--exclude command to specify conditions for which to select objects. For more information, see cp.
      • Copy all objects that are not in the jpg format
        ./ossutil cp oss://my-bucket1/path oss://my-bucket2/path --exclude "*.jpg" -r
      • Copy all objects that contain abc in their names and are not in the jpg or txt format
        ./ossutil cp oss://my-bucket1/path oss://my-bucket2/path --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r
    • Copy an object and modify its storage class
      • Set the storage class of an existing object to Archive
        ./ossutil cp oss://my-bucket/path/0104_6.jpg oss://my-bucket/path/0104_6.jpg --meta X-oss-Storage-Class:Archive
      • Set the storage class of all files in an existing folder to Standard
        ./ossutil cp oss://my-bucket/path/ oss://my-bucket/path/ --meta X-oss-Storage-Class:Standard -r
        Notice
        • You cannot use the cp command to convert the storage class of an object from Archive to any other storage class. You must first use the restore command to restore the object to the readable state, and then use the cp command to modify its storage class.
        • When you run the cp command to overwrite an object, it is considered overwritten and you may incur fees. If objects of the IA or Archive storage class are overwritten 30 or 60 days after they are created, early deletion fees will be incurred. For more information, see Billing items.
    • Copy an object and specify the server-side encryption method
      You can specify the server-side encryption method when you copy an object. After you encrypt the object, you can store the object in a bucket. For more information about server-side encryption, see Server-side encryption.
      • Copy an object and set the server-side encryption method to AES256
        ./ossutil cp oss://bucket/path1/a oss://bucket/path2/ --meta=x-oss-server-side-encryption:AES256
      • Copy an object and set the server-side encryption method to KMS
        ./ossutil cp oss://bucket/path1/a oss://bucket/path2/ --meta=x-oss-server-side-encryption:KMS

        When you use KMS to encrypt an object, OSS creates a master key in the KMS console for the object, which will incur a small fee when the KMS API operation is called.

    • Copy a single object and specify the --meta option
      When copying an object, you can use the --meta option to configure object metadata in the header:value#header:value... format.
      ./ossutil cp oss://bucket/path1/a oss://bucket/path2/ --meta=Cache-Control:no-cache
    • Copy multiple objects between different buckets within the same region
      You can add the -r option to the cp command to copy multiple objects between different buckets within the same region.
      ./ossutil cp oss://your_src_bucket/path1/ oss://your_dest_bucket/path2/ -r                                   
    • Perform incremental copy on objects between different buckets within the same region
      If you specify the --update option when copying multiple objects, ossutil only copies the objects that do not exist at the destination or that have a later last modification time than that of the destination objects. The command is as follows:
      ./ossutil cp oss://your_src_bucket/path1/ oss://your_dest_bucket/path2/ -r --update
      This option can be used to perform incremental copy on objects that have not been successfully copied.
    • Copy an object from a bucket that has the pay-by-requester mode enabled to a common bucket
      ./ossutil cp oss://payer/test.mp4 oss://my-bucket/path  --payer=requester
    • Copy an object from a common bucket to a bucket that has the pay-by-requester mode enabled
      ./ossutil cp oss://my-bucket/path/test.mp4 oss://payer --payer=requester
  • Optimize the object upload, download, and copy performance
    You can add the --jobs and --parallel options to the cp command to specify the number of concurrent operations. If the default number of concurrent operations set by ossutil does not meet your performance requirements, you can modify the values of these two options to adjust the performance.
    • The --jobs option specifies the number of concurrent tasks when multiple files are uploaded, downloaded, or copied.
    • The --parallel option specifies the number of concurrent operations performed on a large object during multipart upload, download, or copy.

    ossutil calculates the number of concurrent operations based on the object size by default. This option does not work for small objects. The threshold for large objects to be uploaded, downloaded, or copied in multipart mode can be specified through the --bigfile-threshold option. When you upload, download, or copy multiple objects, the actual number of concurrent operations is calculated by multiplying the number of jobs by the number of concurrent operations.

    Warning
    • We recommend that you adjust the number of concurrent operations to a value smaller than 100 if the resources of your ECS instance or server (such as network bandwidth, memory, and CPU) are limited. If resources such as the network, memory, and CPU are not fully occupied, you can increase the number of concurrent operations.
    • If there are too many concurrent operations, the upload, download, and copy performance of ossutil may degrade or an EOF error may occur due to inter-thread resource switching and snatching. To resolve this issue, you must adjust the values of the --jobs and --parallel options as required. To perform pressure testing, set the two options to small values first, and slowly adjust them to the optimal values.

Common options

The following table describes the options you can add to the cp command.
Option Description
-r, --recursive Recursively performs operations on objects in a bucket. If this option is specified, commands that support this option will perform operations on all objects in a bucket that meet the specified conditions. If this option is not specified, the commands will only perform operations on a single specified object.
-f, --force Forces an operation without prompting the user for confirmation.
-u, --update ossutil only uploads, downloads, or copies the objects that do not exist at the destination or that have a later last modification time than that of the destination objects.
--output-dir Specifies the directory in which output objects are located. Output objects include report objects generated due to errors that occur when you use the cp command to copy multiple objects. The default value is the ossutil_output directory in the current directory.
--bigfile-threshold Specifies the object size for which to start resumable data transfer. Unit: Byte. Valid values: non-negative integers. Default value: 100 MB.
--part-size Specifies the size of each part during multipart upload, download, or copy of large objects. Unit: Byte. ossutil calculates the appropriate part size based on the object size by default. You can set this option to any positive integer if you need to optimize performance or are operating under special constraints.
--checkpoint-dir Specifies the checkpoint directory. Default value: .ossutil_checkpoint. When a resumable data transfer fails, ossutil automatically creates a directory and records the checkpoint information in the directory. When the resumable transfer succeeds, ossutil deletes this directory. If this option is specified, ensure that the specified directory can be deleted.
--range Specifies the range of the object to be downloaded. This option value must be in the 3-9, 3-, or -9 format.
--encoding-type Specifies the encoding type of the object name. If this option is specified, this value must be url. If this option is not specified, the object name is not encoded. Bucket names cannot be URL-encoded.
--include Includes objects that match a specified string, such as *.jpg.
--exclude Excludes objects that match a specified string, such as *.txt.
--meta Configures the metadata of an object in the [header:value#header:value...] format. Example: Cache-Control:no-cache#Content-Encoding:gzip For more information, see set-meta.
--acl Configures the ACL for an object. Default value: default. Valid values:
  • default: The object inherits the ACL for the bucket it belongs to.
  • private
  • public-read
  • public-read-write
--snapshot-path If you specify the --snapshot-path option when uploading or downloading multiple objects, ossutil takes a snapshot of the upload or download and stores the snapshot information in the specified directory. When this option is specified the next time objects are uploaded or downloaded, ossutil will read the snapshot information from the specified directory to perform incremental upload or download.
Notice
  • The --snapshot-path option is used in certain scenarios to accelerate the incremental upload or download of objects. This option can be used when the number of objects is large and no other users modify the corresponding objects in OSS during two uploads. This option cannot be used to copy objects.
  • The --snapshot-path option records the local lastModifiedTime of uploaded or downloaded objects, and then compares that recorded lastModifiedTime with that of objects to be uploaded or downloaded next time to determine which objects can be skipped. When using this option, make sure that the corresponding objects in OSS are not modified during the two uploads or downloads. In other scenarios where objects are updated in OSS during the two uploads or downloads, use the --update option to perform incremental upload or download on objects.
  • ossutil does not automatically delete snapshot information from the directory specified by snapshot-path. If the snapshot information is not necessary to retain, remove the directory.
  • Additional overheads are required to read and write snapshot information. We recommend that you do not use this option in any of the following scenarios: The number of objects to be uploaded or downloaded is small. The network is properly connected. Other users need to perform operations on those objects. In this case, you can use the --update option to perform incremental upload or download.
  • The --update and --snapshot-path options can be used together. ossutil first uses the --snapshot-path option to determine whether to skip the upload or download of an object. If the upload or download is not skipped by --snapshot-path, ossutil then uses the --update option to determine whether to skip the upload or download of the object.
--disable-crc64 Disables CRC64. ossutil enables CRC64 during data transmission by default.
--maxupspeed Specifies the maximum upload speed in KByte/s. The default value is 0, which indicates that there is no maximum upload speed.
--payer Specifies the payer of the request. To enable the pay-by-requester mode, set this option to requester.
--partition-download Used to download object partitions. The value of this option is in the "partition number:total number of partitions" format. A value of 1:5 indicates that ossutil downloads partition 1 out of the total five partitions. Partitioning rules for objects are determined by ossutil. This option divides an object to be downloaded into multiple partitions that can be downloaded by multiple ossutil commands. Each ossutil command downloads its own partition. You can run multiple ossutil commands on different ECS instances in parallel.
-j, --jobs Specifies the number of concurrent operations performed across multiple objects. Valid values: 1 to 10000. Default value: 3.
--parallel Specifies the number of concurrent operations performed on a single object. Valid values: 1 to 10000.The value of this option is determined by ossutil based on the operation type and object size by default.
--loglevel Specifies the log level. The default value is null, indicating that no log files are generated. Valid values:
  • info: generates prompt logs.
  • debug: generates detailed logs that contain corresponding HTTP request and response information.
--retry-times Specifies the number of times an operation is retried if the operation fails. Valid values: 1 to 500. Default value: 10.
--proxy-host Specifies the URL of the proxy server. HTTP, HTTPS, and SOCKS5 are supported. An example of the URL is http://120.79. **.**:3128 or socks5://120.79. **. **:1080.
--proxy-user Specifies the username of the proxy server. The default value is null.
--proxy-pwd Specifies the password of the proxy server. The default value is null.
Note For more information about common options, see View all supported options.