This topic describes how to run the cp command to download objects from Object Storage Service (OSS) to your on-premises device.

Notice Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.

Command syntax

./ossutil64 cp cloud_url file_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxdownspeed <value>]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--range <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--snapshot-path <value>]
[--disable-crc64]
[--payer <value>]
[--partition-download <value>]
[-j, --job <value>]
[--parallel <value>]
[--version-id <value>]

The following table describes the parameters that you can configure to run the cp command.

Option Description
cloud_url Specifies the path of the object that you want to download from OSS. Format: oss://bucketname/objectname. Example: oss://examplebucket/examplefile.txt.
file_url Specifies the on-premises path to which you want to download the object from OSS. Examples: The path of the on-premises file in Linux is /localfolder/examplefile.txt. The path of the on-premises file in Windows is D:\localfolder\examplefile.txt.
-r, --recursive The recursive operations. If you specify this option in a command, ossutil performs operations on all objects in a bucket that meet the specified condition. If you do not specify this option in a command, ossutil performs operations only on the specified object.
-f --force Specifies the operation to forcefully perform. The operation is performed without a prompt for confirmation.
-u, --update Specifies that ossutil downloads an object from OSS only when the destination file does not exist in the on-premises device or when the last modified time of the source object is later than that of the destination file.
--maxdownspeed Specifies the maximum download speed. Unit: KB/s. The default value is 0, which indicates that the download speed is unlimited.
--disable-ignore-error Specifies that errors are not ignored during batch operations.
--only-current-dir Specifies that only objects in the current directory are downloaded. Subdirectories in the current directory and objects in these subdirectories are not downloaded.
--bigfile-threshold Specifies the maximum size of objects that can be downloaded by using resumable download. Unit: bytes.

Default value: 104857600. This value is equivalent to 100 MB.

Valid values: 0 to 9223372036854775807.
--part-size Specifies the part size. Unit: bytes. By default, ossutil determines the part size based on the object size.

Valid values: 1 to 9223372036854775807.
--checkpoint-dir Specifies the directory in which the log information about resumable download tasks is stored. If a resumable download task fails, ossutil automatically creates the .ossutil_checkpoint directory and stores the checkpoint information in the directory. ossutil deletes the .ossutil_checkpoint directory after the task is completed. Therefore, if you specify a checkpoint directory, make sure that the directory can be deleted.
--range Specifies that a specific range of the object content is downloaded and stored as a new object in the destination on-premises path. The minimum start value of the range is 0, which indicates the byte 0 of the content of the object. You can specify a value for this parameter in one of the following formats:
  • Data range

    For example, 3-9 indicates a range from byte 3 to byte 9, which includes byte 3 and byte 9.

  • Start value

    For example, 3- indicates a range from byte 3 to the end of the object, which includes byte 3.

  • End value

    For example, -9 indicates a range from byte 0 to byte 9, which includes byte 9.

--encoding-type Specifies the method that is used to encode the names of objects. Set the value to url. If you do not specify this parameter, the names of objects are not encoded.
--include Specifies that the command takes effect on all objects that meet the specified conditions.
Note You cannot use this option to specify that the command takes effect on directories, such as ./ossutil64 cp oss://examplebucket/destfolder/ localfolder/ --include dir/. Otherwise, an error --include or --exclude does not support format containing dir info is reported.
--exclude Specifies that the command takes effect on all objects that do not meet the specified conditions.
Note You cannot use this option to specify that the command takes effect on directories, such as ./ossutil64 cp oss://examplebucket/destfolder/ localfolder/ --exclude dir/. Otherwise, an error --include or --exclude does not support format containing dir info is reported.
--meta Specifies the object metadata in the header:value#header:value format. Example: Cache-Control:no-cache#Content-Encoding:gzip. For more information about object metadata, see set-meta (manage object metadata).
--acl Specifies the access control list (ACL) of an object. Default value: private. Default value: private. Valid values:
  • default: The ACLs of the objects are the same as the ACL of the bucket in which the objects are stored.
  • private: Only the bucket owner can perform read and write operations on objects in the bucket. Other users cannot access the objects in the bucket.
  • public-read: Only the bucket owner can perform write operations on objects in the bucket. Other users, including anonymous users, can perform only read operations on the objects in the bucket. This may result in unauthorized access to the data in your bucket and high fees. Proceed with caution when you set this parameter to public-read.
  • public read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. This may result in unauthorized access to the data in your bucket and high fees. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. Therefore, we recommend that you do not set the object ACL to public-read-write except in special cases.
--snapshot-path Specifies the directory in which the snapshots of downloaded objects are stored. In the next download task, ossutil reads the snapshots in this directory to download only incremental objects.
--disable-crc64 Specifies that CRC-64 is disabled. By default, ossutil enables CRC-64 during data transmission.
--payer The payer of the request. If you want the requester who accesses the resources in the specified path to pay for the traffic and request fees, set this parameter to requester.
--partition-download Specifies the partition in which the object that you want to download is stored. The value of this option is in the partition number: the total number of partitions format. A value of 1:5 specifies that the current command downloads partition 1 and the object is split into five partitions. Partitions are numbered starting from 1. Partitioning rules for objects are determined by ossutil. This option splits an object into multiple partitions that can be concurrently downloaded by multiple ossutil commands. Each ossutil command downloads its own partition.
-j, --job The number of concurrent tasks that are performed across multiple objects. Valid values: 1 to 10000. Default value: 3.
--parallel The number of concurrent tasks run to upload a single object. Valid values: 1 to 10000. By default, if you do not configure this option, ossutil specifies the value of this option based on the operation type and the object size.
--version-id Specifies the version ID of the object that you want to download. You can use this option for objects only in a bucket for which versioning is enabled.

The -j, --jobs and --parallel options in the preceding command syntax can be used to adjust performance if the default number of concurrent tasks does not meet your performance requirements. By default, ossutil calculates the number of concurrent operations based on the object size. When you download multiple large objects, the actual number of concurrent tasks is calculated by multiplying the value of -j, --jobs by the value of --parallel.

  • We recommend that you adjust the number of concurrent tasks to a value less than 100 if the resources such as network bandwidth, memory, and CPU of your ECS instance or server are limited. If resources such as the network bandwidth, memory, and CPU are available, you can increase the number of concurrent operations.
  • If the number of concurrent tasks is too large, the download performance of ossutil may decrease, or an end-of-file (EOF) error may occur because thread resources are switched and threads compete for resources. To resolve this issue, you must adjust the values of the -j, --jobs and --parallel options based on the actual available resources. To perform stress testing, specify a small value for the two options before you incrementally increase them to the optimal values.

Sample environment

In this topic, objects are downloaded from OSS to your on-premises directory in Linux. You can change the parameter settings in the examples based on your operating system and environment. Sample environment:

  • Operating system: Linux
  • Destination bucket: examplebucket
  • Directory of the destination bucket: destfolder
  • Local file: examplefile.txt. This file is stored in the root directory.
  • On-premises directory: localfolder. This directory is stored in the root directory.

Download a single object

If you do not specify a name for the destination object, the name of the source object is used by default. If you specify a name for the destination object, the specified name is used in the on-premises device.

  • You can run the following command to save the destination object with the name of the source object:
    ./ossutil64 cp oss://examplebucket/destfolder/examplefile.txt localfolder/
  • You can run the following command to save the destination object with a specified name:
    ./ossutil64 cp oss://examplebucket/destfolder/examplefile.txt localfolder/example.txt

Download multiple objects

You cannot specify multiple object names in ossutil to download multiple objects at the same time. You can download multiple objects at the same time by using the following methods:

  • Download objects of a specific type

    If you want to download multiple objects whose names contain the same prefix or suffix, you can use the --include and --exclude options to download objects that meet specific conditions.

    • You can run the following command to download all objects that are not in the JPG format: 
      ./ossutil64 cp oss://examplebucket/destfolder/ localfolder/ --exclude "*.jpg" -r
    • You can run the following command to download all objects that contain abc in their names and are not in the JPG format or TXT format: 
      ./ossutil64 cp oss://examplebucket/destfolder/ localfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r
  • Download a directory and its subdirectories

    You can run the following command to download all objects in a directory, including subdirectories in the directory: 

    ./ossutil64 cp -r oss://examplebucket/destfolder/ localfolder/

    If a batch download task fails or you want to perform an incremental download task, you can use the --update option that can be shortened to -u to skip objects that have been downloaded. If an object does not have the same name as on-premises files, or the object is last modified later than the on-premises file that has the same name, ossutil downloads the object. If an object has the same name as on-premises files, and the on-premises file that has the same name is last modified later than the object, ossutil skips the object. Example: 

    ./ossutil64 cp -r oss://examplebucket/destfolder/  localfolder/  --update
  • Download a directory (excluding its subdirectories)

    If you want to download only the current directory and do not want to download its subdirectories, use the --only-current-dir option. Example: 

    ./ossutil64 cp oss://examplebucket/destfolder/ localfolder/ --only-current-dir -r

Limit download speed

To limit the maximum download speed, you can specify the --maxdownspeed parameter. Unit: KB/s. Examples:
  • You can run the following command to download objects from OSS and set the --maxdownspeed parameter to 1 MB/s:
    ./ossutil64 cp oss://examplebucket/destfolder/examplefile.txt  localfolder/ --maxdownspeed 1024
  • You can run the following command to download directories from OSS and set the value of the --maxdownspeed parameter to 1 MB/s: 
    ./ossutil64 cp -r  oss://examplebucket/destfolder/  localfolder/ --maxdownspeed 1024

Download a range of content of an object

To download a range of the content of an object, specify the --range parameter. For example, you can run the following command to download the 10th to 20th characters of the examplefile.txt object to your on-premises device as an object:
./ossutil64 cp oss://examplebucket/destfolder/examplefile.txt localfolder/  --range=10-20
Succeed: Total num: 1, size: 11. OK num: 1(download 1 objects).

Download a directory and generate snapshot information

If you specify the --snapshot-path parameter when you download multiple objects, ossutil takes a snapshot of the objects and stores the snapshot information in a specified directory. If you specify this parameter, ossutil reads the snapshot information from the specified directory and downloads only incremental objects next time you download the objects. For more information, see Generate snapshots for objects when you upload the objects. 
./ossutil64 cp -r oss://examplebucket/destfolder/ localfolder/ --snapshot-path=path

Download a specified version of an object from a versioned bucket

After you enable versioning for a bucket, objects that are overwritten or deleted in the bucket are saved as previous versions. You can run the following cp command to which the --version-id parameter is added to download a specified version of an object:
./ossutil64 cp oss://my-bucket/test.jpg localfolder/ --version-id  CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk
Notice
  • Before you use the --version-id parameter, you must run the ls --all-versions command to obtain the version ID of the object.
  • The --version-id parameter can be used for objects only in buckets for which versioning is enabled. For more information about the command used to enable versioning for a bucket, see bucket-versioning.

Common options

If you use ossutil to switch to a bucket that is located in another region, add the -e option to the command to specify the endpoint of the region in which the specified bucket is located. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, you can add the -i option to the command to specify the AccessKey ID of the specified account, and add the -k option to the command to specify the AccessKey secret of the specified account.

For example, you can run the following command to download the exampleobject.txt object in the root directory of the examplebucket bucket to the on-premises directory named localfolder. The source bucket is located in the China (Shanghai) region and is owned by another Alibaba Cloud account.
./ossutil64 cp oss://examplebucket/exampleobject.txt  localfolder/  -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

For more information about other common options that you can specify in the sync command, see Common options.