This topic describes how to run the cp command to download objects from Object Storage Service (OSS) to the local computer.

Note Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.

Command syntax

./ossutil64 cp cloud_url file_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxdownspeed <value>]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--range <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--snapshot-path <value>]
[--disable-crc64]
[--payer <value>]
[--partition-download <value>]
[-j, --job <value>]
[--parallel <value>]
[--version-id <value>]

The following table describes the parameters that you can configure when you run this command to download objects from OSS.

Parameter Description
cloud_url The path of the object that you want to download from OSS. Specify the parameter in the oss://bucketname/objectname format. Example: oss://examplebucket/examplefile.txt.
file_url The local path of the object after you download the object from OSS. Examples: The local path in Linux is /localfolder/examplefile.txt. The local path in Windows is D:\localfolder\examplefile.txt.
-r, --recursive Specifies recursive operations. If this option is specified, commands that support this option are run to perform operations on all objects in a bucket that meet the specified conditions. If this option is not specified, commands that support this option are run to perform operations only on the specified object.
-f --force Specifies the command to forcibly run without prompting the user for confirmation.
-u, --update Specifies that ossutil downloads an object from OSS only when the destination file does not exist in the local computer or when the last modified time of the source object is later than that of the destination file.
--maxdownspeed Specifies the maximum download speed. Unit: KB/s. The default value is 0, which indicates that the download speed is not limited.
--disable-ignore-error Specifies that errors are not ignored during batch operations.
--only-current-dir Specifies that only objects in the current directory are downloaded. Subdirectories in the current directory and objects in these subdirectories are not downloaded.
-bigfile-threshold Specifies the maximum size of objects that can be downloaded by using resumable upload or download. Unit: bytes.

Default value: 104857600 (100 MB).

Valid values: 0 to 9223372036854775807.

--part-size Specifies the size of each part. Unit: bytes. By default, ossutil determines the size of each part based on the object size.

Valid values: 1 to 9223372036854775807.

--checkpoint-dir Specifies the directory in which the log objects of resumable upload or download tasks are stored. Default value: .ossutil_checkpoint. When a resumable download task fails, ossutil creates this directory and stores the checkpoint information about the task in this directory. ossutil deletes the directory after the task is completed. Therefore, if you specify a checkpoint directory, make sure that ossutil can delete the directory.
--range Specifies that a specific range of the content of the object is downloaded and stored as a new file in the destination local path. The minimum start value of the range is 0, which indicates the byte 0 of the content of the object. You can set this parameter in one of the following formats:
  • Specify a range

    For example, the value of 3-9 indicates a range from byte 3 to byte 9, which includes byte 3 and byte 9.

  • Specify the start value

    For example, the value of 3- indicates a range from byte 3 to the end of the object, which includes byte 3.

  • Specify the end value

    For example, the value of -9 indicates a range from byte 0 to byte 9, which includes byte 9.

--encoding-type Specifies the method used to encode the names of objects. Valid values: url. If you do not specify this parameter, the names of objects are not encoded.
--include Specifies that the command applies to all objects that meet the specified conditions.
--exclude Specifies that the command applies to all objects that do not meet the specified conditions.
--meta Specifies the object metadata in the header:value#header:value format. Example: Cache-Control:no-cache#Content-Encoding:gzip. For more information about object metadata, see set-meta.
--acl Specifies the access control list (ACL) of the object to download. Default value: private. Valid values:
  • default: The ACL of the object is the same as the ACL of the bucket in which the object is stored.
  • private: Only the bucket owner can perform read and write operations on objects in the bucket. Other users cannot access the objects in the bucket.
  • public-read: Only the bucket owner can perform write operations on objects in the bucket. Other users, including anonymous users, can perform only read operations on the objects in the bucket. If you set this parameter to public-read, the objects may be unexpectedly accessed, which results in out-of-control costs. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. Therefore, we recommend that you do not set the object ACL to this value except in special cases.
  • public-read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. If you set this parameter to public-read-write, data leakage may occur and you may be charged additional fees. Exercise caution when you set this parameter to public-read-write.
--snapshot-path Specifies the directory in which the snapshots of downloaded objects are stored. In the next download task, ossutil reads the snapshots in this directory to download only incremental objects.
--disable-crc64 Specifies that CRC-64 is disabled. By default, CRC-64 is enabled when you use ossutil to transfer data.
--payer Specifies the payer of the traffic and request fees charged to download the content of the object. If you want the requester who accesses the resources in the specified path to pay for the traffic and request fees charged during downloads, set this parameter to requester.
--partition-download Specifies the partition in which the object that you want to download is stored. The value of this option is in the partition number: the total number of partitions format. You can run multiple ossutil commands on different machines at the same time. A value of 1:5 indicates that ossutil downloads partition 1 out of the five partitions. Partitions are numbered from 1. Partitioning rules for objects are determined by ossutil. This option splits an object into multiple partitions that can be concurrently downloaded by multiple ossutil commands. Each ossutil command downloads its own partition.
-j, --job Specifies the number of concurrent tasks performed across multiple objects. Valid values: 1 to 10000. Default value: 3.
--parallel Specifies the number of concurrent tasks performed for a single object. Valid values: 1 to 10000. By default, if you do not set this parameter, ossutil sets the value of this parameter based on the operation type and the size of the object to download.
--version-id Specifies the version ID of the object that you want to download. You can use this option for objects only in a bucket that has versioning enabled.

When the default configuration of the number of concurrent tasks based on the -j, --jobs and --parallel options cannot meet your performance requirements, you can configure these two parameters based on your requirements. By default, ossutil calculates the number of concurrent operations based on the size of the object to download. When you download multiple large objects, the actual number of concurrent tasks is calculated by multiplying the number of concurrent jobs by the number of concurrent operations.

  • We recommend that you adjust the number of concurrent tasks to a value smaller than 100 if the resources such as network bandwidth, memory, and CPU of your Elastic Compute Service (ECS) instance or server are limited. If resources such as the network bandwidth, memory, and CPU are available, you can increase the number of concurrent operations.
  • If the number of concurrent tasks is too large, the download performance of ossutil may decrease, or an end-of-file (EOF) error may occur because thread resources are switched and threads compete for resources. To resolve this issue, you must adjust the values of the -j, --jobs and --parallel options based on the actual conditions of machines. When you perform stress testing, set a small value for the two options before you incrementally increase them to the optimal values.

Sample environment

In this topic, objects are downloaded from OSS to your local directory in Linux. You can modify the parameters in the following examples based on your operating system and environment. Sample environment:

  • Operating system: Linux
  • Destination bucket: examplebucket
  • Directory of the destination bucket: desfolder
  • Local file: examplefile.txt. This file is stored in the root directory of the local computer.
  • Local directory: localfolder. This directory is stored in the root directory of the local computer.

Simple download

  • Download a single object

    If the name of the local file is not specified for the object you want to download, the original name of the object is used by default. If the name of local file is specified, the file is saved to your local computer based on the specified name.

    • You can run the following command to save the local file based on the original name of the object to download:
      ./ossutil64 cp oss://examplebucket/desfolder/examplefile.txt localfolder/
    • You can run the following command to save the local file based on a specified name:
      ./ossutil64 cp oss://examplebucket/desfolder/examplefile.txt localfolder/example.txt
  • Download a directory
    ./ossutil64 cp -r oss://examplebucket/desfolder/ localfolder/
  • Download an object from a bucket that has pay-by-requester enabled
    ./ossutil64 cp oss://my-bucket/test.mp4 localfolder/ --payer=requester
  • Download multiple objects without downloading the objects that already exist on your local computer
    When you download multiple objects from OSS, specify the --update option, which can be shortened to -u. Then, oosutil only downloads objects that do not have the same name as a local file and objects that have the same name as a local file and have a more recent last modified time than the local file. You can run the following command to download multiple objects that do not exist on your local computer:
    ./ossutil64 cp -r oss://examplebucket/desfolder/  localfolder/  --update                           
    This option can be used to download objects that fail to download or to skip downloaded objects when you download incremental objects.
  • Download objects only in the current directory and ignore subdirectories
    ./ossutil64 cp oss://examplebucket/desfolder/ localfolder/ --only-current-dir -r

Limit download speed

To limit the maximum download speed, you can specify the --maxdownspeed parameter. Unit: KB/s. Examples:
  • You can run the following command to download objects from OSS and set the value of the --maxdownspeed parameter to 1 MB/s:
    ./ossutil64 cp oss://examplebucket/desfolder/examplefile.txt  localfolder/ --maxdownspeed 1024
  • You can run the following command to download directories from OSS and set the value of the --maxdownspeed parameter to 1 MB/s:
    ./ossutil64 cp -r  oss://examplebucket/desfolder/  localfolder/ --maxdownspeed 1024

Range download

To specify a range of the content of an object to download, specify the --range parameter. For example, you can run the following command to download the 10th to 20th characters of the examplefile.txt object to your local computer:
./ossutil64 cp oss://examplebucket/desfolder/examplefile.txt localfolder/  --range=10-20
Succeed: Total num: 1, size: 11. OK num: 1(download 1 objects).

Download a directory and generate snapshot information

If you specify the --snapshot-path option when you download multiple objects, ossutil takes a snapshot of the objects and stores the snapshot information in a specified directory. Next time the objects are downloaded while this option is specified, ossutil reads the snapshot information from the specified directory and downloads only incremental objects. For more information, see Generate snapshots for uploaded objects.
./ossutil64 cp -r oss://examplebucket/desfolder/ localfolder/ --snapshot-path=path                                

Download multiple objects that meet specified conditions

To download objects that meet specified conditions, specify the --include and --exclude parameters. For more information, see Batch upload files that meet specified conditions.
  • You can run the following command to download all objects that are not in the JPG format:
    ./ossutil64 cp oss://examplebucket/desfolder/ localfolder/ --exclude "*.jpg" -r
  • You can run the following command to download all objects that contain abc in their names and are not in the JPG or TXT format:
    ./ossutil64 cp oss://examplebucket/desfolder/ localfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r

Download a specified version of an object from a versioning-enabled bucket

After you enable versioning for a bucket, objects that are overwritten or deleted in the bucket are saved as previous versions. You can run the following cp command to which the --version-id parameter is added to download a specified version of an object:
./ossutil64 cp oss://my-bucket/test.jpg localfolder/ --version-id  CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk
Notice
  • To use the --version-id parameter, you must run the ls --all-versions command to query the version IDs of the object.
  • The --version-id parameter can be used for objects only in versioning-enabled buckets. For more information about the command used to enable versioning for a bucket, see bucket-versioning.

Common options

To use ossutil to manage buckets that are located in different regions, you can use the -e option to use the endpoint of the specified bucket. To use ossutil to manage buckets that are owned by different Alibaba Cloud accounts, you can use the -i option to use the AccessKey ID of the specified account, and use the -k option to use the AccessKey secret of the specified account.

For example, you can run the following command to download the exampleobject.txt object in the root directory of the examplebucket bucket to the local directory named localfolder. The source bucket is located in the China (Shanghai) region and is owned by another Alibaba Cloud account.
./ossutil64 cp oss://examplebucket/exampleobject.txt  localfolder/  -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

For more information about other common options that you can use for the cp command, see Common options.