Run the cp command to download objects from OSS, Run the cp command to download objects from OSS - Object Storage Service

You can use the cp command to download resources such as images and videos from Object Storage Service (OSS) to your local device. For example, you can download multiple objects, download objects at a limited download speed, and download a specific version of an object in a versioned bucket.

Usage notes

To download an object, you must have the oss:GetObject or oss:ListObjects permission. For more information, see Attach a custom policy to a RAM user.
For ossutil V1.6.16 and later, you can directly use ossutil as the binary name in the command line. You do not need to update the binary name based on the system. For ossutil earlier than V1.6.16, you must specify a binary name that corresponds to the system. For more information, see ossutil command reference.

Command syntax

ossutil cp cloud_url file_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxdownspeed <value>]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--range <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--snapshot-path <value>]
[--disable-crc64]
[--payer <value>]
[--partition-download <value>]
[-j, --job <value>]
[--parallel <value>]
[--version-id <value>]
[--start-time <value>]
[--end-time <value>]

The following table describes the parameters and options in the syntax.

Parameter/Option	Description
cloud_url	The path of the OSS object. The path format is `oss://bucketname/objectname`. Example: `oss://examplebucket/examplefile.txt`.
file_url	The path of the local file. Examples: `/localfolder/examplefile.txt` in Linux and `D:\localfolder\examplefile.txt` in Windows.
-r, --recursive	Specifies recursive operations. If you specify this option, ossutil performs the operation on all objects in the bucket that meet the specified condition. If you do not specify this option, ossutil performs the operation only on the specified object.
-f --force	Forces the operation without requiring confirmation.
-u, --update	Specifies that ossutil downloads an object from OSS only when the destination file does not exist in the local device or when the last modified time of the source object is later than that of the destination file.
--maxdownspeed	The maximum download speed. Unit: KB/s. The default value is 0, which specifies that the download speed is unlimited.
--disable-ignore-error	Specifies that errors are not ignored during batch operations.
--only-current-dir	Specifies that only objects in the current directory are downloaded. Subdirectories in the current directory and objects in these subdirectories are not downloaded.
--bigfile-threshold	The object size threshold for using resumable download. If the size of an object exceeds the specified object size threshold, the object is downloaded by using resumable download. Unit: bytes. Default value: 104857600 (100 MB). Valid values: 0 to 9223372036854775807.
--part-size	The part size. Unit: bytes. By default, ossutil determines the part size based on the object size. Valid values: 1 to 9223372036854775807.
--checkpoint-dir	The directory in which the information of the resumable download task is stored. If the task fails, ossutil automatically creates a directory named `.ossutil_checkpoint` and saves the checkpoint information in the directory. After the task is successful, ossutil deletes the directory. If you specify a checkpoint directory, make sure that the directory can be deleted.
--range	Specifies that a specific range of the object content is downloaded and stored as a new file in the destination path. The minimum start value of the range is 0, which indicates the byte 0 of the content of the object. You can specify a value for this option in one of the following formats: A range with a start value and end value For example, a value of `3-9` indicates a range from byte 3 to byte 9. Start value For example, a value of `3-` indicates a range from byte 3 to the end of the object. End value For example, a value of `-9` indicates a range from byte 0 to byte 9.
--encoding-type	The encoding of the object names. Set the value to url. If you do not specify this parameter, the names of objects are not encoded.
--include	Includes all objects that meet the specified conditions. For more information, see Options --include and --exclude.
--exclude	Excludes all objects that meet the specified conditions. For more information, see Options --include and --exclude.
--meta	Sets object metadata. The value is in the `header:value#header:value` format. Example: `Cache-Control:no-cache#Content-Encoding:gzip`. For more information about object metadata, see set-meta.
--acl	The access control list (ACL) of the objects. Valid values: default: The ACL of the objects is the same as the ACL of the bucket in which the objects are stored. This is the default ACL setting for objects. private: Only the bucket owner can perform read and write operations on objects in the bucket. Other users cannot access the objects in the bucket. public-read: Only the bucket owner can perform read and write operations on the objects in the bucket. All users, including anonymous users, can perform read operations on the objects in the bucket. This may result in unauthorized access to the data in your bucket and high fees. Proceed with caution when you set the ACL to public-read. public-read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. This may result in unauthorized access to the data in your bucket and high costs. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. We recommend that you do not set the ACL to public-read-write unless necessary.
--snapshot-path	Specifies the directory in which the snapshots of downloaded objects are stored. In the next download task, ossutil reads the snapshots in this directory to download only incremental objects.
--disable-crc64	Disables 64-bit cyclic redundancy check (CRC-64). By default, ossutil enables CRC-64 during data transmission.
--payer	The payer of the fees that are generated by the operation. If you want the requester who accesses the resources in the specified path to pay the fees that are generated by the operation, such as traffic and request fees, set this option to requester.
--partition-download	The partition that you want to download. The value of this option is in the `Number of partition to download:Total number of partitions` format. For example, the value 1:5 specifies that the object is split into five partitions and that the current command downloads partition 1. Partitions are numbered starting from 1. Partitioning rules for objects are determined by ossutil based on the built-in algorithm. This option splits an object into multiple partitions that can be concurrently downloaded by multiple ossutil commands. Each ossutil command downloads its own partition.
-j, --job	The number of concurrent tasks that can be performed in a multi-object operation. Valid values: 1 to 10000. Default value: 3.
--parallel	The number of concurrent tasks that can be performed on a single object. Valid values: 1 to 10000. If you do not configure this option, ossutil specifies a value based on the operation type and the object size.
--version-id	The version ID of the object that you want to download. You can use this option for objects only in a versioned bucket.
--start-time	The beginning of the time range to filter objects by the last modified time. The value is a timestamp that follows the UNIX time format. Objects whose last modified time is earlier than the UNIX timestamp are ignored. Note Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.
--end-time	The end of the time range to filter objects by the last modified time. The value is a timestamp that follows the UNIX time format. Objects whose last modified time is later than the UNIX timestamp are ignored. Note If both --start-time and --end-time are specified, the command takes effect only on objects whose last modified time is within the range specified by --start-time and --end-time. Only ossutil 1.7.18 and later support this parameter. For more information about how to update ossutil, see update.

If the default settings of the options do not meet your performance requirements, you can use the -j, --jobs and --parallel options to adjust performance. By default, ossutil calculates the value of --parallel based on the object size. When you download multiple large objects, the actual number of concurrent tasks is the value of -j, --jobs multiplied by the value of --parallel.

We recommend that you adjust the number of concurrent tasks to a value less than 100 if the resources such as the network bandwidth, memory, and CPU of your ECS instance or server are limited. If some of your resources such as the network bandwidth, memory, and CPU are left idle, you can increase the number of concurrent operations.
If the number of concurrent tasks is too large, the download performance of ossutil may decrease because threads switch and compete for resources. In addition, an end-of-file (EOF) error may occur. To prevent these issues, you can adjust the values of the -j, --jobs and --parallel options based on your available resources. To perform stress testing, we recommend that you initially specify small values for the two options and gradually increase the values to find the optimal values.

Sample environment

In this topic, objects are downloaded from OSS to your local directory in Linux. If you want to use the sample commands in your actual business scenarios, change the option and parameter settings based on your operating system and environment. Sample environment:

Operating system: Linux
Bucket: examplebucket
Directory in the bucket: destfolder
Local file: examplefile.txt (stored in the root directory)
Local directory: localfolder (within the root directory)

Download a single object

If you do not specify a name for the downloaded object, the name of the downloaded object is the same as the original object. If you specify a name for the downloaded object, the downloaded object uses the specified name.

Download the object with its original name:

ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/

Download the object with the specified name:

ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/example.txt

Download multiple objects at a time

You cannot directly specify multiple object names in the cp command to download multiple objects at a time. However, you can run the cp command to download multiple objects at a time by using the following methods:

Download objects of a specific type
If you want to download multiple objects whose names have a particular pattern, such as objects with the same prefix or suffix in their names, you can use the --include and --exclude options to download objects that meet the specified conditions.
- Run the following command to download all objects that are not in the JPG format:
```
ossutil cp oss://examplebucket/destfolder/ localfolder/ --exclude "*.jpg" -r
```
- Run the following command to download all objects that contain abc in their names and are not in the JPG format or TXT format:
```
ossutil cp oss://examplebucket/destfolder/ localfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r
```
Download a directory and its subdirectories
You can run the following command to download all objects in a directory, including subdirectories in the directory:
```
ossutil cp -r oss://examplebucket/destfolder/ localfolder/
```
If a batch download task fails or you want to perform an incremental download task, you can use the --update option or its shortened form -u to skip objects that have been downloaded. If an object does not have the same name as a local file, or the object is last modified later than the local file that has the same name, ossutil downloads the object. If an object has the same name as a local file, and the local file is last modified later than the object, ossutil skips the object. Sample command:
```
ossutil cp -r oss://examplebucket/destfolder/  localfolder/  --update                           
```
Download a directory (excluding its subdirectories)
If you want to download only the current directory and do not want to download its subdirectories, use the --only-current-dir option. Sample command:
```
ossutil cp oss://examplebucket/destfolder/ localfolder/ --only-current-dir -r
```
Download a directory (with timestamps specified in the command)
The following sample command downloads objects whose last modified time is within the range from 10:09:18 (UTC+8) October 31, 2023 to 12:55:58 (UTC+8) October 31, 2023 from the specified directory and its subdirectories: Sample command:
```
ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --start-time 1698718158 --end-time 1698728158
```
Download all objects in the root directory of a bucket
The following sample command downloads all objects in the root directory of a bucket named examplebucket to the local directory named localfolder/:
```
ossutil cp -r oss://examplebucket  localfolder/
```

Download an object with the maximum download speed limited

To limit the maximum download speed, specify the --maxdownspeed option. Unit: KB/s. Sample commands:

The following sample command downloads the specified object from OSS, with the maximum download speed limited to 1 MB/s:
```
ossutil cp oss://examplebucket/destfolder/examplefile.txt  localfolder/ --maxdownspeed 1024
```
The following sample command downloads the specified directory from OSS, with the maximum download speed limited to 1 MB/s:
```
ossutil cp -r  oss://examplebucket/destfolder/  localfolder/ --maxdownspeed 1024
```

Download a range of the content of an object

To download a range of the content of an object, specify the --range option. For example, you can run the following command to download the 10th to 20th characters of the examplefile.txt object to your local device as a file:

ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/  --range=10-20
Succeed: Total num: 1, size: 11. OK num: 1(download 1 objects).

Download multiple objects at a time and generate snapshot information

If you specify the --snapshot-path option when you download multiple objects at a time, ossutil generates snapshots for the objects and stores the snapshot information in the specified directory. If you specify this option, ossutil reads the snapshot information from the specified directory and downloads only incremental objects next time you download the objects. For more information, see Generate snapshots for objects when you upload the objects.

ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --snapshot-path=path

Download the specified version of an object from a versioned bucket

After you enable versioning for a bucket, objects that are overwritten or deleted in the bucket are saved as previous versions. You can specify the --version-id option in the cp command to download the specified version of the object.

ossutil cp oss://my-bucket/test.jpg localfolder/ --version-id  CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk

Important

To use the --version-id option, you need to run the ls --all-versions command to obtain the version IDs of the object.
The --version-id option can be specified in the command to download objects only from a versioned bucket. For more information about how to enable versioning for a bucket, see bucket-versioning.

Common options

If you use ossutil to switch to a bucket that is located in another region, add the -e option to specify the endpoint of the region in which the bucket is located. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, add the -i option to specify the AccessKey ID of the specified account, and add the -k option to specify the AccessKey secret of the specified account.

Note

The endpoint that you specify must correspond to the region in which the bucket is located. For more information, see Regions and endpoints.

For example, you can run the following command to download the exampleobject.txt object in the root directory of the examplebucket bucket, which is owned by another Alibaba Cloud account and located in the China (Shanghai) region, to the local directory named localfolder:

ossutil cp oss://examplebucket/exampleobject.txt  localfolder/  -e oss-cn-shanghai.aliyuncs.com -i yourAccessKeyID  -k yourAccessKeySecret

For more information about common options, see Common options.