Copy objects - Object Storage Service - Alibaba Cloud Documentation Center

This topic describes how to run the cp command to copy an Object Storage Service (OSS) object to another directory in the same bucket or to other buckets that are located in the same region as the source object. The cp command can be used to copy objects but not parts that are generated by multipart copy tasks in progress.

Usage notes

Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.
You cannot run the cp command to copy objects to a bucket that belongs to another Alibaba Cloud account or to a bucket that is located in a different region. If you want to copy objects to a bucket that belongs to another Alibaba Cloud account or to a bucket that is located in a different region, use ossimport or Data Online Migration.

Command syntax

./ossutil64 cp cloud_url cloud_url
[-r, --recursive]
[-f --force]
[-u --update]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--encoding-type <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--disable-crc64]
[--payer <value>]
[-j, --job <value>]
[--parallel <value>]
[--version-id <value>]
[--start-time <value>]
[--end-time <value>]

The following table describes the parameters and options in the syntax.

Item	Description
cloud_url	The paths of the source and destination objects. The format is `oss://bucketname/objectname`. For example, to copy the source object srcobject.jpg from the examplebucket bucket to the destination object destobject.jpg in the same bucket, set the path of the source object to `oss://examplebucket/srcobject.jpg` and the path of the destination object to `oss://examplebucket/destobject.jpg`.
-r, --recursive	Specifies recursion. If you configure this option, ossutil performs operations on all objects in a bucket that meet the specified condition. If you do not configure this option, ossutil performs operations only on the specified object.
-f --force	Forces an operation without requiring user confirmation.
-u, --update	Specifies that ossutil copies the source object only when the object does not exist in the destination bucket or when the last modified time of the source object is later than that of the destination object.
--disable-ignore-error	Specifies that errors are not ignored during batch operations.
--only-current-dir	Specifies that only objects in the current directory are copied. Subdirectories in the current directory and objects in these subdirectories are not copied.
-bigfile-threshold	Specifies the object size threshold for using resumable upload. Unit: bytes. Default value: 100 MB. Valid values: 0 to 9223372036854775807.
--part-size	Specifies the part size. Unit: bytes. By default, ossutil determines the part size based on the object size. Valid values: 1 to 9223372036854775807.
--checkpoint-dir	Specifies the directory in which the log information of resumable upload tasks is stored. If a resumable upload task fails, ossutil automatically creates a directory named `.ossutil_checkpoint` and saves the checkpoint information in the directory. After the resumable upload task is successful, ossutil deletes the directory. If you specify a checkpoint directory, make sure that the directory can be deleted.
--encoding-type	The method used to encode the names of objects. Set the value to url. If you do not configure this option, the names of objects are not encoded.
--include	Specifies that the command applies to all objects that meet the specified conditions.
--exclude	Specifies that the command applies to all objects that do not meet the specified conditions.
--meta	Specifies object metadata in the `header:value#header:value` format. Example: `Cache-Control:no-cache#Content-Encoding:gzip`. For more information about the metadata, see set-meta (manage object metadata).
--acl	Specifies the access control list (ACL) of the objects. Default value: private. Valid values: default: The ACL of the objects is the same as that of the bucket in which the objects are stored. private: Only the bucket owner can perform read and write operations on objects in the bucket. Other users cannot access the objects in the bucket. public-read: Only the bucket owner can perform write operations on objects in the bucket. Other users, including anonymous users, can perform only read operations on the objects in the bucket. This may result in unrestricted access to the data in your bucket and unexpected high fees. If a user uploads prohibited data or information, your legitimate interests and rights may be infringed. Therefore, we recommend that you do not set the object ACL to public-read except in special cases. public-read-write: All users, including anonymous users, can perform read and write operations on the objects in the bucket. This may result in unauthorized access to the data in your bucket and high fees. Proceed with caution when you set this parameter to public-read-write.
--disable-crc64	Specifies that CRC-64 is disabled. By default, ossutil enables CRC-64 during data transmission.
--payer	Specifies the payer of the request. If you want the requester who accesses the resources in the specified path to pay the traffic and request fees, set this option to requester.
-j, --jobs	Specifies the number of concurrent tasks that can be performed across multiple objects. Valid values: 1 to 10000. Default value: 3.
--parallel	Specifies the number of concurrent tasks performed on a single object. Valid values: 1 to 10000. By default, if you do not specify this option, ossutil sets the value of this option based on the operation type and the object size.
--version-id	The version ID of the object that you want to copy. You can use this option for objects only in a bucket for which versioning is enabled.
--start-time	Specifies the start of the time range to filter objects by last modified time. The value is a UNIX timestamp. Objects whose last modified time is earlier than the UNIX timestamp are ignored.
--end-time	Specifies the end of the time range to filter objects by last modified time. The value is a UNIX timestamp. If you configure this option, objects whose last modified time is later than the timestamp are ignored. Note If both --start-time and --end-time are specified, the command applies only to objects whose last modified time is within the range specified by --start-time and --end-time.

The -j, --jobs and --parallel options in the preceding command syntax can be used to adjust performance if the default number of concurrent tasks does not meet your performance requirements. By default, ossutil calculates the number of concurrent operations based on the object size. When you copy multiple large objects, the actual number of concurrent tasks is calculated by multiplying the number of concurrent jobs by the number of concurrent operations.

We recommend that you adjust the number of concurrent tasks to a value smaller than 100 if your Elastic Compute Service (ECS) instance or server has limited resources such as network bandwidth, memory, and CPU. If the usage of resources such as network bandwidth, memory, and CPU is low, you can increase the number of concurrent operations.
Thread resources are switched and compete for resources. If the number of concurrent tasks is excessively large, the copy performance of ossutil may decrease or an end-of-file (EOF) error may occur. To prevent this issue, you can adjust the values of the -j, --jobs and --parallel options based on the actual available resources. To perform stress testing, specify small values for the two options before you incrementally increase them to the optimal values.

Sample environment

In this topic, objects are copied between different directories or buckets in Linux. You can modify the parameter settings in the examples based on your operating system and environment. Sample environment:

Operating system: Linux
Source bucket: examplebucket1
Directory 1 in the source bucket: srcfolder1
Directory 2 in the source bucket: srcfolder2
Destination bucket: examplebucket2
Directory in the destination bucket: desfolder

Whether to copy the source directory

If the source path does not end with a forward slash (/), all objects whose names match the specified prefix are copied to the destination bucket. If the source path ends with a forward slash (/), only the objects in the specified directory are copied to the destination bucket.

For example, the following objects are stored in the srcfolder1 directory in the source bucket named examplebucket1:

srcfolder1/exampleobject1.txt
srcfolder1/exampleobject2.png
srcfolder1/dir1/
srcfolder1/dir1/exampleobject3.jpg
srcfolder1/dir2/
srcfolder1/dir2/exampleobject4.jpg

Copy objects from a path that does not end with a forward slash (/)

./ossutil64 cp oss://examplebucket1/srcfolder1  oss://examplebucket2 -r

The following objects are copied from the source bucket to the destination bucket named examplebucket2:

srcfolder1/exampleobject1.txt
srcfolder1/exampleobject2.png
srcfolder1/dir1/
srcfolder1/dir1/exampleobject3.jpg
srcfolder1/dir2/
srcfolder1/dir2/exampleobject4.jpg

Copy objects from a path that ends with a forward slash (/)

./ossutil64 cp oss://examplebucket1/srcfolder1/  oss://examplebucket2 -r

The following objects are copied from the source bucket to the destination bucket named examplebucket2:

exampleobject1.txt
exampleobject2.png
dir1/
dir1/exampleobject3.jpg
dir2/
dir2/exampleobject4.jpg

Simple copy

Copy a single object

./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket1/srcfolder2/

Copy incremental objects
If you specify the --update option when you copy multiple objects, ossutil copies the objects only when the destination objects do not exist, or the last modified time of the source objects is later than that of the destination objects. Sample command:
```
./ossutil64 cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/path2/ -r --update
```
This option can be used to copy objects that fail to be copied or to skip copied objects when you copy incremental objects.
Copy multiple objects at a time
The following command copies objects in the srcfolder1 directory whose last modified time is within the range of 10:09:18 (UCT+8) October 31, 2023 to 12:55:58 (UTC+8) October 31, 2023.
```
./ossutil64 cp -r oss://examplebucket1/srcfolder1/ oss://examplebucket2/path2/ --start-time 1698718158 --end-time 1698728158
```
Rename an object
```
./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket1/example.txt                        
```
When you run the cp command to rename an object, the original object still exists. You can delete the original object after you rename the object.

Copy only objects in the current directory and ignore subdirectories

./ossutil64 cp oss://examplebucket1/srcfolder1/ oss://examplebucket1/srcfolder2/ --only-current-dir -r

Modify the metadata of an object
When you copy an object, you can use the --meta option to modify the metadata of the object. The value of this option is in the header:value#header:value... format.
```
./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket1/ --meta=Cache-Control:no-cache
```
Copy an object from a bucket for which pay-by-requester is enabled to a bucket for which pay-by-requester is disabled
```
./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/  --payer=requester
```

Copy multiple objects that meet specified conditions

To copy objects that match specified conditions, you can use --include and --exclude.

Copy all objects that are not in the JPG format

./ossutil64 cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/desfolder/ --exclude "*.jpg" -r

Copy all objects that contain abc in their names and are not in the JPG or TXT format

./ossutil64 cp oss://examplebucket1/srcfolder1/ oss://examplebucket2/desfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt" -r

Modify the storage class of an object

When you overwrite an object, you can use the --meta option to convert the storage class of the object. The valid values of the X-oss-Storage-Class parameter in the cp command are:

Standard: the Standard storage class
IA: the Infrequent Access storage class
Archive: the Archive storage class
ColdArchive: the Cold Archive storage class
DeepColdArchive: the Deep Cold Archive storage class

For more information about storage classes, see Overview.

Convert the storage class of the specified object to Archive

./ossutil64 cp oss://examplebucket1/srcfolder1/examplefile.txt oss://examplebucket1/srcfolder1/examplefile.txt --meta X-oss-Storage-Class:Archive

Convert the storage class of all objects in the specified directory to Standard
```
./ossutil64 cp oss://examplebucket1/srcfolder1/ oss://examplebucket1/srcfolder1/ --meta X-oss-Storage-Class:Standard -r
```
Important
- Before you can convert an Archive, Cold Archive, or Deep Cold Archive object to a Standard or IA object by using the cp command, run the restore command to restore the object. If you enable real-time access of Archive objects, you can convert the storage class of Archive objects without the need to restore them. For more information, see Real-time access of Archive objects.
- If you convert the storage class of an object to IA, Archive, Cold Archive, or Deep Cold Archive and the object is stored for less than the minimum storage duration, you are charged for the entire minimum storage duration. For more information, see Storage fees.
- When you run the cp command to convert the storage class of an object that is larger than 100 MB, ossutil calculates an appropriate part size based on the object size by default. If the part size cannot meet your requirements, you can use --part-size to change the part size. Make sure that the number of parts is smaller than 10,000.

Modify the tags of an object

You can add --tagging to the cp command to modify the tags of an object. Separate the tags with ampersands (&). Sample command:

./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket1/ --tagging "abc=1&bcd=2&……"

For more information about object tagging, see object-tagging (add, modify, query, and delete object tags).

Copy and encrypt an object

When you copy an object, you can specify the server-side encryption method to encrypt the object and store the encrypted object in a bucket. For more information, see Server-side encryption.

Copy an object and set the server-side encryption method to AES-256

./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket1/srcfolder2/ --meta=x-oss-server-side-encryption:AES256

Copy an object and set the server-side encryption method to Key Management Service (KMS)
```
./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/ --meta=x-oss-server-side-encryption:KMS
```
Important
When you use KMS to encrypt an object, OSS creates a customer master key (CMK) in KMS for the object. You are charged a small amount of fees when the KMS API operation is called. For more information, see Billing of KMS.

Copy an object, set the encryption method of the object to SSE-KMS, and then specify a CMK ID

./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket2/desfolder/ --meta=x-oss-server-side-encryption:KMS#x-oss-server-side-encryption-key-id:7bd6e2fe-cd0e-483e-acb0-f4b9e1******

Recover objects in a versioning-enabled bucket

After you enable versioning for a bucket, objects that are overwritten or deleted in the bucket are saved as previous versions. You can add --version-id to the cp command to make a previous version of an object the latest version. Sample command:

./ossutil64 cp oss://examplebucket1/examplefile.txt oss://examplebucket2/ --version-id  CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk

Important

To use the --version-id option, you must run the ls --all-versions command to query version IDs of the object.
The --version-id option can be used only for objects in versioning-enabled buckets. For more information about how to enable versioning for a bucket, see bucket-versioning.

Common options

If you use ossutil to switch to a bucket that is located in another region, add the -e option to the command to specify the endpoint of the region in which the specified bucket is located. If you use ossutil to switch to a bucket that belongs to another Alibaba Cloud account, you can add the -i option to the command to specify the AccessKey ID of the specified account, and add the -k option to the command to specify the AccessKey secret of the specified account.

For example, you can run the following command to copy the srcobject.png object in the root directory of the source bucket named examplebucket to the destination bucket named destbucket. The source bucket is located in the China (Shanghai) region and is owned by another Alibaba Cloud account.

./ossutil64 cp oss://examplebucket/srcobject.png  oss://destbucket  -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

For more information about other common options that you can use for the sync command, see View options.