You can run the sync command to synchronize objects stored in Object Storage Service (OSS) to your computer.
- Binary name
Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.
- Number of synchronized files
You can run the sync command to synchronize up to one million local files to OSS at a time. If the sync command is used to synchronize more than one million local files to OSS at a time, the over max sync numbers 1000000. error message is returned.
- Differences between the sync and cp commands
- The sync command synchronizes objects in a specified directory by recursively synchronizing all objects and subdirectories in the directory. The cp command recursively synchronizes objects only when the -r option is specified.
- The sync command supports the --backup-dir option. When you run the sync command to synchronize data to OSS, you can add this option to the command to specify a directory in OSS to store objects that exist in the destination but do not exist in the source. The cp command does not support the --backup-dir option.
- The sync command does not support the --version-id option. Therefore, you cannot run the sync command to synchronize previous versions in versioned buckets. The cp command supports the --version-id option.
Except for the preceding differences, the sync command can be run in the same way as the cp command. For more information about how to run the cp command, see Download objects.
./ossutil64 sync cloud_url file_url [-f --force] [-u --update] [--maxdownspeed <value>] [--delete] [--backup-dir <value>] [--enable-symlink-dir] [--disable-all-symlink] [--disable-ignore-error] [--only-current-dir] [--output-dir <value>] [--bigfile-threshold <value>] [--part-size <value>] [--checkpoint-dir <value>] [--range <value>] [--encoding-type <value>] [--snapshot-path <value>] [--include <value>] [--exclude <value>] [--disable-crc64] [--payer <value>] [-j, --job <value>] [--parallel <value>] [--retry-times <value>]
The following table describes the parameters that you can configure to run the sync command to synchronize OSS objects to your computer.
|cloud_url||Specifies the path of the directory in which the objects you want to synchronize are
stored. The path is in the following format:
|file_url||Specifies the path of the local directory to which you want to synchronize objects.
|-f --force||Forces an operation without prompting the user for confirmation.|
|-u, --update||Specifies that ossutil synchronizes objects from the source only when the objects do not exist in the destination or when the last modified time of the object is later than that of the corresponding objects in the destination.|
|--maxdownspeed||Specifies the maximum download speed. Unit: KB/s. Default value: 0, which indicates that the download speed is unlimited.|
|--delete||Specifies that only the synchronized objects are retained in the destination local
path. Other objects in the destination local path are deleted.
Warning To add the --delete option to the command, we recommend that you enable versioning for the bucket to prevent data from being accidentally deleted. For more information about versioning, see Overview.
|--backup-dir||Specifies the directory used to store objects that exist in the destination local path but do not exist in the source OSS bucket.|
|--enable-symlink-dir||Specifies whether to synchronize subdirectories to which symbolic links point.|
|--disable-all-symlink||Specifies that all objects and subdirectories to which symbolic links in the directory point are not synchronized.|
|--disable-ignore-error||Specifies that errors are not ignored during batch operations.|
|--only-current-dir||Specifies that only objects in the current directory are synchronized. Subdirectories in the current directory and objects in these subdirectories are not synchronized.|
|--output-dir||Specifies the directory in which output objects are stored. Output objects are reports that are generated when errors occur during batch synchronization of objects. By default, these reports are stored in the ossutil_output directory of the current directory.|
|-bigfile-threshold||Specifies the maximum size of objects that can be synchronized by using resumable
upload or download. Unit: byte.
Default value: 104857600 (100 MB).
Valid values: 0 to 9223372036854775807.
|--part-size||Specifies the part size. Unit: byte. By default, ossutil determines the part size
based on the object size.
Valid values: 1 to 9223372036854775807.
|--checkpoint-dir||Specifies the directory in which the log information of resumable upload or download
tasks is stored. Default value:
|--range||Specifies that a specific range of the object is synchronized and stored as a new
file in the destination local path. The minimum start value of the range is 0, which
indicates the byte 0 of the object. You can set this parameter in one of the following
|--encoding-type||Specifies the method used to encode the names of objects. Valid values: url. If you do not specify this parameter, the names of objects are not encoded.|
|--snapshot-path||Specifies the directory in which the snapshots of synchronized objects are stored. In the next synchronization task, ossutil reads the snapshots in this directory for incremental synchronization.|
|--include||Specifies that the command applies to all objects that meet the specified conditions.|
|--exclude||Specifies that the command applies to all objects that do not meet the specified conditions.|
|--disable-crc64||Disables CRC-64 in synchronization.|
|--payer||Specifies the payer of the traffic and request fees charged when the command is run. If you want the requester who accesses the resources in the specified path to pay for the traffic and request fees charged during queries, set this parameter to requester.|
|-j, --job||Specifies the number of concurrent tasks performed across multiple objects. Valid values: 1 to 10000. Default value: 3.|
|--parallel||Specifies the number of concurrent tasks performed for a single object. Valid values: 1 to 10000. By default, if you do not set this parameter, ossutil sets the value of this parameter based on the operation type and object size.|
|--retry-times||Specifies the number of retries after the command fails to be run. Default value: 10. Valid values: 1 to 500.|
The localdir directory of a bucket named examplebucket contains two objects named a.txt and b.txt and a subdirectory named C. A local file named d.txt is stored in the destdir directory of the local root directory. The following structure shows the objects and directories in your computer and OSS before synchronization:
examplebucket Local root directory └── localdir/ └── destdir/ ├── a.txt └── d.txt ├── b.txt └── C/
- Synchronize the localdir directory from examplebucket to your computer.
After you run the preceding command, the a.txt and b.txt objects and the C directory are added to the destfolder directory of your computer. The following structure shows the objects and directories in your computer and OSS after synchronization:
./ossutil64 sync oss://examplebucket/localdir/ destdir/
examplebucket Local root directory └── localdir/ └── destdir/ ├── a.txt ├── a.txt ├── b.txt ├── b.txt └── C/ ├── d.txt └── C/
- Synchronize the localdir directory of examplebucket to a local directory named destdir. The --backup-dir parameter is set to a directory named backup in the command to specify that all objects exist in the destdir directory but do
not exist in localdir are deleted from destdir and are saved in the backup directory.
./ossutil64 sync oss://examplebucket/localdir/ destdir/ --delete --backup-dir backup/After you run the command, the localdir directory of examplebucket is synchronized to the destdir directory in your computer. Objects that exist in the destdir directory and do not exist in localdir are moved to the backup directory. Only the synchronized objects named a.txt and b.txt and directory named C are stored in the destdir directory after synchronization. The d.txt object that exists in the destdir directory before synchronization is moved to the backup directory. The following structure shows the objects and directories in your computer and OSS after synchronization by specifying the backup directory:
examplebucket Local root directory └── localdir/ ├── destdir/ ├── a.txt │ ├── a.txt ├── b.txt │ ├── b.txt └── C/ │ └── C/ └── backup/ └──d.txt
- If the preceding commands are successful, a similar output is returned to indicate
the number of synchronized objects, the sizes of synchronized objects, and the time
used for the synchronization tasks:
Succeed: Total num: 2, size: 750,081. OK num: 2(upload 2 files). average speed 1641000(byte/s)
To use ossutil to manage buckets that are located in different regions, you can use the -e option to use the endpoint of the specified bucket. To use ossutil to manage buckets that are owned by different Alibaba Cloud accounts, you can use the -i option to use the AccessKey ID of the specified account, and use the -k option to use the AccessKey secret of the specified account.
./ossutil64 sync oss://examplebucket/srcfolder/ examplefolder/ -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA**** -k 67DLVBkH7EamOjy2W5RVAHUY9H****
For more information about other common options that you can use for the sync command, see Common options.