Synchronize local files to OSS

Usage notes

• Binary name

  Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.

• Number of synchronized files

  You can run the sync command to synchronize up to one million local files to OSS at a time. If the sync command is used to synchronize more than one million local files to OSS at a time, the over max sync numbers 1000000. error message is returned.

• Differences between the sync and cp commands
  • The sync command synchronizes files in a specified directory by recursively synchronizing all files and subdirectories in the directory. The cp command recursively synchronizes files only when the -r option is specified.
  • If you run the sync command to synchronize data to an OSS bucket, you can add the --delete option to the command to delete all objects that do not exist in the source from the destination bucket. In this case, only the synchronized objects are retained in the destination bucket. The cp command does not support the --delete option.
  • The sync command does not support the --version-id option. Therefore, the sync command cannot be used to synchronize previous versions in versioned buckets. The cp command supports the --version-id option.

  Except for the preceding differences, the sync command can be used in the same way as the cp command. For more information about how to use the cp command, see cp (Upload objects).

Command syntax

./ossutil64 sync file_url cloud_url
[-f --force]
[-u --update]
[--delete]
[--enable-symlink-dir]
[--disable-all-symlink]
[--disable-ignore-error]
[--only-current-dir]
[--output-dir <value>]
[--bigfile-threshold <value>]
[--part-size <value>]
[--checkpoint-dir <value>]
[--encoding-type <value>]
[--snapshot-path <value>]
[--include <value>]
[--exclude <value>]
[--meta <value>]
[--acl <value>]
[--maxupspeed <value>]
[--disable-crc64]
[--payer <value>]
[-j, --job <value>]
[--parallel <value>]
[--retry-times <value>]
[--tagging <value>]

The following table describes the parameters that you can configure when you run this command.

Examples

Two files named d.txt and e.png are stored in the localfolder directory of the local root directory. The destfolder directory of a bucket named examplebucket contains two objects named a.txt and b.txt and a subdirectory named C. The following structure shows the files and directories in local disks and OSS before synchronization:

Local root directory                  examplebucket
    └── localfolder         └── destfolder/
            ├── d.txt               ├── a.txt
            ├── e.png               ├── b.txt
                                       └── C/

The following examples show how to use the sync command to synchronize local files to OSS in different scenarios:

• Synchronize the localfolder directory from the local disk to examplebucket.

  ./ossutil64 sync localfolder/  oss://examplebucket/destfolder/

  After you run the preceding command, the d.txt and e.png objects are added to the destfolder directory of examplebucket. The following structure shows the files and directories in local disks and OSS after synchronization:

  Local root directory                  examplebucket
      └── localfolder         └── destfolder/
              ├── d.txt               ├── a.txt
              ├── e.png               ├── b.txt
                                         ├── d.txt 
                                         ├── e.png 
                                         └── C/
                          

• Synchronize the localfolder directory from the local disk to examplebucket and retain only the synchronized objects in the destfolder directory of examplebucket.

  You can add the --delete option in the command to delete all objects that do not exist in localfolder from the destfolder. In this case, only the synchronized objects are retained in destfolder.

  ./ossutil64 sync localfolder/  oss://examplebucket/destfolder/ --delete

  After you run the command, the localfolder directory is synchronized from the local disk to examplebucket. The a.txt and b.txt objects and the C subdirectory within the destfolder directory of examplebucket are deleted. Only the synchronized d.txt and e.png files are retained in the destfolder directory of examplebucket. The following structure shows the files and directories in local disks and OSS after synchronization:

  Local root directory                  examplebucket
      └── localfolder         └── destfolder/
              ├── d.txt                ├── d.txt
              ├── e.png                ├── e.png 

• Synchronize the localfolder from the local disk to examplebucket without confirmation.

  By default, when you synchronize a local folder to an OSS bucket, if the OSS bucket contains objects that have the same name as files in the local directory, OSS prompts you to confirm whether you want to overwrite the existing objects. To receive confirmation during synchronization, you can run the following command to synchronize the localfolder directory from the local disk to examplebucket:

  ./ossutil64 sync localfolder/ oss://examplebucket/destfolder/
  cp: overwrite "oss://examplebucket/destfolder/d.txt"(y or N)?

  If you confirm that the objects in examplebucket can be overwritten, you can add the -f,--force option to the command to force the overwrite operation. You can run the following command to synchronize the localfolder directory from the local disk to examplebucket without confirmation:

  ./ossutil64 sync localfolder/ oss://examplebucket/destfolder/ -f,--force

  After you run the command, the d.txt and e.png objects are added to the destfolder directory of examplebucket. The following structure shows the files and directories in local disks and OSS after synchronization:

  Local root directory                  examplebucket
      └── localfolder         └── destfolder/
              ├── d.txt               ├── a.txt
              ├── e.png               ├── b.txt
                                         ├── d.txt 
                                         ├── e.png
                                         └── C/
                      

• If the preceding commands are successful, an output similar to the following is returned to indicate the number of synchronized objects, the sizes of synchronized objects, and the time used for the synchronization tasks:

  Succeed: Total num: 2, size: 750,081. OK num: 2(upload 2 files).
  
  average speed 1641000(byte/s)

Common options

To use ossutil to manage buckets that are located in different regions, you can use the -e option to use the endpoint of the specified bucket. To use ossutil to manage buckets that are owned by different Alibaba Cloud accounts, you can use the -i option to use the AccessKey ID of the specified account, and use the -k option to use the AccessKey secret of the specified account.

./ossutil64 sync srcfolder/ oss://examplebucket/testfolder/ -e oss-cn-shanghai.aliyuncs.com -i LTAI4Fw2NbDUCV8zYUzA****  -k 67DLVBkH7EamOjy2W5RVAHUY9H****

For more information about other common options, see Common options.