How to synchronize local files to OSS - Object Storage Service

You can use the sync command to synchronize local files to Object Storage Service (OSS).

Usage notes

To synchronize local files to OSS, you must have the oss:PutObject, oss:ListObjects, and oss:DeleteObject permissions. For more information, see Attach a custom policy to a RAM user.
Differences between the sync and cp commands
- If you run the sync command to synchronize data from a local source to an OSS bucket, you can add the --delete option to the command to delete all files that do not exist in the local source but exist in the bucket. This way, only the synchronized files are retained in the bucket. The cp command does not support the --delete option.
- The sync command scans the local source and the OSS bucket at the same time and then copies the local files. The cp command scans the source and copies objects at the same time.
- The default and the maximum number of source files for the sync command are 1 million and 5 million respectively. No limit is imposed on the number of source objects for the cp command.

Command syntax

ossutil sync file_url cloud_url

Parameter	Data type	Description
file_url	string	The path of the local directory from which you want to synchronize files. Examples: `/localfolder/` in Linux and `D:\localfolder\` in Windows.
cloud_url	string	The OSS directory to which you want to synchronize the local files. The path is in the `oss://bucketname/path/` format. Example: `oss://examplebucket/exampledir/`. If the value of `cloud_url` does not end with a forward slash (/), ossutil automatically adds one at the end of the value.
--acl	string	The access control list (ACL) of the destination object. Valid values: private: The ACL is private. public-read: The ACL is public-read. public-read-write: The ACL is public-read-write. default: The object inherits the ACL from the bucket.
--backup-dir	string	The backup directory.
--bigfile-threshold	int	The object size threshold for multipart upload, download, and copy. Default value: 104857600.
--cache-control	string	The caching behavior in web browsers when the object is downloaded.
--checkpoint-dir	string	The directory in which the checkpoint file is stored. Default value: `.ossutil_checkpoint/`.
--checksum	/	Specifies that only when one of the following conditions is met, the source object is copied: a. the size of the source object is different from that of the destination object; b. the CRC-64 of the source object is different from that of the destination object, but the size of the source object is the same as that of the destination object. This parameter takes effect only for object copy operations.
--content-disposition	string	The display mode for the object.
--content-encoding	string	The encoding method used to encode the destination object.
--content-type	string	The content type of the destination object.
--copy-props	string	The attributes that you want to synchronize from the local files. Valid values: none metadata default
--delete	/	Deletes objects that exist in the destination path but does not exist in the source path.
-d, --dirs	/	Matches objects and subdirectories in the current directory, instead of matching objects recursively in subdirectories.
--encoding-type	string	The encoding method used to encode the object name. Valid value: url.
--end-with	string	Returns objects whose names are alphabetically before or the same as the value of the --end-with parameter.
--exclude	stringArray	The exclusion rule for paths or file names.
--exclude-from	stringArray	Reads the exclusion rule from a file.
--expires	string	The absolute expiration time of the cached content.
--files-from	stringArray	Reads a list of local files from a file, with empty lines and comment lines ignored.
--files-from-raw	stringArray	Reads a list of local files from a file.
--filter	stringArray	The filter rule for paths or file names.
--filter-from	stringArray	Reads the filter rule for paths or file names from a file.
-f, --force	/	Forces the operation without requiring confirmation.
--include	stringArray	The inclusion rule for paths or file names.
--include-from	stringArray	Reads the inclusion rule from a file.
-j, --job	int	The number of concurrent tasks that can be performed for a multi-file operation. Default value: 3.
--list-objects	/	Specifies that the ListObjects operation is called to list objects.
--min-age	Duration	Transfers files that were last modified at least the specified amount of time ago. The default unit is s (seconds). You can also specify one of the following units: ms, s, m, h, d, w, M, and y. For example, `--min-age 2d` specifies that the tool transfers files that were last modified at least 2 days ago.
--max-age	Duration	Transfers files that were last modified no earlier than the time period defined by this parameter. The default unit is s (seconds). You can also specify one of the following units: ms, s, m, h, d, w, M, and y.
--min-mtime	Time	Transfers files that were last modified before the UTC time defined by this parameter. For example, `--min-mtime "2023-02-06 14:00:00"` specifies that the tool transfers files that were last modified before 14:00:00 on February 6, 2023.
--max-mtime	Time	Transfers files that were last modified after the UTC time defined by this parameter.
--max-size	SizeSuffix	The maximum size of a file that can be transferred. By default, the size is in bytes (B). You can suffix the numerical value with a 1024-based unit: B for byte, K for Kibibyte, M for Mebibyte, G for Gibibyte, T for Tebibyte, or P for Pebibyte.
--max-sync-num	int	The maximum number of files that can be synchronized from the local source to the OSS bucket. Default value: 2000000. Maximum value: 5000000.
--metadata	strings	The user metadata of the object in the key=value format.
--metadata-directive	string	The method used to configure the metadata of the object. Valid values: COPY REPLACE
--metadata-exclude	stringArray	The exclusion rule for object metadata.
--metadata-filter	stringArray	The filter rule for object metadata.
--metadata-filter-from	stringArray	Reads the metadata filter rule from a file.
--metadata-include	stringArray	The inclusion rule for object metadata.
--min-size	SizeSuffix	The minimum size of a file that can be transferred. By default, the size is in bytes (B). You can suffix the numerical value with a 1024-based unit: B for byte, K for Kibibyte, M for Mebibyte, G for Gibibyte, T for Tebibyte, or P for Pebibyte.
--no-progress	/	Specifies that the progress is not displayed.
--page-size	int	The maximum number of objects that can be returned per page. Default value: 1000. Valid values: 1 to 1000.
--parallel	int	The number of concurrent tasks for internal operations on a single object.
--part-size	SizeSuffix	The part size. By default, ossutil calculates the appropriate part size based on the local file size. The part size ranges from 100 KiB to 5 GiB.
-r, --recursive	/	Specifies that the command is recursively run on objects. If this parameter is specified, the command performs the operation on all matching objects in the bucket. If this parameter is not specified, the command performs the operation only on the specified objects in the specified path.
--request-payer	string	The payer of the request. If pay-by-requester is enabled, specify this parameter. Valid value: requester.
--size-only	/	Copies a source file only if it has a different size from the destination object.
--start-after	string	Specifies that objects whose names are alphabetically after the value of the --start-after parameter are returned.
--storage-class	string	The storage class of the object. Valid values: Standard IA Archive ColdArchive DeepColdArchive
--tagging	strings	The tags of the object in key=value pairs.
--tagging-directive	string	The method used to configure the tags of the destination file. Valid values: COPY REPLACE
-u, --update	/	Copies a source file only if its last modified time is later than that of its destination object.
--ignore-existing	/	Ignores existing destination objects.

Note

For more information, see Command-line options.

Examples

Synchronize the localfolder directory from the local device to the destfolder directory of the examplebucket bucket
```
ossutil sync D:/localfolder/ oss://examplebucket/destfolder/
```
Synchronize the files whose names contain the .txt suffix in the localfolder directory from the local device to the destfolder directory of the examplebucket bucket
```
ossutil sync D:/localfolder/ oss://examplebucket/destfolder/ --include "*.txt"
```
Synchronize files whose names do not contain the .txt suffix in the localfolder directory from the local device to the destfolder directory of the examplebucket bucket
```
ossutil sync D:/localfolder/ oss://examplebucket/destfolder/ --exclude "*.txt"
```
Synchronize the localfolder directory from the local device to the destfolder directory of the examplebucket bucket and delete the objects that do not exist in the localfolder directory from the destfolder directory
Add the --delete option to delete all objects that do not exist in the localfolder directory from the destfolder directory. This way, only the synchronized objects are retained in the destfolder directory.
```
ossutil sync D:/localfolder/ oss://examplebucket/destfolder/ --delete
```