Object Storage Service:sync (Synchronize OSS files to a local computer)

File count limits

When you run the sync command without the --delete option, no limit is imposed on the number of files you can synchronize. If you include --delete, you can synchronize up to 1 million files at a time. Exceeding this limit returns the error: over max sync numbers 1000000.

Differences between sync and cp

Except for these differences, the sync and cp commands work in a similar way. For more information about cp, see Download objects.

Parameters

Sync behavior

File filtering

Performance and concurrency

Resumable download

Symbolic link handling

Data integrity and error handling

Output and encoding

Billing

Example 1: Basic synchronization

Synchronize the localdir directory from examplebucket to the local destdir directory:

ossutil sync oss://examplebucket/localdir/ destdir/

After synchronization, the objects a.txt, b.txt, and subdirectory C are added to destdir. The existing file d.txt is preserved because --delete is not specified.

examplebucket           Local root directory
└── localdir/             └── destdir/
       ├── a.txt                ├── a.txt
       ├── b.txt                ├── b.txt
       └── C/                   ├── d.txt
                                └── C/

Example 2: Synchronize with deletion and backup

Synchronize localdir to destdir and remove files from destdir that do not exist in localdir. Move the removed files to a backup directory instead of permanently deleting them:

ossutil sync oss://examplebucket/localdir/ destdir/ --delete --backup-dir backup/

After synchronization, destdir contains only the objects from localdir. The file d.txt, which existed only in destdir, is moved to the backup directory:

examplebucket              Local root directory
└── localdir/               ├── destdir/
       ├── a.txt            │     ├── a.txt
       ├── b.txt            │     ├── b.txt
       └── C/               │     └── C/
                            └── backup/
                                   └──d.txt

Example 3: Synchronize only specific file types

Synchronize only .txt files from localdir, excluding all other file types:

ossutil sync oss://examplebucket/localdir/ destdir/ --include "*.txt"

Example 4: Exclude specific file types

Synchronize all objects from localdir except .log files:

ossutil sync oss://examplebucket/localdir/ destdir/ --exclude "*.log"

Example 5: Incremental synchronization

Synchronize only objects that are new or have been modified since the last synchronization:

ossutil sync oss://examplebucket/localdir/ destdir/ -u

With --update (-u), ossutil skips objects whose last modified time in OSS is the same as or earlier than the corresponding local file.

Example 6: Synchronize only the top-level directory

Synchronize only the objects directly in localdir, without recursing into subdirectories:

ossutil sync oss://examplebucket/localdir/ destdir/ --only-current-dir

Example 7: Speed up repeated synchronization with snapshots

Use --snapshot-path to store synchronization state. On subsequent runs, ossutil reads the snapshot to identify changed objects and synchronizes only those, reducing the time needed for repeated synchronization of the same directory:

ossutil sync oss://examplebucket/localdir/ destdir/ --snapshot-path /tmp/oss-sync-snapshot/

Example 8: Tune concurrency for large-scale synchronization

When you synchronize a directory that contains both small and large files, adjust --job and --parallel to optimize throughput:

• --job controls how many objects are processed at the same time. Increase this value when you need to synchronize many small files.

• --parallel controls how many parts of a single large object are downloaded at the same time. Increase this value when you need to download a few large files.

ossutil sync oss://examplebucket/data/ /local/data/ --job 10 --parallel 5

To limit download speed to 10,240 KB/s (10 MB/s) and avoid saturating your network bandwidth:

ossutil sync oss://examplebucket/data/ /local/data/ --maxdownspeed 10240

Success output

When the synchronization completes successfully, ossutil returns output similar to the following. Note that the output uses "upload" regardless of the sync direction; this is the standard ossutil output format.

Succeed: Total num: 2, size: 750,081. OK num: 2(upload 2 files).

average speed 1641000(byte/s)