Alibaba Cloud ECS data migration to Alibaba Cloud OSS

Practical background

In the operation and maintenance work, there are data file migration scenarios, such as migrating the data of Alibaba Cloud ECS to OSS, and this operation can be implemented through ossutil.

First run the following code on the server you want to migrate to install the tool

When the console outputs the following content, the installation is successful.

The content is installed to the root directory by default, and then grant permissions.

In the next step, you can perform OSS related configuration, and enter the configuration interface through the command ./ossutil64 config

It will ask you to enter the address of the configuration file, you can keep the default /root/.ossutilconfig .

Configuration view

Enter the command ./ossutil64 to view the help document directly.

root@ ~]# ./ossutil64
Usage: ossutil [command] [args...] [options...]
Please use ossutil help command to display the help of command

Commands:
mb cloud_url [options]
Create buckets
ls [cloud_url] [options]
List Buckets or Objects
rm cloud_url [options]
Delete Bucket or Objects

Upload files

The next step is to upload the file, that is, migrate the data to OSS, and use the cp command in ossutil64.

This command is used to upload local files or folders to OSS.

When uploading files with the cp command, multipart upload and resumable upload are used by default. If the file upload process is accidentally interrupted and the file upload is not completed, the uploaded part will be stored in the OSS storage space (Bucket) in the form of Parts. Be careful to delete this part when you no longer need it.

The command format is as follows, among which -r and -u are the most commonly used.

./ossutil64 cp file_url cloud_url
[-r, --recursive]
[-f --force]
[-u --update]
[--maxupspeed ]
[--enable-symlink-dir]
[--disable-all-symlink]
[--disable-ignore-error]
[--only-current-dir]
[--bigfile-threshold ]
[--part-size ]
[--checkpoint-dir ]
[--encoding-type ]
[--include ]
[--exclude ]
[--meta ]
[--acl ]
[--snapshot-path ]
[--disable-crc64]
[--disable-dir-object]
[--payer ]
[--tagging ]
[-j, --job ]
[--parallel ]

The following are the more commonly used operations.

Upload a single file

When uploading a file, if you do not specify a file name to upload to OSS, the original file name will be used for saving by default; if you specify a file name, it will be saved in OSS according to the specified file name.

Only upload files inside the folder

When using the cp command, add the -r option to upload only the files in the local folder to the OSS specified path. For uploading local folder contents

Upload folders and files in folders

When using the cp command, add the -r option and add the local folder name under the path specified by OSS, so that the local folder and the files in the folder can be uploaded to the path specified by OSS.

Upload folder and skip existing files

When batch upload fails and retransmits, you can specify the --update (can be abbreviated as -u) option to skip files that have been uploaded successfully to achieve incremental uploads.

Only upload files in the current directory, ignoring subdirectories
Meaning as the title says:

Upload speed limit

When uploading files, combine the --maxupspeed option to limit the maximum upload speed in KB/s.

Upload files and set the speed limit to 1 MB/s

Upload a folder and set the speed limit to 1 MB/s