Parameters for online migration tasks - Data Online Migration

This topic describes the parameters that you need to configure when you use Data Online Migration to migrate data. This helps you smoothly and efficiently perform a data migration.

Important

The information about third-party cloud services in this topic is for reference only. Alibaba Cloud does not guarantee or make commitments to the performance and reliability of third-party tools, or the potential impacts of operations on the tools.

Alibaba Cloud OSS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from or to an Alibaba Cloud Object Storage Service (OSS) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information about the supported endpoints, see Regions and endpoints. If you migrate data from a third-party cloud service to OSS, data is uploaded to OSS over an internal network.
Bucket
The name of the source or destination OSS bucket. Make sure that the bucket name is not prefixed or suffixed with invalid characters such as spaces, line feeds, and tab characters.

Prefix
The prefix of the data address. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
- If you want to migrate data from OSS, specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated.
- If you want to migrate data to OSS, migrate the source data to the specified directory.
Role authorization
To ensure data security, we recommend that you create a Resource Access Management (RAM) role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and attach policies to the role. For more information, see Preparations.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and attach policies to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
AccessKey ID and AccessKey secret
The AccessKey pair that is used to access the OSS bucket. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. Use the AccessKey pair of an Alibaba Cloud account or a RAM user. You cannot use a Security Token Service (STS) token. For more information, see AccessKey Pair Management.
- If you want to migrate data from OSS, you must grant the RAM user the permissions to query and read data from all resources in the OSS bucket from which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the source bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:ListObjects",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "oss:GetObject",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```
- If you want to migrate data to OSS, you must grant the RAM user full permissions on the OSS bucket to which you want to migrate data.
  Note
  The following policy is for reference only. Replace mybucket with the name of the destination bucket.
  For more information about RAM policies for OSS, see Common examples of RAM policies.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "oss:*",
            "Resource": [
                "acs:oss:*:*:mybucket",
                "acs:oss:*:*:mybucket/*"
            ]
        }
    ]
}
```

AWS S3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an Amazon Simple Storage Service (Amazon S3) bucket.

Note

If access to the Amazon S3 bucket from which you want to migrate data is restricted, you cannot use Data Online Migration to migrate data from the bucket. Before you migrate data, disable the access restriction.

Endpoint
The endpoint of the region in which the data address resides. For more information about the valid formats of Amazon S3 endpoints, see Amazon S3 endpoints in the Amazon S3 documentation. HTTPS and HTTP are supported.
Bucket
The name of the Amazon S3 bucket that stores the source data. Make sure that the bucket name is not prefixed or suffixed with invalid characters such as spaces, line feeds, and tab characters.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access key ID and secret access key
The access key pair that is used to access the Amazon S3 bucket. To ensure data security, we recommend that you create an Identity and Access Management (IAM) user, grant the required permissions to the IAM user, and then create an access key pair of the IAM user for a data migration task. After the migration task is complete, delete the access key pair. For more information, see Manage access keys for IAM users.
The amount of data and the number of objects that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of objects that you want to migrate.

Azure Blob

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Microsoft Azure Blob Storage (Azure Blob) container.

Domain name
The uniform resource identifier (URI) of the resource in Azure Blob. Format: https://<Storage account>.blob.core.windows.net. For more information, see Resource URI Syntax in the Azure Blob documentation.
Storage Account, Secret, and Connection Strings
The name, access key, and connection strings of the storage account that has the read permissions on the source data. For more information, see View account access keys in the Azure Blob documentation.
Container
The name of the Azure Blob container that stores the source data. The name is case-sensitive and must be the same as what you specify when you create the container. For more information, see Create a container.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.

Tencent COS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Tencent Cloud Object Storage (COS) bucket.

If you migrate data from a Tencent COS bucket to an OSS bucket, only Tencent COS V5 is supported. For more information, see Introduction.

Parameters:

Region
The abbreviation of the region in which the data address resides. For more information, see Regions and Access Endpoints.
Bucket
The name of the COS bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
SecretId and SecretKey
The API key that is used to access the COS bucket. To ensure data security, we recommend that you create an API key for a data migration task and delete the API key after the migration task is complete. For more information, see Access Key.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the Tencent COS console. For more information, see Viewing Data Overview.

Qiniu Cloud KODO

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Qiniu Cloud Object Storage (KODO) bucket.

Domain name
The domain name of the KODO bucket that stores the source data. Format: http://Domain name. Examples:
```
http://oy4jki81y.bkt.clouddn.com
http://78rets.com1.z0.glb.clouddn.com
http://cartoon.u.qiniudn.com
```
Only the preceding three types of domain names are supported. For more information, see Domain name management.
Bucket
The name of the KODO bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKey and SecretKey
The AccessKey and the SecretKey that are used to access the KODO bucket. To ensure data security, we recommend that you create an AccessKey and a SecretKey for a data migration task and delete the AccessKey and the SecretKey after the migration task is complete. For more information, see QiniuToken.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the Qiniu Cloud KODO console. For more information, see Bucket overview.

Baidu Cloud BOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Baidu Object Storage (BOS) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and Endpoints.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access Key ID and Secret Access Key
The Access Key ID and the Secret Key that are used to access the BOS bucket. To ensure data security, we recommend that you create an Access Key ID and a Secret Key for a data migration task and delete the Access Key ID and the Secret Key after the migration task is complete. For more information, see How to Get AKSK.
Data Size and File Count
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the BOS console. For more information, see Manage Bucket.

HTTP/HTTPS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from an HTTP or HTTPS source.

Inventory Path
The path of the list file. For more information, see the Step 1: Upload list files section of the "Preparations" topic.
Inventory Domain Name
The OSS endpoint or Amazon S3 endpoint of the region in which the data address resides.
- For more information about the supported endpoints, see Regions and endpoints.
- For more information about the valid formats of Amazon S3 endpoints, see Amazon S3 endpoints.
Authorize Role (list file in OSS)
To ensure data security, we recommend that you create a RAM role and attach the policies required for data migration to the RAM role based on the principle of least privilege.
- Migrate data within the same Alibaba Cloud account
  - Create a RAM role in the Data Online Migration console.
  - Manually create a RAM role and attach policies to the role. For more information, see Preparations.
- Migrate data across Alibaba Cloud accounts
  - Manually create RAM roles and attach policies to the roles. For more information, see Attach policies to RAM roles for cross-account data migration.
Inventory AccessKeyId and Inventory SecretAccessKey (list file in Amazon S3)
To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an access key pair of the IAM user for a data migration task. After the migration task is complete, delete the access key pair. For more information, see Manage access keys for IAM users.

USS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from UPYUN Storage Service (USS).

Domain Address
The endpoint of USS SDK. For more information, see Request method.
Note
If you do not know the network type of your USS service, use http://v0.api.upyun.com.
Service Name
The name of the USS bucket that stores the source data.
Migration Folder
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Operator Name and Operator Secret
The operator name and the operator secret that are used to access the USS bucket. To ensure data security, we recommend that you create an operator and grant the operator the read permissions. For more information, see Storage management.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the USS console. For more information, see Object management.

Kingsoft Cloud KS3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Kingsoft Standard Storage Service (KS3) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints.
Bucket
The name of the KS3 bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
AccessKeyId and SecretAccessKey
The AccessKey that is used to access the KS3 bucket. To ensure data security, we recommend that you create an IAM user, grant the required permissions to the IAM user, and then create an AccessKey of the IAM user for a data migration task. After the migration task is complete, delete the AccessKey. For more information, see Create an AccessKey for an IAM user.
The amount of data and the number of objects that you want to migrate. Obtain values for the two parameters in the KS3 console. For more information, see Bucket.

Huawei Cloud OBS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Huawei Cloud Object Storage Service (OBS) bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and Endpoints.
Bucket
The name of the OBS bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access key ID and secret access key
The access key that is used to access the OBS bucket. To ensure data security, we recommend that you create an access key for a data migration task and delete the access key after the migration task is complete. For more information, see Managing Access Keys for an IAM User.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the OBS console. For more information, see Viewing Bucket Information.

Volcano Engine TOS

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Volcano Engine Tinder Object Storage (TOS) bucket.

Domain Name
The S3 endpoint of the region in which the data address resides. For more information, see Regions and endpoints.
Bucket
The name of the TOS bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Access Key ID and Secret Access Key
The Access Key that is used to access the TOS bucket. To ensure data security, we recommend that you create an Access Key for a data migration task and delete the Access Key after the migration task is complete. For more information, see Manage Access Keys.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the TOS console. For more information, see View bucket information.

UCloud US3

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a UCloud US3 bucket.

Endpoint
The endpoint of the region in which the data address resides. For more information, see Regions and endpoints.
Bucket
The name of the UCloud US3 bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Public Key and Private Key
The API public key and the API private key that are used to access the UCloud US3 bucket. To ensure data security, we recommend that you create an API public key and an API private key for a data migration task and delete the API public key and the API private key after the migration task is complete. For more information, see Token management.
The amount of data and the number of objects that you want to migrate. Get values for the two parameters in the UCloud US3 console. For more information, see View the information about a bucket.

Google Cloud Storage

This section describes the parameters that you need to configure when you use Data Online Migration to migrate data from a Google Cloud Storage bucket.

Bucket
The name of the Google Cloud Storage bucket that stores the source data.
Prefix
The prefix of the data address. Specify a prefix to migrate only the objects in the specified directory. The object whose name is the prefix is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, the value can be docs/.
Key File
The key file of the service account. For more information, see Create a service account key.
The amount of data and the number of objects that you want to migrate. You must configure the Data Size and File Count parameters based on the actual amount of data and the actual number of objects that you want to migrate.