This topic describes the parameters that you can specify when you use Data Online Migration to migrate data.

Alibaba Cloud Object Storage Service

This section describes the parameters that you can specify when you migrate data to or from Alibaba Cloud Object Storage Service (OSS).
  • OSS Endpoint
    The endpoint. The following table lists the valid formats of OSS endpoints.
    No. Format Description
    1 http://oss-cn-hangzhou.aliyuncs.com Upload or download data over the Internet by using HTTP.
    2 http://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data over the internal network by using HTTP.
    3 https://oss-cn-hangzhou.aliyuncs.com Upload or download data over the Internet by using HTTPS.
    4 https://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data over the internal network by using HTTPS.
    If you migrate data from third-party cloud services to OSS, OSS uploads data over the internal network. The following formats are invalid.OSS_EndpointFor more information, see Regions and endpoints.
  • OSS Bucket

    The name of the bucket. The prefix or the suffix of the bucket name cannot contain invalid characters such as spaces, line feeds, and tab keys.

  • OSS Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • AccessKey Id and AccessKey secret

    The AccessKey pair that is used to migrate data. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. You can use the AccessKey pair of an Alibaba Cloud account or a RAM user. You cannot use the AccessKey pair of a temporary user. For more information, see Access control.

    If you migrate data from OSS, you can attach the following policy to a RAM user to grant the required permissions to the RAM user.Permissions policy (OSS as the source)
    If you migrate data to OSS, you can attach the following policy to a RAM user to grant the required permissions to the RAM user.Permissions policy (OSS as the destination)

    You can create a RAM user for a data migration job, and use the AccessKey pair that is generated for the RAM user. After the migration job is completed, you can delete the AccessKey pair.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. You can view the storage usage and the number of the files in the OSS console.

AWS S3

This section describes the parameters that you can specify when you migrate data to or from AWS Simple Storage Service (AWS S3).
Note AWS S3 whitelists are not supported by Data Online Migration. If whitelists are configured in AWS S3, delete the whitelists or submit a ticket before you migrate data.
  • Endpoint
    Data Online Migration supports only the following AWS S3 regions and endpoints. These endpoints use HTTPS and HTTP protocols.
    RegionName Region Endpoint
    US East (Ohio) us-east-2 http://s3.us-east-2.amazonaws.com

    http://s3-us-east-2.amazonaws.com

    US East (N. Virginia) us-east-1 http://s3.us-east-1.amazonaws.com

    http://s3-us-east-1.amazonaws.com

    US West (N. California) us-west-1 http://s3.us-west-1.amazonaws.com

    http://s3-us-west-1.amazonaws.com

    US West (Oregon) us-west-2 http://s3.us-west-2.amazonaws.com

    http://s3-us-west-2.amazonaws.com

    Canada (Central) ca-central-1 http://s3.ca-central-1.amazonaws.com

    http://s3-ca-central-1.amazonaws.com

    Asia Pacific (Seoul) ap-northeast-2 http://s3.ca-central-1.amazonaws.com

    http://s3-ca-central-1.amazonaws.com

    Asia Pacific (Osaka-Local) ap-northeast-3 http://s3.ap-northeast-2.amazonaws.com

    http://s3-ap-northeast-2.amazonaws.com

    Asia Pacific (Singapore) ap-southeast-1 http://s3.ap-southeast-1.amazonaws.com

    http://s3-ap-southeast-1.amazonaws.com

    Asia Pacific (Sydney) ap-southeast-2 http://s3.ap-southeast-2.amazonaws.com

    http://s3-ap-southeast-2.amazonaws.com

    Asia Pacific (Tokyo) ap-northeast-1 http://s3.ap-northeast-1.amazonaws.com

    http://s3-ap-northeast-1.amazonaws.com

    China (Beijing) cn-north-1 http://s3.cn-north-1.amazonaws.com.cn
    China (Ningxia) cn-northwest-1 http://s3.cn-northwest-1.amazonaws.com.cn
    Europe (Frankfurt) eu-central-1 http://s3.eu-central-1.amazonaws.com

    http://s3-eu-central-1.amazonaws.com

    Europe (Ireland) eu-west-1 http://s3.eu-west-1.amazonaws.com

    http://s3-eu-west-1.amazonaws.com

    Europe (London) eu-west-2 http://s3.eu-west-2.amazonaws.com

    http://s3-eu-west-2.amazonaws.com

    Europe (Paris) eu-west-3 http://s3.eu-west-3.amazonaws.com

    http://s3-eu-west-3.amazonaws.com

    South America (São Paulo) sa-east-1 http://s3.sa-east-1.amazonaws.com

    http://s3-sa-east-1.amazonaws.com

  • Bucket

    The name of the bucket. The prefix or the suffix of the bucket name cannot contain invalid characters such as spaces, line feeds, and tab keys.

  • OSS Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • AccessKeyId and SecretAccessKey

    The access key that is used to migrate data. The access key in AWS Identity and Access Management (IAM) consists of an access key ID and an access key secret. To obtain an access key, perform the following operations. Log on to the AWS IAM console, create an IAM user, attach the AmazonS3ReadOnlyAccess permission policy to the IAM user, and then create an access key for the IAM user. After the migration job is completed, you can delete the IAM user.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

Azure Blob

This section describes the parameters for Microsoft Azure Blob. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Storage Account, Secret, and Connection Strings

    To view the details about the storage account, access keys, and connection strings, perform the following operations. Log on to the Microsoft Azure console. In the left-side navigation pane, click Storage accounts and select the storage account. In the Settings section, click Access keys. Then, you can view the details about the storage account, access keys, and connection strings.

  • Container

    The name of an Azure container.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. An example is used to describe how to specify the two parameters. Log on to the Microsoft Azure console. In the left-side navigation pane, choose testbucket2018 - Blob > container - Property. On the container-property page, click Calculate Size to view the container size. Specify the two parameters based on the container size.

Tencent Cloud Object Storage

Data Online Migration supports Tencent Cloud Object Storage (COS) V4 and uses the COS API of version 4 to access the COS service. This section describes the parameters for Tencent COS. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Region
    The abbreviation of the region or the zone where the bucket is deployed. Tencent COS V4 supports the regions and zones that are listed in the following table. For more information, see Regions.
    Region or zone Abbreviation
    Beijing Zone 1 tj
    Beijing bj
    Shanghai sh
    Guangzhou gz
    Chengdu cd
    Singapore sgp
    Hong Kong (China) hk
    Toronto ca
    Frankfurt ger
  • Bucket

    The name of the COS bucket. The COS bucket name uses the user-specified bucket name-APPID format. APPID is a Tencent Cloud account identifier. An example of the COS bucket name is tony-1234567890. Note that the bucket name that you specify in the Data Transport console is tony.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. The prefix must start and end with a forward slash (/), for example, /docs/.

  • APPID

    The APPID. To view the APPID, log on to the COS console and go to the Account Info page.

  • Secret Id and Secret Key

    The API key that is used to migrate data. An API key in COS consists of a SecretId and a SecretKey. To view or create an API key, log on to the COS console, choose Cloud Services > Management Tools > API Key Management. On the API Key Management page, you can view or create an API key. We recommend that you create an API key for a data migration job. We recommend that you delete the API key after the migration job is completed.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, log on to the COS console, and choose Cloud Services > Storage > Cloud Object Storage. On the Object Storage page, choose Monitoring Report > Basic Statistics. On the page that appears, you can view the amount of data and the number of objects that you want to migrate to the destination bucket.

Qiniu Cloud-Object Storage

This section describes the parameters for Qiniu Cloud-Object Storage (KODO). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint

    The endpoint. To view the endpoints, log on to the Qiniu Cloud console, navigate to the Object Storage page, and then select the bucket.

    The endpoints use the http://<Domain name> format.
    http://oy4jki81y.bkt.clouddn.com
    http://78rets.com1.z0.glb.clouddn.com
    http://cartoon.u.qiniudn.com
    Note Only the endpoints of the three types in this figure are valid.
  • Bucket

    The name of the KODO bucket.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key and Secret Key
    The Access Key and the Secret Key that are used to migrate data. To view the Access Key and the Secret Key, log on to the Qiniu Cloud console and choose Personal Panel > Key Management.
    Note You must specify an Access Key and a Secret Key that are in use.
  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, perform the following operations. Log on to the Qiniu Cloud console, navigate to the Object Storage page, and then select the bucket. On the page that appears, click the Content Management tab.

Baidu Object Storage

This section describes the parameters for Baidu Object Storage (BOS). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint
    The endpoint. You can use Data Transport to migrate data from the BOS service that is deployed in the China (Beijing), China (Guangzhou), or China (Suzhou) region. The following table lists the BOS endpoints that are supported by Data Online Migration.
    Abbreviation Domain name Protocol Endpoint
    North China - Beijing bj.bcebos.com HTTP or HTTPS http://bj.bcebos.com
    South China - Guangzhou gz.bcebos.com http://gz.bcebos.com
    East China - Suzhou su.bcebos.com http://su.bcebos.com

    To view the region where the bucket is deployed, log on to the BOS console, and choose Cloud Services > BOS-Bucket Management.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Access Key

    The Access Key ID and the Secret Access Key that are used to migrate data. To view the Access Key ID and the Secret Access Key, log on to the BOS console and choose Authorization > Access Key.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, perform the following operations. Log on to the BOS console and choose Cloud Services > BOS-Bucket Management > Bucket Overview. On the page that appears, view the storage usage of the bucket. Estimate the number of files that you want to migrate based on the storage usage. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

HTTP HTTPS

This section describes the parameters for HTTP and HTTPS sources. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • File Path
    The file path. Note that a list file has two columns.
    • The first column includes a list of HTTP and HTTPS URLs. You must encode the special characters that are included in the URLs. Data Transport uses the HTTP GET method to download a file and use the HTTP HEAD method to obtain the file metadata from an HTTP or HTTPS URL.
    • The second column includes a list of file names. After a file is migrated, the name of the corresponding object consists of a prefix and the file name. Separate the two columns with a tab key (\t).
    Each line includes a file name. Separate lines with line feeds (\n). The following examples show the valid format.
    http://127.0.0.1/docs/my.doc    docs/my.doc
    http://127.0.0.1/pics/my.jpg    pics/my.jpg
    http://127.0.0.1/exes/my.exe    exes/my.exe

    File path format

    After the migration file is generated, upload it to Alibaba Cloud OSS. Data Transport downloads the list file and migrates the files based on the specified URLs. The file path uses the oss://{Bucket}/{The name of the list file} format, as shown in the following example.
    oss://mybucket/httplist.txt
  • List Access Endpoint
    The endpoint. The following table lists the valid formats of OSS endpoints.
    No. Format Description
    1 http://oss-cn-hangzhou.aliyuncs.com Upload or download data over the Internet by using HTTP.
    2 http://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data over the internal network by using HTTP.
    3 https://oss-cn-hangzhou.aliyuncs.com Upload or download data over the Internet by using HTTP.
    4 https://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data over the internal network by using HTTP.
    For more information about OSS endpoints, see Regions and endpoints.
  • List Access AK and List Access SK

    The credentials that are used to download the list file. You can use the credentials of the primary account or a sub-user. If you use the credentials of a sub-user, you must grant the GetObject permissions to the sub-user.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

USS

This topic describes the parameters for UPYUN Storage Service (USS). You may need to configure these parameters when you use the Data Online Migration service.
  • Domain Address
    The USS endpoint that is used as an access point for SDKs, or the domain name in a request URL when you call a RESTful API operation. You can use one of the endpoints that are listed in the following table.
    Note To view the network type, visit http://v0.api.upyun.com.
    Network Type Endpoint Description
    Intelligent Routing http://v0.api.upyun.com Recommended
    China Telecom http://v1.api.upyun.com N/A
    China Unicom or China Netcom http://v2.api.upyun.com N/A
    China Mobile (Tietong) http://v3.api.upyun.com N/A
  • Service Name

    The service name. To view the service name, log on to the UPYUN console, and navigate to the UPYUN Storage Service page.

  • Migration Folder

    The folder in which the files are migrated. The folder is specified by a prefix. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Operator Name and Operator Secret

    The name of the operator. To view the operator details or add an operator, log on to the UPYUN Cloud console, click your account name in the upper-right corner of the page, and select Account Management from the drop-down list. On the page that appears, you can view the operator details or add an operator. You can add an operator for a data migration job and specify a corresponding password for the operator. To allow the operator to migrate data, you must grant the operator the read permissions.

    To grant the operator the required permissions, perform the following operations. On the UPYUN Storage Service page, click the name of the bucket, navigate through Configurations > Content Management, and then click Authorize. In the dialog box that appears, select the required operator and click OK.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, log on to the UPYUN console and navigate through UPYUN Storage Service > Configurations. On the page that appears, view the amount of data and the number of files stored in the bucket. Estimate the amount of data and the number of files that you want to migrate based on the results. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

Kingsoft Standard Storage Service

This section describes the parameters for Kingsoft Standard Storage Service (KS3). You may need to configure these parameters when use Data Transport to migrate data.
  • Endpoint
    The following table lists the mappings between KS3 regions and endpoints.
    Region Endpoint
    BEIJING ks3-cn-beijing.ksyun.com
    SHANGHAI ks3-cn-shanghai.ksyun.com
    HONGKONG ks3-cn-hk-1.ksyun.com
    China (Hangzhou) kss.ksyun.com
    RUSSIA ks3-rus.ksyun.com
    SINGAPORE ks3-sgp.ksyun.com
    Note To view the region where the bucket is deployed, log on to the KS3 console, and navigate through Storage & CDN > Object Storage > My Buckets > Bucket Details.
  • Bucket

    The name of the KS3 bucket.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Key

    The Access Key ID and the Secret Access Key that are used to migrate data. To create an Access Key ID and a Secret Access Key, log on to the KS3 console, navigate through Identities and Management > User Management. On the page that appears, create an IAM user, and attach the KS3FullAccess policy to the IAM user to grant the IAM user the required permissions. On the IAM user details page, create an AccessKey ID and a Secret Access Key. You can delete the IAM user after the migration job is completed.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, log on to the KS3 console and navigate through Storage & CDN > Object Storage > My Buckets > Bucket Details. On the page that appears, view the amount of data and the number of files stored in the bucket. Estimate the amount of data and the number of files that you want to migrate based on the results. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

Huawei Object Storage Service

This section describes the parameters for Huawei Object Storage Service (OBS). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint
    The following table lists the regions that are supported by Huawei OBS. For more information, see Regions and endpoints.
    Region name Abbreviation Endpoint Protocol type
    CN North-Beijing4 cn-north-4 obs.cn-north-4.myhuaweicloud.com HTTPS and HTTP
    CN North-Beijing1 cn-north-1 obs.cn-north-1.myhuaweicloud.com
    CN East-Shanghai2 cn-east-2 obs.cn-east-2.myhuaweicloud.com
    CN East-Shanghai1 cn-east-3 obs.cn-east-3.myhuaweicloud.com
    CN South-Guangzhou cn-south-1 obs.cn-south-1.myhuaweicloud.com
    CN Southwest-Guiyang1 cn-southwest-2 obs.cn-southwest-2.myhuaweicloud.com
    AP-Bangkok ap-southeast-2 obs.ap-southeast-2.myhuaweicloud.com
    AP-Hong Kong ap-southeast-1 obs.ap-southeast-1.myhuaweicloud.com
    AP-Singapore ap-southeast-3 obs.ap-southeast-3.myhuaweicloud.com
    AF-Johannesburg af-south-1 obs.af-south-1.myhuaweicloud.com
    Note To view the region where the bucket is deployed, log on to the KS3 console, and navigate through Storage & CDN > Object Storage > My Buckets > Bucket Details.
  • Bucket

    The name of the Huawei OBS bucket.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Access Key

    The access key that is used to migrate data. An access key in Huawei Cloud consists of an access key ID and an access key secret. To view or create an access key, log on to the Huawei Cloud console. In the upper-right corner of the console, move the pointer over the account name and click My Credentials. On the page that appears, navigate through My Credentials > Access Keys > Create Access Key > . If you have not specified an access key during the first logon, click Create Access Key to create an access key.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, perform the following operations. Log on to the Huawei Cloud console, choose Service List > Object Storage Service > Object Storage, and then select the bucket that you want to migrate. In the Basic Statistics section, check the size of the bucket and the number of files that you want to migrate.

UCloud UFile

The section describes the parameters for UCloud UFile. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Region
    The region where the UCloud UFile service is deployed. UCloud UFile supports the regions that are listed in the following table. For more information, see the Regions and zones.
    Region name Abbreviation
    China (Beijing) cn-bj
    China (Hong Kong) hk
    China (Guangzhou) cn-gd
    China (Shanghai 2) cn-sh2
    US (Los Angeles) us-ca
    Singapore (Singapore) sg
    Indonesia (Jakarta) idn-jakarta
    Nigeria (Lagos) afr-nigeria
    Brazil (Sao Paulo) bra-saopaulo
    UAE (Dubai) uae-dubai
    Vietnam (Ho Chi Minh City) vn-sng
    China (Taipei) tw-tp
    India (Mumbai) ind-mumbai
    US (Washington) us-ws
    Germany (Frankfurt) ge-fra
  • Bucket

    The name of the UFile bucket.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Public Key and Private Key

    The API public key and the API private key that are used to migrate data. To view the API public key and the API private key, log on to the UCloud console. Then, navigate to the API key management page, and view the API public key and the API private key.

  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. Before you specify the two parameters, perform the following operations. Log on to the UCloud console and navigate to the UFile page. On this page, click Bucket and click the name of the bucket that stores the source data. On the Overview tab, view the storage usage of the target bucket. Estimate the amount of data and the number of files that you want to migrate on the storage usage. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.

GCP

This section describes the parameters for Google Cloud Platform (GCP). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.

  • Bucket

    The name of the bucket.

  • Prefix

    The prefix. The files in the directory that have the specified prefix are migrated. If the name of a file is the same as the specified prefix, the file is not migrated. The prefix cannot start with a forward slash (/). However, the prefix must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Key File
    1. Log on to the GCP console.
    2. In the left-side navigation pane, click Service Accounts.
    3. On the Service Accounts page, find the service account. In the Actions column, choose Create a key > Create Key.
    4. In the Create private key for "username" dialog box, select JSON as the key type, and click CREATE.
    5. GCP automatically downloads a JSON key file. Upload the key file to the Data Transport console.
  • Data Size and File Count

    Specify the amount of data and the number of files that you want to migrate. To ensure a successful migration, you must specify an appropriate data amount and the number of files based on an estimation of the data amount.