This topic describes the parameters that are used when you use Data Transport to migrate data.

Alibaba Cloud Object Storage Service

This section describes the parameters for Alibaba Cloud Object Storage Service (OSS). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • OSS Endpoint
    The OSS endpoint. The following table lists the valid formats of the OSS endpoints.
    No. Format Description
    1 http://oss-cn-hangzhou.aliyuncs.com Upload or download data by using HTTP over the Internet.
    2 http://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data by using HTTP over the internal network.
    3 https://oss-cn-hangzhou.aliyuncs.com Upload or download data by using HTTP over the Internet.
    4 https://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data by using HTTP over the internal network.
    If you migrate data from OSS, Data Transport automatically downloads data over the internal network. If you migrate data to OSS, OSS uploads data over the internal network. The following formats are invalid.OSS_EndpointFor more information about OSS endpoints, see Regions and endpoints.
  • OSS Bucket

    The name of the bucket. The prefix or the suffix of the bucket name cannot contain invalid characters such as spaces, line feeds, and tab keys.

  • OSS Prefix

    The OSS prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • AccessKey Id and AccessKey Secret

    The AccessKey pair that is used to migrate data. An AccessKey pair in Alibaba Cloud Resource Management Access (RAM) consists of an AccessKey ID and an AccessKey secret. You can use the AccessKey pair of an Alibaba Cloud account or a RAM user. You cannot use the AccessKey pair of a temporary user. For more information, see Access control.

    If you migrate data from OSS, you can attach the following policy to a RAM user to grant the RAM user the required permissions.Permission policy (OSS as the source)
    If you migrate data to OSS, you can attach the following policy to a RAM user to grant the RAM user the required permissions.Permission policy (OSS as the destination)

    You can create a RAM user for a data migration job, and use the AccessKey pair that is generated for the RAM user. After the data migration is complete, you can delete the AccessKey pair.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated in the OSS bucket. You can view the storage usage and the number of the files for the target OSS bucket in the OSS console.

AWS S3

This section describes the parameters for AWS Simple Storage Service (AWS S3). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
Note Currently, AWS S3 whitelists are not supported by Data Transport. If whitelists are configured in AWS S3, delete the whitelists or submit a ticket before data migration.
  • Endpoint
    Data Transport supports only the following AWS S3 regions and endpoints. These endpoints use HTTPS and HTTP protocols.
    Region name Region Endpoint
    US East (Ohio) us-east-2 http://s3.us-east-2.amazonaws.com

    http://s3-us-east-2.amazonaws.com

    US East (N. Virginia) us-east-1 http://s3.us-east-1.amazonaws.com

    http://s3-us-east-1.amazonaws.com

    US West (N. California) us-west-1 http://s3.us-west-1.amazonaws.com

    http://s3-us-west-1.amazonaws.com

    US West (Oregon) us-west-2 http://s3.us-west-2.amazonaws.com

    http://s3-us-west-2.amazonaws.com

    Canada (Central) ca-central-1 http://s3.ca-central-1.amazonaws.com

    http://s3-ca-central-1.amazonaws.com

    Asia Pacific (Seoul) ap-northeast-2 http://s3.ca-central-1.amazonaws.com

    http://s3-ca-central-1.amazonaws.com

    Asia Pacific (Osaka-Local) ap-northeast-3 http://s3.ap-northeast-2.amazonaws.com

    http://s3-ap-northeast-2.amazonaws.com

    Asia Pacific (Singapore) ap-southeast-1 http://s3.ap-southeast-1.amazonaws.com

    http://s3-ap-southeast-1.amazonaws.com

    Asia Pacific (Sydney) ap-southeast-2 http://s3.ap-southeast-2.amazonaws.com

    http://s3-ap-southeast-2.amazonaws.com

    Asia Pacific (Tokyo) ap-northeast-1 http://s3.ap-northeast-1.amazonaws.com

    http://s3-ap-northeast-1.amazonaws.com

    China (Beijing) cn-north-1 http://s3.cn-north-1.amazonaws.com.cn
    China (Ningxia) cn-northwest-1 http://s3.cn-northwest-1.amazonaws.com.cn
    Europe (Frankfurt) eu-central-1 http://s3.eu-central-1.amazonaws.com

    http://s3-eu-central-1.amazonaws.com

    Europe (Ireland) eu-west-1 http://s3.eu-west-1.amazonaws.com

    http://s3-eu-west-1.amazonaws.com

    Europe (London) eu-west-2 http://s3.eu-west-2.amazonaws.com

    http://s3-eu-west-2.amazonaws.com

    Europe (Paris) eu-west-3 http://s3.eu-west-3.amazonaws.com

    http://s3-eu-west-3.amazonaws.com

    South America (São Paulo) sa-east-1 http://s3.sa-east-1.amazonaws.com

    http://s3-sa-east-1.amazonaws.com

  • Bucket

    The name of the bucket. The prefix or the suffix of the bucket name cannot contain invalid characters such as spaces, line feeds, and tab keys.

  • OSS Prefix

    The OSS prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • AccessKeyId and SecretAccessKey

    The access key that is used to migrate data. The access key in AWS Identity and Access Management (IAM) consists of an access key ID and a secret access key. To obtain an access key, perform the following operations. Log on to the AWS IAM console, create an IAM user, attach the AmazonS3ReadOnlyAccess permission policy to the IAM user, and then create an access key for the IAM user. After the data migration is complete, you can delete the IAM user.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. To ensure a successful migration, you must specify an appropriate data size and the number of files based on an accurate estimation of the actual data amount.

Azure Blob

This section describes the parameters for Microsoft Azure Blob. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Storage Account, Secret, and Connection Strings

    To view the details about the storage account, access keys, and connection strings, perform the following operations. Log on to the Microsoft Azure console. In the left-side navigation pane, click Storage accounts, and select the target storage account. In the Settings section, click Access keys. Then, you can view the details about the storage account, access keys, and connection strings.

  • Container

    The name of an Azure container.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. An example is used to describe how to specify the two parameters. Log on to the Microsoft Azure console. In the left-side navigation pane, choose testbucket2018 - Blob > container - Property. On the container-property page, click Calculate Size to view the container size. Based on the container size, specify the two parameters.

Tencent Cloud Object Storage

Data Transport supports Tencent Cloud Object Storage (COS) V4 and uses the COS API of version 4 to access the COS service. This section describes the parameters for Tencent COS. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Region
    The abbreviation of the region or the zone where the target bucket is deployed. Tencent COS V4 supports the regions and zones that are listed in the following table. For more information, see Regions.
    Region or zone Abbreviation
    Beijing Zone 1 tj
    North China (Beijing) bj
    East China (Shanghai) sh
    South China (Guangzhou) gz
    Southwest China (Chengdu) cd
    Southeast Asia Pacific (Singapore) sgp
    Hong Kong/Macao/Taiwan (Hong Kong, China) hk
    North America (Toronto) ca
    Europe (Frankfurt) ger
  • Bucket

    The name of the COS bucket. The COS bucket name uses the user-specified bucket name-APPID format. APPID is a Tencent Cloud account identifier. An example of the COS bucket name is tony-1234567890. Note that the bucket name that you specify in the Data Transport console is tony.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. The prefix must start and end with a forward slash (/), for example, /docs/.

  • APPID

    The APPID. To view the APPID, log on to the COS console and navigate to the Account Info page.

  • Secret Id and Secret Key

    The API key that is used to migrate data. An API key in COS consists of a SecretId and a SecretKey. To view or create an API key, log on to the COS console, choose Cloud Services > Management Tools > API Key Management. On the API Key Management page, you can view or create an API key. We recommend that you create an API key for a data migration job. We recommend that you delete the API key after the data migration is complete.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the COS console, and choose Cloud Services > Storage > Cloud Object Storage. On the Object Storage page, choose Monitoring Report > Basic Statistics. On the page that appears, you can view the amount of data and the number of objects to be migrated in the target bucket.

Qiniu Cloud-Object Storage

This section describes the parameters for Qiniu Cloud-Object Storage (KODO). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint

    The endpoint. To view the endpoints, log on to the Qiniu Cloud console, navigate to the Object Storage page, and then select the target bucket. On the page that appears, you can view the endpoints in the Converged CDN Test Domain and the Converged CDN Accelerated Domain sections.

    The endpoints use the http://<Domain name> format, as shown in the following figure.
    http://oy4jki81y.bkt.clouddn.com
    http://78rets.com1.z0.glb.clouddn.com
    http://cartoon.u.qiniudn.com
    Note Only the endpoints of the three types in this figure are valid.
  • Bucket

    The name of the KODO bucket.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key and Secret Key
    The Access Key and the Secret Key that are used to migrate data. To view the Access Key and the Secret Key, log on to the Qiniu Cloud console and choose Personal Panel > Key Management.
    Note You must specify an Access Key and a Secret Key that are in use.
  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the Qiniu Cloud console, navigate to the Object Storage page, and then select the target bucket. On the page that appears, click the Content Management tab. On this tab, you can view the number of files and the amount of data stored in the bucket.

Baidu Object Storage

This section describes the parameters for Baidu Object Storage (BOS). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint
    The endpoint. Currently, you can use Data Transport to migrate data from the BOS service that is deployed in the China (Beijing), the China (Guangzhou), or the China (Suzhou) region. The following table lists the BOS endpoints that are supported by Data Transport.
    Region Domain name Protocol Endpoint
    China (Beijing) bj.bcebos.com HTTP or HTTPS http://bj.bcebos.com
    China (Guangzhou) gz.bcebos.com http://gz.bcebos.com
    China (Suzhou) su.bcebos.com http://su.bcebos.com

    To view the region where the bucket is deployed, log on to the BOS console, and choose Cloud Services > BOS-Bucket Management.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Access Key

    The Access Key ID and the Secret Access Key that are used to migrate data. To view the Access Key ID and the Secret Access Key, log on to the BOS console and choose Authorization > Access Key.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the BOS console and choose Cloud Services > BOS-Bucket Management > Bucket Overview. On the page that appears, view the storage usage of the target bucket. Based on the storage usage, estimate the number of files to be migrated. To ensure a successful migration, you must specify the number of files based on an accurate estimation.

HTTP and HTTPS

This section describes the parameters for HTTP and HTTPS sources. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • File Path
    The file path. Note that a list file has two columns.
    • The first column includes a list of HTTP and HTTPS URLs. You must encode the special characters that are included in the URLs. Data Transport uses the HTTP GET method to download a file and use the HTTP HEAD method to obtain the file metadata from an HTTP or HTTPS URL, respectively.
    • The second column includes a list of file names. After a file is migrated, the name of the corresponding object consists of a prefix and the file name. Separate the two columns with a tab key (\t).
    Each line includes a file name. Separate lines with a line feed (\n). The following examples show the valid format.
    http://127.0.0.1/docs/my.doc    docs/my.doc
    http://127.0.0.1/pics/my.jpg    pics/my.jpg
    http://127.0.0.1/exes/my.exe    exes/my.exe

    File path format

    After the migration file is generated, upload it to Alibaba Cloud OSS. Data Transport downloads the list file and migrates the files based on the specified URLs. The file path uses the oss://{Bucket}/{The name of the list file} format, as shown in the following example.
    oss://mybucket/httplist.txt
  • List Access Endpoint
    The endpoint. The following table lists the valid formats of the OSS endpoints.
    No. Format Description
    1 http://oss-cn-hangzhou.aliyuncs.com Upload or download data by using HTTP over the Internet.
    2 http://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data by using HTTP over the internal network.
    3 https://oss-cn-hangzhou.aliyuncs.com Upload or download data by using HTTP over the Internet.
    4 https://oss-cn-hangzhou-internal.aliyuncs.com Upload or download data by using HTTP over the internal network.
    For more information about OSS endpoints, see Regions and endpoints.
  • List Access AK and List Access SK

    The credentials that are used to download the list file. You can use the credentials of the primary account or a subuser. If you use the credentials of a subuser, you must grant the subuser the GetObject permissions.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. To ensure a successful migration, you must specify an appropriate data size and the number of files based on an accurate estimation of the actual data amount.

UPYUN Storage Service

This topic describes the parameters for UPYUN Storage Service (USS). You may need to configure these parameters when you use the Data Transport service.
  • Domain Address
    The USS endpoint that is used as an access point for SDKs, or the domain name in a request URL when you call a RESTful API operation. You can use one of the endpoints that are listed in the following table.
    Note To view the network type, visit http://v0.api.upyun.com.
    Network type Endpoint Description
    Intelligent routing http://v0.api.upyun.com Recommended
    China Telecom http://v1.api.upyun.com N/A
    China Unicom or China Netcom http://v2.api.upyun.com N/A
    China Mobile or China Mobile Tietong http://v3.api.upyun.com N/A
  • Service Name

    The service name. To view the service name, log on to the UPYUN console, and navigate to the UPYUN Storage Service page.

  • Migration Folder

    The folder in which the files are to be migrated. The folder is specified by a prefix. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Operator Name and Operator Secret

    The name of the operator. To view the operator details or add an operator, log on to the UPYUN Cloud console, click your account name in the upper-right corner, and select Account Management from the drop-down list. On the page that appears, you can view the operator details or add an operator. You can add an operator for a data migration job and specify a corresponding password for the operator. To allow the operator to migrate data, you must grant the operator the read permissions.

    To grant the operator the required permissions, perform the following operations. On the UPYUN Storage Service page, click the name of the bucket to be migrated, navigate through Configurations > Content Management, and then click Authorize. In the dialog box that appears, select the required operator and click OK.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the UPYUN console and navigate through UPYUN Storage Service > Configurations. On the page that appears, view the amount of data and the number of files stored in the bucket. Based on the results, estimate the amount of data and the number of files to be migrated. To ensure a successful migration, you must specify the parameters based on an accurate estimation.

Kingsoft Standard Storage Service

This section describes the parameters for Kingsoft Standard Storage Service (KS3). You may need to configure these parameters when use Data Transport to migrate data.
  • Endpoint
    The following table lists the mappings between KS3 regions and endpoints.
    Region Endpoint
    China (Beijing) ks3-cn-beijing.ksyun.com
    China (Shanghai) ks3-cn-shanghai.ksyun.com
    China (Hong Kong) ks3-cn-hk-1.ksyun.com
    China (Hangzhou) kss.ksyun.com
    Russia ks3-rus.ksyun.com
    Singapore ks3-sgp.ksyun.com
    Note To view the region where the bucket is deployed, log on to the KS3 console, and navigate through Storage & CDN > Object Storage > My Buckets > Bucket Details.
  • Bucket

    The name of the KS3 bucket.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Key

    The Access Key ID and the Secret Access Key that are used to migrate data. To create an Access Key ID and a Secret Access Key, log on to the KS3 console, navigate through Identities and Management > User Management. On the page that appears, create an IAM user, and attach the KS3FullAccess policy to the IAM user to grant the IAM user the required permissions. On the IAM user details page, create an AccessKey ID and a Secret Access Key. You can delete the IAM user after the data migration is complete.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the KS3 console and navigate through Storage & CDN > Object Storage > My Buckets > Bucket Details. On the page that appears, view the amount of data and the number of files stored in the bucket. Based on the results, estimate the amount of data and the number of files to be migrated. To ensure a successful migration, you must specify the parameters based on an accurate estimation.

Huawei Object Storage Service

This section describes the parameters for Huawei Object Storage Service (OBS). You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Endpoint
    The following table lists the regions that are supported by Huawei OBS. For more information, see Regions and endpoints.
    Region name Abbreviation Endpoint Protocol type
    CN North-Beijing4 cn-north-4 obs.cn-north-4.myhuaweicloud.com HTTPS and HTTP
    CN North-Beijing1 cn-north-1 obs.cn-north-1.myhuaweicloud.com
    CN East-Shanghai2 cn-east-2 obs.cn-east-2.myhuaweicloud.com
    CN East-Shanghai1 cn-east-3 obs.cn-east-3.myhuaweicloud.com
    China (Guangzhou) cn-south-1 obs.cn-south-1.myhuaweicloud.com
    CN South-Guangzhou cn-southwest-2 obs.cn-southwest-2.myhuaweicloud.com
    AP-Bangkok ap-southeast-2 obs.ap-southeast-2.myhuaweicloud.com
    AP-Hong Kong ap-southeast-1 obs.ap-southeast-1.myhuaweicloud.com
    AP-Singapore ap-southeast-3 obs.ap-southeast-3.myhuaweicloud.com
    AF-Johannesburg af-south-1 obs.af-south-1.myhuaweicloud.com
  • Bucket

    The name of the Huawei OBS bucket.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Access Key ID and Secret Access Key

    The access key that is used to migrate data. An access key in Huawei Cloud consists of an access key ID and a secret access key. To view or create an access key, log on to the Huawei Cloud console. In the upper-right corner of the console, move the pointer over the account name and click My Credentials. On the page that appears, navigate through My Credentials > Access Keys > Create Access Key. If you have not specified an access key during the first logon, click Create Access Key to create an access key.

  • Data Size and File Count
    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the Huawei Cloud console, choose Service List > Object Storage Service > Object Storage, and then select the bucket to be migrated. In the Basic Statistics section, view the storage usage and the number of the files (objects) to be migrated in the bucket. The following figure shows an example.HWCloud_storage

UCloud UFile

The section describes the parameters for UCloud UFile. You may need to specify these parameters when you use the Data Transport console to create data migration jobs.
  • Region
    The region where the UCloud UFile service is deployed. UCloud UFile supports the regions that are listed in the following table. For more information, see the Regions and zones.
    Region name Abbreviation
    China (Beijing) cn-bj
    China (Hong Kong) hk
    China (Guangzhou) cn-gd
    China (Shanghai 2) cn-sh2
    US (Los Angeles) us-ca
    Singapore sg
    Indonesia (Jakarta) idn-jakarta
    Nigeria (Lagos) afr-nigeria
    Brazil (Sao Paulo) bra-saopaulo
    Dubai uae-dubai
    Vietnam (Ho Chi Minh City) vn-sng
    China (Taipei) tw-tp
    India (Mumbai) ind-mumbai
    US (Washington) us-ws
    Germany (Frankfurt) ge-fra
  • Bucket

    The name of the UFile bucket.

  • Prefix

    The prefix. The files in the directory that has the specified prefix are to be migrated. If the name of a file is the same as the specified prefix, the file is not to be migrated. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify the prefix as docs/.

  • Public Key and Private Key

    The API public key and the API private key that are used to migrate data. To view the API public key and the API private key, log on to the UCloud console. Then, navigate to the API key management page, and view the API public key and the API private key.

  • Data Size and File Count

    Specify the amount of data and the number of files to be migrated. Before you specify the two parameters, perform the following operations. Log on to the UCloud console and navigate to the UFile page. On this page, click Bucket and click the name of the target bucket that stores the source data. On the Overview tab, view the storage usage of the target bucket. Based on the storage usage, estimate the amount of data and the number of files to be migrated. To ensure a successful migration, you must specify the number of files based on an accurate estimation.