This topic describes the precautions and procedure for data migration.

Precautions

When creating a migration job, you must note the following issues:
  • A migration job occupies the network resources of the source data address and destination data address. To ensure business continuity, we recommend that you specify a speed limit for a migration job or perform the migration job during off-peak hours.
  • Before a migration job is performed, files at the source data address and the destination data address are checked. The files at the destination data address are overwritten if the source files have the same name as the destination files and have a later modification time. If two files have the same name but different content, you must change the name of one file or back up the files.
  • Data Transport only allows you to migrate data of a single bucket at a time, rather than all data for an account.

Step 1: Create a source data address

  1. Log on to the Data Transport console.
  2. Choose Data Online Migration > Data Address, and click Create Data Address.
  3. In the Create Data Address dialog box, configure the parameters and click OK.
    Parameter Required Description
    Data Type Yes Select YOU PAI.
    Data Name Yes A data name is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    Domain Address Yes An endpoint of UPYUN Storage Service (USS) that is used as an access point for SDKs or the domain name in a requested URL when you call a RESTful API operation.
    You can use one of the following endpoints:
    • http://v0.api.upyun.com (recommended)
    • http://v1.api.upyun.com (China Telecom)
    • http://v2.api.upyun.com (China Unicom)
    • http://v3.api.upyun.com (China Mobile Tietong)
      Notice UPYUN sets a flow limit by default when you migrate large amounts of data. When you migrate large amounts of data, we recommend that you contact UPYUN customer service to remove the flow limit. Alternatively, you can use a Content Delivery Network (CDN) domain to migrate data.
    Use CDN No Specify whether to use a CDN domain to migrate data.
    • Do not use: Migrate data by using the endpoint that is specified in the Domain Address parameter.
    • Use: Migrate data by using the specified CDN domain that is configured for the USS bucket.
      Note
      • By using a CDN, you can increase the migration speed and avoid the impact of flow limits that are set by UPYUN.
      • If you want to use an HTTPS-based domain name, you must enable the HTTPS protocol for the UPYUN CDN service. For more information, see Manage HTTPS settings.
    Service Name Yes You can log on to the UPYUN Cloud console to view the service name on the UPYUN Storage Service page.
    Migration Folder Yes
    • Migrate all data: indicates that all data in a bucket is migrated.

      When you migrate all data, you do not need to specify a folder.

    • Migrate partial data: indicates that files in a folder are migrated. The folder is specified by a prefix. A prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, docs/.
    Operator Name and Operator Secret Yes
    1. Log on to the UPYUN Cloud console, click your account name in the upper-right corner, and select Account Management from the drop-down list. On the Account Management page that appears, click Create Operator. You must grant the read permission to the operator.
    2. On the UPYUN Storage Service page, click the name of a bucket. On the Configurations page that appears, select Content Management, and click Authorize. In the dialog box that appears, select the required operator and click OK.
  4. You must apply for whitelist permissions because this feature is in the public preview phase. Click Application.
  5. Enter the required information and submit the application for using this feature. After the application is approved, you will receive a short message service (SMS) notification.

Step 2: Create a destination data address

  1. Choose Data Online Migration > Data Address and click Create Data Address.
  2. In the Create Data Address dialog box, configure the parameters and click OK.
    Parameter Required Description
    Data Type Yes Select OSS.
    Data Region Yes Select a region where the destination data address is located.
    Data Name Yes A data name is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    OSS Endpoint Yes
    Select an endpoint based on the region where your data is located.
    • You can use an HTTP endpoint to access OSS from the Internet, for example, http://oss-cn-endpoint.aliyuncs.com.
    • You can also use an HTTPS endpoint to access OSS from the Internet, for example, https://oss-cn-endpoint.aliyuncs.com.
    For more information about OSS endpoints, see Regions and endpoints.
    Note When migrating third-party data to OSS, you can only access OSS from a public endpoint.
    AccessKeyId and AccessKeySecret Yes Enter an AccessKey pair that is used to migrate data. For more information, see Create an AccessKey pair.
    OSS Bucket Yes Select a bucket to store migrated data.
    OSS Prefix No An OSS prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, data/to/oss/. Do not specify the parameter if you want to migrate data to the root directory of a bucket.
    Notice If the name of a source file starts with a forward slash (/), you must specify an OSS prefix. If no OSS prefix is specified, the migration job will fail. Assume that the name of a file to be migrated is /test/test.png. You must specify oss/ for the OSS prefix. After a migration job is complete, the name of the OSS file changes from /test/test.png to oss//test/test.png.

Step 3: Create a migration job

  1. Choose Data Online Migration > Migration Jobs, and click Create Job.
  2. In the Create Job dialog box, read the Terms of Migration Service, select I understand the above terms and conditions, and apply for opening Data Transport, and then click Next.
    Then, the Fee Reminder dialog box appears.OSS_billing request
  3. In the Create Job dialog box, configure the parameters and click Next.
    Parameter Required Description
    Job Name Yes A job name must be 3 to 63 characters in length and can contain lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
    Source Data Address Yes Select the source data address that you have created.
    Destination Data Address Yes Select the destination data address that you have created.
    Notice If the source data address and the destination data address are located in different countries or regions, you must submit a ticket to request permissions to create a cross-country or cross-region migration job. You must ensure that your business is legitimate, data transit conforms to local rules and regulations, and data does not include illegal information.
    Specified Directory No
    • Do not filter: All data at the source data address is migrated.
    • Exclude: The files and subdirectories in the excluded directories are not migrated.
    • Contain: Only the files and subdirectories in the specified directories are migrated.
    Note
    • A directory cannot start with a forward slash (/) or a backslash (\), and cannot contain double slashes (//), double periods (..), or double quotation marks ("). The character string that consists of all the specified directory names cannot exceed 10 KB in size.
    • A directory must end with a forward slash (/), for example, docs/.
    • You can specify a maximum of 20 directories of the Exclude or Contain type.
    Migration Type Yes
    • Full: specifies a full migration job. You must specify the Start Time Point of File parameter. Files with the last modification time later than the specified start time point are migrated. After the files are migrated, the migration job is closed. You can submit the job again if the data at the source data address changes. In this case, Data Transport only migrates the data that is changed after the previous job.
    • Incremental: specifies an incremental migration job. To perform an incremental job, specify the Migration Interval and Migration Times parameters based on your needs. You must specify the Start Point Time of File parameter. Files with the last modification time later than the specified start time point are migrated during the first migration. After the first migration is complete, an incremental migration is performed based on the migration interval. An incremental migration job only migrates files that are created or modified after the previous migration started and before this migration starts. Assume that you specify N for the migration times. Full migration is performed once. Then, incremental migration is performed (N-1) times. For example, you can set the migration interval to 1, the migration times to 5, and the start time point to 2019-03-05 08:00. The present time is 2019-03-10 08:00. When the first migration starts, Data Transport migrates files that are last modified between 2019-03-05 08:00 and 2019-03-10 08:00. Assume that the first migration requires 1 hour to complete. The second migration starts at 2019-03-10 10:00. The two hours from 8:00 to 10:00 include the time period that the first migration requires (1 hour) and the migration interval (1 hour). During the second migration, files that are last modified between 2019-03-10 08:00 and 2019-03-10 10:00 are migrated. The migration job includes a full migration and four incremental migrations.
    Notice Before you start a migration job, Data Transport compares files of the source data address with those of the destination data address. If a source file has the same name as a destination file, the destination file is overwritten when one of the following conditions is met:
    • The content types of the source file and the destination file are different.
    • The source file is updated after the previous migration.
    • The size of the source file is different from that of the destination file.
    Start Time Point of File Yes
    • All: All files are migrated.
    • Assign: Files that are created or modified after the specified time are migrated. For example, if you set the start time point to 2018-11-01 08:00:00, only files that are created or modified after 2018-11-01 08:00:00 are migrated. Files that are created or modified before the specified time are skipped.
    Migration Interval Yes (only for incremental migration) The default value is 1 hour and the maximum value is 24 hours.
    Migration Times Yes (only for incremental migration) The default value is 1 and the maximum value is 30.
  4. On the Performance tab, navigate to the Data Prediction section and specify the Data Size and File Count parameters.
    Note To ensure a successful migration, you must estimate the amount of data to be migrated. For more information, see Estimate the amount of data to be migrated.
  5. Optional. On the Performance tab, navigate to the Flow Control section, specify the Time Range and Max Flow parameters, and then click Add.
    Note To ensure business continuity, we recommend that you specify the Time Range and Max Flow parameters based on the fluctuation of workloads.
  6. Click Create and wait until the migration job is complete.