This topic describes how to migrate data by using Data Transport.

Precautions

When you perform online migration, take note of the following items:
  • Data Online Migration uses the standard and publicly available API operations provided by the storage service provider of the source data address to access the source data. The data accessed by Data Online Migration depends on the specific API operations that are implemented by the service provider.
  • A migration job occupies the network resources of the source and destination data addresses. To ensure business continuity, we recommend that you specify a rate limit for your migration job or run your migration job during off-peak hours.
  • Before a migration job starts, the system checks the files at the source data address and the destination data address. If the source and destination data addresses have files with the same name, and the last modification time of the source file is later than the last modification time of the destination file, the destination file is overwritten. If both files contain different information, we recommend that you change the name of one file or back up the destination file.
Warning Make sure that you verify the migrated data at the destination data address after the migration task is complete to ensure that the migrated data is correct. If you delete the data at the source data address before you verify the migrated data at the destination data address, you are liable for the losses and consequences caused by any data loss.

Step 1: Create a source data address

  1. Log on to the Alibaba Cloud Data Transport console.
  2. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.
  3. In the Create Data Address panel, configure the following parameters and click OK.
    ParameterRequiredDescription
    Data TypeYesSelect Http/Https.
    Data NameYesEnter a name. The name must be 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    File PathYesSpecify the file path in the format of oss://{bucket}/{Name of a list file}. For more information, see Create a list file.
    List Access EndpointYesEnter a suitable endpoint based on the region where the source bucket resides. For more information, see Regions and endpoints.
    List Access AKYesSpecify the AccessKey pair that is used to migrate data. The AccessKey pair is automatically generated for the RAM user in the preparation process. For more information about the preparation process, see Prepare for a migration job.
    List Access SKYes

Step 2: Create a destination data address

  1. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.
  2. In the Create Data Address panel, configure the following parameters and click OK.
    ParameterRequiredDescription
    Data TypeYesSelect OSS.
    Data NameYesEnter a name. The name must be 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    Data RegionYesSelect the region where the destination data address is located.
    Enable Transfer AccelerationNoFor Data Online Migration to use the transfer acceleration feature provided by Object Storage Service (OSS), you must enable the transfer acceleration feature for buckets. The transfer acceleration feature takes effect within 30 minutes after it is enabled. After you enable the feature, wait for 30 minutes before you create a migration job.
    Important If you use the transfer acceleration feature to access your OSS buckets, you are charged additional transfer acceleration fees. For more information, see Enable transfer acceleration.
    OSS EndpointYesSelect an endpoint of the region where your data is stored. For more information, see Regions and endpoints.
    • http://oss-cn-endpoint.aliyuncs.com: specifies that you use an HTTP-based endpoint to access OSS over the Internet.
    • https://oss-cn-endpoint.aliyuncs.com: specifies that you use an HTTPS-based endpoint to access OSS over the Internet.
    • http://oss-cn-qingdao-internal.aliyuncs.com: specifies that you use a private HTTP-based endpoint to access OSS.
    • https://oss-cn-qingdao-internal.aliyuncs.com: specifies that you use a private HTTPS-based endpoint to access OSS.
    Important For example, a private OSS endpoint is used as the source data address when you create a migration job. In this case, the destination data address must be an Apsara File Storage NAS or OSS data address in the same region.
    AccessKey IdYesSpecify the AccessKey pair that is used to migrate data. For more information, see Prepare for a migration job.
    AccessKey SecretYes
    OSS BucketYesSelect a bucket to store the migrated data.
    OSS PrefixNoThe value cannot start with a forward slash (/) but must end with a forward slash (/). For example, you can specify data/to/oss/ as the OSS prefix. Do not specify this parameter if you want to migrate data to the root directory of the specified bucket.

Step 3: Create a migration job

  1. In the left-side navigation pane, choose Data Online Migration > Migration Jobs. On the page that appears, click Create Job.
  2. In the Create Job panel, read the terms of the migration service and select I agree to the above terms and activate the Data Transport service. Then, click Next.
  3. In the Job Config step, configure the following parameters and click Next.
    ParameterRequiredDescription
    Job NameYesEnter a job name. The name must be 3 to 63 characters in length and can contain only lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
    Source Data AddressYesSelect the source data address that you created.
    Destination Data AddressYesSelect the destination data address that you created.
    Migration TypeYesFull: The system runs a full migration job. After all data is migrated, the migration job ends. If the data at the source data address changes after the migration job is complete, you can submit another full migration job. In this case, the system migrates only the data that has changed since the last migration job.
    Important Before you start a full migration job, the system compares files at the source data address with those at the destination data address. If a source file has the same name as a destination file, the destination file is overwritten if one of the following conditions is met:
    • The Content-Type property values of the source file and the destination file are different.
    • The last modified time of the source file is later than that of the destination file.
    • The size of the source file is different from that of the destination file.
    File Overwrite MethodYesThe method that is used to process the source file whose name is the same as that of a destination file. Valid values:
    • LastModified: If a source file has the same name as a destination file, the system compares the last modified time indicated by the LastModified property of the files.
      • If the last modified time of the source file is later than that of the destination file, the source file is migrated and the destination file is overwritten.
      • If the last modified time of the source file is earlier than that of the destination file, the source file is not migrated.
      • If the last modified time of the two files are the same, the system compares the Size and Content-Type property values of the files.

        If the two files have the same property values, the source file is not migrated. If the values of at least one property are different, the source file is migrated and the destination file is overwritten.

    • Condition: If a source file has the same name as a destination file, the system compares the values of the LastModified, Size, and Content-Type properties of the files.
      • If all property values of the two files are the same, the source file is not migrated.
      • If the two files have different values of at least one property, the source file is migrated and the destination file is overwritten.
    • All: If a source file has the same name as a destination file, the system performs no comparison and directly overwrites the destination file with the source file.
    • No: If a source file has the same name as a destination file, the system performs no comparison and directly skips the source file during the migration.
  4. In the Performance step, navigate to the Data Prediction section and configure the Data Size and File Count parameters.
    Note To ensure a successful migration, we recommend that you estimate the amount of data that the system will migrate as accurately as possible. For more information, see Estimate the amount of data that you want to migrate.
  5. Optional:In the Flow Control section of the Performance step, configure the Time Range and Max Flow(MB/s) parameters and click Add.
    Note To ensure business continuity, we recommend that you configure the Time Range and Max Flow(MB/s) parameters based on the workload peaks and troughs.
  6. Click Create and wait until the migration job is complete.

Success Stories

For examples of migrating data from an HTTP or HTTPS address to an OSS bucket, see Migrate data from a local IDC to OSS.