This topic describes how to migrate data by using Data Transport.

Usage notes

When you perform an online migration, take note of the following items:
  • A migration task occupies the network resources of the source and destination data addresses. To ensure business continuity, we recommend that you configure a rate limit for your migration task or run your migration task during off-peak hours.
  • Before an online migration task starts, the system checks the files at the source data address and the destination data address. If the source and destination have files with the same name, and the last modification time of the file in the source is later than the file in the destination, the file in the destination is overwritten. If both files are important and contain different information, we recommend that you change the name of one file or back up the destination file.
  • Data Online Migration allows you to migrate only the data of a single bucket per task. You cannot migrate all data that belongs to your account at a time.

Step 1: Create a source data address

  1. Log on to the Alibaba Cloud Data Transport console.
  2. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.
  3. In the Create Data Address panel, configure the following parameters and click OK.
    Parameter Required Description
    Data Type Yes Select AWS S3 Inventory.
    Data Name Yes Enter a name. The name must be 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    Endpoint Yes Enter the endpoint for accessing AWS Simple Storage Service (S3). For more information, see Regions and endpoints.
    Bucket Yes The name of the AWS S3 bucket.

    The bucket name cannot contain invalid characters, such as spaces, line breaks, and tab characters.

    Access Key Id Yes Specify the key pair that is used to access the AWS S3 bucket. After the migration is complete, delete the key pair.
    Secret Access Key Yes
    Inventory Endpoint Yes Enter the endpoint for accessing AWS S3 Inventory. For more information, see Regions and endpoints.
    Inventory AccessKey Id Yes Specify the key pair that is used to access AWS S3 Inventory. After the migration is complete, delete the key pair.
    Inventory Secret AccessKey Yes
    InventoryAddress Yes Enter the URL of the AWS S3 inventory lists, such as s3://{bucket}/{ObjectKey}. In this example, bucket indicates the name of the bucket where the inventory lists are located, and ObjectKey is the path of the manifest.json file.
  4. Optional:If the system prompts that this feature is in public preview, click Application to apply for using this feature.
  5. Optional:Enter the required information and submit the application for public preview. After the application is approved, the system sends a text message to notify you.

Step 2: Create a destination data address

  1. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.
  2. In the Create Data Address panel, configure the following parameters and click OK.
    Parameter Required Description
    Data Type Yes Select OSS.
    Data Name Yes Enter a name. The name must be 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    Data Region Yes Select the region where the destination data address is located.
    Enable Transfer Acceleration No For Data Online Migration to use the transfer acceleration feature provided by Object Storage Service (OSS), you must enable the transfer acceleration feature for buckets. The transfer acceleration feature takes effect within 30 minutes after it is enabled. After you enable the feature, wait for 30 minutes before you create a migration job.
    Notice If you use the transfer acceleration feature to access your OSS buckets, you are charged additional fees for transfer acceleration. For more information, see Transfer acceleration.
    OSS Endpoint Yes
    Select an endpoint in the region where your data is stored. For more information, see Regions and endpoints.
    • http://oss-cn-endpoint.aliyuncs.com: indicates that you use an HTTP-based endpoint to access OSS over the Internet.
    • https://oss-cn-endpoint.aliyuncs.com: indicates that you use an HTTPS-based endpoint to access OSS over the Internet.
    Note When you migrate data from third-party cloud services to OSS, you must access OSS from a public endpoint over the Internet.
    AccessKey Id Yes Specify the AccessKey pair that is used to migrate data. For more information, see Preparations.
    AccessKey Secret Yes
    OSS Bucket Yes Select a bucket to store the migrated data.
    OSS Prefix No An OSS prefix cannot start with a forward slash (/) but must end with a forward slash (/). For example, you can specify data/to/oss/ as the OSS prefix. Do not specify this parameter if you want to migrate data to the root directory of the specified bucket.
    Notice If the name of a source file starts with a forward slash (/), you must specify an OSS prefix when you configure the destination data address. If no OSS prefix is specified, the migration job fails. For example, if the name of a file that you want to migrate is /test/test.png, you must specify an OSS prefix, such as oss/. After the migration job is complete, the name of the file changes from /test/test.png to oss//test/test.png.

Step 3: Create a migration job

  1. Choose Data Online Migration > Migration Jobs and click Create Job.
  2. In the Create Job panel, read the terms of the migration service and select I understand the above terms and conditions, and apply for opening data migration service. Then, click Next.
    In the Fee Reminder dialog box, click Yes, Go Ahead. OSS request fee
  3. In the Job Config step, configure the following parameters and click Next.
    Parameter Required Description
    Job Name Yes Enter a job name. The name must be 3 to 63 characters in length and can contain only lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
    Source Data Address Yes Select the source data address that you created.
    Destination Data Address Yes Select the destination data address that you created.
    Notice If the source data address and the destination data address are located in different countries, make sure that you have the permissions to create cross-country migration jobs. To request the permissions, submit a ticket. You must declare that your business is legitimate, the data transmission process conforms to local rules and regulations, and your data does not contain illegal information.
    Specified Directory No Specify the directories whose files and subdirectories are included or excluded during migration.
    • Do not filter: All data from the source data address is migrated.
    • Exclude: The files and subdirectories in the specified directories are not migrated.
    • Contain: Only the files and subdirectories in the specified directories are migrated.
    Note
    • You must configure a whitelist before you set the Specified Directory parameter. For more information, see Advanced features.
    • A directory name cannot start with a forward slash (/) or a backslash (\), and cannot contain double forward slashes (//), double periods (..), or double quotation marks ("). The character string that consists of all the specified directory names cannot exceed 10 KB in length.
    • A directory name must end with a forward slash (/). For example, you can specify docs/ as the directory name.
    • You can specify a maximum of 20 directories of the Exclude or Contain type.
    Migration Type Yes Full: The system runs a full migration job. After all data is migrated, a migration job ends. If the data at the source data address changes after the migration is complete, you can submit another job to migrate all data. In this case, the system migrates only the data that changes.
    Notice Before you start a full migration job, the system compares files at the source data address with those at the destination data address. If a source file has the same name as a destination file, the destination file is overwritten when one of the following conditions is met:
    • The Content-Type property values of the source file and the destination file are different.
    • The last modification time of the source file is later than that of the destination file.
    • The size of the source file is different from that of the destination file.
    File Overwrite Method Yes The method that is used to process the source file whose name is the same as a destination file. Valid values:
    • LastModified: If a source file has the same name as a destination file, the system compares the last modification time indicated by the LastModified property of the files.
      • If the last modification time of the source file is later than that of the destination file, the source file is migrated and the destination file is overwritten.
      • If the last modification time of the source file is earlier than that of the destination file, the source file is skipped during the migration.
      • If the two files have the same last modification time, the system compares the Size and Content-Type property values of the files.

        If the files have the same property values, the source file is skipped in the migration. If the values of at least one property are different, the source file is migrated and the destination file is overwritten.

    • Condition: If a source file has the same name as a destination file, the system compares the values of the LastModified, Size, and Content-Type properties of the files.
      • If all the property values are the same, the source file is skipped during the migration.
      • If the values of at least one property are different, the source file is migrated and the destination file is overwritten.
    • All: If a source file has the same name as a destination file, the system performs no checks and directly overwrites the destination file with the source file.
    • No: If a source file has the same name as a destination file, the system performs no checks and directly skips the source file during the migration.
  4. On the Performance tab, navigate to the Data Prediction section and configure the Data Size and File Count parameters.
    Note To ensure a successful migration, we recommend that you estimate the amount of data to be migrated as accurately as possible. For more information, see Estimate the amount of data to be migrated.
  5. Optional: On the Performance tab, navigate to the Flow Control section, specify the Time Range and Max Flow(MB/s) parameters, and then click Add.
    Note To ensure business continuity, we recommend that you configure the Time Range and Max Flow parameters based on the traffic fluctuations.
  6. Click Create and wait until the migration job is complete.