This topic describes the note and procedure for migrating data.

Note

When you perform an online migration, take note of the following items:

  • A migration job occupies the network resources of the source data address and the destination data address. To ensure business continuity, we recommend that you specify a rate limit for your migration job or run your migration job during an off-peak period.
  • Before a migration job starts, the system checks the files at the source data address and the destination data address. If a source file has the same name as a destination file and the source file is updated later than the destination file, the destination file is overwritten. If the two files have different content, you must change the name of one file or back up the destination file.
  • Data Online Migration allows you only to migrate data of a single bucket at a time. You cannot migrate all data that belongs to your account at the same time.

Step 1: Create a source data address

  1. Log on to the Alibaba Cloud Data Transport console.
  2. In the left-side navigation pane, choose Data Online Migration > Data Address. On the page that appears, click Create Data Address.
  3. In the Create Data Address panel, configure the following parameters and click OK.
    Parameter Required Description
    Data Type Yes Select UCloud US3.
    Data Name Yes Enter a name. The name must be 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    Region Yes Select the region to which the source data address belongs, such as Beijing (cn-bj).
    Note You can log on to the UCloud console and click the name of the bucket that stores the data to be migrated. Choose Overview > Basic Information to view the region of the bucket.
    Bucket Yes Select the bucket that stores the data to be migrated.
    Prefix Yes
    • Migrate All Data: All data in the bucket is migrated.

      If you select this option, you do not need to specify a prefix.

    • Migrate Partial Data: Only the files in the directory that is specified by the prefix are migrated.

      If you select this option, you must specify a prefix. A prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify data/to/oss/ as the prefix.

    Public Key Yes Enter the API public key that is used to migrate data. Delete the key after the migration.
    Private Key Yes Enter the API private key that is used to migrate data. Delete the key after the migration.
  4. Optional:If the system prompts that this feature is in the public preview, click Application to submit a whitelist permission application.
  5. Enter the required information and submit the application for public preview. After the application is approved, the system sends a text message to notify you.

Step 2: Create a destination data address

  1. Choose Data Online Migration > Data Address, and click Create Data Address.
  2. In the Create Data Address panel, configure the parameters and click OK.
    Parameter Required Description
    Data Type Yes Select OSS.
    Data Region Yes Select the region where the destination data address is located.
    Data Name Yes Enter a name that is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    OSS Endpoint Yes
    Select an endpoint based on the region where your data is located.
    • http://oss-cn-endpoint.aliyuncs.com: indicates that you use an HTTP-based endpoint to access Object Storage Service (OSS) over the Internet.
    • https://oss-cn-endpoint.aliyuncs.com: indicates that you use an HTTPS-based endpoint to access OSS over the Internet.
    For more information, see Endpoint.
    Note When you migrate third-party data to OSS, you must access OSS from an Internet endpoint.
    Access Key Id and Access Key Secret Yes Specify the AccessKey pair that is used to migrate data. For more information, see Create an AccessKey pair.
    OSS Bucket Yes Select a bucket to store the migrated data.
    OSS Prefix No An OSS prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify data/to/oss/ as the OSS prefix. Do not specify this parameter if you want to migrate data to the root directory of the specified bucket.
    Notice If the name of a source file starts with a forward slash (/), you must specify an OSS prefix when you configure the destination data address. If no OSS prefix is specified, the migration job fails. For example, if the name of a file to be migrated is /test/test.png, you must specify an OSS prefix, such as oss/. After the migration job is completed, the name of the OSS file changes from /test/test.png to oss//test/test.png.

Step 3: Create a migration job

  1. Choose Data Online Migration > Migration Jobs and click Create Job.
  2. In the Create Job dialog box, read the Terms of Migration Service, select I understand the above terms and conditions, and apply for enabling the data migration service, and then click Next.
    Then, the Fee Reminder message that reminds you of the request fee appears. Click Yes, Go Ahead. OSS request fee
  3. In the Job Config step, configure the following parameters and click Next.
    Parameter Required Description
    Job Name Yes Enter a job name. The name must be 3 to 63 characters in length and can contain only lowercase letters, digits, and hyphens (-). The name cannot start with or end with a hyphen (-).
    Source Data Address Yes Select the source data address that you created.
    Destination Data Address Yes Select the destination data address that you created.
    Notice The source data address can be located in a different country than the destination data address. In this case, before you configure the migration job, you must submit a ticket to request permissions to create cross-country migration jobs. You must ensure that your business is legitimate, the data transit process conforms to local rules and regulations, and your data does not contain illegal information.
    Specified Directory No
    • Do not filter: All data from the source data address is migrated.
    • Exclude: The files and subdirectories in the excluded directories are not migrated.
    • Contain: Only the files and subdirectories in the specified directories are migrated.
    Note
    • A directory name cannot start with a forward slash (/) or a backslash (\) and cannot contain double slashes (//), double periods (..),or double quotation marks ("). The character string that consists of the specified directory names cannot exceed 10 KB in size.
    • A directory must end with a forward slash (/). For example, you can specify docs/ as the directory.
    • You can specify a maximum of 20 directories of the Exclude or Contain type.
    Migration Type Yes
    • Full: The system runs a full migration job. When you run a full migration job, you must configure the Start Time Point of File parameter. The system migrates the data whose last modification time is later than the specified start time at one time. The migration job is closed after all data is migrated. You can submit the job again if the data at the source data address changes. In this case, only the data that is changed after the previous job is migrated.
    • Incremental: The system runs the migration job based on the specified Migration Interval and Migration Times parameters. You must configure the Start Time Point of File parameter. During the first migration, the system migrates the full data that is modified after the specified start time. After the first migration is complete, incremental migrations are performed based on the specified Migration Interval parameter. For each incremental migration, the system migrates only the data that is created or modified after the previous migration started and before the current migration starts. If you set Migration Times to N, full migration is performed once and then incremental migration is performed (N - 1) times. For example, you set Migration Interval to 1 hour, Migration Times to 5, and Start Time Point of File to 08:00, March 5, 2019. In this case, the current time is 08:00, March 10, 2019. When the first migration starts, the system migrates the files that are last modified between 08:00, March 5, 2019 and 08:00, March 10, 2019. In this example, the first migration requires 1 hour to complete and the second migration starts at 10:00, March 10, 2019. The two hours from 08:00 to 10:00 include the 1 hour that the first migration requires and the migration interval of 1 hour. During the second migration, the system migrates the files that are last modified between 08:00, March 10, 2019 and 10:00, March 10, 2019. The migration job consists of one full migration and four incremental migrations.
    Notice Before you start a migration job of the Full or Incremental type, the system compares files at the source data address with those at the destination data address. If a source file has the same name as a destination file, the destination file is overwritten when one of the following conditions is met:
    • The content types of the source file and the destination file are different.
    • The source file is updated after the previous migration.
    • The size of the source file is different from that of the destination file.
    Start Time Point of File Yes
    • All: All files are migrated.
    • Assign: Only the files that are created or modified after the specified time are migrated. For example, if you set the start time to 08:00:00, November 01, 2018, only the files that are created or modified after 08:00:00, November 01, 2018 are migrated. Files that were created or modified before the specified time are skipped.
    Migration Interval Yes (only for incremental migrations) The default value is 1 hour, and the maximum value is 24 hours.
    Migration Times Yes (only for incremental migrations) The default value is 1, and the maximum value is 30.
    File Overwrite Method Yes The method of how to process a source file that has the same name as a destination file during migration. Valid values:
    • LastModified
    • Condition: If a source file is named the same as a destination file, the system compares the values of the LastModified, Size, and Content-Type attributes of the source file with those of the destination file.
      • If the values of all the LastModified, Size, and Content-Type attributes for the two files are the same, the source file is skipped and not migrated.
      • If the values of at least one of the LastModified, Size, and Content-Type attributes for the two files are different, the destination file is overwritten with the source file.
    • All: If a source file has the same name as a destination file, the system directly overwrites the destination file with the source file.
    • If a source file has the same name as a destination file, the system directly skips the source file.
  4. On the Performance tab, navigate to the Data Prediction section and configure the Data Size and File Count parameters.
    Note An accurate estimation of the data amount helps you migrate data as expected. For more information, see Estimate the amount of data to be migrated.
  5. Optional. On the Performance tab, navigate to the Flow Control section, configure the Time Range and Max Flow parameters, and then click Add.
    Note To ensure business continuity, we recommend that you configure the Time Range and Max Flow parameters based on the workload peaks and troughs.
  6. Click Create and wait until the migration job is completed.