This topic describes the notes and procedure for data migration.

Notes

When you perform an online migration, take note of the following issues:
  • A migration task occupies the network resources of the source and destination data addresses. To ensure business continuity, we recommend that you configure a rate limit for your migration task or run your migration task during off-peak hours.
  • Before an online migration task starts, the system checks the files at the source data address and the destination data address. If the source and destination have files with the same name, and the last modification time of the file in the source is later than the file in the destination, the file in the destination is overwritten. If both files are important and contain different information, we recommend that you change the name of one file or back up the destination file.

Step 1: Create a source data address

  1. Log on to the Alibaba Cloud Data Transport console.
  2. Choose Data Online Migration > Data Address, and click Create Data Address.
  3. In the Create Data Address panel, configure the following parameters and click OK.
    Parameter Required Description
    Data Type Yes Select NAS.
    Data Region Yes Select a region where an Elastic Compute Service (ECS) instance is deployed.
    Data Name Yes Enter a name that is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    NAS Type Yes Select Others.
    VPC Yes Select a virtual private cloud (VPC) to which the ECS instance is connected.
    Switches Yes Select a vSwitch to which the ECS instance is attached.
    NAS Address Yes Enter the private IP address of the ECS instance.
    Sub Folder Yes Enter the path of the shared folder that stores the data to be migrated.
    Connection Method Yes Select a protocol type for the shared folder.
    Connection Password Optional Specify whether a password is required to access the shared folder.
    • No Password: No username and password are required to access the shared folder.
    • Use Password: A username and password are required to access the shared folder. You must enter a valid username and password.
    Note For Windows-based instances, you must enter the username and password. For Linux-based instances, you can specify the parameter based on your needs.
    Note For more information about the status of a new data address, see Data address status.
  4. If the system prompts that this feature is in the public preview, you must apply for whitelist permissions. Click Application.
  5. Enter the required information and submit the application for public preview. After the application is approved, you will receive a short message service (SMS) notification.

Step 2: Create a destination data address

  1. Choose Data Online Migration > Data Address, and click Create Data Address.
  2. In the Create Data Address panel, configure the parameters and click OK. The following table describes the parameters.
    Parameter Required Description
    Data Type Yes Select OSS.
    Data Region Yes Select the region where the destination data address is located.
    Data Name Yes Enter a name that is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    OSS Endpoint Yes Select an endpoint based on the region where your data is located. For more information, see Endpoint.
    Access Key Id and Access Key Secret Yes Specify the AccessKey pair that is used to migrate data. For more information, see Create a Resource Access Management (RAM) user and grant required permissions to the RAM user.
    OSS Bucket Yes Select a bucket to store the migrated data.
    OSS Prefix No An Object Storage Service (OSS) prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify data/to/oss/ as the OSS prefix. Do not specify this parameter if you want to migrate data to the root directory of the specified bucket.

Step 3: Create a migration job

  1. Choose Data Online Migration > Migration Jobs, and click Create Job.
  2. In the Create Job panel, read the Terms of Migration Service and select I understand the above terms and conditions, and apply for opening data migration service. Click Next.
  3. In the Fee Reminder dialog box, click Yes, Go Ahead.
  4. On the Job Config tab, configure the parameters and click Next.
    Parameter Required Description
    Job Name Yes Enter a job name. The name must be 3 to 63 characters in length and can contain only lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
    Source Data Address Yes Select the source data address that you created.
    Destination Data Address Yes Select the destination data address that you created.
    Notice If the source data address is located in a different country than the destination data address, make sure that you have the permissions to create cross-country migration jobs. To request the permissions, submit a ticket. You must ensure that your business is legitimate, the data transit process conforms to local rules and regulations, and your data does not contain illegal information.
    Specified Directory No
    • Do not filter: All data from the source data address is migrated.
    • Exclude: The files and subdirectories in the excluded directories are not migrated.
    • Contain: Only the files and subdirectories in the specified directories are migrated.
    Note
    • A directory name cannot start with a forward slash (/) or a backslash (\) and cannot contain double slashes (//), double periods (..), or double quotation marks ("). The character string that consists of the specified directory names cannot exceed 10 KB in size.
    • A directory name must end with a forward slash (/). For example, you can specify docs/ as the directory name.
    • You can specify a maximum of 20 directories of the Exclude or Contain type.
    Migration Type Yes
    • Full: The system runs a full migration job. If this option is selected, the system migrates data that is modified after the start time specified by the Start Time Point of File parameter at one time. The migration job is closed after all data is migrated. If the data at the source data address changes, you can submit the job again. In this case, only the data that is changed after the previous job is migrated.
    • Incremental: The system runs a migration job based on the specified Migration Interval and Migration Times parameters. If this option is selected, the system first performs a full migration to migrate all data that is modified after the start time specified by the Start Time Point of File parameter. After the first migration is complete, incremental migrations are performed based on the specified Migration Interval parameter. For each incremental migration, the system migrates only the data that is created or modified after the previous migration started and before the current migration starts. If you set Migration Times to N, full migration is performed once and then incremental migration is performed (N - 1) times. For example, you set Migration Interval to 1 hour, Migration Times to 5, and Start Time Point of File to 08:00, March 5, 2019, and the current time is 08:00, March 10, 2019. When the first migration starts, the system migrates the files that are last modified between 08:00, March 5, 2019 and 08:00, March 10, 2019. In this example, the first migration requires 1 hour to complete and the second migration starts at 10:00, March 10, 2019. The duration from 08:00 to 10:00 includes the 1 hour that the first migration requires and the migration interval of 1 hour. During the second migration, the system migrates the files that are last modified between 08:00, March 10, 2019 and 10:00, March 10, 2019. The migration job consists of one full migration and four incremental migrations.
    Notice Before you start a migration job of the Full or Incremental type, the system compares files at the source data address with those at the destination data address. If a source file has the same name as a destination file, the destination file is overwritten when one of the following conditions is met:
    • The content types of the source file and the destination file are different.
    • The source file is updated after the previous migration.
    • The size of the source file is different from that of the destination file.
    Start Time Point of File Yes
    • All: All files are migrated.
    • Assign: Only the files that are created or modified after the specified time are migrated. For example, if you set the start time to 08:00:00, November 01, 2018, only the files that are created or modified after 08:00:00, November 01, 2018 are migrated. Files that are created or modified before the specified time are skipped.
    Migration Interval Yes (only for incremental migrations) The default value is 1 hour and the maximum value is 24 hours.
    Migration Times Yes (only for incremental migrations) The default value is 1 and the maximum value is 30.
    File Overwrite Method Yes The method that is used to process a source file that has the same name as a destination file during migration. Valid values:
    • LastModified: If a source file has the same name as a destination file, the system compares the values of the LastModified property of the files.
      • If the value of the LastModified property of the source file is smaller than that of the destination file, the source file is skipped and not migrated.
      • If the value of the LastModified property of the source file is greater than that of the destination file, the destination file is overwritten with the source file.
      • If the values of the LastModified property of the two files are equal, the system checks whether the values of the Size and Content-Type properties of the files are equal.

        If the values of the Size and Content-Type properties of the files are equal, the source file is skipped and not migrated. If the values of one or both properties are different between the files, the destination file is overwritten with the source file.

    • Condition: If a source file has the same name as a destination file, the system compares the values of the LastModified, Size, and Content-Type properties of the files.
      • If the values of the LastModified, Size, and Content-Type properties of the files are equal, the source file is skipped and not migrated.
      • If the values of at least one of the LastModified, Size, and Content-Type properties are different between the files, the destination file is overwritten with the source file.
    • All: If a source file has the same name as a destination file, the system performs no checks and directly overwrites the destination file with the source file.
    • No: If a source file has the same name as a destination file, the system performs no checks and directly skips the source file.
  5. Click Next. The Performance tab appears.
    • If you select Full or Incremental for the job type, specify the Data Size and File Count parameters.
      Note To ensure a successful migration, estimate the amount of data to be migrated as accurately as possible.
    • If you select Sync for the job type, specify the Subtask file count and Subtask file size parameters.
      • Subtask file count: A migration job can be divided into multiple subtasks based on the specified Subtask file count parameter. A maximum of 20 subtasks can run at a time. Set an appropriate number of files for each subtask to reduce the amount of time that is required to run a migration job. The default value is 1000. Assume that you need to migrate 10,000 files. If you set the Subtask file count parameter to 500, the migration job is divided into 20 subtasks that run at the same time. If you set the Subtask file count parameter to 100, the migration job is divided into 100 subtasks. Each time the system migrates the files, 20 subtasks run at the same time and the remaining subtasks are queued.
      • Subtask file size: A migration job can be divided into multiple subtasks based on the specified Subtask file size parameter. A maximum of 20 subtasks can run at a time. Set an appropriate size of files for each subtask to reduce the amount of time that is required to run a migration job. The default value is 1 GB. Assume that you need to migrate files with a total size of 40 GB. If you set the Subtask file size parameter to 2 GB, the migration job is divided into 20 subtasks that run at the same time. If you set the Subtask file size parameter to 1 GB, the migration job is divided into 40 subtasks. Each time the system migrates the files, 20 subtasks run at the same time and the remaining subtasks are queued.
      Note Subtasks are generated when the specified Subtask file count or Subtask file size parameter value is reached. If the specified Subtask file count parameter value is reached but the specified Subtask file size parameter value is not reached, subtasks are generated based on the number of files. If the specified Subtask file count parameter value is not reached but the specified Subtask file size parameter value is reached, subtasks are generated based on the file size. Assume that you set the Subtask file count parameter to 1000 and the Subtask file size parameter to 1 GB. If the number of files reaches 1,000 but the file size does not reach 1 GB, subtasks are generated based on the number of files. If the file size reaches 1 GB but the number of files does not reach 1,000, subtasks are generated based on the file size.
  6. Optional. On the Performance tab, navigate to the Flow Control section, specify the Time Range and Max Flow parameters, and then click Add.
    Note To ensure business continuity, we recommend that you specify the Time Range and Max Flow parameters based on the workload peaks and troughs.
  7. Click Create and wait until the migration job is completed.

View the status of a data address

After you create a data address for an ECS instance, one of the following states is displayed for the data address:
  • Normal: The data address is created.
  • Creating: It takes time to create the first data address for the ECS instance. Wait for 3 minutes. If the state of the data address remains Creating for a long time, you can click Refresh in the upper-right corner to update the status.
  • Invalid: An error has occurred when you create the data address. You can verify the configuration information and check whether Data Online Migration is allowed to access the shared files of the ECS instance. If this issue persists, you can submit a ticket.