This topic describes the procedure for data migration.

Notes

When you perform an online migration, take note of the following issues:
  • A migration job occupies the network resources of the source data address and the destination data address. To ensure business continuity, we recommend that you specify a rate limit for a migration job or run the migration job during an off-peak period.
  • Before a migration job starts, the system checks the files at the source data address and the destination data address. If a source file has the same name as and is updated later than a destination file, the destination file is overwritten. If the two files have different content, you must change the name of one file or back up the files.
  • Symbolic links that exist at the source data address are skipped during migration.

Step 1: Create a source data address

  1. Log on to the Alibaba Cloud Data Transport console.
  2. Choose Data Online Migration > Data Address, and click Create Data Address.
  3. In the Create Data Address panel, configure the parameters and click OK. The following table describes the parameters.
    Parameter Required Description
    Data Type Yes Select NAS.
    Data Region Yes Select the region where the Network Attached Storage (NAS) file system is located.

    If you select Alibaba Cloud for NAS Type, select the region where the Apsara File Storage NAS file system is located. If you select Others for NAS Type, select the region of the Alibaba Cloud virtual private cloud (VPC) to which the third-party NAS file system is attached.

    Data Name Yes Enter a name that is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    NAS Type Yes Select the source of the NAS file system based on the actual scenario.
    • Alibaba Cloud: a NAS file system that is created on Alibaba Cloud.
    • Others: a NAS file system that is created on a third-party platform. You must mount the file system within an Alibaba Cloud VPC.
    File System Yes (only for Apsara File Storage NAS) Select the corresponding NAS file system.
    Mount Point Yes (only for Apsara File Storage NAS) Select the mount point of the NAS file system.
    Notice You can mount a NAS file system only within a VPC. The classic network is not supported.
    VPC Yes (only for third-party NAS services) Select a VPC to which the NAS file system is attached.
    Switches Yes (only for third-party NAS services) Select a vSwitch for the NAS file system.
    NAS Address Yes (only for third-party NAS services) Enter the IP address of the NFS server. This IP address must be accessible to the VPC.
    Sub Folder Optional Enter the directory where the data to be migrated is located. If you do not set this parameter, the default value is the root directory (/).
    Notice If you specify a subdirectory, you must make sure that the subdirectory exists on the NAS server. Otherwise, the data address fails to be created.
    Connection Method Yes (only for third-party NAS services) Select a protocol type for NAS.
    Connection Password Yes (only for third-party NAS services) Specify whether a password is required.
    • No Password: No password is required to access the NAS service.
    • Use Password: A username and password are required to access the NAS service. You must enter a valid username and password.
    Note For more information about the status of a new data address, see Data address status.
  4. If the system prompts that this feature is in the public preview, you must apply for whitelist permissions. Click Application.
  5. Enter the required information and submit the application for public preview. After the application is approved, you will receive a short message service (SMS) notification.

Step 2: Create a destination data address

  1. Choose Data Online Migration > Data Address, and click Create Data Address.
  2. In the Create Data Address panel, configure the parameters and click OK. The following table describes the parameters.
    Parameter Required Description
    Data Type Yes Select OSS.
    Data Region Yes Select the region where the destination data address is located.
    Data Name Yes Enter a name that is 3 to 63 characters in length. Special characters are not supported, except for hyphens (-) and underscores (_).
    OSS Endpoint Yes Select an endpoint based on the region where your data is located. For more information, see Endpoint.
    Access Key Id and Access Key Secret Yes Specify the AccessKey pair that is used to migrate data. For more information, see Create and authorize a RAM user.
    OSS Bucket Yes Select a bucket to store the migrated data.
    OSS Prefix No An OSS prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, you can specify data/to/oss/ as the OSS prefix. Do not specify this parameter if you want to migrate data to the root directory of the specified bucket.

Step 3: Create a migration job

  1. Choose Data Online Migration > Migration Jobs, and click Create Job.
  2. In the Create Job panel, read the Terms of Migration Service and select I understand the above terms and conditions, and apply for opening data migration service. Click Next.

    Then, the Fee Reminder message that reminds you of the request fee appears. Click Yes, Go Ahead.

    OSS_billing request
  3. On the Job Config tab, configure the parameters and click Next.

    The following table describes the parameters.

    Parameter Required Description
    Job Name Yes Enter a name that is 3 to 63 characters in length. The name can contain only lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-).
    Source Data Address Yes Select the source data address that you have created.
    Destination Data Address Yes Select the destination data address that you have created.
    Notice The source data address may be located in a different country than the destination data address. In this case, before you configure the migration job, you must submit a ticket to request permissions to create cross-country migration jobs. You must commit that your business is legitimate, data transit conforms to local rules and regulations, and your data does not contain illegal information.
    Specified Directory No
    • Do not filter: All the data at the source data address is migrated.
    • Exclude: The files and subdirectories in the excluded directories are not migrated.
    • Contain: Only the files and subdirectories in the specified directories are migrated.
    Note
    • A directory cannot start with a forward slash (/) or a backslash (\), and cannot contain double slashes (//), double periods (..), or double quotation marks ("). The character string that consists of all the specified directory names cannot exceed 10 KB in size.
    • A directory must end with a forward slash (/). For example, you can specify docs/ as the directory.
    • You can specify a maximum of 20 directories of the Exclude or Contain type.
    Migration Type Yes Before you start a migration job, the system compares files at the source data address with those at the destination data address. If a source file has the same name as and updated earlier than a destination file, the source file is skipped during migration. All the other files are migrated.
    • Full: The system runs a full migration job. After all data is migrated, a migration job is closed. You can submit the job again if the data at the source data address changes. In this case, the system migrates only the data that is changed after the previous job.
    • Incremental: The system runs the migration job based on the specified Migration Interval and Migration Times parameters. All data is migrated during the first migration. After the first migration is complete, incremental migrations are performed based on the migration interval. For each incremental migration, the system migrates only the data that is created or modified after the previous migration started and before this migration starts. If you set Migration Times to N, full migration is performed once and then incremental migration is performed (N - 1) times. Assume that you set the migration interval to 1 hour and migration times to 5. The system performs one full migration and four incremental migrations at an interval of 1 hour.
    • Sync: The system synchronizes the data from the source data address to the destination data address. A synchronization job continues to run based on the specified Job Period parameter until you stop the job. When a synchronization job is performed for the first time, data is synchronized based on the specified Start Time Point of File parameter. After the first synchronization is complete, the system synchronizes the files whose last modification time is later than the start time of the last synchronization each time a synchronization interval ends. Assume that the first synchronization is performed at 08:00, November 01, 2018. For the second synchronization, files that are created or modified after 08:00, November 01, 2018 are synchronized.
      Notice You can select Sync only if the source data address and the destination data address are located in the same region. Otherwise, this option is unavailable.
    Migration Interval Yes (only for incremental migration) The default value is 1 hour and the maximum value is 24 hours.
    Migration Times Yes (only for incremental migration) The default value is 1 and the maximum value is 30.
    Start Time Point of File Yes (only for data synchronization)
    • All: All files are synchronized.
    • Assign: Only the files that are created or modified after the specified time are synchronized. For example, if you set the start time to 08:00:00, November 01, 2018, only the files that are created or modified after 08:00:00, November 01, 2018 are synchronized. Files that are created or modified before the specified time are skipped.
    Start Time of Job Yes (only for data synchronization)
    • Immediately: The system synchronizes the data immediately after the migration job is created.
    • Schedule: The system synchronizes the data at the scheduled time.
    Job Period Yes (only for data synchronization) The time interval between two synchronization jobs. A synchronization job starts each time an interval ends. Valid units: hour, day, and week.
    Don't trigger new task if another task running Yes (only for data synchronization) Specify whether to start a new synchronization job if the last synchronization job is still running when the synchronization interval ends. You must use this parameter together with Job Period. Assume that Job Period is set to 1 hour and this parameter is not selected. In this case, the next synchronization job runs regardless of whether the last synchronization job is completed within 1 hour. This parameter is selected by default.
  4. Click Next. The Performance tab appears.
    • If you select Full or Incremental for the job type, specify the Data Size and File Count parameters.
      Note To ensure a successful migration, estimate the amount of data to be migrated as accurately as possible.
    • If you select Sync for the job type, specify the Subtask file count and Subtask file size parameters.
      • Subtask file count: A migration job can be divided into multiple subtasks based on the specified Subtask file count parameter. A maximum of 20 subtasks can run at a time. Set an appropriate number of files for each subtask to reduce the amount of time that is required to run a migration job. The default value is 1000. Assume that you need to migrate 10,000 files. If you set the Subtask file count parameter to 500, the migration job is divided into 20 subtasks that run at the same time. If you set the Subtask file count parameter to 100, the migration job is divided into 100 subtasks. Each time the system migrates the files, 20 subtasks run at the same time and the remaining subtasks are queued.
      • Subtask file size: A migration job can be divided into multiple subtasks based on the specified Subtask file size parameter. A maximum of 20 subtasks can run at a time. Set an appropriate size of files for each subtask to reduce the amount of time that is required to run a migration job. The default value is 1 GB. Assume that you need to migrate files with a total size of 40 GB. If you set the Subtask file size parameter to 2 GB, the migration job is divided into 20 subtasks that run at the same time. If you set the Subtask file size parameter to 1 GB, the migration job is divided into 40 subtasks. Each time the system migrates the files, 20 subtasks run at the same time and the remaining subtasks are queued.
      Note Subtasks are generated when the specified Subtask file count or Subtask file size parameter value is reached. If the specified Subtask file count parameter value is reached but the specified Subtask file size parameter value is not reached, subtasks are generated based on the number of files. If the specified Subtask file count parameter value is not reached but the specified Subtask file size parameter value is reached, subtasks are generated based on the file size. Assume that you set the Subtask file count parameter to 1000 and the Subtask file size parameter to 1 GB. If the number of files reaches 1,000 but the file size does not reach 1 GB, subtasks are generated based on the number of files. If the file size reaches 1 GB but the number of files does not reach 1,000, subtasks are generated based on the file size.
  5. Optional. On the Performance tab, navigate to the Flow Control section, specify the Time Range and Max Flow parameters, and then click Add.
    Note To ensure business continuity, we recommend that you specify the Time Range and Max Flow parameters based on the workload peaks and troughs. The default value of the Time Range parameter is 06:00 to 12:00. The default value of the Max Flow parameter is 5 MB/s.
  6. Click Create. Wait until the migration job is completed.

View the status of a data address

After a NAS data address is created, one of the following states is displayed for the data address:
  • Normal: The data address is created.
  • Creating: It takes time to create the first NAS data address. Wait for about 3 minutes. If the state of the data address remains Creating for a long time, you can click Refresh in the upper-right corner to update the status.
  • Invalid: An error has occurred when you create the data address. You can verify the configuration information and check whether Data Online Migration is allowed to access the NAS service. If this issue persists, you can contact Alibaba Cloud technical support.