This topic describes how to migrate data between Alibaba Cloud NFS file systems.

Prerequisites

A source NFS file system that has data stored on it and has a virtual private cloud (VPC) mount target is available.

Preparations

  1. View information about the source mount target.
    Before you migrate data, view information about the mount target of the source file system and the VPC where the mount target resides. For more information, see View mount targets.
    Note If the mount target of your NFS file system resides in the classic network, you must create a VPC mount target for the file system. For more information, see Add a mount target.
  2. Configure the destination mount target.
    You can use one of the following three methods to configure a mount target for the destination NFS file system:
    • Create a new file system in the destination region and zone, and a mount target is automatically created for the file system.
    • Find a mount target that has been created for the destination file system.
    • Create a mount target for the destination file system.
    Note If the destination file system and the source file system reside in the same region, make sure that the destination mount target and the source mount target reside in the same VPC to facilitate migration.
    1. Create an NFS file system.
      For more information about how to create a file system when no file system is available, see Create a file system.
      Note If you purchase a pay-as-you-go NFS Capacity or NFS Performance file system, select the same VPC and VSwitch that are used by the source mount target. Then, a mount target is automatically created. After a new file system is created, you can buy a storage plan and bind the storage plan to the file system to reduce costs.
    2. Find an existing mount target.

      If the destination file system has a mount target that resides in the same VPC as the source mount target, you can use this mount target. For more information, see View mount targets.

    3. Create a mount target.
      For more information about how to create a mount target for a file system, see Add a mount target. You must create a mount target for the destination file system in the following scenarios:
      • You want to migrate data to an existing file system but the mount targets created for the file system do not reside in the same VPC as the source mount target.
      • After a new file system is created, no mount target is automatically created.
      Note When you create a mount target for the destination file system, we recommend that you select the VPC where the source mount target resides.

Migration

Purchase a new ECS instance and mount the two NFS file systems on the ECS instance. Then, use the rsync tool to copy the data for migration. The detailed procedure for data migration is described as follows.

  1. Mount the source and destination file systems on the ECS instance.
    Notice We recommend that you purchase a new ECS instance for migration. If you use an existing ECS instance to migrate data, the migration may occupy CPU and network bandwidth resources that have been allocated to current workloads.
    Log on to the ECS console and click Create Instance. On the Basic Configurations page that appears, configure the instance as follows:
    • Region: Select the region and zone where the source file system resides.
    • Instance Type: Select the minimum specification.
    • Image: Select CentOS 7.6.
    • Storage: Click Add NAS File System under Apsara File Storage NAS File System. The following figure shows the detailed configurations.
      Note
      • If the source and destination mount targets reside in the same VPC, you can configure the Apsara File Storage NAS file system mount information in the Storage section of the Basic Configurations page. After the ECS instance is started, the source and destination Apsara File Storage NAS file systems are automatically mounted on the instance.
      • If the source and destination mount targets do not reside in the same VPC, you need to configure the source file system on the Basic Configurations page. After you create an ECS instance, you can manually mount the destination file system on the ECS instance. For more information, see Enable a cross-VPC mount for a file system.
      storage_adding
    After you have created an ECS instance and mounted the source and destination Apsara File Storage NAS file systems on the ECS instance, run the following command to check whether the mount is successful.
    mount | grep nas.aliyuncs.com
    The following figure is displayed if the mount is successful. The source file system is mounted on the /mnt/volumeA directory and the destination file system is mounted on the /mnt/volumeB directory.Mount_succeeded
  2. Install tools for data migration
    Run the following command to install tools for data migration:
    sudo yum install -y rsync tmux
    Note
    • The rsync tool is used to copy data.
    • The tmux tool is used to check the installation progress of rsync. For more information, see Tmux User Guide.
  3. Migrate existing data.
    Run the following command to synchronize existing data stored on the source file system to the destination file system:
    tmux 
    sudo rsync -avP /mnt/volumeA/ /mnt/volumeB/
    Notice
    • The directory of the source file system in the rsync command must end with a forward slash (/). Otherwise, the destination directory to which data is synchronized cannot match the desired directory.
    • The tmux command creates a new tmux session. You can enter rsync in a tmux session to view the data migration progress. If the ECS instance is disconnected during migration, log on to the ECS instance again and run the tmux attach command to resume the tmux session and check the migration progress.
    • A test has proven that 320 minutes are required for migrating 1 million files with a total size of 100 GB (100 KB each) from the source file system to the destination.
  4. Migrate incremental data
    If applications running on other ECS instances writes data to the source file system during data migration, you need to migrate the incremental data to the destination file system after the existing data is migrated.
    1. Stop running applications that use the source file system
      To ensure no data is written to the source file system during incremental data migration, you must stop all applications on ECS instances and containers that use the source file system before synchronizing incremental data.
      Notice
      • After you stop all applications that use the source file system, do not manually delete data from the source file system. Otherwise, data may be lost during incremental data migration.
      • Do not perform this operation during peak hours. You can run the fuser -mv<dir> command to find the IDs of processes that read and write data to the source file system.
    2. Synchronize incremental data
      Run the following command to synchronize incremental data generated during the migration of existing data to the destination file system.
      sudo rsync -avP --delete /mnt/volumeA/ /mnt/volumeB/
      Notice
      • The --delete option in the command is used to delete data from the destination file system that is deleted from the source during incremental data migration. Use this option with caution in case that you accidentally delete other data from the destination file system.
      • The rsync command scans the source directory before synchronizing incremental data. Therefore, the amount of time needed to migrate a small amount of data may be longer than expected.
  5. Check the migration result.
    After the migration is complete, run the following command to check whether the data stored on the destination file system is consistent with the source file system.
    sudo rsync -rvn /mnt/volumeA/ /mnt/volumeB/
    If the data in the two file systems is consistent, no file path is returned, as shown in the following figure.Migration_result

Switch workloads to the new file system

If you need to switch your current workloads from the original file system to the new file system after data migration, unmount the original file system from all ECS instances and containers and mount the new file system on the instances and containers.

To mount the new file system on an ECS instance, perform the following steps:
  1. Run the mount | grep nas.aliyuncs.com command to record the mount information of the existing Apsara File Storage NAS file system. Note the directory <dir> on which the file system is mounted.
  2. Run the fuser -mv<dir> command to find the IDs of processes that read and write data to the file system and run the kill -9 command to stop these processes.
  3. Run the umount<dir> command to unmount the original file system.
  4. Mount the new file system on the original <dir> directory. For more information, see the parameters in Mount an NFS file system.
  5. Restart the processes that access the new file system to check whether they can perform normal read/write operations on the file system.
  6. Modify the automatic mount information in the /etc/fstab file by replacing the original mount target with the new mount target.
To mount the new file system on Kubernetes containers, perform the following steps:
  1. Modify the YAML configuration file of dynamic volumes or persistent volumes by replacing the original mount target with the new mount target.
  2. Use the modified configuration file to generate new pods. Make sure that the new file system is mounted on the pods and the pods can perform normal read/write operations on the file system.
  3. Recycle all the pods that use the original file system.
Notice After workloads are switched to the new file system, retain the data stored on the original file system for at least one week. Do not delete data from the original file system immediately. This prevents data loss caused by accidental deletion or accidental synchronization.