This topic describes how to migrate data between Apsara File Storage NAS Network File System (NFS) file systems.

Prerequisites

A source NFS file system is created and contains data. A mount target of the virtual private cloud (VPC) type is created for the NFS file system.

Prepare for data migration

  1. View information about the source mount target.
    Before you migrate data, you must view the information about the mount target of the source file system and the VPC in which the mount target resides. For more information, see View mount targets
    Note If the mount target of your NFS file system is created in the classic network, you must create a mount target in a VPC for the file system. For more information, see Create a mount target.
  2. Specify the destination mount target.
    Note If the destination file system and the source file system reside in the same region, make sure that the destination mount target and the source mount target reside in the same VPC. This ensures a successful migration task.
    You can use one of the following three methods to specify a mount target for the destination NFS file system:
    • Create a file system in the destination region and zone. A mount target is automatically created for the file system. For more information, see Create a General-purpose NAS file system in the NAS console.
      Note When you purchase a pay-as-you-go NFS Capacity or a pay-as-you-go NFS Performance file system, select the same VPC and vSwitch that are used by the source mount target. A mount target is automatically created for the file system. After you create a file system, you can buy a storage plan and bind the storage plan to the file system to reduce costs.
    • Find a mount target that is created for the destination file system. For more information, see View the endpoint of a mount target.
    • Create a mount target for the destination file system. For more information, see Create a mount target.
      You must create a mount target for the destination file system in the following scenarios:
      • You want to migrate data to an existing file system but the mount targets created for the file system do not reside in the same VPC as the source mount target.
      • After you create a file system, no mount target is automatically generated.

Migrate data

Purchase a new Elastic Compute Service (ECS) instance and mount the two NFS file systems on the ECS instance. Then, use the rsync tool to copy the data that you want to migrate. Procedure

  1. Mount the source file system and destination file systems on the ECS instance.
    Notice If you want to migrate data, we recommend that you purchase a new ECS instance. If you use an existing ECS instance to migrate data, the migration task may consume CPU and network bandwidth resources that are allocated to the workloads that are running.
    Log on to the ECS console and click Create Instance. On the Basic Configurations page, configure the following parameters:
    • Region: Select the region and zone in which the source file system resides.
    • Instance Type: Select the minimum specification.
    • Image: Select CentOS 7.6.
    • Storage: Click Add NAS File System under Apsara File Storage NAS File System. The following figure shows the detailed configurations.
      Note
      • If the source mount target and destination mount targets reside in the same VPC, you can configure the mount information of NAS file systems in the Storage section of the Basic Configurations page. When you start the ECS instance, the source and destination NAS file systems are automatically mounted on the instance.
      • If the source mount target and destination mount targets do not reside in the same VPC, you must configure the source file system on the Basic Configurations page. After you create an ECS instance, you can manually mount the destination file system on the ECS instance. For more information, see Mount a file system across VPCs or regions.
      Add a NAS file system
    After you create an ECS instance and mount the source NAS file system and destination NAS file system on the ECS instance, run the following command to check whether the mount is successful:
    mount | grep nas.aliyuncs.com
    If the mount is successful, the output that is similar to the following information appears. The source file system is mounted on the /mnt/volumeA directory, and the destination file system is mounted on the /mnt/volumeB directory. Mount succeeded
  2. Install the tools that are used to migrate data.
    Run the following command to install the tools that are used to migrate data:
    sudo yum install -y rsync tmux
    Note
    • The rsync tool is used to copy data.
    • The tmux tool is used to check the installation progress of rsync. For more information, see tmux User Guide..
  3. Migrate existing data.
    Run the following command to synchronize existing data that is stored in the source file system to the destination file system:
    tmux
    sudo rsync -avP /mnt/volumeA/ /mnt/volumeB/
    Notice
    • The source path in the rsync command must end with a forward slash (/). Otherwise, the source path does not match the destination path after the data is synchronized.
    • The tmux command creates a new tmux session. You can enter rsync in a tmux session to view the data migration progress. If the ECS instance is disconnected during migration, log on to the ECS instance and run the tmux attach command to resume the tmux session and check the migration progress.
    • The result of a test indicates that 320 minutes are required to migrate 1 million files with a total size of 100 GB (100 KB each) from the source file system to the destination file system.
  4. Migrate incremental data.
    If applications that run on other ECS instances write data to the source file system during data migration, you must migrate the incremental data to the destination file system after the existing data is migrated.
    1. Stop running applications that use the source file system.
      To ensure that no data is written to the source file system during incremental data migration, you must stop all running applications on ECS instances and containers that use the source file system before you synchronize incremental data.
      Notice
      • After you stop all applications that use the source file system, do not delete data from the source file system. Otherwise, data may be lost during incremental data migration.
      • We recommend that you migrate incremental data during off-peak hours. You can run the fuser -mv<dir> command to find the IDs of processes that read and write data to the source file system.
    2. Synchronize incremental data.
      Run the following command to synchronize the incremental data that is generated when you migrate existing data to the destination file system:
      sudo rsync -avP --delete /mnt/volumeA/ /mnt/volumeB/
      Notice
      • The --delete option in the command is used to delete data from the destination file system that is deleted from the source during incremental data migration. To avoid the accidental deletion of data from the destination file system, proceed with caution when you use this option.
      • The rsync command scans the source directory before the command synchronizes incremental data. Therefore, the time required to migrate a small amount of data may be longer than expected.
  5. Check the migration result.
    After the migration is complete, run the following command to check whether the data that is stored in the destination file system is consistent with the data that is stored in the source file system.
    sudo rsync -rvn /mnt/volumeA/ /mnt/volumeB/
    If the data in the two file systems is consistent, no file path is returned, as shown in the following figure. NFS file system migration result

Switch workloads to the new file system

If you want to switch your current workloads from the original file system to the new file system after you migrate data, unmount the original file system from all ECS instances and containers and mount the new file system on the instances and containers.

  • To mount the new file system on an ECS instance, perform the following steps:
    1. Run the mount | grep nas.aliyuncs.com command to record the mount information of the existing NAS file system. Take note of the directory <dir> on which the file system is mounted.
    2. Run the fuser -mv<dir> command to find the IDs of processes that read and write data to the file system. Then, run the kill -9 command to stop the processes.
    3. Run the umount<dir> command to unmount the original file system.
    4. Mount the new file system on the original <dir> directory. For more information about the mount parameters, see Manually mount an NFS file system.
    5. Restart the processes that access the new file system to check whether the processes can perform read and write operations on the file system as expected.
    6. Modify the automatic mount information in the /etc/fstab file by replacing the original mount target with the new mount target.
  • To mount the new file system on Kubernetes containers, perform the following steps:
    1. Modify the YAML configuration file for dynamic volumes or persistent volumes by replacing the original mount target with the new mount target.
    2. Use the modified configuration file to generate new pods. Make sure that the new file system is mounted on the pods and the pods can perform read and write operations on the file system as expected.
    3. Recycle all of the pods that use the original file system.
Notice After you switch workloads to the new file system, retain the data that is stored in the original file system for at least one week. If you immediately delete data from the original file system, data loss may occur due to accidental deletion or accidental synchronization.