Upload data to and download data from an NFS file system

Configure an ECS instance

In Solution 1: Migrate data by using an SFTP client and Solution 2: Migrate data by using the rsync command-line tool, you must configure an Elastic Compute Service (ECS) instance as an intermediate node to access Apsara File Storage NAS over the Internet. We recommend that you purchase a new ECS instance as an intermediate node to mount a NAS file system. You can also mount the NAS file system on an existing ECS instance.

• We recommend that you use a newly purchased ECS instance to mount the NAS file system as an intermediate node. For information about how to mount a NAS file system, see Mount NAS file systems when you purchase an ECS instance.

  Table 1. Basic configurations

  Parameter	Description
  Image	We recommend that you select CentOS 8.0 64-bit.

  Table 2. Networking

  Parameter	Description
  Public IP address	Select Assign Public IP Address.
  Bandwidth Billing	Select Pay-By-Traffic.
  Peak Bandwidth	The peak bandwidth. Maximum value: 100 Mbit/s. We recommend that you set this parameter to 100 Mbit/s.

• You can also mount the NAS file system on an existing ECS instance and use the ECS instance as an intermediate node. For more information, see Mount an NFS file system.

Solution 1: Migrate data by using an SFTP client

1. Install an SFTP client.
   Download and install an SFTP client based on your local operating system. Multiple SFTP versions are available. In the following example, FileZilla is used.
2. Establish a connection between the SFTP client and the ECS instance.
   1. Open and configure the FileZilla client. The following table describes the parameters. Click Quickconnect to establish a connection.
      After the local client and the ECS instance are connected, the local file system is displayed in the left pane. The file system that is mounted on the ECS instance is displayed in the right pane.

      Parameter	Description
      Host	The public IP address of the ECS instance, for example, 192.0.2.1.
      Username	The username, for example, root.
      Password	The password, for example, the logon password of the root user.
      Port	The port number used by SFTP. Default value: 22.

   2. In the Remote site field in the right pane, change the path to the directory where the NAS file system is mounted, for example, /mnt. Press Enter to view the list of files in the NAS file system.
      
3. Upload and download data
   • Drag and drop one or more files or directories from the left pane to the right pane to upload the data.
   • Drag and drop one or more files or directories from the right pane to the left pane to download the data.

Solution 2: Migrate data by using the rsync command-line tool

1. Install the rsync tool.
   • Linux
     • If you are using a CentOS or Red Hat operating system, run the following command to install rsync by using the yum package manager:

       sudo yum install rsync

     • If you are using an Ubuntu or Debian operating system, run the following command to install rsync by using the apt package manager:

       sudo apt-get install rsync

   • macOS
     Download and install the Homebrew package manager, and then run the following command to install rsync:

     brew install rsync

   • Windows

     Download and install the Cygwin simulation environment. During the installation process, you can search for and install rsync. You can also download, compile, and install rsync.

2. Upload data.
   Run the following command to upload a specified local directory to the NAS file system in incremental synchronization mode:

   rsync -avP DirToSync/ root@1.2.3.4:/mnt/DirToSync/

   Replace the parameters in the command with the actual values. The following table describes the parameters.

   Parameter	Description
   DirToSync	The name of the local directory that you want to upload.
   root	The owner of the destination directory in the NAS file system.
   1.2.3.4	The public IP address of a Linux ECS instance on which an NFS file system is mounted.
   /mnt	A path in the ECS instance. The path is used to mount the NAS file system.

   Note The source path in the rsync command must end with a forward slash (/). Otherwise, the source path does not match the destination path after the data is synchronized.
3. Download data.
   Run the following command to download data from the NAS file system to a local directory:

   rsync -avP root@1.2.3.4:/mnt/DirToSync/ DirToSync/

4. Configure automatic upload.
   You can configure crontab for your local Linux or macOS operating system to upload data at a regular interval by running rsync commands.

   • For Linux operating systems:
     1. Establish a password-free connection between the local server and the ECS instance. For more information, see Connect to a Linux instance by using an SSH key pair.
        Run the following command to check whether the connection is successful:

        ssh -i ~/.ssh/ecs.pem root@1.2.3.4

        Note ~/.ssh/ecs.pem is the path of the key file on your local server.
     2. Configure crontab.
        Run the crontab -e command to open the editor and configure a scheduled time to upload tasks. The following example shows the sample settings:

        0 23 * * * rsync -av -e "ssh -i ~/.ssh/ecs.pem" ~/Documents/ root@1.2.3.4:/mnt/Documents/

        The preceding crontab configuration allows Linux operating systems to automatically upload data from the local Documents directory to the NAS file system at 23:00:00 every day. You can modify the parameters in the crontab configuration based on your business requirements.
   • For macOS:
     1. Configure access permissions on the /usr/sbin/cron directory.

        Go to System Preferences. In the window that appears, choose Security & Privacy > Privacy > Full Disk Access, click Unlock, and then click +. In the window that appears, select Macintosh HD Directory, press cmd+ shift+.to show the hidden directories, and then select /usr/sbin/cron.

     2. Establish a password-free connection between the local server and the ECS instance. For more information, see Connect to a Linux instance by using an SSH key pair.
        Run the following command to check whether the connection is successful:

        ssh -i ~/.ssh/ecs.pem root@1.2.3.4

        Note ~/.ssh/ecs.pem is the path of the key file on your local server.
     3. Configure crontab.
        Run the crontab -e command to open the editor and configure a scheduled time to upload tasks. The following example shows the sample settings:

        0 23 * * * rsync -av -e "ssh -i ~/.ssh/ecs.pem" ~/Documents/ root@1.2.3.4:/mnt/Documents/

        The preceding crontab configuration allows Linux operating systems to automatically upload data from the local Documents directory to the NAS file system at 23:00:00 every day. You can modify the parameters in the crontab configuration based on your business requirements.

   Note If the resync tool cannot meet your requirements, you can use the fpsync tool to migrate your data in multiple threads. For more information, see Appendix: Use the fpsync command-line tool to migrate data in multiple threads.

Solution 3: Upload data to OSS, and then migrate the data to NAS

Solution 4: Mount the NAS file system to an on-premises database

If your on-premises database frequently reads and writes a large amount of data to and from the NAS file system, you must create a physical connection and mount the NAS file system on the database. For more information, see Access an Apsara File Storage NAS file system from a local data center by using VPN Gateway and Access an Apsara File Storage NAS file system from a local IDC by using NAT Gateway.

Access uploaded data

After the data is uploaded, you can mount the NAS file system on the ECS instance or the container to which your business belongs. This allows you to share the data in the NAS file system.

For example, you can mount an NFS file system on a Linux ECS instance and access data in the NAS file system the same way you access your local data. For more information, see Mount an NFS file system.

You can also deploy business applications on the cloud. Then, you can read and write a large amount of data to and from the NAS file system by using programs on multiple compute nodes. For example, you can Use NGINX as a proxy for Apsara File Storage NAS.

Appendix: Use the fpsync command-line tool to migrate data in multiple threads

In the following example, a Linux operating system is used to describe how to use the fpsync command-line tool to migrate data.

1. Download and install the fpsync tool.

   wget -N https://github.com/martymac/fpart/archive/fpart-1.1.0.tar.gz -P /tmp
   tar -C /tmp/ -xvf /tmp/fpart-1.1.0.tar.gz
   cd /tmp/fpart-fpart-1.1.0
   sudo yum install -y automake libtool
   autoreconf -i
   ./configure
   make
   sudo make install
   sudo yum install parallel -y
   printf "will cite" | parallel --bibtex
   sudo yum install -y rsync

2. Copy the entire directory.

   fpsync -n 10 -f 10000 /data/src/ /data/dst/

   Note For more information about the fpsync tool, see fpsync.