This topic describes how to upload local data to an NFS file system and download data from an NFS file system to a local server over the Internet.

Configure an ECS instance

In Solution 1: Migrate data by using an SFTP client and Solution 2: Migrate data by using the rsync command-line tool, you must configure an Elastic Compute Service (ECS) instance as an intermediate node to access Apsara File Storage NAS over the Internet. We recommend that you purchase a new ECS instance as an intermediate node to mount a NAS file system. You can also mount the NAS file system on an existing ECS instance.

Note Uploading data to or downloading data from a NAS file system consumes the Internet bandwidth of an ECS instance. We recommend that you purchase a new ECS instance and use it to upload and download data.
  • We recommend that you use a newly purchased ECS instance to mount the NAS file system as an intermediate node. For information about how to mount a NAS file system, see Mount NAS file systems when you purchase an ECS instance.
    Table 1. Basic configurations
    Parameter Description
    Image We recommend that you select CentOS 8.0 64-bit.
    Table 2. Networking
    Parameter Description
    Public IP address Select Assign Public IP Address.
    Bandwidth Billing Select Pay-By-Traffic.
    Peak Bandwidth The peak bandwidth. Maximum value: 100 Mbit/s. We recommend that you set this parameter to 100 Mbit/s.
  • You can also mount the NAS file system on an existing ECS instance and use the ECS instance as an intermediate node. For more information, see Mount an NFS file system.
Note You are not charged for the inbound traffic of an elastic IP address (EIP). However, you are charged for the outbound traffic of an EIP. Therefore, you are not charged for the traffic that is generated when you upload data to the NAS file system over the Internet. However, you are charged for the traffic that is generated when you download data from the NAS file system. For more information, see EIP billing.

Solution 1: Migrate data by using an SFTP client

If you need to upload and download a small number of files, we recommend that you install and use an SFTP client on your local server. This solution has the following benefits:
  • Supports multiple operating systems.
  • Provides a graphical user interface (GUI).
  1. Install an SFTP client.
    Download and install an SFTP client based on your local operating system. Multiple SFTP versions are available. In the following example, FileZilla is used.
  2. Establish a connection between the SFTP client and the ECS instance.
    1. Open and configure the FileZilla client. The following table describes the parameters. Click Quickconnect to establish a connection.
      After the local client and the ECS instance are connected, the local file system is displayed in the left pane. The file system that is mounted on the ECS instance is displayed in the right pane. U-D NFS0201
      Parameter Description
      Host The public IP address of the ECS instance, for example, 192.0.2.1.
      Username The username, for example, root.
      Password The password, for example, the logon password of the root user.
      Port The port number used by SFTP. Default value: 22.
    2. In the Remote site field in the right pane, change the path to the directory where the NAS file system is mounted, for example, /mnt. Press Enter to view the list of files in the NAS file system.
      UD-NFS0202
  3. Upload and download data
    • Drag and drop one or more files or directories from the left pane to the right pane to upload the data.
    • Drag and drop one or more files or directories from the right pane to the left pane to download the data.

Solution 2: Migrate data by using the rsync command-line tool

If you need to frequently upload and download a large number of files, we recommend that you install the rsync command-line tool on your local server. This solution has the following benefits:
  • After a file is uploaded or downloaded, the metadata of the file remains unchanged, including the owner and permission information.
  • You can perform incremental synchronization.
  • You can configure crontab for your local Linux or macOS operating system to automatically back up data to the NAS file system.
  1. Install the rsync tool.
    • Linux
      • If you are using a CentOS or Red Hat operating system, run the following command to install rsync by using the yum package manager:
        sudo yum install rsync
      • If you are using an Ubuntu or Debian operating system, run the following command to install rsync by using the apt package manager:
        sudo apt-get install rsync
    • macOS
      Download and install the Homebrew package manager, and then run the following command to install rsync:
      brew install rsync
    • Windows

      Download and install the Cygwin simulation environment. During the installation process, you can search for and install rsync. You can also download, compile, and install rsync.

  2. Upload data.
    Run the following command to upload a specified local directory to the NAS file system in incremental synchronization mode:
    rsync -avP DirToSync/ root@1.2.3.4:/mnt/DirToSync/
    Replace the parameters in the command with the actual values. The following table describes the parameters.
    Parameter Description
    DirToSync The name of the local directory that you want to upload.
    root The owner of the destination directory in the NAS file system.
    1.2.3.4 The public IP address of a Linux ECS instance on which an NFS file system is mounted.
    /mnt A path in the ECS instance. The path is used to mount the NAS file system.
    Note The source path in the rsync command must end with a forward slash (/). Otherwise, the source path does not match the destination path after the data is synchronized.
  3. Download data.
    Run the following command to download data from the NAS file system to a local directory:
    rsync -avP root@1.2.3.4:/mnt/DirToSync/ DirToSync/
  4. Configure automatic upload.
    You can configure crontab for your local Linux or macOS operating system to upload data at a regular interval by running rsync commands.
    • For Linux operating systems:
      1. Establish a password-free connection between the local server and the ECS instance. For more information, see Connect to a Linux instance by using an SSH key pair.
        Run the following command to check whether the connection is successful:
        ssh -i ~/.ssh/ecs.pem root@1.2.3.4
        Note ~/.ssh/ecs.pem is the path of the key file on your local server.
      2. Configure crontab.
        Run the crontab -e command to open the editor and configure a scheduled time to upload tasks. The following example shows the sample settings:
        0 23 * * * rsync -av -e "ssh -i ~/.ssh/ecs.pem" ~/Documents/ root@1.2.3.4:/mnt/Documents/
        The preceding crontab configuration allows Linux operating systems to automatically upload data from the local Documents directory to the NAS file system at 23:00:00 every day. You can modify the parameters in the crontab configuration based on your business requirements.
    • For macOS:
      1. Configure access permissions on the /usr/sbin/cron directory.

        Go to System Preferences. In the window that appears, choose Security & Privacy > Privacy > Full Disk Access, click Unlock, and then click +. In the window that appears, select Macintosh HD Directory, press cmd+ shift+.to show the hidden directories, and then select /usr/sbin/cron.

      2. Establish a password-free connection between the local server and the ECS instance. For more information, see Connect to a Linux instance by using an SSH key pair.
        Run the following command to check whether the connection is successful:
        ssh -i ~/.ssh/ecs.pem root@1.2.3.4
        Note ~/.ssh/ecs.pem is the path of the key file on your local server.
      3. Configure crontab.
        Run the crontab -e command to open the editor and configure a scheduled time to upload tasks. The following example shows the sample settings:
        0 23 * * * rsync -av -e "ssh -i ~/.ssh/ecs.pem" ~/Documents/ root@1.2.3.4:/mnt/Documents/
        The preceding crontab configuration allows Linux operating systems to automatically upload data from the local Documents directory to the NAS file system at 23:00:00 every day. You can modify the parameters in the crontab configuration based on your business requirements.
    Note If the resync tool cannot meet your requirements, you can use the fpsync tool to migrate your data in multiple threads. For more information, see Appendix: Use the fpsync command-line tool to migrate data in multiple threads.

Solution 3: Upload data to OSS, and then migrate the data to NAS

If you need to upload a large amount of data to the NAS file system, you can upload the data to OSS and then migrate the data from OSS to NAS. We recommend that you use this solution if the Internet bandwidth of the ECS instance cannot meet your requirements. For more information, see Implement migration.
Note
  • Migration of data from OSS to NAS is in public preview. If you need to migrate your business data, you can to apply for the whitelist access.
  • After you migrate your business data, the metadata may be modified. However, you can run rsync commands to restore the metadata.

Solution 4: Mount the NAS file system to an on-premises database

If your on-premises database frequently reads and writes a large amount of data to and from the NAS file system, you must create a physical connection and mount the NAS file system on the database. For more information, see Access an Apsara File Storage NAS file system from a local data center by using VPN Gateway and Access an Apsara File Storage NAS file system from a local IDC by using NAT Gateway.

Access uploaded data

After the data is uploaded, you can mount the NAS file system on the ECS instance or the container to which your business belongs. This allows you to share the data in the NAS file system.

For example, you can mount an NFS file system on a Linux ECS instance and access data in the NAS file system the same way you access your local data. For more information, see Mount an NFS file system.

You can also deploy business applications on the cloud. Then, you can read and write a large amount of data to and from the NAS file system by using programs on multiple compute nodes. For example, you can Use NGINX as a proxy for Apsara File Storage NAS.

Appendix: Use the fpsync command-line tool to migrate data in multiple threads

In the following example, a Linux operating system is used to describe how to use the fpsync command-line tool to migrate data.

  1. Download and install the fpsync tool.
    wget -N https://github.com/martymac/fpart/archive/fpart-1.1.0.tar.gz -P /tmp
    tar -C /tmp/ -xvf /tmp/fpart-1.1.0.tar.gz
    cd /tmp/fpart-fpart-1.1.0
    sudo yum install -y automake libtool
    autoreconf -i
    ./configure
    make
    sudo make install
    sudo yum install parallel -y
    printf "will cite" | parallel --bibtex
    sudo yum install -y rsync
  2. Copy the entire directory.
    fpsync -n 10 -f 10000 /data/src/ /data/dst/
    Note For more information about the fpsync tool, see fpsync.