This topic describes the file transfer principle and typical transfer methods on Unix-like, Linux, and Windows platforms in Alibaba Cloud ECS. Additionally, this topic compares these methods to help you choose appropriate file transfer methods that meet your specific requirements.

File transfer principle

File transfer, also known as file data communication, is a form of information transfer that transmits file data between data sources and data sinks. In a file transfer process, the OS extracts file data to the memory for temporary storage, and then duplicates the data to the destination. Encryption adds a secure layer to a file, while duplication transfers the encrypted file as a whole to another location. Decryption is needed only when a compressed package is opened. A large file cannot be transferred as a whole between hosts immediately because the transfer is a continuous process. If any interruption occurs during the transfer, the file will not exist in the destination path. If multiple files are transferred, they are transferred separately and sequentially. If any interruption occurs during the transfer, the files that are being transferred or have not yet been transferred will fail, but the transferred files are transferred successfully. A compressed package is considered as one file regardless of how many files the package contains.

Multiple file transfer tools, such as Netcat, FTP, SCP, and NFS, can be used to transfer files. The following sections describe the features and usage of some typical file transfer tools.

Netcat

Netcat is a powerful and versatile networking tool with optimal file transfer capabilities.

Parameter descriptions

Parameter Description
-g <gateway> Specifies up to eight long-distance communication gateways for the router.
-G <number of indicators> Specifies the number of source routing indicators. The value is a multiple of 4.
-i <delay in seconds> Specifies the time interval for sending messages and scanning the communications port.
-l Enables the listening mode to control received data.
-o <output file> Specifies the name of the file where the transferred data is dumped and saved in hexadecimal character codes.
-P <communication port> Specifies the communication port used by the local host.
-r Specifies the communication port between the local host and the remote host.
-u Enables the UDP transfer protocol.
-v Shows the command running process.
-w <timeout in seconds> Specifies the waiting time for a connection.
-z Enables the zero input/output mode, which is used only for scanning the communications port.
-n Uses IP addresses instead of the DNS.

Examples of usage

1. Scan ports 21-24 (for example, IP address 192.168.2.34).
nc -v -w 2 192.168.2.34 -z 21-24

Response example:

nc: connect to 192.168.2.34 port 21 (tcp) failed: Connection refused
Connection to 192.168.2.34 22 port [tcp/ssh] succeeded!
nc: connect to 192.168.2.34 port 23 (tcp) failed: Connection refused
nc: connect to 192.168.2.34 port 24 (tcp) failed: Connection refused
2. Copy files from 192.168.2.33 to 192.168.2.34.
  • Run the following command at 192.168.2.34: nc-l 1234 > test.txt.

  • Run the following command at 192.168.2.33: nc192.168.2.34 < test.txt.

3. Run the following nc commands to operate Memcached as needed:

  • To store data, run the command printf “set key 0 10 6rnresultrn” |nc 192.168.2.34 11211.

  • To obtain data, run the command printf “get keyrn” |nc 192.168.2.34 11211.

  • To delete data, run the command printf “delete keyrn” |nc 192.168.2.34 11211.

  • To view the status, run the command printf “statsrn” |nc 192.168.2.34 11211.

  • To simulate the top command to view the status, run the command watch “echo stats” |nc 192.168.2.34 11211.

  • To clear the cache, run the following command:
    printf “flush_allrn” |nc 192.168.2.34 11211       # This operation cannot be undone.

SCP

The use of Secure Copy (SCP) commands is similar to that of RCP commands. The difference is that SCP commands provide higher security protection by prompting users to enter a password for verification. Therefore, we recommend that you use SCP commands instead of RCP commands. SCP commands use SSH to transfer data and use the same authentication model as SSH to provide the same security protection. SSH is a reliable protocol that provides security for remote logon sessions and other network services. With SSH, you can effectively prevent information leakage during remote management. SCP is an SSH-based application. Therefore, it requires that the machines involved in data transfer support SSH.

Features

Similar to RCP, SCP can retain the file attributes on a specific file system and retain the copied subdirectories that need recursion.

SCP provides better file transfer confidentiality. Overall, SCP is suitable for users with high data security requirements.

Examples of usage

If you do not want to enter your username and password every time you use SCP commands to copy files between two machines, you can configure SSH.

To generate an RSA key, run the following command:


Response example

When you are prompted to enter the path and password to save the key, you can press Enter to use the default path and a null password. Then, the generated public key is saved in /.ssh/id_rsa.pub, and the private key is saved in /.ssh/id_rsa. You can copy the content of the public key from this key pair to the /.ssh/authorized_keys file in the machine that you want to access. In this way, you do not need to enter your password when you next access this machine.

Copy a file between two Linux hosts
Basic command format:
scp [optional parameter] file_source file_target
To copy a file from a local directory to a remote directory, run one of the following four commands:
scp local_file remote_username@remote_ip:remote_folder
scp local_file remote_username@remote_ip:remote_file
scp local_file remote_ip:remote_folder
scp local_file remote_ip:remote_file
Note In the first and second commands, user names are specified and the password must be entered after the commands are executed. In the first command, a remote directory is specified and the file name remains the same. In the second command, a file name is specified.

In the third and fourth commands, user names are not specified and the password must be entered after the commands are executed. In the third command, a remote directory is specified and the file name remains the same. In the fourth command, a file name is specified.

To copy a file from a remote directory to a local directory, run the following commands:

scp root@www.cumt.edu.cn:/home/root/others/music /home/space/music/i.mp3
scp -r www.cumt.edu.cn:/home/root/others/ /home/space/music/

Rsync

Rsync is a file synchronization and transfer tool for Linux or Unix. As an alternative to RCP, Rsync may be used through RSH or SSH, or run in daemon mode. In daemon mode, the Rsync server opens port 873 for client connections. During client connections, the Rsync server will verify the password. If the password is correct, the file can be transferred. During the first connection, the entire file is transferred. During the subsequent connections, only incremental data of the file is synchronized.

Rsync Installation methods
Note You can use the package manager of your OS to install Rsync.
sudo apt-get  install  rsync      # Install Rsync online in Debian and Ubuntu.
slackpkg  install  rsync          # Install Rsync online using Slackware.
yum install rsync                 # Install Rsync in Fedora and Red Hat.
Install Rsync through source code compilation:
wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
tar xf rsync-3.0.9.tar.gz
cd rsync-3.0.9
./configure && make && make install
Parameter descriptions
Parameter Description
-v Specifies the output mode.
-a Specifies the archive mode. It meas that files are transferred recursively and all file attributes are retained. This parameter is equivalent to the combined parameter -rlptgoD.
-r Transfers subdirectories recursively.
-l Retains soft links.
-p Retains file permissions.
-t Retains file time information.
-g Retains file group information.
-o Retains file owner information.
-D Retains device file information.
-H Retains hard links.
-S Processes sparse files explicitly to save space for DST files.
-z Compresses backup files during transfer.
Six work modes of Rsync
  • To copy local files from the /home/coremail directory to the /cmbak directory, run the following command:

    rsync -avSH /home/coremail/ /cmbak/
  • To copy files from a local machine to a remote machine, run the following command:

    rsync -av /home/coremail/ 192.168.11.12:/home/coremail/
  • To copy files from a remote machine to a local machine, run the following command:

    rsync -av 192.168.11.11:/home/coremail/ /home/coremail/
  • To copy files from a remote Rsync server (running in daemon mode) to a local machine, run the following command:

    rsync -av root@172.16.78.192::www /databack
  • To copy files from a local machine to a remote Rsync server (running in daemon mode), run the following command. This work mode is started when the DST path information contains the “::” delimiter.

    rsync -av /databack root@172.16.78.192::www
  • To show the file list of a remote machine, run the following command:

    rsync -v rsync://192.168.11.11/data
Description of the Rsync configuration file
cat/etc/rsyncd.conf              # The contents are as follows:
port = 873                       # Specify the port number.
uid = nobody                     # Specify the UID of the daemon process when the module transfers files.
gid = nobody                     # Specify the GID of  the daemon process when the module transfers files.
use chroot = no                  # Use chroot to enter the directories in the file system.
max connections = 10             # Specify the maximum concurrent connections.
strict modes = yes               # Specify whether to check the permissions of password-protected files.
pid file = /usr/local/rsyncd/rsyncd.pid       # Specify PID files.
lock file = /usr/local/rsyncd/rsyncd.lock     # Specify the lock file that supports the maximum concurrent connections. By default, the lock file is /var/run/rsyncd.lock.
motd file = /usr/local/rsyncd/rsyncd.motd     #Define server information and write the rsyncd.motd file.
log file = /usr/local/rsyncd/rsync.log        # Specify the log of the Rsync server.
log format = %t %a %m %f %b
syslog facility = local3
timeout = 300
[conf]                                   # custom module
path = /usr/local/nginx/conf             # Specify the directory to be backed up.
comment = Nginx conf
ignore errors                            # Ignore some IO errors.
read only = no                           # To allow the client to upload files, set the value to no. Otherwise, set the value to yes.
write only = no                          # To allow the client to download files, set the value to no. Otherwise, set the value to yes.
hosts allow = 192.168.2.0/24             # Specify an allowed IP address.
hosts deny = *                           # Specify a denied IP address.
list = false                             # Use the module list upon request.
uid = root
gid = root
auth users = backup                      # Specify a connection user name, which is irrelevant to Linux user names.
secrets file = /etc/rsyncd.pass          # Specify the password file.