This topic describes the file transfer principle and typical transfer methods on Unix-like, Linux, and Windows platforms in Alibaba Cloud ECS. Additionally, this topic compares these methods to help you choose appropriate file transfer methods that meet your specific requirements.
File transfer principle
File transfer, also known as file data communication, is a form of information transfer that transmits file data between data sources and data sinks. In a file transfer process, the OS extracts file data to the memory for temporary storage, and then duplicates the data to the destination. Encryption adds a secure layer to a file, while duplication transfers the encrypted file as a whole to another location. Decryption is needed only when a compressed package is opened. A large file cannot be transferred as a whole between hosts immediately because the transfer is a continuous process. If any interruption occurs during the transfer, the file will not exist in the destination path. If multiple files are transferred, they are transferred separately and sequentially. If any interruption occurs during the transfer, the files that are being transferred or have not yet been transferred will fail, but the transferred files are transferred successfully. A compressed package is considered as one file regardless of how many files the package contains.
Multiple file transfer tools, such as Netcat, FTP, SCP, and NFS, can be used to transfer files. The following sections describe the features and usage of some typical file transfer tools.
Netcat is a powerful and versatile networking tool with optimal file transfer capabilities.
|-g <gateway>||Specifies up to eight long-distance communication gateways for the router.|
|-G <number of indicators>||Specifies the number of source routing indicators. The value is a multiple of 4.|
|-i <delay in seconds>||Specifies the time interval for sending messages and scanning the communications port.|
|-l||Enables the listening mode to control received data.|
|-o <output file>||Specifies the name of the file where the transferred data is dumped and saved in hexadecimal character codes.|
|-P <communication port>||Specifies the communication port used by the local host.|
|-r||Specifies the communication port between the local host and the remote host.|
|-u||Enables the UDP transfer protocol.|
|-v||Shows the command running process.|
|-w <timeout in seconds>||Specifies the waiting time for a connection.|
|-z||Enables the zero input/output mode, which is used only for scanning the communications port.|
|-n||Uses IP addresses instead of the DNS.|
Examples of usage
nc -v -w 2 192.168.2.34 -z 21-24
nc: connect to 192.168.2.34 port 21 (tcp) failed: Connection refused Connection to 192.168.2.34 22 port [tcp/ssh] succeeded! nc: connect to 192.168.2.34 port 23 (tcp) failed: Connection refused nc: connect to 192.168.2.34 port 24 (tcp) failed: Connection refused
Run the following command at 192.168.2.34:
nc-l 1234 > test.txt.
Run the following command at 192.168.2.33:
nc192.168.2.34 < test.txt.
3. Run the following nc commands to operate Memcached as needed:
To store data, run the command
printf “set key 0 10 6rnresultrn” |nc 192.168.2.34 11211.
To obtain data, run the command
printf “get keyrn” |nc 192.168.2.34 11211.
To delete data, run the command
printf “delete keyrn” |nc 192.168.2.34 11211.
To view the status, run the command
printf “statsrn” |nc 192.168.2.34 11211.
To simulate the top command to view the status, run the command
watch “echo stats” |nc 192.168.2.34 11211.
To clear the cache, run the following command:
printf “flush_allrn” |nc 192.168.2.34 11211 # This operation cannot be undone.
The use of Secure Copy (SCP) commands is similar to that of RCP commands. The difference is that SCP commands provide higher security protection by prompting users to enter a password for verification. Therefore, we recommend that you use SCP commands instead of RCP commands. SCP commands use SSH to transfer data and use the same authentication model as SSH to provide the same security protection. SSH is a reliable protocol that provides security for remote logon sessions and other network services. With SSH, you can effectively prevent information leakage during remote management. SCP is an SSH-based application. Therefore, it requires that the machines involved in data transfer support SSH.Features
Similar to RCP, SCP can retain the file attributes on a specific file system and retain the copied subdirectories that need recursion.
SCP provides better file transfer confidentiality. Overall, SCP is suitable for users with high data security requirements.Examples of usage
If you do not want to enter your username and password every time you use SCP commands to copy files between two machines, you can configure SSH.
To generate an RSA key, run the following command:
When you are prompted to enter the path and password to save the key, you can press Enter to use the default path and a null password. Then, the generated public key is saved in /.ssh/id_rsa.pub, and the private key is saved in /.ssh/id_rsa. You can copy the content of the public key from this key pair to the /.ssh/authorized_keys file in the machine that you want to access. In this way, you do not need to enter your password when you next access this machine.Copy a file between two Linux hosts
scp [optional parameter] file_source file_target
scp local_file remote_username@remote_ip:remote_folder scp local_file remote_username@remote_ip:remote_file scp local_file remote_ip:remote_folder scp local_file remote_ip:remote_file
In the third and fourth commands, user names are not specified and the password must be entered after the commands are executed. In the third command, a remote directory is specified and the file name remains the same. In the fourth command, a file name is specified.
To copy a file from a remote directory to a local directory, run the following commands:
scp firstname.lastname@example.org:/home/root/others/music /home/space/music/i.mp3 scp -r www.cumt.edu.cn:/home/root/others/ /home/space/music/
Rsync is a file synchronization and transfer tool for Linux or Unix. As an alternative to RCP, Rsync may be used through RSH or SSH, or run in daemon mode. In daemon mode, the Rsync server opens port 873 for client connections. During client connections, the Rsync server will verify the password. If the password is correct, the file can be transferred. During the first connection, the entire file is transferred. During the subsequent connections, only incremental data of the file is synchronized.Rsync Installation methods
Install Rsync through source code compilation:
sudo apt-get install rsync # Install Rsync online in Debian and Ubuntu. slackpkg install rsync # Install Rsync online using Slackware. yum install rsync # Install Rsync in Fedora and Red Hat.
wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz tar xf rsync-3.0.9.tar.gz cd rsync-3.0.9 ./configure && make && make install
|-v||Specifies the output mode.|
|-a||Specifies the archive mode. It meas that files are transferred recursively and all file attributes are retained. This parameter is equivalent to the combined parameter -rlptgoD.|
|-r||Transfers subdirectories recursively.|
|-l||Retains soft links.|
|-p||Retains file permissions.|
|-t||Retains file time information.|
|-g||Retains file group information.|
|-o||Retains file owner information.|
|-D||Retains device file information.|
|-H||Retains hard links.|
|-S||Processes sparse files explicitly to save space for DST files.|
|-z||Compresses backup files during transfer.|
To copy local files from the /home/coremail directory to the /cmbak directory, run the following command:
rsync -avSH /home/coremail/ /cmbak/
To copy files from a local machine to a remote machine, run the following command:
rsync -av /home/coremail/ 192.168.11.12:/home/coremail/
To copy files from a remote machine to a local machine, run the following command:
rsync -av 192.168.11.11:/home/coremail/ /home/coremail/
To copy files from a remote Rsync server (running in daemon mode) to a local machine, run the following command:
rsync -av email@example.com::www /databack
To copy files from a local machine to a remote Rsync server (running in daemon mode), run the following command. This work mode is started when the DST path information contains the “::” delimiter.
rsync -av /databack firstname.lastname@example.org::www
To show the file list of a remote machine, run the following command:
rsync -v rsync://192.168.11.11/data
cat/etc/rsyncd.conf # The contents are as follows: port = 873 # Specify the port number. uid = nobody # Specify the UID of the daemon process when the module transfers files. gid = nobody # Specify the GID of the daemon process when the module transfers files. use chroot = no # Use chroot to enter the directories in the file system. max connections = 10 # Specify the maximum concurrent connections. strict modes = yes # Specify whether to check the permissions of password-protected files. pid file = /usr/local/rsyncd/rsyncd.pid # Specify PID files. lock file = /usr/local/rsyncd/rsyncd.lock # Specify the lock file that supports the maximum concurrent connections. By default, the lock file is /var/run/rsyncd.lock. motd file = /usr/local/rsyncd/rsyncd.motd #Define server information and write the rsyncd.motd file. log file = /usr/local/rsyncd/rsync.log # Specify the log of the Rsync server. log format = %t %a %m %f %b syslog facility = local3 timeout = 300 [conf] # custom module path = /usr/local/nginx/conf # Specify the directory to be backed up. comment = Nginx conf ignore errors # Ignore some IO errors. read only = no # To allow the client to upload files, set the value to no. Otherwise, set the value to yes. write only = no # To allow the client to download files, set the value to no. Otherwise, set the value to yes. hosts allow = 192.168.2.0/24 # Specify an allowed IP address. hosts deny = * # Specify a denied IP address. list = false # Use the module list upon request. uid = root gid = root auth users = backup # Specify a connection user name, which is irrelevant to Linux user names. secrets file = /etc/rsyncd.pass # Specify the password file.