Mount a CPFS file system on Alibaba Cloud Linux - Cloud Parallel File Storage

This topic describes how to install the CPFS-POSIX client on an Alibaba Cloud Linux 2 operating system and mount a Cloud Parallel File Storage (CPFS) file system.

Background information

The target Elastic Compute Service (ECS) instances in this topic are the compute nodes on which a CPFS file system is mounted.
CPFS file systems are compatible with POSIX-based APIs. You can install the CPFS-POSIX client to mount and use a CPFS file system. The CPFS-POSIX client is a custom service that you can install on the Alibaba Cloud Linux 2 operating system. If you want to access a CPFS file system using the NFS protocol, see Use the CPFS-NFS client to mount a file system.
After you add a POSIX mount target, CPFS automatically creates three control plane node ECS instances. The instances are named <FSID>-<GENID>-qr-001, <FSID>-<GENID>-qr-002, and <FSID>-<GENID>-qr-003. The first control plane node ECS instance in the list, <FSID>-<GENID>-qr-001, is the default installation node for the CPFS-POSIX client.

Prerequisites

A file system is created. For more information, see Create a file system.
A POSIX mount target is added. For more information, see Add a POSIX mount target.
You have created one or more ECS instances that run Alibaba Cloud Linux 2. The instances must meet the following requirements:
- Version: Alibaba Cloud Linux 2.1903 LTS 64-bit
- Kernel version: 4.19.91-27.4.al7.x86_64 or earlier.
  Run the uname -r command to check the kernel version.

Step 1: Configure the security group of the target ECS instance

Before you mount the file system on the prerequisite ECS instances, add the security group of the CPFS-POSIX control plane node ECS instances to the target ECS instances. This prevents remote connection and data transmission failures between the target ECS instances and the CPFS cluster.

Obtain the security group information of the CPFS-POSIX control plane node ECS instance.
1. Log on to the NAS console.
2. In the navigation pane on the left, choose File System > File System List.
3. In the upper-left corner of the page, select the resource group and region where the destination file system resides.
4. Click the destination file system to go to its details page, and then click Mount Targets.
5. In the Client Management node section, click the ID of the first ECS instance in the list to open its details page.
6. On the Instance Details page, click the Security Groups tab and record the Security Group ID/Name of the CPFS-POSIX control plane node.
  The name of the control plane node security group is in the format of <FSID>-<GENID>-qr-sg.
  <FSID> is the ID of the CPFS file system. <GENID> is the serial number of the mount target, which increments each time a CPFS mount target is created.
Add the security group of the CPFS-POSIX control plane node ECS instance to the destination ECS instance.
1. In the navigation pane on the left, choose Instances & Images > Instances.
2. On the Instances page, click the ID of the ECS instance on which you want to mount the CPFS file system. Click the Security Groups tab, and then click Change Security Group.
3. In the Change Security Group dialog box, select the security group of the control plane node (<FSID>-<GENID>-qr-sg) that you recorded from the Security Group drop-down list, and then click Confirm.

Step 2: Install the CPFS-POSIX client

Log on to the NAS console.
In the navigation pane on the left, choose File System > File System List.
In the upper-left corner of the page, select the resource group and region of the target file system.
Click the target file system. On the file system details page, click Mount.
In the Client Management node section, click the first control plane node ECS instance in the list to go to its details page.
Click Connect to log on to the CPFS-POSIX client installation node.
Enable password-free logon between the control plane nodes and the target ECS instances.
Note
Before you configure password-free logon between the control plane nodes and multiple ECS instances, set the same logon password for the target ECS instances. Otherwise, you must create the client.list file and repeat Steps 10 to 12 multiple times.
1. Create a client.list file and add the internal IP addresses of the target ECS instances.
  1. Create the client.list file.
```
touch client.list
```
  2. Run the vi client.list command to open the client.list file. Then, add the internal IP addresses of the target ECS instances.
```
<Internal IP address of target ECS 1> 
<Internal IP address of target ECS 2>
<Internal IP address of target ECS 3>
```
    Example:
```
192.168.1.249
192.168.1.250
192.168.1.251
```
    Note
    You can add one or more ECS instances. If you add multiple ECS instances, enter the internal IP address of each target ECS instance on a new line.
  3. Save the file and exit.
2. Create a qr.list file and add the IP addresses of the control plane nodes.
  1. Create the qr.list file.
```
touch qr.list
```
  2. Run the vi qr.list command to open the qr.list file. Then, add the IP addresses of the control plane nodes. The following output is an example:
```
10.0.2.41
10.0.2.42
10.0.2.43
```
    Find the IP addresses of the CPFS-POSIX client control plane nodes in the Client Management node section.
  3. Save the file and exit.
3. Configure the auto_ssh.sh script.
  1. Create the auto_ssh.sh file.
```
touch auto_ssh.sh
```
  2. Run the vi auto_ssh.sh command to open the auto_ssh.sh file. Then, add the following content.
```
#!/usr/bin/expect
set timeout 10
set username [lindex $argv 0]
set password [lindex $argv 1]
set hostname [lindex $argv 2]
spawn ssh-copy-id -i /root/.ssh/id_rsa.pub $username@$hostname
expect {
    #first connect, no public key in ~/.ssh/known_hosts
    "Are you sure you want to continue connecting (yes/no)?" {
        send "yes\r"
        expect "password:"
        send "$password\r"
    }
    #already has public key in ~/.ssh/known_hosts
    "password:" {
        send "$password\r"
    }
    "Now try logging into the machine" {
        #it has authorized, do nothing!
    }
    "already exist on the remote" {
        #it has authorized, do nothing!
    }
}
expect eof
```
  3. Save the file and exit.
Run the following command to grant execute permissions for the auto_ssh.sh file.
```
chmod +x auto_ssh.sh
```
Run the following command to record the password of the target ECS instances in the client.list file.
```
PASSWD='yourpassword'
```
Note
If the target ECS instances use different passwords, you must create a separate client.list file for each instance and repeat Steps 8 to 10.
Run the following command to configure password-free logon for the control plane nodes.
```
cat client.list | xargs -I {} ./auto_ssh.sh root $PASSWD {}
```

Run the following commands to remotely install the CPFS client on the ECS instances specified in the client.list file. The installation is successful when the Complete! message is returned.

region_id=`curl http://100.100.100.200/latest/meta-data/region-id`;
wget https://cpfs-${region_id}-pre.oss-${region_id}-internal.aliyuncs.com/cpfs/CPFS2.3.4-CentOS.tar.gz -O /root/CPFS2.3.4-CentOS.tar.gz;
cat client.list | xargs -I {} scp /root/CPFS2.3.4-CentOS.tar.gz {}:/root/
pssh -ih client.list "tar xzvf /root/CPFS2.3.4-CentOS.tar.gz"
pssh -ih client.list "cd /root/CPFS2.3.4-CentOS/CentOS7;yum install -y gpfs.adv-*.x86_64.rpm gpfs.base-*.x86_64.rpm gpfs.docs-*.noarch.rpm gpfs.gpl-*.noarch.rpm gpfs.gskit-*.x86_64.rpm gpfs.gss.pmsensors-*.x86_64.rpm gpfs.license.dm-*.x86_64.rpm gpfs.msg.en_US-*.noarch.rpm"

Run the following commands to remotely compile the kernel extension for the ECS instances specified in the client.list file.
```
pssh -ih client.list "/usr/bin/yum -y install cpp gcc-c++ elfutils-libelf-devel" 
pssh -ih client.list "export LINUX_DISTRIBUTION=KERNEL_ORG_LINUX; /usr/lpp/mmfs/bin/mmbuildgpl"
```
After the configuration is complete, you can create a custom image of the environment. To add new ECS instances to access the CPFS file system, use the image to create the instances, which eliminates the need to reconfigure the environment. For more information about how to create a custom image, see Create a custom image.

Step 3: Mount the file system

In the navigation pane on the left, click Instances.
On the Instances page, find the CPFS-POSIX client installation node ECS instance (qr-001). In the Actions column, click Connect to log on to the node.
Find the logon password for the CPFS-POSIX client installation node ECS instance in the Client Management node section.

Run the following commands to update the hosts file on the control plane nodes and target ECS instances.

pssh -ih client.list "hostname;ifconfig eth0 | grep -v inet6 | grep inet | awk '{print \$2}'" | grep -v SUCC | sed 'N;s/\n/ /' > /tmp/ip-host-client.list
pssh -ih qr.list "hostname;ifconfig eth0 | grep -v inet6 | grep inet | awk '{print \$2}'" | grep -v SUCC | sed 'N;s/\n/ /' > /tmp/ip-host-qr.list
cat /tmp/ip-host-client.list | awk '{print $2,$1,$1"t_MAGICTAG"}' > /etc/client-hosts.append
cat /tmp/ip-host-qr.list | awk '{print $2,$1,$1"t_MAGICTAG"}' > /etc/qr-hosts.append
pssh -ih client.list "cp /etc/hosts hosts.bak"
pssh -ih qr.list "cp /etc/hosts hosts.bak"
cat client.list | xargs -I {} scp /etc/client-hosts.append {}:/etc/
cat client.list | xargs -I {} scp /etc/qr-hosts.append {}:/etc/
cat qr.list | xargs -I {} scp /etc/client-hosts.append {}:/etc/
pssh -ih client.list "cat /etc/client-hosts.append >> /etc/hosts"
pssh -ih client.list "cat /etc/qr-hosts.append >> /etc/hosts"
pssh -ih qr.list "cat /etc/client-hosts.append >> /etc/hosts"

Run the following commands to add the target ECS instances to the CPFS cluster.

Run the commands.

mmaddnode -N client.list
mmchlicense client --accept -N client.list
mmchnode --perfmon -N client.list
mmstartup -N client.list
mmgetstate -a

View the sample response.

If the state of a target ECS instance node changes to active, the node is successfully added to the CPFS cluster.

 Node number  Node name                            GPFS state  
---------------------------------------------------------------
           1  cpfs-2989198****323d3-000001-qr-001  active
           2  cpfs-2989198****323d3-000001-qr-002  active
           3  cpfs-2989198****323d3-000001-qr-003  active
           4  iZ0jl91t3p5kehvr6i5****              active

Run the following command to confirm that the node list contains the target ECS instances.

Run the command.
```
mmlscluster
```