What do I do if the "error while loading shared libraries" error occurs when I start the SSH service -

Problem description

The SSH service fails to start. An error message indicates the libcrypto.so.1.1 shared library is missing.

error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory.

Cause

The SSH service and its authentication modules dynamically load the libcrypto.so.1.1 shared library at runtime. This error occurs if the library file or its symbolic link's target file is missing, moved, or corrupted. Without this dependency, the SSH service cannot start.

Solutions

Important

If a previous snapshot of the system disk is available, first create a new snapshot to back up your current data. Then, roll back the system disk using the previous snapshot and verify that the SSH service starts correctly.

If no previous snapshot is available, you need a healthy Elastic Compute Server (ECS) instance that runs the same operating system and is in the same zone as the problematic instance. You can restore the missing library file by attaching the system disk from the problematic instance to the healthy one.

Step 1: Detach the system disk

Make sure the problematic instance is in the Stopped state, then perform the following steps:

To prevent data loss, create a snapshot of the system disk.
Go to ECS console - Instances. In the top navigation bar, select the target region and resource group.
Click the ID of the problematic instance to open the Instance Details page, then click the Block Storage tab.
In the Actions column for the System Disk, choose > Detach.
In the Detach Cloud Disk dialog box, confirm the details and click OK. The disk is successfully detached when the instance status changes to No System Disk.

Step 2: Attach the disk to a healthy instance as a data disk

Make sure the healthy instance is in the Running state, then perform the following steps:

Attach the problematic instance's system disk to the healthy instance.
1. Click the ID of the healthy instance to open its details page.
2. Click the Block Storage tab, then click Attach Cloud Disk.
3. On the Attach Cloud Disk page, select the detached system disk in the Disk section, then click Next.
4. On the Partition Disk and Create and Mount File Systems page, select Configure Later.
Click Connect, the choose Workbench. Log on to the instance as prompted to open the terminal.

Mount the file system.

Identify the partition name of the problematic disk.

sudo lsblk -f

vda                                                      
├─vda1                                                   
├─vda2 vfat         7938-FA03                            /boot/efi
└─vda3 ext4   root  33b46ac5-7482-4aa5-8de0-60ab4c3a4c78 /
vdb                                                      
├─vdb1                                                   
├─vdb2 vfat         7938-FA03                            
└─vdb3 ext4   root  33b46ac5-7482-4aa5-8de0-60ab4c3a4c78

In this example, vdb is the problematic disk and vdb3 is the root partition to mount. The partitions are as follows:

vdb1/vdb2: Contain system boot files. You can ignore these.
vdb3: Contains operating system files and data. This partition must be mounted.

Create a directory and mount the file system.

sudo mkdir <mount_directory> && sudo mount /dev/<partition_name> <mount_directory>

Parameter	Description
`<partition_name>`	The root partition name of the problematic disk that you identified in the previous step.
`<mount_directory>`	A custom, empty directory path that starts with `/`. Important Mounting the file system to a non-empty directory makes the original files in that directory inaccessible.

For example, to mount the target partition vdb3 to a new directory named /test, run sudo mkdir /test && sudo mount /dev/vdb3 /test.

Verify that the file system is mounted.
Run the sudo lsblk command. If a mount directory (MOUNTPOINT) appears for the target partition, the file system is mounted.

Step 3: Repair the libcrypto.so.1.1 file

This topic uses Alibaba Cloud Linux 3.2104 as an example. On the healthy instance, repair the shared library file in the mounted path.

Find the path to libcrypto.so.1.1.
The name of the library file may vary depending on the operating system. Search for the file name that appears in your specific error message.
```
sudo find / -name libcrypto.so.1.1
```
Sample output:
```
/usr/lib64/libcrypto.so.1.1
```
The sample output confirms that the library file is in the /usr/lib64/ directory.
Check the target of the symbolic link.
```
ll /usr/lib64/libcrypto.so.1.1
```
Sample output:
```
lrwxrwxrwx. 1 root root 19 Nov 20  2024 /usr/lib64/libcrypto.so.1.1 -> libcrypto.so.1.1.1k
```
This example shows that libcrypto.so.1.1 is a symbolic link that points to the libcrypto.so.1.1.1k file. Therefore, copy the healthy libcrypto.so.1.1.1k file to the same path on the problematic disk.
Restore the library file.
Replace <mount_directory> with the actual mount directory you created.
```
sudo cp /usr/lib64/libcrypto.so.1.1.1k <mount_directory>/usr/lib64/
```
Based on the example, you would run sudo cp /usr/lib64/libcrypto.so.1.1.1k /test/usr/lib64/.
Set the correct permissions and recreate the symbolic link.
1. Set the file permissions and ownership.
  Replace <mount_directory> with the actual mount directory.
```
# Set permissions to rwxr-xr-x
sudo chmod 755 <mount_directory>/usr/lib64/libcrypto.so.1.1.1k

# Set owner and group to root
sudo chown root:root <mount_directory>/usr/lib64/libcrypto.so.1.1.1k
```
  Based on the example, you would run sudo chmod 755 /test/usr/lib64/libcrypto.so.1.1.1k and sudo chown root:root /test/usr/lib64/libcrypto.so.1.1.1k.
2. Recreate the symbolic link.
```
sudo ln -sf /usr/lib64/libcrypto.so.1.1.1k <mount_directory>/usr/lib64/libcrypto.so.1.1
```
  Based on the example, you would run sudo ln -sf /usr/lib64/libcrypto.so.1.1.1k /test/usr/lib64/libcrypto.so.1.1.

Step 4: Reattach the disk to the problematic instance as the system disk

Unmount the file system.
Replace <mount_directory> with the actual mount directory.
```
sudo umount <mount_directory>
```
Based on the example, you would run sudo umount /test.
Detach the repaired system disk.
1. Return to the ECS console and go to the details page of the healthy instance. Click the Block Storage.
2. In the Actions column for the repaired system disk, click Detach.
3. In the Detach Cloud Disk dialog box, click OK.
Attach the repaired system disk back to the original instance.
1. Go to the details page of the problematic instance, click the Block Storage tab, then click Attach Cloud Disk.
2. On the Attach Cloud Disk page, select the repaired system disk as the Disk, configure the Logon Credentials, then click Next.
3. On the Partition Disk and Create and Mount File Systems page, select Configure Later.
Start the ECS instance.

Step 5: Verify the results

Log on to the problematic ECS instance and restart the SSH service to verify that it runs correctly.

To stop billing, you can release the healthy instance after you confirm the service is restored.

Recommendations

Handle core system files with caution: Before you delete, move, or modify core system files, always create a snapshot to back up your data. Do not delete system files that you are not familiar with.
Use standard software installation methods: Always use the system's native package manager, such as yum or apt, to install or uninstall software. This ensures that dependencies are handled automatically and helps prevent damage to shared libraries.