Cloud Backup:Troubleshoot Cloud Backup client status exceptions

Symptoms

In the Cloud Backup console, you may see the following:

1. The client status is "Stopped", and the following message appears: "The Cloud Backup client is not connected properly."

2. A backup job fails with the error message: "The backup fails because the backup client loses connection."

Causes

The following issues can cause the client status exception and make backup jobs fail:

• The client program does not run as expected. For example, the client process has stopped.

• The network connection between the client and the Cloud Backup service is interrupted, for example by network unreachability, firewall rules, or incorrect client settings.

• The client fails to register, for example after the NIC of the on-premises server is replaced.

• The client is stopped while a backup job is running. The backup process ends unexpectedly, the Cloud Backup service cannot obtain the job status, and the job times out.

Solution

1. Check whether the Cloud Backup client is running.

   Windows

   On the supported Windows versions, the Cloud Backup client is available in a new version and an old version, which use different service names:

   • New version: Alibaba Cloud Hybrid Backup Service

   • Old version: Aliyun Hybrid Backup Service

   Check the status of the Cloud Backup service:

   • Check the service status

     Note

     Before you change the startup type of a service, make sure that you understand its function. An unintended change can affect the operating system. Some services require administrator privileges.

     1. Open the Services console in either of the following ways:

        • Press Win + R to open the Run dialog box, and then enter services.msc.

        • In the Windows search bar, enter Services, and then click the Services app.

     2. The Services console lists all Windows services on the server.

        • To start a service, double-click its name to open the properties dialog box, and then click Start.

        • To stop a service, double-click its name to open the properties dialog box, and then click Stop.

     3. Find the Cloud Backup service. If the client runs as expected, the service is in the Running state.

        • If the client is not running, right-click the service and choose Restart.

        • If the client still does not run, uninstall and then reinstall the client. For more information, see Uninstall the Cloud Backup client and Download and activate the Windows client.

   Linux

   On the supported Linux versions, the Cloud Backup client is available in a new version and an old version, which run as different processes:

   • New version: hbrclient

   • Old version: hybridbackup

   New Cloud Backup client (hbrclient)

   Check the status of the client:

   • Check the process status

     Run systemctl status hbrclient, service hbrclient status, or ps axu|grep hbrclient to check the client status. If the output contains active or run, the client is running.

   • Check whether the port is open

     Run netstat -lpt | grep hbrclient to check the port status. If the port is open, the output shows the LISTEN state.

     tcp6       0      0 [::]:43565              [::]:*                  LISTEN      1727/hbrclient

   Old Cloud Backup client (hybridbackup)

   Check the status of the client:

   • Check the process status

     Run ps axu|grep hybridbackup. If the output lists a process, the client is running.

   • Check whether port 8011 is open

     Run lsof -i:8011 to check the port status. If the port is open, the output shows the LISTEN state. If the command returns command not found, run yum install lsof or apt install lsof to install lsof, and then try again.

     COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
     hybridbac 11477 username    5u  IPv6 10625414      0t0  TCP *:8011 (LISTEN)

   Restart or reinstall the client

   • If the client is not running, try restarting the Cloud Backup service.

     Note

     The commands vary across Linux distributions.

     Restart commands for the new client:

     • systemctl restart hbrclient

     • systemctl restart hbrclientupdater

     Restart commands for the old client:

     • systemctl restart hybridbackup

     • systemctl restart updater

   • If the client still does not run, uninstall and then reinstall the client. For more information, see Uninstall the Cloud Backup client and Download and activate the Linux client.

   For information about the client logs, see Where are Cloud Backup client logs stored?

2. Check the logs and the network.

   Check the logs and the network status of the client, and then reconfigure and reactivate the client.

   1. Check the log file.

      Note

      The default installation paths of the new client are as follows. Use the actual installation path.

      • Windows log path: Local disk (C:) > Program Files > Aliyun Hybrid Backup Service Client > logs

      • Linux log path: /opt/alibabacloud/hbrclient/logs

      1. Open the log file and search for Failed to register client. AppError: ErrorCode=.

      2. If the ErrorCode is InvalidInstanceId, the network is working, but the client fails to register.

         Root cause: Operations on the ECS instance, such as replacing the system disk or restarting it after a patch installation, can invalidate its encrypted InstanceID. The instance is then treated as invalid, and client registration fails.

         Handling: An old client with an invalidated InstanceID cannot be recovered by reactivation. Install a new Cloud Backup client and create a new backup plan. To keep the backup data of the old client, do not delete the old client from the client list.

      3. If the ErrorCode is not InvalidInstanceId, restart the Cloud Backup service. On Linux, run service hbrclient restart.

      4. If the issue persists, go to Step 2 to check the network status of the server.

      5. If the client still fails to register, submit a ticket to get support.

      6. If Program Stopped! appears in the log file, the client was stopped while a backup job was running. The backup process ended unexpectedly, the Cloud Backup service could not obtain the job status, and the job timed out. You can check whether the Cloud Backup client is running. If the client is running, subsequent backup jobs run as expected.

         

   2. Check the network status of the server.

      1. We recommend that you use the Diagnostic tool for Cloud Backup clients to monitor the network environment of the client. For common network issues, see Common network issues.

      2. Open the network.log file of the client.

      3. If Failed on ping appears in the log file, a network issue exists, such as an incorrect firewall or route configuration. Contact the network administrator and configure the public endpoint or the ECS internal endpoint for the Cloud Backup region based on the list in Check the network connectivity.

      4. If Failed on ping does not appear, go to Step 4.

   3. Check whether the NIC of the server was replaced.

      Contact the network administrator to confirm whether the NIC of the server that hosts the client was replaced.

      1. A NIC replacement changes the MAC address of the server, so the client cannot register and its status becomes abnormal. If the NIC was replaced, reactivate the client as described in Step 6.

      2. If the NIC was not replaced, go to Step 5.

   4. If you use the old client, check whether the server was cloned.

      Contact the network administrator to confirm whether the server that hosts the client was cloned.

      1. If the server was cloned, the installed client was cloned with it, so the client on the cloned server fails to register.

         • If the client on the original server is still in use, uninstall the client on the cloned server and then reinstall it.

         • If the client on the original server is no longer in use, reactivate the client on the cloned server as described in Step 6.

      2. If the server was not cloned, go to Step 5.

   5. Check whether the operating system of the server was reinstalled.

      Note

      After the operating system of the server is reinstalled, the client loses contact with the Cloud Backup console, and its status becomes abnormal. If you then reinstall and register the client, a second client with the same name appears in the console.

      1. Stop the new Cloud Backup client service first.

      2. In the Cloud Backup console, obtain the reactivation command and token of the old client. Reactivation is available only after the client has been disconnected for more than 1 hour. If reactivation is unavailable, wait and try again.

      3. Run the reactivation command in the client installation directory on the server.

      4. After a short while, the old client is displayed as Activated in the Cloud Backup console.

      5. In the Cloud Backup console, delete the new client whose status is abnormal.

   6. If the client status is still abnormal, reactivate the Cloud Backup client.

      If the client status is still abnormal after reactivation, operations on the ECS instance, such as replacing the system disk, reinstalling the operating system, or restarting the instance after a patch installation, may have invalidated its encrypted InstanceID. The registration of the old client is no longer valid and cannot be restored by using a reactivation token. Perform the following steps:

      1. Install a new Cloud Backup client on the same ECS instance.

      2. Create a new backup plan with the new client to restore backup protection.

      3. Keep the record of the old client in the client list so that you can still view its historical backup data. If you delete the record, you can no longer access that data.

      Note

      After the encrypted InstanceID of an ECS instance is invalidated, the old client cannot be recovered. Force-reinstalling the client or using the console Reinstall action does not resolve this issue. The Reinstall action depends on Cloud Assistant. If Cloud Assistant is not installed on the ECS instance or is unavailable, the client cannot be installed automatically. You can confirm this issue in the client registration logs described in Step 1: if a log file contains Failed to register client. AppError: ErrorCode=InvalidInstanceId, the encrypted InstanceID has been invalidated.

      1. Log on to the Cloud Backup console.

      2. Select the data source for file backup.

         • ECS File Backup Standard Edition

           In the left-side navigation pane, choose Backup > ECS File Backup Standard Edition.

         • Local File Backup

           In the left-side navigation pane, choose Backup > Local File Backup.

      3. In the top navigation bar, select the region where the client resides.

      4. Find the client that you want to reactivate. In the Actions column, choose More > Reactivate Client.

      5. In the Reactivate Client panel, copy the command and run it in the client installation directory on the server. Wait until the client is reactivated.

      After the client is reactivated, the Client Status changes to Activated.

   7. If the issue persists after these checks, submit a ticket to get support.