Quick troubleshooting method for remote desktop failure to connect to Windows instance -

Overview

This topic describes how to troubleshoot problems that the remote desktop cannot connect to a Windows instance.

Detail

Alibaba Cloud reminds you that:

Before you perform operations that may cause risks, such as modifying instance configurations or data, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.

If you modify the configurations and data of instances including but not limited to ECS and RDS instances, we recommend that you create snapshots or enable RDS log backup.

If you have authorized or submitted security information such as the logon account and password in the Alibaba Cloud Management console, we recommend that you modify such information in a timely manner.

If you cannot remotely log on to your ECS instance, we recommend that you check whether you can remotely log on to the ECS console through the Alibaba Cloud VNC. For more information, see remotely log on to a Windows instance through VNC. Use the following troubleshooting method to troubleshoot the remote connection failure of the Windows instance.

Step 1: log on to the instance by using the management Terminal
Step 2: Check the logon password
Step 3: Check the port and security group
Step 4: Check the remote desktop service
Step 5: check the network
Step 6: check the CPU load, bandwidth, and memory usage
Step 7: check the firewall configuration
Step 8: set the system security policy
Step 9: check the configuration of the remote terminal service
Step 10: check your antivirus software
Step 11: try to restart the instance

Step 1: log on to the instance by using the management Terminal

If you cannot connect to the instance remotely for any reason, try to use the remote connection function provided by Alibaba Cloud to make sure that the instance is still responding and is not completely down, and then troubleshoot the fault by reason.

Log on to the ECS console. In the left-side navigation pane, choose instances> remote connection.
When you connect to the instance for the first time or forget the password, clickchange the VNC connection passwordto modify the password for the remote connection.
Then, connect to the instance by using the remote connection password.

Step 2: Check the logon password

Make sure that the logon password is correct and you have reset the password before. Check whether the instance password has not been restarted after the instance password is reset. If there is a record of instance password modification but no record of instance restart, follow these steps to restart the instance.

Log on to the ECS console. In the left-side navigation pane, choose instances.
At the top of the page, select a region, and choose more>Instance status>Restart the instance and then click OK.

Step 3: Check the port and security group

Check whether the port is normal and whether the security group rule has restrictions.

Seehow to view and modify the default port of a remote desktop for a Windows instance, check whether the remote connection port of the instance is modified. If the logon method is changed or the modified port number is not allowed in the ECS Security group rules, follow these steps to allow the modified port number.
Note: by default, the ECS Security group rules allow port 3389. After modifying the port of the remote desktop, you must allow the modified port in security group rules.
1. Log on to the ECS console.
2. Locate the instance and click manage to go to the instance details page. In the left-side navigation pane, click security groups. Find the security group and click add rules in the actions column.
3. Select the direction in which the rule is applied.
4. On the security group rules page, you can add security group rules in one of the following ways. For more information, see add security group rules.
  - Method 1: Quickly create security group rules
    - Authorization policy: allow
    - Port range: RDP(3389)
    - Authorization objects: 0.0.0.0/0 (allow access from all IP addresses)
  - Method 2. Manually add security group rules
    - Authorization policy: allow
    - Priority: 1 (a security rule has the highest priority. A smaller number indicates a higher priority.)
    - Protocol type: TCP
    - Port range: set to 33899 if the custom remote port is 33899
    - Authorization objects: 0.0.0.0/0 (allow access from all IP addresses)
5. Use IP: port to connect to the remote desktop. The connection method is similar to the following.
Run the following command to test whether the Port obtained in the previous step is normal: If the Port test fails, seehow to troubleshoot the port failure when you can ping the server.
```
telnet [$IP] [$Port]
```
Note:
- [$IP] indicates the IP address of a Windows instance.
- [$Port] indicates the RDP Port number of a Windows instance.
The command output is as follows: telnet 192.168.0.1 4389. Normally, the returned result is similar to the following.
```
Trying 192.168.0.1 ...
Connected to 192.168.0.1  4389.
Escape character is '^]'
```
Check whether the Windows remote port setting is out of the range. If it is out of the range, you need to change the port to a range between 0 and 65535, and there are no occupied ports. For more information, see the following operations.
1. Log on to the instance and select start>Run, enter regedit, and then click confirm.
2. Open the registry editor and choose #server\wds\ rdpwd \TDs \TCP.
3. Double-click the PortNumber click decimal, a new course is developed, which combines original port by the "113322" shall be modified as between 0 and 65535 and not to the current port conflict and those of the ports, e.g., 5588, etc., Port.
  Note: 113322 is the port number displayed on the right of PortNumber.
4. Then, open the parameter server.
5. Double-click PortNumber, click decimal, and change the original port "113322" to the same as the port number in step 3.
6. Then, restart the host to confirm that the remote connection is successful.

Step 4: Check the remote desktop service

You can check whether the remote desktop service is enabled on the Windows Server. Perform the following operations:

Remotely log on to a Windows instance by using Alibaba Cloud VNC. For more information, see remotely log on to a Windows instance through VNC.
Right-click my computer and select properties>Advanced system settings.
In the system properties dialog box, click the remote tab. In the remote desktop section, confirm that allow remote connection to this computer is selected.
To improve system security, you may mistakenly disable key services that the remote desktop service depends on, resulting in remote desktop service exceptions. You can perform the following operations to check the IP address.
1. Log on to the Windows instance.
2. Select start>Run. Enter msconfig and click OK.
3. In the dialog box that appears, click the general tab, select start normally, and then restart the server.

Step 5: check the network

If you cannot remotely connect to a Windows instance, check whether the network is normal.

Use the computer connection comparison test in other network environment (different network segments or different operators) to determine whether it is a local network problem or a server problem. If the problem is caused by the local network or carrier, contact the local IT personnel or carrier. If the network interface controller driver is abnormal, reinstall it. Troubleshoot local network faults and proceed to the next step.
Run the ping command on the client to test the network connectivity with the instance.
- When a network exception occurs, seehow to capture data when a network exception occursin the case of troubleshooting.
- When ping is lost or ping fails, see the link test method when using the ping command for packet loss or disconnection for troubleshooting.
- If intermittent packet loss occurs and the network of the ECS instance remains unstable, see use the ping command to test intermittent packet loss of the IP address of an ECS instance.
Test the connectivity to the client by using the ping command on the instance. If a "general fault" error occurs, see the message "general fault" appears when you ping the internet address of a Windows instance.

Step 6: check the CPU load, bandwidth, and memory usage

Check whether the CPU load is too high. If yes, follow this step to solve the problem. If no, go to the next step.
- Check whether the CPU load is too high. Log on to the instance from the terminal on the instance details page and check whether the Windows Update operation is being performed in the background.
- Run Windows Update to install the latest Microsoft security patch.
- High CPU load is a normal result if the application needs a large amount of disk access, network access, and high computing. You can try to upgrade the instance specification to solve the resource bottleneck problem.
- For solutions to high CPU load, seesolution to high CPU usage of Windows ECS instances.
The failure of remote connection may be caused by insufficient public network bandwidth. The Specific Troubleshooting method is as follows. To solve this problem, restart the ECS instance and restart it. For more information, see manual renewal, or see auto-renewal.
1. Log on to the ECS console.
1. Locate the instance and click manage in the actions column. On the instance details page that appears, you can view network monitoring data.
1. Check whether the server bandwidth is "1K" or "0K". If you did not purchase a public network bandwidth when purchasing an instance, then upgrade the public network bandwidth, and did not select the required bandwidth when purchasing the instance, the bandwidth becomes "1K".
After you enter the password to log on to the remote connection, the desktop cannot be displayed and you can exit without any error messages. This problem may be caused by insufficient memory on the server. Check the memory usage of the server. The specific operation is as follows.

1. Use the remote connection function in the console to log on to a Windows instance.

1. Select start>Control panel>Management tools, double-click event viewer. Check whether any warning log of insufficient memory resources exist. If the system displays a log error indicating insufficient memory, seehow to solve the problem of insufficient virtual memory in Windows.

Step 7: check firewall configurations

You can perform this troubleshooting only when you have been authorized to disable the firewall. Check whether the firewall is disabled. If it is not disabled, modify the firewall configuration policy. For more information, seehow to configure a firewall for remote connection to a Windows instance. After the operation is completed, connect to the instance again to confirm that the connection is successful. This topic describes how to enable the firewall for a Windows Server 2012 when you log on for the first time. For a newly purchased Windows 2012 instance, you can connect to the server for the first time. Connect to the server and activates the system after, you will be prompted as follows in the picture information, the user needs to click is if you click no, the server automatically turns on the internet firewall connection directly off. To solve this problem, follow these steps.

Use the consoleremote connectionfunction to log on to a Windows instance.

In the menu bar, click start.>Control panel.

Select the View method as the small icon, and click Windows Firewall.

In the Windows Firewall window, click advanced settings.

In the displayed window, clickinbound rules, right-click to the bottom, right-clickremote desktop-user mode (TCP-In), and selectstart a rule.

Return to the previous page and click Windows Firewall attributes.
Select enable (recommended), and click apply.

Note: We recommend that you enable all the domain configuration files, dedicated configuration files, and public configuration files tabs.

For more information about Firewall settings, seeset the firewall for remote connection to Windows instances.

Step 8: configure system security policies

You can check whether the Windows Server has security policies that block Remote Desktop Connection. The specific operation is as follows.

Use the consoleremote connectionfunction to log on to a Windows instance.

Select start>Control panel>Management tools and double-click the local security policy.

In the displayed window, clickip security policiesto check whether there are related security policies.
If yes, right-click the policy and select delete, or double-click the IP address's security policy to reconfigure it to allow Remote Desktop Connection. Then, use Remote Desktop Connection.

Step 9: remote terminal service configuration check

The failure to connect to the remote desktop of a Windows instance may be caused by the following remote terminal service configuration exceptions.

Exception 1: The server-side self-signed certificate is damaged.

If the client is a Windows 7 or later system, it tries to establish a TLS connection with the server. If the self-signed certificate used for TLS connection on the server is damaged, the remote connection fails.

Use the consoleremote connectionfunction to log on to a Windows instance.

Select start>Management tools>Remote Desktop Services, and then double-click remote desktop session host configuration.

Select RDP-Tcp. In the RDP-Tcp properties dialog box that appears, change the security layer to RDP security layer.
In the actions column, click disable connection. Click enable connection.

Exception 2: The connection to the remote desktop session host is disabled.

When you run the netstat command to query for the port, the port is not listened properly. Use the consoleremote connectionafter logging on to a Windows instance, the remote desktop RDP connection property configuration file is disabled. Reference server side of self-signed certificate damage find RDP connection property profile, if RDP-Tcp is disabled, click enable connection can.

Exception 3: Terminal Server role configuration

When you access a Windows instance from a remote desktop, the following message appears.
This is generally due to the installation and configuration of terminal server on the server, but no valid access authorization is configured. See the following two solutions.
- The remote desktop prompts "no remote desktop authorization server can provide license" error
- Remote login to Windows instance reported "remote desktop user group does not have this permission" error

Step 10: check anti-virus software

The failure to connect to the remote desktop may be caused by the setting of third-party anti-virus software, which can be solved in the following ways. Here are two cases of remote access failure caused by watchdog configuration.

If anti-virus software is executed in the background, you can log on to the terminal on the instance details page to upgrade anti-virus software to the latest version or delete it directly.
Use anti-virus software commercial edition or use Microsoft Safety Scanner free Microsoft security tools to scan anti-virus programs in security mode. For more information, see security scan programs.

Case 1: security dog blacklist interception

If the following situations occur after watchdog security is installed, check whether security settings or corresponding interception are configured in the protection software.

The client cannot connect to a Windows instance from the remote desktop, but can connect to a Windows instance from other regions.
The IP address of the server cannot be pinged, and the route is tracked through tracert command. It is found that the server cannot be reached.
Alibaba Cloud security does not intercept local public IP addresses.

You can open the server security dog for inspection and select the network firewall. Click Super blacklist in the rule settings column. If an instance public IP address exists in the blacklist, delete the blacklist rule and add the public IP address to the Super whitelist.

Note: if the threshold of Alibaba Cloud security is too low, the public IP address of the instance may be blocked. We recommend that you increase the cleaning threshold to avoid blocking the public IP address of an instance. For more information, seeanti-DDoS basic.

Case 2: abnormal security watchdog program

Use the consoleremote connectionafter you log on to a Windows instance, an error message is displayed in the lower-right corner of the system desktop. A similar output is displayed. This problem may be caused by an exception in Fortinet software. You can uninstall watchdog security software on Windows and restart the server to restore the network.

Step 11: try to restart the instance

If you cannot connect to the instance by using the remote connection function provided by Alibaba Cloud, restart the instance. A restart operation will stop your instance from running and interrupt business. Exercise caution when performing this operation.

Note: before restarting an instance, you must create a snapshot for the instance to back up data or create an image. For more information about how to create a snapshot, see create a snapshot.

Log on to the ECS console. In the left-side navigation pane, click instances.
At the top of the page, select a region and choose more>Instance status>Restart the instance and click OK.

Documentation

[Recommended] Remote Desktop cannot connect to Windows instance

Application scope

Elastic Compute Service