Symptoms
Service access issues: Service response times increase significantly, leading to request timeouts or causing services to become inaccessible.
High monitoring metrics: In the ECS console or CloudMonitor, your instance's outbound Internet bandwidth consistently exceeds 80%, approaching or reaching the bandwidth limit. You may also see a sudden increase in network connections.
Monitoring alerts: You receive an SMS or email alert that your network bandwidth usage has exceeded the preset alert threshold.
Causes
Abnormal processes or malicious programs: The instance is infected with cryptocurrency mining programs, Trojans, or acts as a bot or zombie in a DDoS attack. These programs generate substantial abnormal network traffic.
Network attacks: Malicious access attempts, such as application-layer CC attacks, DDoS attacks, or brute-force attacks, target the instance's public-facing ports. These attacks can saturate the inbound bandwidth with invalid requests.
Insufficient instance network specifications: As your business grows, the instance's configured bandwidth is insufficient for normal service traffic, creating a network performance bottleneck.
Solution
First, use the sar tool to identify the NIC with abnormal traffic, then use the iftop tool (to identify the peer IP address consuming bandwidth) or the nethogs tool (to identify the high-traffic process) to analyze the NIC traffic, and finally take appropriate measures based on the nature of the process and IP address.
Step 1: Locate the high-traffic NIC
Use the sar tool to identify the high-traffic NIC to pinpoint the troubleshooting target.
Log on to an ECS instance using a VNC connection.
Go to ECS console - Instances. In the top navigation bar, select the target region and resource group.
Go to the details page of the target instance. Click Connect and select VNC. Enter the username and password to log on to the ECS instance.
Gather network interface statistics.
# -n DEV: Reports network device statistics # 1 5: Samples every 1 second, 5 times in total sudo sar -n DEV 1 5Identify the high-traffic NIC.
Focus on the
Averagevalue oftxkB/s(average outbound bandwidth). Compare the values to identify theIFACE(NIC name) with the highest value.rxkB/srepresents the average inbound bandwidth.
In this example, the
eth0NIC has the highesttxkB/s. Therefore,eth0is the high-traffic NIC.
Step 2: Analyze and resolve high traffic
Analyze the NIC traffic.
iftop: Monitors NIC traffic from a connection perspective, allowing you to identify the IP addresses and ports that generate the most traffic with the local machine.Web services can use
iftopto locate high-traffic IP addresses, and use tools such aslogwatchto analyze their web logs to determine traffic compliance.nethogs: Monitors NIC traffic on a per-process basis to identify the process that consumes the most network bandwidth.
Iftop
Install the
iftoptool.For Alibaba Cloud Linux and CentOS systems:
sudo yum install -y iftopFor Ubuntu and Debian systems:
sudo apt update sudo apt install -y iftop
Monitor the high-traffic NIC.
Replace
<IFACE>with the name of the high-traffic NIC identified in Step 1.# -i <IFACE>: Specifies the NIC to monitor as <IFACE> # -P: Displays port numbers sudo iftop -i <IFACE> -PFor example, if the high-traffic NIC is named
eth0, runsudo iftop -i eth0 -P.Analyze the NIC traffic to find the peer IP address consuming the most bandwidth.

The real-time traffic information is sorted in descending order, where the
=>symbol indicates the rate at which the local machine sends data to the peer IP address. In the example, the average outbound traffic from the local machine to the IP address140.205.11.xover the past 2 seconds is4.32Mb/s.Press
qto exit theiftopinterface.View the process corresponding to the port.
You can obtain the
<bandwidth-consuming peer IP address>from the previous step.sudo netstat -antp | grep <PEER_IP_ADDRESS>Example output:

In this example, the local IP address is
172.16.0.x, the peer IP address is140.205.11.x, the corresponding process isnginx: worker, and the PID is2282.
Nethogs
Example
Install the nethogs tool.
For Alibaba Cloud Linux and CentOS systems:
sudo yum install -y nethogsFor Ubuntu and Debian systems:
sudo apt update sudo apt install -y nethogs
Monitor the high-traffic NIC.
Replace
<IFACE>with the name of the high-traffic NIC identified in Step 1.# The default monitoring interval is 1 second. You can use -d to specify the monitoring interval. sudo nethogs <IFACE>For example, for a high-traffic NIC named
eth0, runsudo nethogs eth0.Analyze the NIC traffic.

Here,
SENTindicates the rate at which the local machine sends data to the peer IP address (the receiver). In the example, the process with the highest traffic usage isnginx: worker process. The outbound traffic is approximately 696 KB/s, and the process ID is2282. Pressqto exit the tool's interactive interface.
Choose a solution based on the process or peer IP address.
If the identified process (such as a download tool like wget or curl, or an unknown program) exhibits suspicious behavior, or if it communicates with a malicious peer IP address:
Terminate the abnormal process: Run the
sudo kill -15 <PID>command. Replace<PID>with the PID of the identified high-traffic process.ImportantBefore you terminate a process, ensure it is not a critical business process to avoid service disruption.
Block the malicious IP address: Use security group rules to block known malicious IP addresses. If you are experiencing an application-layer CC attack where the source IP addresses are widely distributed, security group rules offer limited protection. In this case, we recommend that you deploy a Web Application Firewall (WAF) for specialized protection. For more information, see Add an ECS instance to WAF to defend against CC attacks.
Scan for and remove malicious programs: Enable the virus detection and removal feature in Security Center to perform a full scan of the instance and remove any detected malicious programs.
Respond to high-frequency DDoS attacks: During high-frequency DDoS or CC attacks, peak traffic may exceed the blackhole threshold of Anti-DDoS Origin. This causes your cloud product to enter a blackhole state and become inaccessible. In this scenario, you must enable Anti-DDoS Pro and Anti-DDoS Premium to ensure continuous availability.
If a legitimate business process is causing the high traffic, this is likely due to normal business demand:
Upgrade bandwidth: The instance's current bandwidth is a bottleneck. Upgrade the instance bandwidth.
Optimize the application: Check your application code for potential optimizations, such as reducing unnecessary data transfers, adding caching, or compressing data.
Rate limiting: If your service allows, you can use tools such as
iptablesto limit the traffic rate for specific IP addresses or ports, preventing a single user or service from consuming all available bandwidth.
If you find no suspicious processes consuming bandwidth but overall usage remains high, the total traffic volume has likely exceeded the instance's network capacity. You should upgrade the instance bandwidth.
Next steps
Periodically collect system metrics for ongoing analysis. For more information, see Use the atop tool to monitor Linux system metrics.
Configure instance monitoring and alerting to receive notifications about future risks and anomalies.