By Shantanu Kaushik
Cloud storage plays an important role in every aspect of cloud computing. Storage is the basis of any successful application deployment, database solution, information sharing, or any other cloud storage-based scenario. In the previous article of this series, we discussed the need for proper storage optimization and the cloud monitoring solution by Alibaba Cloud.
Optimizations enroll the need for Cloud Monitor to gather important metrics and make adjustments to your cloud service for a better and more productive event cycle. In this article, we will continue discussing error diagnosis, using network logging tools, E2E tracking/diagnosis, troubleshooting, and OSS sandboxing.
Storage errors could boil down to a large-scale outage. To prevent this, you need to be as proactive as possible with error diagnosis, including server return with error information to the client when the client-side applications are faulty. Alibaba Cloud Monitor records these errors and displays detailed reports to outline and showcase the complete error information when required and requested.
You can always retrieve the information for specific errors by filtering for the required data type. You can also include individual requests from server log, network log, and client log. The best way to diagnose an error is to combine the error information, including the Object Storage Service (OSS) error code, HTTS status code, and OSS error detail.
You can read more about OSS error responses here. This will give you a detailed explanation of error codes and possible causes.
The Object Storage Service (OSS) provides server and network logging. You can log user requests to track end-to-end requests and the details associated with these requests. Various issues can be backtracked using the network logging function. You can retrieve the user application data, storage, and network logs. These logs can provide detailed information to assist with diagnosing issues related to traffic conditions between the server and the client.
An end-to-end process starts with a client request that is processed over a network and sent to the Alibaba Cloud OSS server for processing and execution. This end-to-end tracking process ensures the diagnosis of potential problems and troubleshooting.
Let’s outline some common performance issues and their solutions:
1. Low End-to-End Latency (Average)
There could be an issue where end-to-end latency is low, but the client request latency is too high. In this scenario, the client-side will experience request delay. This could be due to:
2. High Average End-to-End Latency
High end-to-end latency may be caused by the slow performance of client applications. This could be the result of slow network speeds or a limited number of available connections.
Check to see if there are a large number of connections in the using
TIME_WAIT. If there are, you can adjust the core parameters to overcome this issue. Alternatively, when the number of available connections is insufficient, you need to address the parameters that might be affecting the client CPU, memory, and network resources. You should increase the number of threads or connections to overcome this issue.
You can also try and optimize the client-side application and adjust the application configuration to implement an asynchronous access method. You can analyze the application performance and optimize it if necessary.
System monitoring is a vital tool to find a cause and fix it. You can analyze and pinpoint the resource on the client-side that is causing the bottlenecks. The second step would be to optimize the resource usage or increase client resources, such as CPU, memory, and network bandwidth.
High end-to-end latency due to network factors is generally short-lived. You can use tools like Wireshark to investigate these network problems and fix them.
If you notice a sudden spike in storage capacity and you cannot find a valid reason, such as an increase in upload requests, it could be due to:
When your Alibaba OSS bucket comes under attack, the Object Storage Service automatically sends the bucket to a sandbox. Let me explain: Here, Sandbox is an isolation mechanism for buckets that are under attack or are compromised. It is a part of the safety operations developed by Alibaba Cloud to ensure that your operations and data will remain safe, even if a bucket has been compromised.
If your bucket is under attack, OSS automatically adds the attacked bucket to the sandbox. The bucket in the sandbox can still respond to requests, but the service quality of your application will not be normal.
To prevent the Object Storage Service from adding your bucket to the sandbox, you can use Anti-DDoS Pro.
Another solution is to configure a reverse proxy using Alibaba Cloud Elastic Compute Service (ECS) and configure the Anti-DDoS Pro instance. The IP address resolved from the default domain of a bucket changes dynamically for added security measures. To use a fixed IP address to access the bucket, an ECS instance can be used to set up a reverse proxy and attach the elastic IP of an ECS instance to an Anti-DDoS Pro instance to ward off DDoS attacks and HTTP floods.
Cloud storage is highly elastic and offers enhanced performance when compared to traditional storage practices. When undergoing a technological paradigm shift, the inclusion of more agile practices is required. Storage plays a crucial role. Maintaining that storage by monitoring it closely and troubleshooting problems is highly recommended.
When it comes to containerization and microservices with DevOps, a lot of things depend on storage. Alibaba Cloud has devised architectures to support and uplift the overall service quality to ensure uninterrupted service and high-scalability usage scenarios.
We will discuss the complete architecture and usage scenarios of the Apsara File Storage NAS solution by Alibaba Cloud.
Alibaba Clouder - November 17, 2020
Alibaba Clouder - November 17, 2020
Alibaba Clouder - November 12, 2020
Alibaba Clouder - November 26, 2020
Alibaba Clouder - October 29, 2020
Alibaba Clouder - March 5, 2021
Deploy custom Alibaba Cloud solutions for business-critical scenarios with Quick Start templates.Learn More
Provides scalable, distributed, and high-performance block storage and object storage services in a software-defined manner.Learn More
An encrypted and secure cloud storage service which stores, processes and accesses massive amounts of data from anywhere in the worldLearn More
Plan and optimize your storage budget with flexible storage servicesLearn More
More Posts by Alibaba Clouder