How to Test Cloud Server-Part I
Created#More Posted time:Dec 11, 2015 16:05 PM
How to Test Cloud Server-Part 1
by Mr. Tian Jiu, from AliCloud
With rapid service growth, the system's performance is increasingly highlighted, and the selection of cloud servers, system architecture and testing service support capability has become a major consideration of many clients. This article hereby introduces how to test the cloud server's performance, so as to solve your puzzles.
2 Testing Type
The cloud server's performance can be measured and compared based on the manufacturer's benchmark performance testing results and the in-service performance testing results. Benchmark performance testing refers to testing the performance of hardware (such as CPU, memory, disk and network) and underlining software (such as MySQL, MongoDB, Redis, Cassandra and Hadoop) of the cloud server.
In-service performance testing refers to testing the support capability of the system by simulating the pressure at the peak traffic after the service system is
deployed, and testing whether the performance indicators meet relevant requirements during high concurrency at peak hours by simulating the user
3 Performance Testing
For whichever performance testing, certain procedures shall be followed, and relevant performance testing techniques and tools used. Major testing procedures include: testing preparation, testing implementation, data analysis and optimization, and testing report. Major performance testing techniques include testing indicators, typical services, service scenarios, monitoring, result analysis, optimization and verification. Performance testing tools provide
automatic means and various service scenarios to verify the performance indicators of the system.
3.1 Benchmark Performance Testing
3.1.1 Demand Evaluation
The required performance indicators of hardware or underlining software of cloud servers provided by cloud manufacturers are evaluated through survey. Major indicators to be evaluated include:
CPU: Time and delay for file system to create, delete and access content blocks in CPU.
Memory: Data block reading rate.
Disk: Rate and delay for file system to create, delete, read and write data blocks.
Network: Transmission rate and delay of network data packets.
MySql: Online transaction processing capacity.
3.1.2 Environment Setup
Apply for necessary cloud server resources from cloud manufacturers for environment deployment. The benchmark testing tool Perfkit can be installed on the cloud server or other servers.
Perfkit, a free open-source cloud testing platform tool developed and released by Google, can be used for benchmark testing of cloud products such as Google's Compute
Engine, Amazon AWS, Microsoft Azure and Alibaba's AliCloud.
Perfkit provides many parameter options for benchmark testing of the cloud server. When the testing is finished, the results are stored as a json file. You can open the json file using Perfkit Explorer. You can obtain the user manual at: https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/
3.1.4 Result Analysis and Report
Results generated by Perfkit not only can be analyzed and optimized, but also can be compared with benchmarks of other manufacturers' cloud products.
3.2 In-service Performance Testing
3.2.1 Demand Evaluation
The required support capabilities of service systems are determined by evaluating the service growth from the following aspects:
> Typical services: Identify the most typical services of the system. Test at least 70% of the typical services to obtain the proportion of each service at peak hours.
> Environment: Difference in production environment architecture, configuration and testing environment. Is any third-party system used in the testing?
> Testing indicators: Required testing indicators of the system, such as TPS, service response time and success rate; server resource indicators such as CPU, memory, disk and network (please note that these indicators are different from those for benchmark testing. The main consideration here is whether the resource indicators have reached the bottleneck); middleware-related indicators such as JVM, thread pool and JDBC; database-related indicators such as the hit rate, lock wait, deadlock and SQL efficiency.
> Typical service scenarios: Peak-hour scenarios, such as hybrid service scenario, abrupt service change scenario, exception scenario, reliability scenario, and stability scenario.
> Problems in the production system: What are the typical problems during production? What are the causes and frequency? Is it necessary to include such problems in the in-service performance testing?
3.2.2 Testing Preparation
> Prepare the testing schemes and policies after the demand survey.
> Apply for the cloud server environment, deploy the application system, and ensure the consistency between the testing environment and the production environment, so as to ensure successful service testing. You need to apply for press environment resources and deploy the press tools unless an additional performance pressure testing platform is provided by the cloud manufacturer.
> Prepare the basic data volume of the system, which must be at the same order of magnitude as the data volume of the production system.
> Prepare the testing script and parametric data to simulate the real service operation in actual environment.
> Prepare to deploy the Monitor to monitor the servers.
> Prepare test cases, make trial run, and ensure all preparations are done.
Based on the performance test cases, start tests on the tested system using the press platform, observe changes in server resources and relevant service indicators, judge whether the system satisfies the performance indicator requirements and the bottlenecks of the system, and conduct optimization and verification.
3.2.4 Result Analysis and Report
Collect relevant performance testing data, analyze the results and prepare the testing report.
4 Tool Introduction
4.1 Benchmark Testing Tool
Perfkit is the mainstream tool for cloud server benchmark testing at present. It has the following advantages:
> A third-party open source tool having high reliability
> Supporting high customizability with flexible configuration parameters. Perfkit can simulate user scenarios to the maximum extent, and well satisfy the client's performance testing requirements before cloudification. It also contains dozens of benchmark testing tools (such as sysbench and ycsb) and is continuously improved and optimized during the open-source projects of Google.
> Easy to install and use
> Cost effective. The cloud server resources can be released immediately after the testing.
4.2 In-service Performance Testing Tool
The major in-service performance testing tools include AliCloud Testing (the international version is being planned), Loadrunner, ab, jmeter, etc.
AliCoud Performance Testing: The performance testing tool platform developed by AliCloud is not only applicable for performance testing of AliCloud server, but also applicable
for performance testing of any cloud server connected to Internet. The tool has the following advantages:
> Professional: The tool offers distributed concurrency tests without an upper limit of pressure. It can simulate service scenarios to expose all system performance defects.
> Easy to use: The platform provides a press, sparing the need of installing software. The testing scripts simplify scenarios, saving both time and effort. Users can know how to use the tool quickly without any difficulty.
> Economical: With free quota, the tool can be used free of charge. The tool can be used to evaluate the capacity in advance, promoting service development. It also improves user experience, helping to increase market share.
> Reliable: The tool supports disaster recovery, having the reliability of up to 99.99%. The testing results are real and accurate. The tool also offers multiple security measures to ensure the data security.
It is a professional performance testing tool with powerful functions, but it is expensive and not easy to use.
ab: It is a free open-source performance testing tool with simple functions. It cannot be used to test complicated services or used in hybrid service scenarios, and the
simulation is untrue.
It is a free open-source performance testing tool with many functions, but it cannot be used for large-scale distributed tests, and is error prone.
5 5-Minute Course for Perfkit for AliCloud
pls click How to test cloud server-Part 2 to continue your reading of this article
[Cloudy edited the post at Dec 11, 2015 18:28 PM]