Although rapid iterative development has become a popular approach to software development, many development teams are unsure of the implementation of QA through this model. Traditional QA practices, such as manual tests and regression tests, cannot adapt to the iterative rhythm, which requires multiple online launches in a day. Development teams often face a dilemma: sacrificing quality for speed, or regularly fixing the release while compromising business agility.
What are some QA best practices to minimize risks while keeping up with the pace of rapid iterative development? Let us explore the successful QA practices of the RippleTek wireless technology team from 2014 to 2016.
The introduction of specialized testing personnel brings about additional communication costs and creates bottlenecks when multiple systems are headed for parallel release. Therefore, RippleTek did not designate specialized testing personnel but conducted functional tests with developers. The pre-launch validation process is described below:
When the merge request passes the code review and is eligible for launch, developers can launch the feature online by themselves. To reduce risks while improving efficiency and comfort, the entire launch action only requires a single click by the developer in the CI system. This will launch the feature online fully automatically. In addition to the one-click deployment, the O&M infrastructure should provide the following basic functions:
In principle, it is best to avoid rollbacks unless a severe problem that impacts availability occurs and no other options are available. Despite the rarity of rollbacks, your design should support rollback and any changes made should be as retractable as possible. If developers discover an unexpected problem following the launch of a new product version, they can revert the product to the previous version with only one click.
RippleTek uses O&M bots for collecting and analyzing system logs, and monitoring the trends of the number of error logs. When the number of error logs of a service increases sharply, the system sends an alarm to the service developers and O&M personnel. In this manner, developers will receive alarms if there are problems with the new code. Within a few minutes after the launch, the developers can decide whether to fix the product or to perform a version rollback.
If the change released in a basic function module affects the global function, you can adopt a gray release to minimize risks. In gray release, developers initially use the new code with a small amount of traffic. The running status of the code is observed for a specified period. After confirming that the new code does not experience exceptions, you can then apply it to the entire network.
Data-driven QA is a testing method that constantly monitors the current system status to detect for anomalies. In data-driven QA, the implementation effects of new and old versions of the code are compared through observation, analysis, and statistics.
Based on this definition, the Error Monitoring and Push Alarms function can be categorized as a data-driven QA method. Other commonly used data-driven QA methods include:
In rapid iterative development, QA should not be an independent process before business changes or releases. The ideal QA process must permeate development, O&M, and data analysis processes. With adaptive QA practices, your online production environment can support dozens of releases a day while mitigating risks.
Alibaba Clouder - April 20, 2017
Alibaba Clouder - September 1, 2020
Alibaba Clouder - March 8, 2017
Alibaba Clouder - December 25, 2020
Hiteshjethva - March 5, 2021
Alibaba Clouder - September 9, 2020
More Posts by Alibaba Clouder