Application and Change-Based Delivery Model - Alibaba DevOps Practice Part 16

This article is from Alibaba DevOps Practice Guide written by Alibaba Cloud Yunxiao Team

This article introduces some of Alibaba's best practices in the delivery phase, including:

The application and change-centered delivery process
Check items and checkpoints based on changes
R&D mode selection based on application features

Application and Change

Before going into specific practices, let's introduce Alibaba's application and change model:

An application is the carrier of R&D activities. It includes a series of elements, such as code base, deployment environment, configuration items, change management, release pipeline, and O&M. The entire development and delivery process of a requirement can be completed within the application. You can set different roles for an application. Each role has responsibilities in all stages of the R&D process, including the role in charge of releasing the online environment and the role that has the permission to modify the test configuration.

Changeis an abstract concept and an important part of an application. All modifications to online services can be called changes. The changes belong to a specific application. A change can contain one or more different types of content. The most common type of content change is a code change.

A change has a lifecycle. A typical change lifecycle is developing > releasing > completed. If multiple application changes are being developed and released at the same time, a coordination mechanism is required. For example, a mechanism is required to determine whether to integrate them on a temporary integration branch or a frequently used development branch. We call this branch collaboration mode the R&D mode.

A change is usually created to implement requirements and repair defects. Multiple applications may need to be modified for only one requirement. That means you need to create changes for multiple applications. However, if you want to fix a simple defect, you may only modify one application, which only involves one change. Therefore, work items (requirements and defects) and changes are in a one-to-many relationship.

Let's look at the work items and changes from two perspectives:

The Perspective of Application Deployment: Changes to one or multiple applications are involved each time an application is deployed online. That means multiple work items may be involved.
The Perspective of Releasing Work Items: If you want to release a work item, multiple applications may need to be deployed in sequence.

The two perspectives overlap. Which perspective should a platform or tool focus on?

If it focuses more on applications. If you want to complete the release of a work item, the development team needs to coordinate the deployment sequence of multiple applications, which may incur certain communication costs.
If it focuses more on work items, the following scenario may occur. An acceptance environment may be deployed for a group of changes of one work item while the changes of other work items are still waiting to be implemented. Thus, the overall delivery throughput may be affected.

If you want to take both of the perspectives into account, you will inevitably implement the major version system or the train system, making the delivery cycle of work items longer.

Alibaba Cloud DevOps is an application-oriented tool. It mainly ensures the deployment process and quality of individual applications so the Development Team can arrange the deployment pace flexibly.

For example, if the release of a work item involves three changes to three applications, you can deploy the first change on Monday and the second change on Tuesday. These two changes will not cause any visible changes online. If the Product Team expects the function to be online on Thursday, you can deploy the third change and officially release the requirement. The separation of application deployment and application release reduces the risks of centralized deployment. The prerequisite requires a relatively complete automated quality assurance and checkpoint mechanism.

Change-Based Check Items and Checkpoints

Change is the carrier of delivery. Therefore, you can set up check items for a change to ensure the delivery quality. The most common check items include:

Whether the code review is passed
Whether the security scan is passed
Whether the unit test or UI test is passed

Based on these check items, quality checkpoints are usually planted on the following lifecycle nodes:

From Under Development to Being Released: Specific quality standards (such as unit test pass rate and coverage rate) must be met before the change is allowed to enter the being released state and enter the test environment for verification.
Released in an Environment: At this time, the change enters the integration pipeline and is built. However, more verification conditions must be met to be deployed to an environment. For example, the change can only be officially released if the security audit is passed.

The application owner can decide the checkpoints to be set up and the nodes to host these checkpoints. We recommend starting multiple check items at the same time to improve efficiency as soon as possible.

You can configure checkpoints globally for all applications on the release platform and allow some of the checkpoints to be enabled automatically. For example, if the platform detects that your code has dependencies with a JAR package that has security problems, you have to fix it. Otherwise, the online deployment will fail. This way, some global risks can be controlled.

The change-based and application-based release pipeline provides a framework for checking and checkpoint setup. Specific checks are provided by other specialized platforms, such as the testing platform, the security scanning platform, and the compliance scanning platform for specific businesses.

R&D Mode

As mentioned above, we need to adopt different R&D modes to enable collaborative development and the release of multiple changes in an application. The main difference between different R&D modes lies in the strategy of branch merging. Currently, Alibaba mainly adopts the following three R&D modes:

Master Branch Mode
Aone-Flow Branch Mode
Git Flow Mode

Application sizes vary, and the parallelism for requirement development is also different.

In Alibaba, many applications changes are not released in parallel. This type of application is suitable for master branch mode or Git Flow mode. Some applications have high parallelism of change, and the release pace may be different. In the industry, this problem is generally solved using a switch. In Alibaba, a branch mode called Aone-Flow is used.

In Aone-Flow branch mode, the typical workflow for developers is listed below:

After you commit changes, Aone-Flow will create an integration branch or reuse an existing branch to merge the changes to this integration branch automatically.
When you find that your change has problems and cannot be released, you can exit the release. Then, Aone-Flow creates a new integration branch automatically to merge other changes that need to be released.
After the integration branch gets deployed officially, the branch will be merged to the master branch. All changes to the integration branch are marked completed and tagged with Git Tag.
When rollback is required, you can use the Git revert command to roll back the online version and code based on system records. This can prevent you from accidentally releasing master code with defects.

Here is a simple comparison between the master branch mode and Aone-Flow mode:

	Master Branch or Git Flow Mode	Aone-Flow
Support for Parallel Feature Development	A functional switch is required, which costs a lot.	When entering/exiting release, you may need to resolve conflicts between branches repeatedly.
Support for Refactoring	After code modification, you must push the changes.	Refactoring is prone to causing conflicts, which may need to be solved repeatedly. The solution is to release the refactored changes online as soon as possible so they can be merged into master.
Support for "quick launch"	Code that passed automated tests is merged into the public branch, such as the master branch. When the release is needed, all these functions will be released. Even if you implement automated testing, there could still be a risk. This may cause people to be reluctant to release frequently, or the release mode becomes a release train mode.	You can select changes to be released during the release. Therefore, each change is not constrained by other changes. The release of a single change has a relatively lower risk. In addition, the aforementioned problem of "repeatedly resolving branch conflicts" will urge developers to release changes as soon as possible.

You can see the advantages and disadvantages of the master branch mode and Aone-Flow mode. In Alibaba, more than half of the teams have adopted the Aone-Flow mode to deliver features in a faster and more independent way.

Summary

After reading the article, take the Aone-Flow mode as an example and use the figure below to illustrate the phases necessary to launching a requirement:

It only draws three environments: daily, pre-release, and formal environments. There will also be a gray release environment and a safe production environment during the actual process to avoid online problems and failures.

Alibaba has also adopted some techniques in the development phase of changes to isolate test environments to help developers carry out feature-level isolation testing and joint commissioning. Please see the isolated test environment section for more information. We get a relatively safe requirement delivery process through the control and process.

Community

Application and Change-Based Delivery Model - Alibaba DevOps Practice Part 16

Application and Change

Change-Based Check Items and Checkpoints

R&D Mode

Summary

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Bastionhost

Managed Service for Grafana

Mobile Testing

DevOps Solution