This article is from Alibaba DevOps Practice Guide written by Alibaba Cloud Yunxiao Team
We have functional and performance requirements for deliverables to ensure the software delivery quality. These requirements can be reflected from the data generated during the delivery process, including code review data, security scanning data, and regression test results. All these types of data are included in the deliverables (or product). We call these types of data the metadata of the product.
Metadata refers to the kind of data that does not change once generated by a system. As metadata is tamper-proof and traceable, it becomes the necessary basic data in the release process.
For example, from the perspective of the architecture, Alibaba’s middle platform applications rely on internal packages from many business teams, but the quality of these packages is hard to control. How can we solve this problem?
A method is to present data in a more panoramic and three-dimensional way from the dimensions of single applications and application dependency trees. Let’s take code review as an example. Products of middle platform applications contain many internal packages developed by business teams. However, when reviewing middle platform applications, you can only see that the version numbers of internal packages in the pom file have been changed. You cannot see any code changes. Reviewers need to see the code changes behind these version numbers and the information related to these code changes, such as related requirements, code detection results, and unit test results.
Most of the dependencies during runtime are determined when an application is built. Many runtime problems are caused by dependencies. The best way is to make the dependency tree generate values, thus transferring risks.
In addition to the dependency tree obtained in the building process, we have other data, such as the quality data generated during testing and the security data generated during security scanning. All of the data is stored in the metadata center. Then, the data is used for automated checkpoint planting in the delivery process through the control policy center. This process usually includes three stages: visibility, controllability, and reliability.
The delivery efficiency can finally be improved from the stage of visibility to the stage of controllability and then to the stage of reliability. Meanwhile, metadata and rules continue to evolve and are gradually accumulated into a knowledge base, becoming one of the most important assets of the company.
Reliable Release Based on Metadata
The preceding figure shows the architecture of reliable release. Reliable release is metadata-centric and product-based. It continuously produces and consumes metadata and works with various automatic access control rules to ensure a continuous, reliable, and safe release.
Metadata is classified into two types: metadata of the package and data that has blood relationships with the package.
The metadata of a package includes basic package information, such as building information, quality data, and security scanning data. It also includes third-party information written using the metadata protocol.
Metadata Details
A product, such as a book or a movie, can be scored. Let’s take JAR packages of an internal library as an example. The score can help users identify and reference the JAR package with the highest quality.
The score includes a system score and a user score. The score is specifically for one version of a product.
The full system score is about ten points, which are mainly given based on the following rules:
The user score is the developers' evaluation of the product. Some products have high system scores, but the interface design is unreasonable with many dependencies. Developers can give a lower score to promote the continuous improvement of the products.
A large product is composed of many smaller products. Products are involved in multiple relationships, such as dependency and combination. The blood relationships of a product refer to the number of dependencies of the product. We recommend using the dependency version with a high score.
Recommend Version Upgrades Automatically
In addition to being used for access and exit on each node in the delivery process, metadata can be used for product governance. Product governance is based on two capabilities of metadata, insights and trends.
The following sections use a JAR package (an internal package) as an example.
Indicators of an internal package include dependency depth, the total number of dependencies, and the total number of versions. Team leaders can find their owner teams in the insight report, find the internal package that needs to be optimized, and perform targeted optimization activities.
Key objects to be handled are generally internal packages with a high level of dependency depth and a large number of dependencies and versions. As shown in the following figure, "com.taod:feent” is a typical object that needs to be handled first.
Find Key Products
Due to the depth of dependencies, the total number of dependencies of an internal package refers to the total number of times this package is referred by applications in the last released GA version. Therefore, we need to select an internal package version that meets the requirements, as shown in the following figure:
Determine the Culprit
Next, check the details of the dependency tree and analyze it again. It is possible that our internal package is also referred to by an internal package that is a key object to be handled. We can check the dependency tree details to know whether there is such a package.
If there is such a package, handle it in the following ways:
The trend report is mainly reflected from the following two aspects:
Trend Analysis
We will scan the product being released and its dependencies based on the faults that have occurred in the past and requirements, such as security and testing quality. Based on the metadata analysis of the product, the released risks are classified, and the users are prompted to know how to fix them.
Risk Notification in the Process
Risk Details
Risks are classified into high, medium, and low risks based on their severity levels. Among them, high and medium risks require special attention. The risk levels are defined below:
High
Medium
A checkpoint will be planted at the last time of official release. aAcording to the risk repair costs, this can make risk detection as early as possible.
The Last Check
The core difference between code-based delivery and product-based delivery is that products are complete and cannot be changed. Thus, a continuous delivery system built based on products and their metadata can achieve reliable release, significantly improving release efficiency and reducing release risks.
Improving the Building Efficiency - Alibaba DevOps Practice Part 17
How to Back Up and Recover Your Data Securely and Efficiently with Alibaba Cloud
863 posts | 196 followers
FollowAlibaba Clouder - November 27, 2020
Alibaba Cloud Community - February 18, 2022
Alibaba Cloud Community - February 5, 2022
Alibaba Cloud Community - March 8, 2022
Alibaba Clouder - December 15, 2020
Alibaba Cloud Community - February 6, 2022
863 posts | 196 followers
Follow
Alibaba Cloud Flow
An enterprise-level continuous delivery tool.
Learn More
DevOps Solution
Accelerate software development and delivery by integrating DevOps with the cloud
Learn More
ChatAPP
Reach global users more accurately and efficiently via IM Channel
Learn More
Secure Content Delivery Solution
Accelerate static and dynamic web content in a fast, reliable, and safe way using Secure DCDN (Dynamic Route for CDN)
Learn MoreMore Posts by Alibaba Cloud Community
