MSE online configuration tag push

Background

In the micro-service scenario, the full-link grayscale, as a low-cost new function verification method, has been more and more widely used. In addition to the grayscale of micro-service instances and traffic, the configuration items in micro-service applications should also have corresponding grayscale capabilities to meet the requirements of grayscale applications for special configurations.

Why do I need to configure label push

Starting from the full link gray scale

In the micro-service scenario, the grayscale publishing of applications faces new challenges. Unlike the single architecture, which packages the application as a whole and then releases the test version, microservice applications are often composed of multiple services. These services are usually developed independently by different teams. A new function usually only involves some services. When testing new features, we only need to publish these services. In order to make the micro-service application run normally, it is also necessary to design a scheme that allows the grayscale traffic to pass through other services that do not need to be published.

This function usually has two solutions: physical environment isolation and logical environment isolation. The former needs to build a network isolated and resource-independent environment for each grayscale environment. In order to ensure the normal operation of the application, it also needs to do redundant publishing for services and various middleware that are not grayscale, as shown in the figure:

There is a lot of waste of machine resources in the physical environment isolation scheme. Therefore, the industry generally adopts the latter, that is, the way of logical environment isolation, as the full-link grayscale solution. It only needs to deploy grayscale services. By calling the flow control on the link, grayscale traffic can flow between the grayscale environment and the formal environment, so as to realize the normal operation of grayscale microservice applications and help the business side to verify new functions. As shown in the figure:

Configure grayscale application scenarios

Many business scenarios may involve the application of configuring grayscale capability. Here are several typical scenarios.

• Canary release

The canary release has been widely implemented in the industry and is a common means of new version release. The canary press conference divides the traffic proportionally. At the beginning, it allocates a small proportion of the traffic for the new version of the instance. After a period of operation, it confirms that the new version is running normally, and then gradually increases the proportion of the traffic allocated until the final full flow cutoff. In this way, we can control the impact area and improve the stability of the system when the new version has problems.

Canary release is usually achieved through flow dyeing and machine marking. The new version of the machine is marked with the canary version, and part of the traffic is marked with the canary version, which eventually evolved into a full-link grayscale solution. The configuration items in the canary version of the application may need to use different configuration values from the old version, which requires the ability to configure grayscale.

• New functions online

When the change involves a large function going online, the stability of the function is often verified by gradually increasing the quantity. A typical scaling method is white list, that is, users/devices configured in the white list can use new functions, while users not in the white list still use the old version. After running online for a period of time, collect feedback from users on the whitelist, optimize the functions and gradually increase the number of users/devices in the whitelist, and release the functions in full after reaching the final stable state.

Users/devices from the whitelist will be marked with special marks and routed to the grayscale environment. If the configuration value in the new function needs to be different from that in the old version, the configuration grayscale needs to be used synchronously.

• Database migration

Database migration is also a common problem in business development. With the rapid growth of business, the original database may no longer meet the future business needs in terms of capacity/performance, so database migration is required. In order to ensure the stability of the migration process, migration is usually gradual. In this process, some traffic will be written to the new database and some traffic will be written to the old database. After the migration is completed, all traffic will be switched to the new database. During the migration process, we can operate on different databases through flow coloring and gray scale configuration.

Problems and solutions

Microservice applications usually introduce the configuration center for configuration management, which provides dynamic configuration push capability, so that applications can dynamically change the running logic without restarting. However, the management dimension of the configuration center is only the configuration item itself, and it cannot sense the environment information of the service instance coming to get the configuration, that is, it cannot distinguish whether the instance requesting the configuration is a formal environment instance or a grayscale environment instance. In this context, if a configuration needs to use different values in the formal environment and grayscale environment, they must be used as different configuration items in the configuration center. We may need to write the following code:

If there are multiple configuration items with different configuration values in the grayscale environment, such code needs to be repeated several times. In more extreme scenarios, if there are more than one set of grayscale environments under test, the configuration values in each set of environments are different, and the code responsible for obtaining the configuration items will be more complex. In addition, there are often multiple services in a grayscale environment, and each service needs to maintain a set of similar codes independently. The final solution is shown in the figure. The configuration values used by the same configuration item in different environments need to be actively differentiated in user applications.

The reason is that the configuration center is unable to perceive the environment information of the service instance, which makes us have to perform this task in the code instead of the configuration center, resulting in the intrusion of the environment information into the business code. Although we can reduce the complexity of using business code by encapsulating the code that gets the configuration, as long as the fact that the configuration center cannot perceive the environment information of the service instance exists, the intrusion of this environment information into business code cannot be avoided.

Function introduction

The newly launched configuration tag push function of MSE sinks the perception of environment information under the configuration management scenario to the platform side, which is the responsibility of the agent. Users can easily use the configuration push capability in the full-link grayscale scenario by simply accessing MSE, and the cumbersome environment information detection logic in the business code is eliminated. As shown in the figure:

The functions provided by the configuration label push include:

• Configuration management of label dimension

You can view various configuration items in the application in the configuration list page. At present, MSE supports the collection of three types of configurations:

A. Use the annotation @ Switch tag configuration class provided by the SDK

B. Configuration items marked with Spring's annotation @ Value

C. The configuration class marked with the annotation @ ConfigurationProperties of SpringBoot

The configuration values of all service instances will be displayed under each configuration item. Users can intuitively see the grayscale environment of different instances through the tag name, and can also view the distribution of configuration values under different grayscale environments

• Application configuration push capability of label dimension

With "Push by Tag", users can easily push persistent new configuration values for all service instances in the specified grayscale environment. Persistence means that even if the application is restarted, the configuration items for the environment will not be lost.

• Dynamic marking of instances in the configuration scenario

In addition to setting labels for service instances through the marking method of MSE when the application is started, users can dynamically add/delete labels for service instances on the MSE console to adapt to configuration item management in different grayscale environments.

• Traceability of the whole configuration tag push process

MSE provides traceability capability for the whole process of label push, including instance label change records and label push records, to help users easily troubleshoot problems in the label push process.

Configure label push practice

Next, we will demonstrate the use process of configuring tag push through practice, which can be completed in three simple steps.

• Preparation

1. Connect applications to MSE microservice governance

2. Enter the MSE console and select the region

3. Go to the governance center>application governance, and enter the newly accessed application

• Step 1: Add a label

The first step to configure tag push is to set the tag for the service instance. The tag of the service instance can be set through - Alicloud.service.tag at startup, and can also be set dynamically on the MSE console.

Next, we will demonstrate the process of dynamic marking for service instances. Select Application Configuration>Tag List. All service instances under the current application will be displayed here. Click the "Add Label" button in the upper left corner, and in the pop-up node marking window, we can select a batch of service instances to mark. The user can filter the instances to be marked through the node IP. In addition to filtering by node IP, MSE provides the ability to filter service instances by host name. After selecting the machine to be marked, enter the name of the label and click "Add Label" to complete the machine marking.

• Step 2: Push by label

After marking the machine, we can push the configuration value for the specified label. Return to the application configuration>configuration list, select the "customName" configuration item, and click "push by tag". In the pop-up push window, we select the label to be pushed, set the configuration value to be pushed, and click Next: Value Comparison to see the difference between the new and old configuration items. Then click "Tag Push" to complete the configuration distribution.

You can see that the configuration item has been successfully pushed to two machine instances with the "gray" tag.

After that, we also perform the same operation on the "gray2" tag and push the "testGray2" configuration value.

The configuration value push for tags is persistent. The user only needs to operate once on the console. After that, when the service instance with the specified tag restarts, MSE will send the persistent configuration value to the application through the agent.

• Step 3: View the distribution of configuration values

Return to the application configuration>configuration list and select the "customName" configuration item. We can see that the two configuration values just pushed for the labels "gray" and "gray2" have taken effect.

Click "Value Distribution". In the pop-up window, you can see the distribution of the configuration value of the configuration item on all machine instances, as well as its persistence value under different labels.

Conclusion

This paper introduces the problems brought by the full-link grayscale scenario to the configuration management, introduces the MSE solution for this scenario, and shows the use process of configuration tag push through practice. In the future, MSE will also do more exploration on configuration governance to help users better solve the problems in micro-service configuration management and improve the stability of micro-service applications.