Microservices Engine (MSE) instances support the canary release feature. If you enable the canary release feature for MSE instances, you can release configurations to a small percentage of nodes to verify the configurations before you officially release the configurations. This reduces the risks caused by configuration pushes. This topic describes how to configure the canary release feature for an MSE Nacos instance.
Prerequisites
An MSE Nacos instance is created. For more information, see Create a Nacos engine.
Only MSE Nacos Developer Edition and Professional Edition support the canary release feature.
We recommend that you use the Nacos client in version 2.x.
Background information
When you use MSE instances to manage application configurations in a centralized manner and you modify a configuration of the application, the new configuration overwrites the original configuration and is pushed to all nodes of the application. If the configuration is invalid, the entire application may fail to run due to the configuration push. This causes high risks.
To address the preceding issue, we recommend that you perform a canary release on several nodes of the application after you modify a configuration of the application and test the nodes to verify the configuration. If the test is passed, you can gradually perform canary release on other nodes. This helps reduce the risks caused by configuration changes.
During the configuration change, you need to select canary nodes to implement a canary release. MSE Nacos instances support IP address-based canary release and tag-based canary release.
IP address-based canary release
IP address-based canary release is a basic canary release method. If you use this method, you can select a list of IP addresses to determine the machines for which a canary release needs to be implemented. In some small-sized business systems, this method can meet the canary release requirements, greatly reduce the risk of configuration pushes, and reduce the failures caused by configuration errors.
Create a beta configuration
Log on to the MSE console, and select a region in the top navigation bar.
In the left-side navigation pane, choose Microservices Registry > Instances.
On the Instances page, click the name of the instance.
In the left-side navigation pane, choose Configuration Management > Configurations.
On the page that appears, find the desired configuration, and click Edit in the Actions column. In the Edit Configuration panel, select IP-based Canary Release for Release Type.
Select one or more IP addresses to which a new configuration is pushed from the Application Node IP Address drop-down list.
You can also manually enter IP addresses. The autocomplete feature is supported when you manually enter IP addresses.
NoteIP addresses are the IP addresses of the nodes that subscribe to the configuration. Separate multiple IP addresses with commas (,).
Click Canary Release after you modify the configuration. In the Comparison of Configuration Content dialog box, confirm the information in Current Official Version Content and Current Release Content, and click Release.
View a beta configuration
Log on to the MSE console, and select a region in the top navigation bar.
In the left-side navigation pane, choose Microservices Registry > Instances.
On the Instances page, click the name of the instance.
In the left-side navigation pane, choose Configuration Management > Configurations.
On the page that appears, find the configuration for which a beta release is performed, and click Edit in the Actions column.
In the Edit Configuration panel, click the Beta(IP) tab to view the information about the beta release.
Related operations
Cancel a beta release: On the Beta(IP) tab of the Edit Configuration panel, click Stop Canary Release to cancel the beta release.
Perform an official release: On the Beta(IP) tab of the Edit Configuration panel, select Full Release for Release Type. In the Comparison of Configuration Content dialog box, confirm the configuration information, and click Full Release. An official release is performed on the configuration, and the current beta release is discontinued.
When you perform an IP address-based canary release, the following issues may occur:
If you use Nacos client 1.x, the server obtains the IP address of the client by using a Server Load Balancer (SLB) instance. In this case, the server may not obtain the accurate IP address of the client.
When a server is deployed in the Kubernetes architecture, if a node is rebuilt, its IP address may change. In this case, the IP address-based canary version may become invalid.
Tag-based canary release
MSE Nacos 2.2.3.3 and later support tag-based canary release. You can configure tags for application nodes on the client and implement a canary release based on tags.
To implement a tag-based canary release, you must upgrade the version of your MSE Nacos instance to 2.2.3.3 or later.
Open source Nacos client 2.3.2 and later support custom tags for canary release. You must upgrade the version of the open source Nacos client to 2.3.2 or later.
To use environment variables to add tags, you must upgrade the version of the Nacos client to 2.4.2 or later.
Configure tags for application nodes on the client
You can configure key-value pairs as tags for application nodes by using .properties files, Java virtual machine (JVM) parameters, or environment variables. If multiple tags have the same key, the tags are prioritized in the following order: Tags in .properties files > Tags specified by JVM parameters > Tags specified by environment variables. nacos.config.gray.label
is the default built-in tag for canary release in Nacos.
//1. Pass tags in a .properties file.
Properties properties = new Properties();
properties.put(PropertyKeyConst.SERVER_ADDR, "your endpoint");
properties.put("project.name", "your app name");
properties.put("nacos.config.gray.label","yourgrayname");
// 2. Configure JVM parameters.
Configure the startup parameter -Dnacos.config.gray.label=yourgrayname.
// 3. Configure environment variables.
Configure the environment variable nacos_config_gray_label=yourgrayname.
String dataId = "gray_test_dataid";
String group = "test-group";
configService.addListener(dataId, group, new Listener() {
@Override
public Executor getExecutor() {
return null;
}
@Override
public void receiveConfigInfo(String configInfo) {
System.out.println("receiveConfig:" + configInfo);
}
});
Implement tag-based canary release on the server
View listener tags
After tags are configured for an application on the client, you can view the listeners of a configuration on the server and the tags carried by each listener.
Implement tag-based canary release
On the details page of the configuration, click Edit Configuration. In the Edit Configuration panel, select Tag-based canary release for the Release Type parameter, and then select an existing key-value pair of application nodes as a tag. You can view the number of application nodes that match the selected key-value pair.
After a tag-based canary version is released, you can click the Listening Query tab on the details page of the configuration to view the configuration version of the current client. You can view the details of the current canary version on the details page of the configuration.
After a canary version is released and stabilizes, you can expand the value range of the tag to gradually expand the scope of the canary release until full release. After you click Full Release in the Edit Configuration panel, the canary versions that are released become invalid. If a business exception occurs during the canary release, you can click Stop Canary Release to cancel the canary release. The system automatically rolls back to the original configuration.
Additional information
Advanced features of tags in an application
In addition to the nacos.config.gray.label
tag, you can configure a tag with multiple key-value pairs. You can also configure custom tags by using a Service Provider Interface (SPI) named Collector. This way, you can specify tags in a flexible manner.
Configure a tag with multiple key-value pairs
Nacos allows you to inject multiple key-value pairs into an application tag. You can specify the
nacos.app.conn.labels
parameter in .properties files and JVM parameters or specify the nacos_app_conn_labels parameter in environment variables. Sample format of the nacos.app.conn.labels parameter:nacos.app.conn.labels="k1=v1,k2=v2,k3=v3"
.// 1. Pass tags in .properties files. Properties properties = new Properties(); properties.put(PropertyKeyConst.SERVER_ADDR, "your endpoint"); properties.put("project.name", "your app name"); properties.put("nacos.app.conn.labels","app=demo,site=hangzhou-c,otherkey=othervaue"); // 2. Configure JVM parameters. Configure the startup parameter -Dnacos.app.conn.labels="app=demo,site=hangzhou-c,otherkey=othervaue". // 3. Configure environment variables. Configure the environment variable nacos_app_conn_labels="app=demo,site=hangzhou-c,otherkey=othervaue". NacosConfigService configService = new NacosConfigService(properties); String dataId = "gray_test_dataid"; String group = "test-group"; configService.addListener(dataId, group, new Listener() { @Override public Executor getExecutor() { return null; } @Override public void receiveConfigInfo(String configInfo) { System.out.println("receiveConfig:" + configInfo); } });
Implement the SPI of custom tags in an application
The Nacos client defines the SPI of custom tags in an application. To implement the SPI of custom tags in an application, you must implement the
com.alibaba.nacos.common.labels.LabelsCollector
interface and publish the specific implementation class as a service.package your.demo.test; import com.alibaba.nacos.common.labels.LabelsCollector; import java.util.HashMap; import java.util.Map; import java.util.Properties; /** * TestLabelsCollector. * * @author yourname */ public class TestLabelsCollector implements LabelsCollector { @Override public String getName() { return "testlables"; } @Override public Map<String, String> collectLabels(Properties properties) { Map<String, String> labels = new HashMap<>(); labels.put("test", "implements your lables logic"); return labels; } @Override public int getOrder() { return 1; } }
Support for multiple canary versions
A configuration may have multiple versions, including the official version, IP-based canary version, and tag-based canary version. Multiple versions can be released during a tag-based canary release. For multiple versions of a configuration, the configuration queries and pushes of the Nacos server are matched based on the following order of precedence: IP-based canary version > Tag-based canary version> Official version.
When you process multiple tag-based canary versions, the system sorts the versions based on the value of the priority field. If a version has a larger field value than other versions, the version has a higher priority. If the field value is the same for the versions, the system sorts the versions based on the version name. We recommend that you specify priorities for different canary versions.
If a tag is configured for an application node, the system preferentially checks whether the node has an IP address-based canary version. If the IP address-based canary version exists, the IP address-based canary version of the current configuration is returned. If the IP address-based canary version does not exist, the system attempts to match a tag-based canary version based on priorities. If the match is successful, the tag-based canary version of the current configuration is returned. If no tag-based canary version is matched, the official version of the current configuration is returned. You can view the currently matched configuration version of each application node on the Listening Query page in the MSE console.
MSE Nacos versions later than 2.3.0 support multiple canary versions of a configuration. The maximum number of canary versions that can be released for a configuration is limited. By default, a maximum of five tag-based canary versions can be released. If you want to release more than five tag-based canary versions, the additional canary versions are blocked.
Tag planning in an application
We recommend that you plan the tags in an application based on the business impact scope. This way, canary release can be gradually performed and risks caused by configuration changes are reduced. Best practices for tag planning:
Use application names as tags. In this case, perform canary release on several non-critical applications. After the canary version stabilizes and no exceptions are detected, gradually perform the canary release on critical applications.
Isolate traffic on the upstream and downstream servers by managing inbound traffic on the business side. For example, you can assign a batch of servers to the internal users of your company, configure specific tags for the servers, and implement canary release on the servers. This way, the canary version can be used by the internal users of the company.
Format of tags in an application
The key-value pair of a tag can contain letters, digits, underscores (_), hyphens (-), and periods (.). Tags in other formats are ignored. If you specify multiple key-value pairs by using the nacos.app.conn.labels
parameter, the key-value pairs must be in the "k1=v1,k2=v2,k3=v3"
format. If a key-value pair in the "k1=v1,k2"
format is used, the k2 parameter is ignored, and the key-value pair is parsed as k1=v1
.