Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog How Do We Implement Full-Process Grayscale Capability on the Database?

How Do We Implement Full-Process Grayscale Capability on the Database?

This article introduces full-process grayscale capability and explains common solutions for full-process grayscale capability on databases with specific examples.

By Shimian

What Is Full-Process Grayscale Capability?

Dependencies between services are complex in the microservice architecture. Sometimes a feature release depends on multiple services being upgraded and launched at the same time. We hope new versions of these services can be simultaneously enabled with low-traffic grayscale verification. This is the unique full-process grayscale scenario in the microservices model. The grayscale verification is performed on multiple different versions of services by building environment isolation from the gateway to the entire backend service.

During the release, we only need to deploy the grayscale version of the service. When traffic is transferred over the call, the gateway, middleware, and microservices that flow through the gateway identify the grayscale traffic and dynamically forward it to the grayscale version of the corresponding service, as shown in the following figure:

1

The preceding figure shows the effect of this scheme. We use different colors to represent the grayscale traffic of different versions. The microservice gateway and microservice need to identifies the traffic and make dynamic decisions according to governance rules. When the service version changes, the forwarding of this call will change in real-time. Compared with the grayscale environment built by using machines, this solution can save a lot of machine costs and O&M manpower but also help developers quickly and accurately control online traffic in real-time.

OpenSergo Traffic Routing Standards

Q: What is OpenSergo?

A: OpenSergo [1] is an open and general service governance standard oriented to a distributed service architecture and covers a full-process heterogeneous ecosystem. It forms a general service governance standard based on industry service governance scenarios and practices. The most important feature of OpenSergo is to define service governance rules in a unified set of configurations /DSL /protocols. It is oriented to a multi-language heterogeneous architecture to achieve full-process ecosystem coverage. No matter whether the language of microservices is Java, Go, Node.js, or other languages, whether it is standard microservices or Mesh access, from gateway to microservices, from database to cache, from service registration and discovery to configuration, developers can use the same set of OpenSergo CRD standard configuration to perform unified governance control for each layer without paying attention to the differences of various frameworks and languages. It can reduce the complexity of isomerization and full-process service governance control.

Q: Why did you introduce OpenSergo to me before learning about full-process grayscale?

A: OpenSergo defines a unified YAML configuration method to implement full-process service governance specifications for distributed architectures. While introducing specifications and standards, we can understand the implementation of technical details. At the same time, we can implement new components with OpenSergo standards.

Traffic routing is to move routes traffic with certain attribute characteristics to a specified destination. Traffic routing is an important part of traffic governance. Developers can implement various scenarios based on traffic routing standards (such as Grayscale release, Canary release, disaster recovery routing, and label routing).

Full-Process Grayscale Example:

2

Traffic routing rules (v1alpha1) are divided into three parts:

  1. Workload Label Rule: Label a group of workloads with the corresponding tags. This can be understood as the corresponding tags for the application or the corresponding storage layer, which is the database load (database and table).
  2. Traffic Label Rule: Tag traffic that has certain attribute characteristics with the corresponding
  3. Make matching routes based on workload labels and traffic labels, and route traffic with specified labels to the matching workloads.

Traffic Labeling

We need to label traffic with certain attributes.

Let’s assume users in the Shenzhen region need to be phased out to the new homepage to test the user location=cn-shenzhen. The cn-shenzhen is located in the location header:

apiVersion: traffic.opensergo.io/v1alpha1
kind: TrafficLabelRule
metadata:
  name: my-traffic-label-rule
  labels:
    app: spring-cloud-zuul
spec:
  selector:
    app: spring-cloud-zuul
  trafficLabel: gray
  match:
  - condition: "=="    # Match Expression
    type: header       # Match the Attribute Type
    key: 'location'   # Parameter Name
    value: cn-shenzhen       # Parameter Value

With the preceding configuration, the location header is cn-shenzhen HTTP traffic, marked with the grayscale mark, which means this traffic is grayscale traffic.

Label the Workload

It is generally necessary for the business in a business system that uses Nacos as service discovery to determine the labeling method based on the microservice framework it uses. If a Java application uses the Spring Cloud microservice development framework, we can add corresponding environment variables to the business container to add labels. For example, if we want to add a version grayscale label to a node, add a traffic.opensergo.io/label: grayscale to the business container, so the framework will add a grayscale label to the node when registering it with Nacos.

We can use WorkloadLabelRule CRD to label some complex workload tagging scenarios (such as database instances and cache instance labels). Example:

apiVersion: traffic.opensergo.io/v1alpha1
kind: WorkloadLabelRule
metadata:
  name: gray-sts-label-rule
spec:
  workloadLabels: ['gray']
  selector:
    db: mse-demo
    table: mse_demo_table_gray

Common Solutions for Full-Process Grayscale Capability on Databases

Scenario 1: Shadow Library

Each one independently maintains a set of independent libraries. Assuming that the name of the library in the baseline environment is mse-demo, the traffic of the grayscale environment can be mapped to the mse-demo-grayscale library. We establish the shadow library corresponding to the environment traffic on the same instance. We maintain the connection pool of each library connection in the business and select the connection of the corresponding shadow library to access according to different traffic standards. This achieves the effect of isolating data from the baseline environment database, thus avoiding the pollution of the baseline environment database caused by the data generated by the grayscale environment traffic.

Scenario 2: Shadow Table

Similar to the shadow library, for the shadow table, the corresponding shadow table is created on the same database on the same instance. In the process of executing SQL, we parse and modify the SQL of grayscale traffic and realize that the SQL of different environment traffic respectively access the corresponding table. Assuming that the name of the table in the baseline environment is mse_demo_table, the traffic in the grayscale environment can be mapped to the mse_demo_table_grayscale table. This enables the isolation of grayscale data and baseline environment data tables.

MSE Database Full-Process Grayscale Capability

MSE [2] provides a data isolation solution that allows users to implement a full-process grayscale capability at the database level without modifying any business code. The following describes the ability of MSE to implement full-process grayscale capability through the shadow table solution based on MySQL data storage.

Prerequisites

  • Application Access to MSE
  • Deploy a Demo Application

You can deploy applications A, B, and C in Alibaba Cloud Container Service. Each application is deployed with a base version and a grayscale version. You can deploy a Nacos Server application for service discovery. Please see the tutorial to complete application deployment [3]

3

Demo Application Introduction: The C application in this demo executes the following statement to the database:

CREATE TABLE `mse_demo_table` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3

The database table creation statements involved:

CREATE TABLE `mse_demo_table` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3
  • To create a shadow table, the database tables involved in our demo are mse_demo_table since we need to create a grayscale environment. Therefore, we need to create a mse_demo_table_grayscale table in advance.
CREATE TABLE `mse_demo_table_gray` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3
;

Step 1: Configure Full-Process Grayscale Rules

You need to configure the full-process release of MSE. Please see Configure full-process grayscale [4] for more information.

Create the following lane rules:

4

Step 2: Configure Full-Process Grayscale Capability of the Database

  • We need to configure the following environment variables to enable/configure the full-process grayscale capability of the database.
Environment Variable Key Field Meaning Description
profiler.micro.service.db.trace.enable true on /false off. Default value: false. Database Full-process Grayscale Switch
profiler.micro.service.db.trace.tables By default, an empty string indicates that all tables are phased out. You can enter multiple tables and separate them with commas (,)
such as mse_demo_table,user_table.		 It indicates which tables are chosen to enable the grayscale capability. If you do not specify this parameter, all database tables will be enabled with grayscale capability.

Step 3: Result Verification

We initiate grayscale requests and find that all traffic requests access the grayscale environment.

curl -H "location: cn-shenzhen" http://106.14.XX.XX/a
Agray[172.18.XX.XX] -> Bgray[172.18.XX.XX] -> Cgray[172.18.XX.XX]%

We use the following SQL to query the shadow table:

SELECT * FROM `mse_demo_table_gray`

It is found that the data in the grayscale environment is inserted into the shadow table.

Not Limited to Full-Process Grayscale Capability

So far, MSE supports Cloud-Native Gateway, ALB, APISIX, Apache Dubbo, Spring Cloud, RocketMQ, and databases. We implement traffic isolation at the data level using shadow tables. In the next step, we will further productize this capability. The full-process grayscale capability will support the cache-level capability.

Service governance is necessary after the transformation of microservices has reached a certain stage. New problems continue to emerge in this process.

  • Does service governance have other capabilities apart from the full-process grayscale capability?
  • Is there a standard definition of service governance capability? What is included in service governance capability?
  • Are there any full-process best practices or standards in multi-language scenarios?
  • How can heterogeneous microservices be managed in a unified manner?

When we were exploring service governance and connecting with other microservices, we found that there are a lot of troubles caused by different governance systems, and the cost of getting through two (or multiple) governance systems was high. To this end, we proposed the OpenSergo project. OpenSergo wants to solve the problem of fragmentation and interoperability of different frameworks and different languages in microservice governance.

The OpenSergo community is cooperating with various communities to discuss and define unified service governance standards. We welcome any developers, communities, and enterprises interested in joining the co-creation of OpenSergo service governance standards.

References

[1] OpenSergo:
https://opensergo.io/

[2] Microservice Engine (MSE):
https://www.alibabacloud.com/product/microservices-engine

[3] Deploy demo applications:
https://www.alibabacloud.com/help/en/microservices-engine/latest/configure-full-link-grayscale-2#h3-url-3

[4] Configure an end-to-end canary release:
https://www.alibabacloud.com/help/en/microservices-engine/latest/configure-full-link-grayscale-2

Database Microservices Computing Grayscale MSE Canary Release Grayscale Release Microservice engine OpenSergo
0 1 0
Share on

Read previous post:

Kubecost is Integrated with Alibaba Cloud Container Service for Kubernetes (ACK)

Read next post:

Java Agent Exploration – appendToSystemClassLoaderSearch Problems

Alibaba Cloud Native Community

480 posts | 48 followers

You may also like

Comments

Alibaba Cloud Native Community

480 posts | 48 followers

Related Products

More Posts by Alibaba Cloud Native Community

See All