Solving Data Consistency Problems under Microservice Architecture

With the rapid development of business, the application monolithic architecture exposes many problems such as poor code maintainability, low fault tolerance, difficult testing, and poor agile delivery capabilities. Microservices emerge as the times require. The birth of microservices solves the above problems on the one hand, but introduces new problems on the other hand. One of the main problems is: how to ensure the consistency of business data between microservices.

This article will use a commodity procurement business to see how to ensure the data consistency of the business through Fescar under Dubbo's microservice architecture. In the examples described in this article, both Dubbo and Fescar's registration and configuration service centers use Nacos. Fescar 0.2.1+ started to support Nacos registration configuration service center.

business description
The business of users purchasing goods, including 3 microservices:

Inventory Service: Deducts the stock quantity of a given item.
Order Service: Generate orders based on purchase requests.
Account Services: User account deductions.
The business structure diagram is as follows:


Inventory Service (StorageService)


public interface StorageService {
/**
* deduct storage count
*/
void deduct(String commodityCode, int count);
}
OrderService

public interface OrderService {
/**
* create order
*/
Order create(String userId, String commodityCode, int orderCount);
}
AccountService (AccountService)

public interface AccountService {
/**
* debit balance of user's account
*/
void debit(String userId, int money);
}
Note: The above three microservices are all deployed independently.

8 Steps to Data Consistency
Step 1: Initialize MySQL database (requires InnoDB storage engine)

Modify the connection information corresponding to StorageService, OrderService, and AccountService in resources/jdbc.properties.

jdbc.account.url=jdbc:mysql://xxxx/xxxx
jdbc.account.username=xxxx
jdbc.account.password=xxxx
jdbc.account.driver=com.mysql.jdbc.Driver
# storage db config
jdbc.storage.url=jdbc:mysql://xxxx/xxxx
jdbc.storage.username=xxxx
jdbc.storage.password=xxxx
jdbc.storage.driver=com.mysql.jdbc.Driver
# order db config
jdbc.order.url=jdbc:mysql://xxxx/xxxx
jdbc.order.username=xxxx
jdbc.order.password=xxxx
jdbc.order.driver=com.mysql.jdbc.Driver
Step 2: Create undo_log (for Fescar AT mode) table and related business table

Relevant table building scripts can be obtained under resources/sql/. Execute the business table building script in dubbo_biz.sql in the corresponding database, and execute the undo_log.sql table building script in each database.

CREATE TABLE undo_log (


`id` bigint(20) NOT NULL AUTO_INCREMENT,
`branch_id` bigint(20) NOT NULL,
`xid` varchar(100) NOT NULL,
`rollback_info` longblob NOT NULL,
`log_status` int(11) NOT NULL,
`log_created` datetime NOT NULL,
`log_modified` datetime NOT NULL,
`ext` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_unionkey` (`xid`,`branch_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `storage_tbl`;
CREATE TABLE `storage_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`commodity_code` varchar(255) DEFAULT NULL,
`count` int(11) DEFAULT 0,
PRIMARY KEY (`id`),
UNIQUE KEY (`commodity_code`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `order_tbl`;
CREATE TABLE `order_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` varchar(255) DEFAULT NULL,
`commodity_code` varchar(255) DEFAULT NULL,
`count` int(11) DEFAULT 0,
`money` int(11) DEFAULT 0,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `account_tbl`;
CREATE TABLE `account_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` varchar(255) DEFAULT NULL,
`money` int(11) DEFAULT 0,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Note: It is necessary to ensure that each physical library contains the undo_log table. Here, one physical library can be used to represent the independent logical library corresponding to the above three microservices.

Step 3: Introduce Fescar, Dubbo and Nacos related POM dependencies


0.2.1
2.6.5
0.0.2



com.alibaba.fescar
fescar-spring
${fescar.version}


com.alibaba.fescar
fescar-dubbo-alibaba
${fescar.version}


dubbo
org.apache.dubbo




com.alibaba
dubbo
${dubbo.alibaba.version}


com.alibaba
dubbo-registry-nacos
${dubbo.registry.nacos.version}

Note: Due to the compatibility problem between apache-dubbo and dubbo-registry-nacos jar, it is necessary to exclude the dependency of apache.dubbo in fescar-dubbo and manually introduce alibaba-dubbo. Subsequent apache-dubbo (2.7.1+) will be compatible with dubbo -registry-nacos. In Fescar, fescar-dubbo jar supports apache.dubbo, and fescar-dubbo-alibaba jar supports alibaba-dubbo.

Step 4: Microservice Provider Spring configuration

Configure the following three microservice Spring configuration files (dubbo-account-service.xml, dubbo-order-service and dubbo-storage-service.xml ) respectively:

Configure Fescar proxy data source






Here, you need to use com.alibaba.fescar.rm.datasource.DataSourceProxy to wrap the Druid data source as a direct business data source. DataSourceProxy is used for intercepting and parsing business SQL and interacting with TC to coordinate transaction operation status.
Configure the Dubbo registry

Configuring Fescar GlobalTransactionScanner




The first parameter of the construction method here is the business custom applicationId. If you deploy multiple microservices on a single machine, you need to ensure that the applicationId is unique.
The second parameter of the constructor is the logical grouping of Fescar transaction services, this Group by configuration center configuration item
service.vgroup_mapping.my_test_tx_group is mapped to the corresponding Fescar-Server cluster name, and then the available service list is obtained according to the cluster name.grouplist.

Step 5: Transaction initiator configuration

Configure the following configuration in dubbo-business.xml:

Configure the Dubbo registry
Same as Step 4
Configuring Fescar GlobalTransactionScanner
Same as Step 4
Add the @GlobalTransactional annotation to the transaction initiator service method

@GlobalTransactional(timeoutMills = 300000, name = "dubbo-demo-tx")
timeoutMills is the overall timeout of the transaction, the default is 60s, and name is the alias of the transaction method signature, which is empty by default. Parameters in the annotation can be omitted.
Step 6: Start Nacos-Server

Download the latest release package of Nacos-Server and unzip it
Run Nacos-server
Linux/Unix/Mac

sh startup.sh -m standalone
Windows

cmd startup.cmd -m standalone
Access the Nacos console:
http://localhost:8848/nacos/index.html#/configurationManagement?dataId=&group=&appName=&namespace

If the access is successful, the Nacos-Server service is running successfully (default account/password: nacos/nacos)

Step 7: Start Fescar-Server

Download the latest release package of Fescar-Server and unzip it
Initialize Fescar configuration
Go to the conf folder of the Fescar-Server decompression directory, confirm the configuration value of nacos-config.txt (generally do not need to be modified), and run the nacos-config.sh script to initialize the configuration after confirmation.

sh nacos-config.sh $Nacos-Server-IP
eg:

sh nacos-config.sh localhost
The final output of the script execution is "init nacos config finished, please start fescar-server.", indicating that the push configuration is successful. If you want to confirm further, you can log in to the Nacos console, configure the list, and filter the configuration items of Group=FESCAR_GROUP.


Modify the Fescar-server service registration method to nacos
Go to registry.conf in the conf folder of the Fescar-Server decompression directory, modify type="nacos" and configure the relevant properties of Nacos.

registry {
# file nacos
type = "nacos"
nacos {
serverAddr = "localhost"
namespace = "public"
cluster = "default"
}
file {
name = "file.conf"
}
}
type: can be configured as nacos and file, no service registration function when configured as file

nacos.serverAddr: Nacos-Sever service address (excluding port number)

nacos.namespace: Nacos registration and configuration isolation namespace

nacos.cluster: the name of the cluster where the service is registered

file.name: type = "file" configuration file name under classpath

Running Fescar-server
Linux/Unix/Mac

sh fescar-server.sh $LISTEN_PORT $PATH_FOR_PERSISTENT_DATA $IP (this parameter is optional)
Windows

cmd fescar-server.bat $LISTEN_PORT $PATH_FOR_PERSISTENT_DATA $IP (this parameter is optional)
LISTENpORT: Fescar-Server service port PATH_FOR_PERSISTENT_DATA: transaction operation record file storage path (existing path)

$IP (optional parameter): used to specify the IP of the Fescar-Server registration service in a multi-IP environment

eg: sh fescar-server.sh 8091 /home/admin/fescar/data/

After running successfully, you can see the service name=serverAddr service registration list in the Nacos console:


Step 8: Start the microservice and test

Modify the business client discovery registration method to nacos
Same as [Modify the registration method of Fescar-server service to nacos] in Step 7
Start DubboAccountServiceStarter
Start DubboOrderServiceStarter
Start DubboStorageServiceStarter
After the startup is complete, you can see the three providers that have been started in the service list of the Nacos console:


Start DubboBusinessTester for testing
Note: The transaction will be rolled back only if an exception is thrown inside the method annotated with @GlobalTransactional. The entire Dubbo service invocation link only needs to be annotated in the service method of the initiator at the beginning of the transaction.

Through the above 8 steps, we have realized the data consistency between the three independent microservices of inventory, order and account in the business of purchasing goods for users.

Reference link:

This article sample address:

https://github.com/fescar-group/fescar-samples/tree/master/nacos

Fescar:

https://github.com/alibaba/fescar

Dubbo:

https://github.com/apache/incubator-dubbo

Nacos:

https://github.com/alibaba/nacos

The author of this article: Qing Ming, nicknamed slievrly in the community, one of the initiators of the Fescar open source project, a core R&D member of Alibaba Middleware TXC/GTS, has been engaged in the core R&D of distributed middleware for a long time, and has rich experience in the field of distributed transactions. Technology accumulation.