Practices for solving data consistency problems in a microservices architecture
With rapid business development, monolithic architectures show many problems, such as low code maintainability, low fault tolerance, difficult testing, and poor agile delivery capabilities. To solve these problems, microservices were born. Although microservices can solve the aforementioned problems, it also brings about new problems, one of which is how to ensure business data consistency between microservices.
This article uses a commodity purchase case to explain how Fescar can ensure business data consistency under the microservices architecture Dubbo. In the example described in this article, the registration, configuration, and service center for both Dubbo and Fescar is Nacos. Fescar later than 0.2.1 supports the Nacos registration, configuration, and service center.
The commodity purchase business includes three microservices:
The business structure is as follows:
StorageService
public interface StorageService {
/**
* deduct storage count
*/
void deduct(String commodityCode, int count);
}OrderService
public interface OrderService {
/**
* create order
*/
Order create(String userId, String commodityCode, int orderCount);
}AccountService
public interface AccountService {
/**
* debit balance of user's account
*/
void debit(String userId, int money);
}Note: The three preceding microservices are all deployed independently.
Modify the connection information for StorageService, OrderService, and AccountService in resources/jdbc. properties.
jdbc.account.url=jdbc:mysql://xxxx/xxxx
jdbc.account.username=xxxx
jdbc.account.password=xxxx
jdbc.account.driver=com.mysql.jdbc.Driver
# storage db config
jdbc.storage.url=jdbc:mysql://xxxx/xxxx
jdbc.storage.username=xxxx
jdbc.storage.password=xxxx
jdbc.storage.driver=com.mysql.jdbc.Driver
# order db config
jdbc.order.url=jdbc:mysql://xxxx/xxxx
jdbc.order.username=xxxx
jdbc.order.password=xxxx
jdbc.order.driver=com.mysql.jdbc.DriverRelated scripts for creating tables can be found under "resources/sql/". Run the business table creation script in dubbo_biz.sql in the corresponding database and run the undo_log.sql table creation script in each database.
CREATE TABLE undo_log (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`branch_id` bigint(20) NOT NULL,
`xid` varchar(100) NOT NULL,
`rollback_info` longblob NOT NULL,
`log_status` int(11) NOT NULL,
`log_created` datetime NOT NULL,
`log_modified` datetime NOT NULL,
`ext` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_unionkey` (`xid`,`branch_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `storage_tbl`;
CREATE TABLE `storage_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`commodity_code` varchar(255) DEFAULT NULL,
`count` int(11) DEFAULT 0,
PRIMARY KEY (`id`),
UNIQUE KEY (`commodity_code`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `order_tbl`;
CREATE TABLE `order_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` varchar(255) DEFAULT NULL,
`commodity_code` varchar(255) DEFAULT NULL,
`count` int(11) DEFAULT 0,
`money` int(11) DEFAULT 0,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `account_tbl`;
CREATE TABLE `account_tbl` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` varchar(255) DEFAULT NULL,
`money` int(11) DEFAULT 0,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;Note: Ensure that each physical database includes the undo_log table. In this example, a physical database can be used to represent the independent logic database of the three preceding microservices.
<properties>
<fescar.version>0.2.1</fescar.version>
<dubbo.alibaba.version>2.6.5</dubbo.alibaba.version>
<dubbo.registry.nacos.version>0.0.2</dubbo.registry.nacos.version>
</properties>
<dependency>
<groupId>com.alibaba.fescar</groupId>
<artifactId>fescar-spring</artifactId>
<version>${fescar.version}</version>
</dependency>
<dependency>
<groupId>com.alibaba.fescar</groupId>
<artifactId>fescar-dubbo-alibaba</artifactId>
<version>${fescar.version}</version>
<exclusions>
<exclusion>
<artifactId>dubbo</artifactId>
<groupId>org.apache.dubbo</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>dubbo</artifactId>
<version>${dubbo.alibaba.version}</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>dubbo-registry-nacos</artifactId>
<version>${dubbo.registry.nacos.version}</version>
</dependency>Note: Because apache-dubbo is currently incompatible with dubbo-registry-nacos jar, the apache.dubbo dependency should be excluded from fescar-dubbo and alibaba-dubbo should be manually added. Later apache-dubbo (2.7.1+) will be compatible with dubbo-registry-nacos. In Fescar, fescar-dubbo jar supports apache.dubbo and fescar-dubbo-alibaba jar supports alibaba-dubbo.
Perform the following configuration in the Spring configuration files of the three microservices (dubbo-account-service.xml, dubbo-order-service and dubbo-storage-service.xml):
<bean id="accountDataSourceProxy" class="com.alibaba.fescar.rm.datasource.DataSourceProxy">
<constructor-arg ref="accountDataSource"/>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="accountDataSourceProxy"/>
</bean>com.alibaba.fescar.rm.datasource.DataSourceProxy is required to wrap Druid data source as the direct business data source. DataSourceProxy is used for intercepting and resolving business SQL as well as interacting with TC to coordinate transaction operation status.
<dubbo:registry address="nacos://${nacos-server-ip}:8848"/><bean class="com.alibaba.fescar.spring.annotation.GlobalTransactionScanner">
<constructor-arg value="dubbo-demo-account-service"/>
<constructor-arg value="my_test_tx_group"/>
</bean>The first parameter of the constructor is the custom business applicationId. If multiple microservices are to be deployed on a single machine, the applicationId must be unique.
The second parameter of the constructor is the logical group of the Fescar transaction service. This group uses the "service.vgroup_mapping.my_test_tx_group" configuration item in the configuration center to map to the corresponding Fescar-Server cluster name, and then obtains the list of available services based on the cluster name ".grouplist".
Make the following configuration in dubbo-business.xml:
@GlobalTransactional(timeoutMills = 300000, name = "dubbo-demo-tx")"timeoutMills" is the total transaction timeout time (60s by default). "name" is the alias of the transaction method signature (set to null by default). All the parameters in the annotation can be omitted.
Linux/Unix/Mac
sh startup.sh -m standaloneWindows
cmd startup.cmd -m standaloneAccess the Nacos console: http://localhost:8848/nacos/index.html#/configurationManagement?dataId=&group=&appName=&namespace
If it's able to be accessed, the Nacos-Server service is running successfully (default account/password: nacos/nacos).
Go to the conf folder under the directory where Fescar-Server is decompressed, confirm the configuration value for the nacos-config.txt (generally modification is not required) and then run the nacos-config.sh script to initialize the configuration.
sh nacos-config.sh $Nacos-Server-IPExample:
sh nacos-config.sh localhost The final output of this script is "init nacos config finished, please start fescar-server." This indicates that the push configuration was successful. If you want to further check the configuration, log on to the Nacos console, go to the configuration list and filter the configuration item with Group=FESCAR_GROUP.
Find registry.conf in the conf folder under the directory where Fescar-Server is decompressed, change the type to nacos and configure the related properties of Nacos.
registry {
# file nacos
type = "nacos"
nacos {
serverAddr = "localhost"
namespace = "public"
cluster = "default"
}
file {
name = "file.conf"
}
}type: It can be configured as nacos and file. When configured as file, service registration is unavailable.
nacos.serverAddr: Nacos-Sever service address (the port number is excluded)
nacos.namespace: the independent namespace for Nacos registration and configuration
nacos.cluster: the cluster name of the registration service
file.name: the name of the configuration file under classpath when type is set to file
Linux/Unix/Mac
sh fescar-server.sh $LISTEN_PORT $PATH_FOR_PERSISTENT_DATA $IP (This parameter is optional)Windows
cmd fescar-server.bat $LISTEN_PORT $PATH_FOR_PERSISTENT_DATA $IP (This parameter is optional)LISTENPORT: the service port of Fescar-Server
PATH_FOR_PERSISTENT_DATA: Path of the transaction operation record file (the path already exists)
$IP (optional): the IP used for specify the registration service of Fescar-Server in a multi-IP environment
eg: sh fescar-server.sh 8091 /home/admin/fescar/data/
After running the server, you can see that the service name in the Nacos console is serverAddr:
Similar to the [Change the Fescar-server service registration method to nacos] section in Step 7
After the startup, in the Nacos console, you can see three providers in the service list:
Note: Only exceptions thrown in the methods with the @GlobalTransactional annotation will trigger transaction rollback. The entire Dubbo service call link only needs to annotate the service method of the original transaction initiator.
The eight preceding steps allow us to ensure data consistency among the three independent microservices in the commodity purchase business: storage, order, and account.
Reference links:
Sample in this article:
https://github.com/fescar-group/fescar-samples/tree/master/nacos
Fescar:
https://github.com/alibaba/fescar
Dubbo:
https://github.com/apache/incubator-dubbo
Nacos:
https://github.com/alibaba/nacos
Qing Ming is one of the initiators of the Open Source Fescar project and one of the core members of the Alibaba middleware TXC/GTS R&D team. The author has been engaged in the core R&D work of distributed middleware for a long time and is very experienced in the distributed transaction field.
2,605 posts | 747 followers
FollowAlibaba Clouder - June 15, 2020
Alibaba Clouder - November 23, 2020
Alibaba Cloud_Academy - August 4, 2022
Alibaba Clouder - December 15, 2020
Alibaba Clouder - March 8, 2021
Alibaba Clouder - August 17, 2021
2,605 posts | 747 followers
FollowLearn More
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn More
Container Registry
A secure image hosting platform providing containerized image lifecycle management
Learn MoreMore Posts by Alibaba Clouder