PolarDB for MySQL supports global consistency (high-performance mode). PolarTrans uses Commit Timestamp Store (CTS) and Remote Direct Memory Access (RDMA) to provide global consistency (high-performance mode) at the kernel level. This ensures that strong consistency is implemented for all read requests destined for any read-only nodes in your cluster. This topic describes the limits, technical solution, and performance data of global consistency (high-performance mode) and how to enable it.
Supported versions
If you want to enable global consistency (high-performance mode), your PolarDB for MySQL cluster must meet one of the following version requirements:
The engine version is 8.0.2 with a revision version of 8.0.2.2.19 or later.
The engine version is 8.0.1 with a revision version of 8.0.1.1.29 or later.
The engine version is 5.7 with a revision version of 5.7.1.0.26 or later.
For more information about how to check the engine version, see Query the engine version.
Usage notes
By default, global consistency (high-performance mode) is enabled for all read-only nodes in a serverless cluster.
Global consistency (high-performance mode) cannot be enabled for read-only nodes in a secondary cluster on a global database network (GDN).
Global consistency (high-performance mode) cannot be enabled for read-only column store nodes.
Global consistency (high-performance mode) is compatible with the fast query cache feature. However, if the modification tracking table (MTT) optimization is enabled for global consistency (high-performance mode) and you enable both the fast query cache feature and global consistency (high-performance mode), the MTT optimization becomes invalid. For more information about the fast query cache feature, see Fast query cache.
Background information
The original architecture of PolarDB consists of one primary node and multiple read-only nodes. By default, the read-only nodes provide eventual read consistency. Physical replication and shared storage technologies can effectively reduce the latency of read-only nodes. However, read-only requests destined for read-only nodes may not read the latest data written on the primary node. In latency-sensitive industries such as finance and games, delayed reading of read-only nodes can cause data inconsistency.
As shown in the preceding figure, multiple services are decoupled by using the microservices framework. After Service A writes data, the system sends a message by using the message queue (MQ) service to notify Service B that the data is successfully written. Service B consumes the message and perceives that the data is successfully written. Service B then issues a read request to read the data for follow-up business flow. If only eventual read consistency is guaranteed, neither Service A nor Service B can guarantee that the latest data written in Step 1 is read. This brings data consistency issues to upper-layer services. The only solution to this situation is forwarding the read-only requests to the primary node to ensure read-after-write consistency.
Global consistency (high-performance mode) technical solution
PolarTrans uses CTS and RDMA to provide strict read consistency for read-only nodes at the kernel level. This ensures that the latest data is always visible on the primary node and read-only nodes and provides the capability of strict read consistency for the cluster.
After PolarTrans is enabled, PolarDB assigns a timestamp to each read or write transaction when the transaction is committed on the primary node. This timestamp can be a logical timestamp or a hybrid timestamp. In addition, the primary node writes the transaction ID and commit sequence number (CSN) to the redo log. Then, read-only nodes replay the redo log to obtain transaction status.
The following section describes how strict read consistency works on read-only nodes:
A client initiates a query.
A read-only node obtains the latest commit timestamp of the primary node by using RDMA one-side remote read.
The read-only node creates a strictly consistent read view based on the maximum commit timestamp of the primary node, and waits until the redo log is replayed to the corresponding checkpoint to obtain the transaction status.
The read-only node determines data visibility based on the strictly consistent read view and returns the result to the client.
In global consistency (high-performance mode), read-only nodes obtain the latest commit timestamp from the primary node by using RDMA one-side remote read and use the commit timestamp in subsequent operations such as calculating latency for the current transaction and building a strongly consistent read view. Global consistency (high-performance mode) is implemented in combination with multiple concurrency optimization methods. Query results between multiple threads can be reused to reduce the number and overheads of remote read operations. The time for transaction status synchronization process is limited within 2%.
Global consistency (high-performance mode) and global consistency
PolarDB provides the following four consistency levels to meet your requirements for consistency in different scenarios: eventual consistency, session consistency, global consistency, and global consistency (high-performance mode).
Global consistency (high-performance mode) is an upgrade of the original global consistency level, and has stricter requirements on strong consistency than global consistency. Therefore, if stricter strong consistency is required in your business system, we recommend that you use global consistency (high-performance mode).
For a PolarDB for MySQL 5.7, 8.0.1, or 8.0.2 cluster, if strict strong consistency is required, global consistency (high-performance mode) is more recommended.
For a PolarDB for MySQL 5.6 cluster, you can select global consistency for strict strong consistency because this version does not support global consistency (high-performance mode).
For more information about how to enable global consistency, see Enable global consistency.
Enable global consistency (high-performance mode)
Log on to the PolarDB console. In the Database Nodes section of the Basic Information page of your cluster, move the pointer over the read-only node for which you want to enable global consistency (high-performance mode) and click Enable Global Consistency.
This feature takes effect only for new connections.
After you enable global consistency (high-performance mode) for a read-only node, the consistency level of other nodes that belong to the same cluster endpoint as this read-only node and for which global consistency (high-performance mode) is not enabled is automatically switched to session consistency. For more information, see Session consistency.
Performance comparison
Test environment
A cluster of PolarDB for MySQL 8.0 Cluster Edition with eight cores and 32 GB of memory
Test tool
sysbench
Test data
25 tables with 250,000 rows in each table
Test results
RDMA performance loss
If no latency exists between a read-only node and the primary node, the RDMA performance overhead of obtaining the current maximum CSN of the transaction on the primary node can be calculated by comparing the data before and after global consistency (high-performance mode) is enabled. The result indicates that after global consistency (high-performance mode) is enabled, the RDMA performance loss is limited within 2%.
Read-only performance
Under normal write load conditions, read-only nodes have latency. Strictly consistent reads require transaction status acquisition and verification, which affects the peak read-only performance of read-only nodes. However, the optimized global consistency (high-performance mode) technical solution can control this impact within 20% and allow read-only nodes to provide strictly consistent reads on par with the primary node.
Read and write performance
In read and write scenarios, after global consistency (high-performance mode) is enabled, more read-only requests can be distributed to read-only nodes by using the cluster endpoint with no impacts on the write performance of the primary node. This improves the read and write throughput of the entire cluster. The following figure shows the performance comparison results of different modes in the sysbench oltp_read_write test, in which RW indicates that all requests are sent to the primary node. In high concurrency scenarios, the cluster performance is 1.7 times higher than that of the RW mode after global consistency (high-performance mode) is enabled.
Performance of scaled read-only nodes
In scenarios with high read/write ratios, such as the sysbench oltp_read_write test, the performance of the cluster can be improved by scaling the read-only nodes. More importantly, scaling the read-only nodes is relatively safer for service continuity than changing cluster specifications. This is because scaling the read-only nodes does not require cluster switchovers or cause service interruptions.
High-specification cluster performance
After global consistency (high-performance mode) is optimized by using the MTT technology, global consistency (high-performance mode) can greatly improve the performance of clusters in highly concurrent write scenarios. A sysbench oltp_read_write stress test is conducted on a cluster that contains one primary node and one read-only node. In the cluster, 512 concurrent threads are executed. After global consistency (high-performance mode) is enabled, the cluster performance is 1.7 times higher than that of the RW mode. If write requests continue to increase, you can improve the cluster performance to millions of queries per second (QPS) by increasing the number of read-only nodes from 1 to 2.
