All Products
Search
Document Center

PolarDB:Global consistency (high-performance mode)

Last Updated:Jul 26, 2023

PolarDB for MySQL supports global consistency (high-performance mode). PolarTrans uses Commit Timestamp Store (CTS) and Remote Direct Memory Access (RDMA) to provide global consistency (high-performance mode) at the kernel level. This ensures that strong consistency is implemented for all read requests destined for any read-only nodes in the cluster. This topic describes the limits, technical solution, enabling method, and performance data of global consistency (high-performance mode).

Limits

  • Your cluster meets one of the following version requirements:

    • A cluster of PolarDB for MySQL 8.0.2 whose revision version is 8.0.2.2.11 or later.

    • A cluster of PolarDB for MySQL 8.0.1 whose revision version is 8.0.1.1.29 or later.

    • A cluster of PolarDB for MySQL 5.7 whose revision version is 5.7.1.0.26 or later.

    Note

    For more information about how to check the cluster version, see Query the engine version.

  • Global consistency (high-performance mode) cannot be enabled for read-only nodes of a secondary cluster in a global database network (GDN).

  • You cannot enable global consistency (high-performance mode) for read-only column store nodes or read-only hot standby nodes.

Background information

In the original architecture of PolarDB for MySQL, there are one primary node and multiple read-only nodes. By default, the read-only nodes provide eventual read consistency. Physical replication and shared storage technologies can effectively reduce the latency of read-only nodes. However, read-only requests destined for read-only nodes may not read the latest data written on the primary node. In latency-sensitive industries such as finance and games, delayed reading of read-only nodes can cause data inconsistency. Strictly consistent read requests

The preceding figure shows that multiple services are decoupled through the microservice framework. After Service A writes data, the system sends a message through Message Queue to notify Service B that the data was written successfully. Service B consumes the message and perceives that the data is written successfully. Service B then issues a read request to read the data for follow-up business flow. If only eventual read consistency can be guaranteed, neither Service A nor Service B can guarantee that the latest data written in Step 1 can be read. This brings data consistency problems to upper-layer services. The only solution to this situation is forwarding the read-only requests to the primary node to ensure read-after-write consistency.

Global consistency (high-performance mode) technical solution

PolarTrans uses CTS and RDMA networks to provide strict read consistency for read-only nodes at the kernel layer. This ensures that the latest data can be synchronized on the primary node and read-only nodes.

After PolarTrans is enabled, PolarDB for MySQL will assign a timestamp to each read or write transaction when the transaction is committed on the primary node. This timestamp can be a logical timestamp or a hybrid timestamp. At the same time, the primary node writes the transaction ID and commit sequence number (CSN) to the redo log. Then, read-only nodes replay the redo log to obtain transaction status.

The following section describes how strict consistency works on read-only nodes:

  1. A client initiates a query.

  2. The read-only node obtains the maximum commit timestamp of the primary node by performing the one-sided RDMA remote read operation.

  3. The read-only node creates a strictly consistent read view based on the maximum commit timestamp of the primary node, and waits until the redo log is replayed to the corresponding checkpoint to obtain the transaction status.

  4. The read-only node determines data visibility based on the strictly consistent read view and returns the result to the client.

In global consistency (high-performance mode), read-only nodes obtain the latest commit timestamp from the primary node by using RDMA one-side remote read and use the commit timestamp in subsequent operations such as calculating latency for the current transaction and building a strongly consistent read view. Global consistency (high-performance mode) is implemented in combination with multiple concurrency optimization methods. Query results between multiple threads can be reused to reduce the number and overheads of remote read operations. The time for transaction status synchronization process is limited within 2%.

Enable global consistency (high-performance mode)

Log on to the PolarDB console. In the Database Nodes section of the Overview page of the cluster, select the read-only nodes for which you want to enable global consistency (high-performance mode) and click Enable Global Consistency. Enable global consistency (high-performance mode)

Performance comparison

  • Test environment

    A cluster of PolarDB for MySQL 8.0 Cluster Edition with 8 cores and 32 GB of memory is used.

  • Test tool

    Sysbench

  • Test data

    25 tables with 250,000 rows in each table.

  • Test results

    • RDMA performance loss

      If no latency exists between the read-only node and the primary node, the RDMA performance overhead of obtaining the current maximum CSN of the transaction on the primary node is compared before and after global consistency (high-performance mode) is enabled. The result indicates that after global consistency (high-performance mode) is enabled, the RDMA performance loss is limited within 2%. RDMA performance loss

    • Read-only performance

      Under normal write load conditions, read-only nodes have latency. Strictly consistent reads require transaction status acquisition and verification, which affects the peak read-only performance of read-only nodes. However, the optimized global consistency (high-performance mode) technical solution can control this impact within 20% and allow read-only nodes to provide strictly consistent reads on par with the primary node. Read-only performance

    • Read and write performance

      In read and write scenarios, after global consistency (high-performance mode) is enabled, more read-only requests can be distributed to read-only nodes by using cluster endpoints while the write performance of the primary node is not affected. This improves the read and write throughput of the entire cluster. The result indicates that the performance comparison results of different modes in the sysbench oltp_read_write test, where RW indicates that all requests are sent to the primary node. In high concurrency scenarios, the cluster performance is 1.7 times higher than that of the RW mode after global consistency (high-performance mode) is enabled. Read and write performance

    • Performance of scaled read-only nodes

      In scenarios with high read/write ratios, such as the sysbench oltp_read_write test, the performance of the cluster can be improved by scaling the read-only nodes. More importantly, scaling the read-only nodes is relatively safer for service continuity than changing cluster specifications. This is because scaling the read-only nodes does not require cluster switchovers or cause service interruptions.

    • High-specification cluster performance

      After global consistency (high-performance mode) is optimized by using the modification tracking table (MTT) technology, global consistency (high-performance mode) can greatly improve the performance of clusters in highly concurrent write scenarios. A sysbench oltp_read_write stress test is conducted on a cluster that contains one primary node and one read-only node. In the cluster, 512 concurrent threads are executed. After global consistency (high-performance mode) is enabled, the cluster performance is 1.7 times higher than that of the RW mode. If write requests continue to increase, the cluster performance can be improved to millions of queries per second (QPS) by increasing the number of read-only nodes from 1 to 2. Read and write performance (88 cores)