PolarDB:What is global consistency (high-performance mode)

Linear Lamport timestamp

To reduce the latency and bandwidth consumption of read requests, an RO node can use the linear Lamport timestamp mechanism. When a request reaches an RO node and another request has already obtained a timestamp from the RW node, the RO node can directly reuse that timestamp. This avoids repeated requests and ensures strong consistency while improving performance.

In the preceding figure, two concurrent read requests r1 and r2 reach an RO node. r2 sends a request to the RW node for a timestamp at t2 and, at t3, obtains the RW node timestamp TS3rw. The relationship between these events is e2TS3rwe3. r1 arrives at t1. Assigning a timestamp to each event on the RO node determines the order of different events on the same RO node. If t1 is earlier than t2, the relationship between the events is e1e2TS3rwe3. In other words, the timestamp obtained by r2 already reflects all updates made before r1 arrived. Therefore, r1 can directly use the timestamp of r2 without obtaining a new timestamp. Based on this principle, each time an RO node obtains a timestamp from the RW node, the RO node saves the timestamp locally and records the time when the timestamp was obtained. If the arrival time of a request is earlier than the time when the locally cached timestamp was obtained, the request can directly use that timestamp.

Hierarchical fine-grained modification tracking

To perform strongly consistent reads, an RO node must first obtain the latest commit timestamp of the current transaction on the RW node, and wait until its logs are replayed to that timestamp before it processes a read request. However, while the RO node waits for log replay, the data required by the current request may already be the latest, and no wait is required. To avoid unnecessary waiting, PolarDBglobal consistency (high-performance mode) uses more fine-grained modification tracking. The RW node maintains three levels of modification information: the global latest modification timestamp, the table-level timestamp, and the page-level timestamp.

When processing a read request, an RO node first obtains the global timestamp to determine data consistency. If the global timestamp does not meet the condition, the RO node further obtains the local timestamp of the target table for more fine-grained verification. If the condition is still not met, the RO node continues to check the timestamp of the target page for a more precise determination. The RO node must wait for log replay only when the page timestamp is still later than the timestamp to which its logs have been replayed. This ensures that the RO node reads the latest data.

On the RW node, all three levels of timestamps are stored in an in-memory hash table to reduce memory consumption. To further optimize memory usage, the timestamps of multiple tables or pages can be mapped to the same hash table entry. To ensure consistency, only a larger timestamp can replace a smaller one. This design ensures that consistency is not compromised even if an RO node obtains a larger timestamp. The following figure shows the implementation, in which TID and PID indicate the table ID and the page ID, respectively. A timestamp obtained by an RO node is cached locally based on the linear Lamport timestamp design so that other eligible requests can use it.

RDMA-based log shipment

In PolarDBglobal consistency (high-performance mode), the RW node remotely writes logs to the RO node cache over one-sided RDMA. This process does not require the CPU of the RO node and has low latency. The following figure shows the implementation. The RO node and the RW node both maintain log buffers of the same size. A background thread of the RW node writes the RW log buffer to the RO log buffer over RDMA. The RO node reads the local log buffer instead of reading files, which accelerates replication synchronization.