Apsara PolarDB runs in a cluster architecture. Each cluster contains a primary node and one or more read-only nodes. Clients can connect to an Apsara PolarDB cluster by using two types of endpoint: cluster endpoints and primary endpoints. We recommend that you use cluster endpoints because the nodes in the cluster can share the same endpoint to achieve read/write splitting.

Apsara PolarDB architecture

Read/write splitting mechanism

Data replication is a simple method to replicate data from the primary node to read-only nodes. You only need to asynchronously transfer the write-ahead logs (WALs) of the primary node to read-only nodes. Data replication enables read-only nodes to process queries. This reduces the load on the primary node and ensures high availability.

However, if read-only nodes are used to process queries, you must consider the following issues:

  • 1. Clients can connect to the primary node and read-only nodes through two different endpoints. You must specify the endpoint for connections in your applications.
  • 2. Data is asynchronously replicated. Data may not be immediately synchronized to read replicas after a client commits data modifications. Therefore, data in read-only nodes may not be up-to-date. This indicates that consistency of data is not guaranteed.

To fix Issue 1, Apsara PolarDB uses the read/write splitting proxy. The proxy can be used to establish connections between clients and Apsara PolarDB. You can use the proxy to parse each query from the clients. The proxy can be used to send write requests, such as UPDATE, DELETE, INSERT, and CREATE, to the primary node. It can also be used to send read requests such as SELECT to read-only nodes.

WAL for read/write splitting in Apsara PolarDB

Data inconsistency caused by the synchronization latency still exists and is not fixed. If you execute the SELECT statement to retrieve data from read-only nodes, the returned data may be inconsistent with the data that is stored on the primary node. If the load on an Apsara PolarDB cluster is light, the synchronization latency can be reduced to less than five seconds. In scenarios that involve a heavy load, the synchronization latency may be increased to a large extent. For example, this issue may occur when you execute data definition language (DDL) statements to add columns on large tables or insert a large amount of data.

Eventual consistency and session consistency

  • Eventual consistency: Apsara PolarDB synchronizes data from the primary node to read-only nodes by using asynchronous physical replication. Updates to the primary node are replicated to read-only nodes. In most cases, data changes are synchronized to read-only nodes with a latency of a few milliseconds. The latency is based on the load of write requests on the primary node. This allows you to achieve eventual consistency through asynchronous replication.
  • Session consistency: Session consistency is used to fix the issue of data inconsistency that occurs before eventual consistency is achieved. Physical replication is fast. Based on this feature, Apsara PolarDB can be used to forward requests to the read-only nodes that have completed asynchronous replication. For more information, see How it works.

Session consistency based on read/write splitting

Apsara PolarDB runs in a read/write splitting architecture. Traditional read/write splitting allows you to ensure only eventual consistency. Latency exists between updates to the primary node and replication to read-only nodes. This may result in different responses that are returned by different nodes for the same query. For example, you can execute the following statements within a session:

INSERT INTO t1(id, price) VALUES(111, 96);
UPDATE t1 SET price = 100 WHERE id=111;
SELECT price FROM t1;

In this example, the result of the last query may be invalid because Apsara PolarDB may send the SELECT request to a read-only node where data has not been updated. To avoid this issue, you must make changes to your applications instead of Apsara PolarDB. The most common solution is to divide your workloads. For the workloads that require high consistency, requests are sent to only the primary node. Otherwise, for the workloads that require eventual consistency, write requests are sent to the primary node and read requests are sent to read-only nodes. However, this solution makes application development more complex, increases the load on the primary node, and compromises the read/write splitting performance.

To address this issue, Apsara PolarDB provides session consistency. Within the same session, read requests are sent to read-only nodes where data has been updated. In this way, statements are used to query only the up-to-date data on read-only nodes.

How it works

How it works

Apsara PolarDB uses a proxy to achieve read-write splitting, load balancing, and read consistency. The proxy can be used to track the log sequence number (LSN) of the redo log on each node. Each time the log stored on the primary node is updated, the LSN of the log is updated as the session LSN. If a new read request is received within the same session, the proxy can be used to compare the session LSN and the LSN of the log stored on each node. Then, the proxy can be used to forward the request to a read-only node where the LSN is equal to or greater than the session LSN. Apsara PolarDB implements physical replication. After the primary node is used to process a write request, it returns the result to the client and replicates data to read-only nodes in parallel. This allows you to use read-only nodes to update data before subsequent read requests are received. Apsara PolarDB achieves read consistency without the need to handle the heavy load on the primary node.

Best practices for the use of consistency levels

We recommend that you use session consistency. This consistency level minimizes the impact on cluster performance and supports most scenarios.

The following solution applies if you want to achieve consistency among multiple sessions:

Use hints to force the primary node to run a specific statement.

eg: /*FORCE_MASTER*/ select * from user;