PolarDB:Product overview

Centralized (Enterprise Edition and Standard Edition)

• Database proxy (Proxy)

  The database proxy is a network proxy service located between the database and applications, which handles all requests when applications access the database. The proxy layer provides security authentication and supports advanced features such as automatic read/write splitting, load balancing, consistency levels, connection pooling, persistent connections, and overload protection.

• Database compute nodes

  • PolarDB uses a multi-node cluster architecture. A one-write-multiple-read cluster has one read/write node and multiple read-only nodes.

  • The read/write node and read-only nodes use an Active-Active Failover approach to provide high availability database services.

  • Compute nodes primarily provide database SQL engine functionality and are available in general-purpose and dedicated specifications.

• Shared distributed storage

  Multiple compute nodes share a single copy of data rather than each compute node storing its own copy, which significantly reduces storage costs. Based on newly built distributed block storage (Distributed Storage) and file system (Distributed Filesystem), storage capacity can be smoothly expanded online, avoiding the storage capacity limitations of a single database server and effectively handling data volumes of hundreds of terabytes.

Distributed

• Database nodes

  • The cluster consists of compute nodes and data nodes. Compute nodes manage cluster metadata and create distributed execution plans. Data nodes store actual data shards.

  • Each data node and compute node in the cluster uses a centralized architecture that decouples storage and computing. The cluster supports database proxy and one-write-multiple-read mode and provides failover mechanisms for compute nodes and data nodes. The cluster allows the addition of read-only nodes to enhance the read capability of individual compute nodes or data nodes.

• Distributed features

  • The cluster supports manual sharding and provides horizontal scaling capabilities, making it suitable for business scenarios that involve data volumes below petabytes (PB).

  • The cluster ensures consistency in distributed transactions.

  • The cluster supports dual-zone deployment. Operations are performed on the active working cluster in the primary zone. In the event of a failure in the primary zone, the hot standby cluster in the secondary zone can seamlessly take over to ensure uninterrupted service.

  • The cluster offers 24/7 non-disruptive operation during maintenance, upgrades, and configuration changes. The cluster supports the addition of heterogeneous compute nodes and data nodes.

Centralized (Standard Edition)

• Data node

  Data nodes are used to persistently store data in PolarDB-X and provide reliable and consistent storage services based on the Paxos protocol. The self-developed Lizard transaction system offers higher availability and approximately 35% performance improvement compared to the MySQL distributed engine.

• Multi-replica architecture

  PolarDB-X uses Paxos to ensure strong consistency between replicas (RPO = 0). Paxos requires each write operation to be confirmed by more than half of the nodes. This ensures that a cluster can continue to provide services as expected even if one of the cluster nodes fails. Paxos can ensure strong consistency by eliminating inconsistency issues among replicas. Replicas are classified into the following roles:

  • Leader

    A leader processes client requests and makes decisions. A leader maintains logs to ensure data consistency and recoverability.

  • Follower

    A follower accepts and executes instructions from the leader. When the leader fails or becomes inaccessible, a follower can be elected to become the new leader.

  • Logger

    Similar to a follower, a logger supports the majority consensus protocol but does not provide data services. When the leader fails or becomes inaccessible, loggers participate in the leader election. A logger may be elected as a temporary leader, but it does not provide data services. After the followers are updated and synchronized their status with the most recent data, the logger gives up leadership.

  • Learner

    A learner passively receives state information from the system and does not participate in the leader election or decision-making process. Therefore, a learner produces minimal overhead on the system.

Distributed (Enterprise Edition)

• Global meta service (GMS)

  Responsible for maintaining globally consistent system metadata such as table metadata, schema metadata, and statistics metadata. GMS manages security-related information such as user accounts and permissions. GMS provides the Timestamp Oracle (TSO) service.

• Compute node

  Compute nodes are the entrance of the system. These nodes use the stateless design, and include models such as the SQL parser, optimizer, and executor. Compute nodes are responsible for distributed data routing, computation, dynamic scheduling, distributed transaction coordination based on the Two-Phase Commit (2PC) protocol, distributed DDL execution, and global secondary index maintenance. Compute nodes provide enterprise-level features such as SQL throttling and the three-role mode.

• Data Node

  Data nodes are responsible for persistent row-oriented data. Data nodes ensure data durability and guarantee strong consistency based on Paxos. Data nodes use MVCC to maintain the visibility of distributed transactions. Data nodes can also meet the requirements of operations that need to push down computing tasks in distributed architectures, such as Project, Filter, Join, and Aggregation.

• Columnar node

  Columnar nodes provide persistent CCIs, builds CCIs based on OSS, and consumes the binary logs of distributed transactions in real time to meet the requirements of real-time updates. When combined with compute nodes, columnar nodes can provide snapshot-consistent query capabilities for CCIs.

• Change data capture (CDC) node

  CDC nodes provide incremental subscription capabilities fully compatible with MySQL binlog format and protocol. CDC nodes also provide primary/secondary replication capabilities compatible with MySQL replication protocol.