This topic describes the architecture and key advantages of PolarDB-X.
Architectural evolution
Currently, the distributed database space has three main technical approaches: sharding, NewSQL native distributed databases, and cloud-native databases. Each approach has unique strengths and characteristics. PolarDB-X builds on the proven stability of DRDS and X-DB technologies, integrates the cloud-native technology of PolarDB, and incorporates NewSQL's capabilities for distributed data consistency to create a new cloud-native, distributed product.
Overall architecture
The overall architecture of PolarDB-X is as follows:
Core components
Global Meta Service (GMS): Provides distributed metadata, issues global timestamps through a Timestamp Oracle (TSO), maintains metadata such as tables, schemas, and statistics, and manages security information such as accounts and permissions.
Compute node (CN): Provides a distributed SQL engine that includes a core optimizer and executor. The stateless SQL engine offers distributed routing and computation, and handles two-phase commit (2PC) coordination for distributed transactions, distributed DDL execution, and global index maintenance.
Data node (DN): Provides the data storage engine. It uses the majority-based Paxos consensus protocol to provide highly reliable storage and provides Multiversion concurrency control (MVCC) for distributed transactions. It also supports pushdown computation to meet distributed query requirements (such as Project, Filter, Join, and Aggregation pushdowns) and can run on both local disks and shared storage.
Change Data Capture (CDC): Provides a primary/secondary replication protocol compatible with the MySQL ecosystem, including the binary log protocol and data format, and supports standard MySQL replication protocols.
Columnar node: Provides persistent columnar indexes. It consumes the binary logs of distributed transactions in real time and builds columnar indexes on object storage. This enables real-time updates and, when combined with compute nodes, provides snapshot-consistent queries on columnar data.
Core modules
Like most traditional standalone relational databases, PolarDB-X consists of a network layer, SQL parsing layer, optimization layer, execution layer, and storage layer. The optimization layer includes logical and physical optimization. The execution layer supports single-node two-phase execution, single-node parallel execution, and multi-node parallel execution. The storage layer uses many optimization and execution techniques found in traditional standalone databases.
Ecosystem tools
PolarDB-X is highly compatible with the MySQL protocol and ecosystem. It supports common tools from the MySQL open-source ecosystem, such as MySQL drivers (JDBC/ODBC), multiple languages (Java, Go, C, C++, Python), data import and export tools, and various client GUIs. PolarDB-X provides a complete ecosystem tooling solution.
Integrated ecosystem tools:
Data Management Service (DMS): A cloud version of Alibaba Group’s database service platform built on over a decade of Alibaba Group's experience. It provides an installation-free, O&M-free, and ready-to-use web-based database management terminal that unifies multiple database types and environments. It helps enterprises establish database DevOps workflows that match the security, efficiency, and compliance standards of Alibaba Group.
Database Autonomy Service (DAS): A cloud service that uses machine learning and expert knowledge to enable database self-perception, self-healing, self-optimization, self-O&M, and self-security. DAS helps users simplify database management and prevent service failures caused by manual errors, ensuring stable, secure, and efficient database services.
Data Transmission Service (DTS): Supports data sources such as relational databases, NoSQL, and big data. It integrates data migration, subscription, and real-time synchronization capabilities. DTS enables long-distance, millisecond-level asynchronous data transmission in public and hybrid cloud scenarios. Its underlying infrastructure uses the same active geo-redundancy architecture used for Alibaba's Double 11 Shopping Festival, providing real-time data streams to thousands of downstream applications.
Data Disaster Recovery: A low-cost, enterprise-grade, cloud-native backup platform. It provides robust protection for databases in various environments, including on-premises data centers, other cloud providers, public clouds, and hybrid clouds.
Product architecture
As a mature cloud product from Alibaba Cloud, PolarDB-X provides comprehensive productized capabilities, including console-based O&M operations, multiple delivery options, complete OpenAPI support, and integrated ecosystem tools.
The deployment architecture of PolarDB-X is based on Kubernetes and runs on high-performance physical servers. The product architecture is designed as follows:
A PolarDB-X instance consists of multiple nodes. The product is purchased on a per-node basis. The initial purchase must include at least two nodes. Scaling out or in is done in one-node increments. Each node is available in various specifications, such as 4c16g, 8c32g, and 16c64g.
Based on resource isolation requirements, PolarDB-X offers the following families: General-purpose, Dedicated, and Dedicated host. For example, with the General-purpose family, different PolarDB-X instances on the same hardware share idle computing resources like CPU. This resource sharing improves cost-effectiveness through economies of scale. In contrast, with the Dedicated family, each PolarDB-X instance has exclusive access to its allocated computing resources, such as CPU, and does not share them with other clusters on the same server, providing more stable performance.
PolarDB-X supports diverse delivery options for public and hybrid clouds. In the public cloud, it supports deployment across multiple regions and availability zones. For network and security, it supports VPC, IP allowlist, asymmetric account passwords, and Transparent Data Encryption (TDE) to ensure data service security. In a hybrid cloud, it supports lightweight delivery based on DBStack, allowing you to deploy and operate the database on your existing hardware.
Key advantages
High availability
Leveraging the X-DB capabilities (X-Paxos consensus protocol) proven over many years during Alibaba's Double 11 Shopping Festival, PolarDB-X provides strong data consistency and delivers a recovery point objective (RPO) of 0 during node failover. It also supports various deployment and disaster recovery options, such as three data centers in the same city or five data centers across three regions with synchronous Paxos replication. Additionally, it supports configurations like three data centers across two zones, geo-disaster recovery, and active geo-redundancy using asynchronous binary log replication. For long-distance data transmission, network performance is optimized using batching and pipelining techniques.
High compatibility
PolarDB-X is highly compatible with MySQL, including its SQL syntax and function types. It uses a global time service to provide globally consistent distributed transactions. By using TSO and two-phase commit (2PC), it delivers full ACID guarantees and supports Read Committed and Repeatable Read isolation levels in a distributed environment. On top of distributed transactions, PolarDB-X provides a global secondary index. It uses transactional multi-writes to ensure strong consistency between the index and the primary table, while a cost-based optimizer (CBO) handles index selection. For metadata and ecosystem integration, PolarDB-X uses Online DDL technology to ensure metadata consistency in a distributed setting. At the hardware layer, it is compatible with and certified to run on major Chinese operating systems and chips, such as Kirin, Kunpeng, and Hygon.
Furthermore, among mainstream distributed databases, database change logs like redo logs and binary logs are often overlooked by vendors. The history of relational databases shows that a rich ecosystem and adherence to standards are crucial for market adoption. PolarDB-X supports a binary log service, fully embracing the MySQL database ecosystem. You can treat PolarDB-X as a standard MySQL database and use the standard binary log dump protocol to retrieve binary logs.
High scalability
PolarDB-X uses a shared-nothing architecture to support horizontal scaling and enables online database scaling. For OLTP workloads, it supports tens of millions of concurrent requests and petabyte-scale data storage. For OLAP workloads, it incorporates Massively parallel processing (MPP) technology. As you add more machines, query performance scales linearly to meet the demands of complex reporting queries, such as those in TPC-H benchmarks.
HTAP
With the proliferation of mobile internet and IoT devices, data is growing at an explosive rate. Traditional solutions for OLTP and OLAP often rely on simple read/write splitting or ETL models, where data is extracted from online databases to a data warehouse on a T+1 basis. This approach leads to high storage costs, poor real-time performance, and complex pipelines and maintenance. PolarDB-X is designed to support HTAP (Hybrid transactional/analytical processing) workloads, allowing TPC-C and TPC-H benchmarks to run simultaneously on a single instance while ensuring that analytical queries do not affect the stability of transactional traffic. At its core, the compute layer accurately identifies TP and AP traffic and intelligently routes them to different replicas by leveraging multi-replica features and consistent read capabilities. The AP path enables Massively parallel processing (MPP) by default, which linearly improves analytical query performance while maintaining isolation. At the storage layer, its pushdown computation capabilities are continuously improved, and a high-performance columnar engine is planned for future releases to enable a true hybrid row-columnar HTAP capability.
Rapid elasticity
By integrating the cloud-native technology of PolarDB, PolarDB-X leverages PolarDB's shared storage and RDMA network optimizations to provide backups in seconds, rapid elasticity, and on-demand storage scaling. Based on shared storage and combined with distributed multi-point write capabilities, it delivers elasticity in seconds without data migration.
Open ecosystem
PolarDB-X fully embraces the MySQL open-source ecosystem, ensuring compatibility with distributed MySQL while maintaining full control over its own codebase. Its architecture is designed to be simple and open, allowing engineers with a MySQL background to handle ongoing operations and maintenance. In addition, PolarDB-X is fully integrated with Alibaba Cloud's database ecosystem, supporting services such as Data Transmission Service (DTS), Data Disaster Recovery, and Data Management Service (DMS) to connect with the broader Alibaba Cloud ecosystem.