As business data volumes and access pressure keep growing, single-node databases may face bottlenecks in storage capacity and read/write performance. The multi-master cluster partitioned tables of PolarDB for MySQL support horizontally splitting data into multiple physical shards by distribution keys, with multiple read/write nodes jointly carrying the business load. This solution combines the elasticity of shared storage with the horizontal scaling capability of distributed computing, and can be used to improve the concurrent throughput capability of large-scale workloads.
Product architecture
Multi-master cluster partitioned tables adopt a storage-compute separation architecture:
Storage layer: The storage layer pools storage resources (shared storage) and provides on-demand resource allocation.
Compute layer: The compute layer allows multiple nodes to read and write at the same time (shared-nothing) to horizontally scale out the read/write performance.
Scope of application
Currently, only the 8.0.2 kernel version of PolarDB for MySQL supports multi-master cluster partitioned tables.
Core advantages and capabilities
Automatic horizontal splitting
When you create a multi-master cluster partitioned table, you can specify a distribution key, and the system automatically distributes the data to different physical shards according to the distribution rules.
Transparent to business
The usage of multi-master cluster partitioned tables is similar to that of regular tables, and applications generally do not need to be aware of the physical distribution of the data.
Horizontal write scale-out
Multiple read/write nodes can carry write tasks at the same time, and compute resources can be flexibly scaled according to the business load.
Integrated distributed DDL
Provides a fully integrated distributed DDL framework, ensuring that all related operations such as adding/removing indexes and modifying column definitions can be completed smoothly in a unified and transparent process, which greatly simplifies the maintenance workflow.
Global read capability
Reads the global data in the cluster through global read-only nodes to carry cross-database and cross-shard read-only queries, without the need to build additional aggregation clusters or DTS synchronization links.
Scenarios
Multi-master cluster partitioned tables are a data organization form provided by Multi-master Cluster (Limitless), mainly designed for application scenarios with large data volumes and high-concurrency reads and writes, such as e-commerce and gaming.
E-commerce transaction scenarios
For large tables, high-concurrency writes, and hotspot workloads, distribution keys are used to distribute the data and access pressure across multiple read/write nodes, reducing the pressure on a single node.
Database and table sharding scenarios
For workloads that have already adopted the database and table sharding architecture, transparent data splitting and integrated distributed DDL capabilities are provided, reducing the costs of application modification and daily maintenance.
Performance comparison
Sysbench was used to run stress tests on two mainstream commercial distributed databases (distributed database 1 and distributed database 2) on the market, as well as the PolarDB multi-master cluster partitioned tables:
Test description: On the purchase pages, purchase distributed database 1, distributed database 2, and the PolarDB multi-master partitioned tables respectively, ensuring that the purchased clusters have the same cost.
Test procedure: Sysbench runs on an ECS test instance with 32 cores and 128 GB of memory, and performance tests are performed on each distributed database through 128 concurrent request threads. Four workload patterns are tested: oltp_point_select, oltp_read_only, oltp_write_only, and oltp_read_write, in order to evaluate the performance of each distributed database.
Test results: The following is the performance comparison of each distributed database, taking QPS as an example:
Under the preceding test environment and workloads, the PolarDB multi-master cluster partitioned tables show good overall performance. The actual performance may be affected by factors such as instance specifications, data scale, data distribution, and business load.