PolarDB:DDL replication

Efficient DML replication implements table consistency between the upstream and downstream clusters of a GDN. DDL replication ensures schema consistency between the primary and secondary clusters of a GDN. Schema consistency between the primary and secondary clusters of a GDN means not only the consistency of the table structure, but also the consistency of all database objects. This ensures that secondary clusters can carry application traffic when a failover occurs. Otherwise, even losing an index can cause a serious performance issue.

DDL types

PolarDB-X supports multiple types of database objects, including not only common MySQL-compatible tables, columns, indexes, views, and functions, but also custom object types such as table groups, sequences, global secondary indexes (GSIs), clustered columnar indexes (CCIs), and custom functions. PolarDB-X also provides custom extended syntax, such as TTL, locality, online modify column (OMC), and local indexes. DDL replication is challenging both in terms of compatibility and the complexity of distributed scenarios. Therefore, PolarDB-X provide the built-in DDL replication capabilities which are characterized by simplicity, reliability, and ease of use.

The following table describes the supported DDL types.

Table group types

PolarDB-X supports two types of table groups: explicit table groups and implicit table groups.

• You can create an explicit table group by executing the CREATE TABLEGROUP statement, delete it by executing the DROP TABLEGROUP statement, and modify it by executing the ALTER TABLEGROUP statement. For more information, see Table groups.

• Implicit table groups are implicitly created by PolarDB-X. When you create a partitioned table, delete a partitioned table, or change the partition type of a table, implicit table groups may be created, modified, or deleted. For implicit table groups, the CREATE TABLEGROUP and DROP TABLEGROUP statements cannot be used, but the ALTER TABLEGROUP statement can be used. Implicit table groups are named maintaining a sequence number list: tg1, tg2, tg3, and so on. Sequence numbers are generated with uncertainty. For example, if partitioned tables T1 and T2 are created in parallel, the sequence numbers of the implicit table groups for these two tables are random. In addition, some sequence numbers may be skipped. For example, if the sequence number of a table group is generated by adding 1 to an existing sequence number and when the DDL task flow is rolled back, the generated sequence number is skipped.

Implicit table groups pose a challenge to primary/secondary replication in PolarDB-X. If the primary cluster and secondary clusters generate their own implicit table groups based on the default rules, the names of the implicit table groups for the same tables in the primary cluster and secondary clusters may be inconsistent, or even the names of all implicit table groups may be confused. For GDNs, implicit table groups are replicated by extending DDL statements. When a DDL statement involves the change of an implicit table group, the extended DDL statement is recorded in the binary log of the primary cluster.

create table if not exists tb1 (
  a int PRIMARY KEY,
  b int,
  c int,
  d varchar(10) UNIQUE,
  INDEX b(b),
  INDEX b_2(b),
  KEY b_3 (b),
  KEY b_4 (b),
  UNIQUE KEY b_5 (b),
  UNIQUE KEY b_6 (b),
  UNIQUE INDEX b_7 (b),
  UNIQUE INDEX b_8 (b),
  INDEX g1(b),
  KEY g2 (b),
  UNIQUE KEY g3 (b),
  UNIQUE INDEX g4 (b)
) DEFAULT CHARACTER SET = utf8mb4 DEFAULT COLLATE = utf8mb4_general_ci;

ALTER TABLE tb1 MERGE PARTITIONS p1, p2 TO p12;

ALTER TABLE tb1 MERGE PARTITIONS p1, p2 TO p12 WITH TABLEGROUP=tg29 IMPLICIT;

Multi-stream replication

In single-stream replication mode, the DDL statements of the primary cluster are replicated to the secondary clusters one by one based on the sequence in the binary log. However, in multi-stream replication scenario, you need to consider coordinating different replication links in DDL replication. Each replication link must wait for other replication links after receiving a DDL statement. The DDL statement can be replicated to secondary clusters only after all links receive it. Otherwise, inconsistency between DML traffic and schemas will occur, and exceptions or data errors are thrown.

For multi-stream replication, PolarDB-X provides the distributed DDL replication engine to implement DDL replication consistency in distributed scenarios using efficient multi-link coordination algorithm.

Limits