PolarDB-X does not provide binary logs. To ensure data quality, we recommend that you take note of the limits when you perform business design, business development, and O&M changes.

Overview

Limits related to business design

  • All tables must have primary keys. Otherwise, data inconsistency may occur (the destination database may contain duplicate data records).
  • We recommend that you do not use the global secondary indexes (GSIs) of PolarDB-X because they are updated asynchronously. If you use GSIs, DTS can guarantee only eventual consistency of data.
  • The databases that you want to synchronize cannot be deployed in mixed mode (where the unit mode and the copy mode are mixed).
    Note In unit mode, users perform read and write operations in their respective unit nodes. Two-way synchronization is implemented between the databases in each unit node and those in the central node. In copy mode, users write data to the databases in the central node. The data is then synchronized to the databases in each unit node.
  • If you use the underlying MySQL databases of PolarDB-X to configure two-way data synchronization tasks, you must convert the Float and Double data types into Decimal for business tables. If you use PolarDB-X as the source instance of a one-way data synchronization task, a data migration task, or a change tracking task, you do not need to convert the Float and Double data types.
  • Data synchronization tasks, data migration tasks, and change tracking tasks do not support the following objects of PolarDB-X: stored procedures, triggers, functions, views, and events.
  • DTS does not support schema synchronization for PolarDB-X. You must manually create objects such as databases and tables in the destination instance.
  • The source PolarDB-X instance must have sufficient capacity to support business growth.
  • If MySQL databases of version 5.7 and 8.0 run on the PolarDB-X instance, you cannot use the PolarDB-X instance as the source instance of a change tracking task. You must configure a change tracking task for each MySQL database to track and consume data from the PolarDB-X instance.

Limits related to database architecture

  • The ApsaraDB RDS for MySQL instances used by a PolarDB-X instance cannot be used by other PolarDB-X instances.
  • For data synchronization or migration between PolarDB-X instances, the source and destination ApsaraDB RDS for MySQL instances must have equivalent deployment. For example, if the source PolarDB-X instance uses four ApsaraDB RDS for MySQL instances, the destination PolarDB-X instance must also use four ApsaraDB RDS for MySQL instances with the same specifications.
  • The sharding rules of the source and destination PolarDB-X instances must be the same. Otherwise, the data synchronization or migration task cannot be created.
  • You can synchronize, migrate, or track the data of business tables in an PolarDB-X instance. You cannot synchronize, migrate, or track the data of metadata tables or system tables in the instance.

Limits related to O&M changes

Category Description Impact and solution
PolarDB-X Change a sharding rule, for example, change the shard key of a database or table or change the number of shards. Not supported. You must perform the following steps to recreate the task:
  1. Stop and delete the original DTS task.
  2. Wait until the changes are completed in the source database. Then, clear the data that has been synchronized or migrated to the destination database.
  3. Configure a data synchronization or migration task for each ApsaraDB RDS for MySQL instance in the PolarDB-X instance.
Change the number of instances at the storage layer, for example, scale out instances and migrate frequently-accessed tables.
Storage layer Change the specifications of instances and switch workloads at the storage layer. DTS tasks are not affected.
Change parameter settings. The parameter settings of the source and destination databases must be the same. To change the parameter settings of instances at the storage layer, you must make sure that new parameters do not affect previous parameters.
Note If you are not sure about the impact of changing parameter settings, you can contact technical support of Database Expert Service.
Change backup and recovery policies, and enable auditing and diagnostics for instances at the storage layer. The change takes effect only on the current instance and does not affect other instances with replication relationships.
DTS task Perform DDL operations. If you configure a DTS task to replicate data from multiple ApsaraDB RDS for MySQL instances in a PolarDB-X instance to the destination database, performing DDL operations may cause task latency.
DDL operations at the database or table level Add tables. Not supported. You must perform the following steps:
  1. After you perform DDL operations to create tables in the destination database, perform the same DDL operations to create tables in the source database.
  2. Add the new tables to the objects of the data synchronization task.
You can write data to the source database only after the preceding operations are completed. If you select tables as the objects of the data synchronization task, you must add the new tables to the synchronization queues on the source and destination instances at the storage layer.
Add fields, add secondary indexes, delete indexes, and modify indexes (except for replacing secondary indexes with unique indexes).
  • If you configure a DTS task based on a PolarDB-X instance, you must perform DDL operations in the destination database and then perform the same DDL operations in the source database.
  • If you configure a DTS task based on ApsaraDB RDS for MySQL instances attached to a PolarDB-X instance, DTS automatically updates the operations. The following operations are supported: Add fields, add secondary indexes, delete indexes, and modify indexes (except for replacing secondary indexes with unique indexes).
Perform other DDL operations. Only the preceding DDL operations are supported.
Switchover
Note Switchover: After you use DTS to synchronize or migrate data from the source database to the destination database, you switch workloads from the source database to the destination database.
Perform a switchover. Before you perform a switchover, make sure that the DTS task is not delayed. Otherwise, data quality issues occur.
Perform a failover that meets the requirements of recovery point objective (RPO).
Note RPO represents the maximum amount of data that can be lost after a recovery from a failure. RPO is measured by time.
Warning Failover: If the source instance or the data center where the source instance resides fails, you can switch workloads to a backup system. A failover is a lossy operation.
If a failure (such as network interruption, equipment failure, or Internet data center failure) occurs and the DTS task is delayed, you may need to perform a failover. In this case, if the difference between the time when the last data entry is updated to the destination database and the time when the failure occurs is less than the RPO, you can perform a failover to recover your business. For example, if the RPO is 5 minutes, the quality of the data within the 5 minutes cannot be guaranteed after you perform a failover. You may need to revise the data to ensure consistency.
Perform a failover that does not meet the requirements of RPO. The DTS task may be delayed because of the following reasons: a large number of DDL operations is performed in the source database, a network failure occurs, and the performance of the destination database is unfavorable. In this case, if the data center fails and the difference between the time when the last data entry is updated to the destination database and the time when the failure occurs is greater than the RPO, we recommend that you wait until the data center recovers before you perform a failover. For example, if the RPO is 5 minutes, the quality of the data within the 5 minutes cannot be guaranteed after you perform a failover. You may need to revise the data to ensure consistency.

Potential risks related to data quality

Some changes or switchover operations may cause data quality issues such as schema inconsistency between the source and destination databases.

  • If data latency occurs between the primary and secondary databases of the source instance, the data written to the primary database is not updated to the secondary database in a timely manner. In this case, if you perform a primary/secondary switchover in the source instance, DTS uses the secondary database of the source instance as the source database for data synchronization, data migration, or change tracking. As a result, the data that is not updated to the secondary database is lost.
  • If the DTS task is resumed from a network failure after you perform a switchover, DTS attempts to synchronize, migrate, or track the data generated before the failure occurs. This mechanism prevents data loss in the destination database. In this case, if the destination tables do not have primary keys, data will be inconsistent between the source and destination databases. If the destination tables have primary keys, data may not be consistent when DTS implements the retry mechanism, but data will remain consistent after the retry ends.
  • The DTS task may be delayed due to network failures and DDL operations.
  • The DTS task may be delayed or interrupted due to changes to the source database, unfavorable performance of the destination database, and schema inconsistency.

Alibaba Cloud cannot solve the preceding issues. You must recreate a DTS task or adjust the source and destination databases.

Suggestions to ensure data quality

  • You must perform all DDL operations with caution. All DDL operations must be confirmed by the technical engineers to comply with the preceding limits.
  • Do not directly perform DDL operations in your program code.