edit-icon download-icon

Create a data synchronization task

Last Updated: Dec 19, 2017

This document introduces how to create a data synchronization task in the DTS console. An example of building a synchronization link between two RDS for MySQL instances is used to describe the procedure.

Prerequisites

The data synchronization function applies restrictions on instance types, synchronization architectures, and trigger objects. Before starting the configuration, make sure that the following restrictions are well met.

Instance types

Currently, real-time synchronization is available for RDS for MySQL instances.

  • The target instance does not support RDS instances that use standard access mode and the Internet addresses.
  • RDS instances in Hong Kong Zone A are unavailable.

Synchronization architectures

Real-time synchronization supports the following types of synchronization architectures:

  • A->B, or unidirectional synchronization between two instances.

    1-to-1 Unidirectional architecture

    The synchronization objects on B must be granted read-only permission only. Otherwise, it may cause synchronization link exceptions.

  • A->B/C/D, or 1-to-N distributed synchronization architecture.

    1-to-N Distributed architecture

    The number of target RDS instances can be set on your demand, but the synchronization objects on the target instances must be granted read-only permission only. Otherwise, it may cause synchronization link exceptions.

  • B/C/D->A, or N-to-1 data convergence architecture.

    N-to-1 Convergence architecture

    The synchronization objects on each synchronization instance must be different from each other, ensuring complementary integration.

The following architectures are NOT available:

  • A->B->C, or cascade architecture.

    Cascade architecture

  • A->B->A, or bidirectional synchronization between Instance A and Instance B.

    Bidirectional architecture

If unsupported architectures are detected in the synchronization configuration, the pre-check returns a Complex Topology error.

Incompatible triggers

For full database synchronization, if the source database contains triggers that can update the content of the synchronized table within the database, data inconsistency may occur between the source and target instances.

Suppose that the source instance contains Table A and Table B. When a row is inserted into Table A, a trigger in Table A inserts a row into Table B. The synchronization task synchronizes the source instance to the target instance, where Table A and Table B are represented as Table A’ and Table B’.

During synchronization, the target instance continuously replicates full amount data from the source instance, including Table A and Table B. If you insert a row (1) into Table A, the trigger in A inserts a row (2) into Table B. The row you inserted (1) and the row inserted by the trigger (2) are synchronized respectively to the target instance Table A’ and Table B’. When the row you inserted (1) is updated to Table A’, the trigger in A’ inserts a row (2’) in Table B’.

As a result, one row insertion (1) in the source Table A triggers two row insertions (2 and 2’) in the target Table B. Therefore, the data between source and target is inconsistent.

To avoid this problem, delete the trigger in the target instance and synchronize Table B from the source instance.

Procedure

Before going ahead, make sure you have at least one synchronization instance available, or you can purchase one on the Data Transmission Service purchase page. For specific instructions, see Purchase DTS instances.

Follow these steps to create a synchronization task in the DTS console.

  1. Log on to the Alibaba Cloud DTS console and enter the Data Synchronization page.

  2. Locate the synchronization instance to be configured and click Configure synchronization instance in the Operation list.

  3. Select the source and target instances. The parameters include:

    • Synchronization task name: Enter a name for the synchronization task. The name is not required to be unique.
    • Source instance information: Select the source Instance ID, and specify the Connection method.
    • Target instance information: Select the target Instance ID, and specify the Connection method.

    Note:

    • The source and target instances must be different instances.
    • The connection methods include Non-encrypted connection and SSL secure connection. The secure connection consumes more CPU resources.
  4. After the configuration, click Authorize whitelist and enter into next step.

  5. Wait for the backend process to add the IP address of the DTS server to the specified RDS instance whitelists and create a synchronization account on the target RDS instance. After the process is complete, click Next.

    • By updating the RDS instance whitelists, the DTS server is granted access permissions. This can avoid connecting failures due to an RDS whitelist restriction. We recommend that you do not remove the server IP addresses from the whitelists of RDS instances during synchronization.
    • DTS automatically creates a synchronization account on the target RDS instance, and names it dtssyncwriter. We recommend that you do not remove this account during synchronization. Otherwise, it may cause synchronization task interruption.
  6. Select the synchronization object (database or table). You can specify multiple databases or tables.

    • If you specify the entire database, all the DDL operations in the database (such as create table and drop view) are replicated to the target database.
    • If you specify a table, only drop/alter/truncate/rename table and create/drop index operations in the table are replicated to the target database.

    Note: The renaming table operation may cause inconsistent data after synchronization.

    For example, assume that the synchronization objects include Table A, but do not include Table B. If the source instance renames A to B during synchronization, all data changes in the source Table B are not replicated to the target instance. To avoid such a problem, choose to synchronize the entire database which includes both Table A and Table B.

  7. Click Next.

  8. Perform Advanced setup. DTS creates a replica of synchronization objects on the target instance as the base-line data.

    The setup includes two types: schema initialization and existing data initialization. Typically, both of the initialization types are required.

  9. Perform Pre-check. DTS examines a predefined checklist for possible causes of failure.

  10. Click Start to start the synchronization link. After being started, the synchronization task is added to the synchronization list.

    Note: The link stays in Synchronization Initialization status for a period of time, depending on the size of the synchronized objects.

    When the initialization is complete, the synchronization instance status becomes Synchronizing, and the synchronization link is established between the source and target instances.

Thank you! We've received your feedback.