Data Transmission Service (DTS) supports two-way data synchronization between two ApsaraDB for MongoDB instances (sharded cluster architecture). This feature is suitable for scenarios such as active geo-redundancy (unit-based) and geo-disaster recovery. This topic describes how to configure two-way data synchronization between ApsaraDB for MongoDB instances.
Prerequisites
- The source and destination ApsaraDB for MongoDB instances are created. For more information, see Create a sharded cluster instance.
Note For more information about the supported versions, see Overview of data synchronization scenarios.
- The available storage space of the destination ApsaraDB for MongoDB instance is larger than the total size of the data in the source ApsaraDB for MongoDB instance.
- The replication.oplogGlobalIdEnabled parameter of the shard and Configserver nodes is set to true. For more information, see Configure database parameters for an ApsaraDB for MongoDB instance.
- The source and destination ApsaraDB for MongoDB instances are deployed in a sharded cluster architecture. You must create databases
and collections to be sharded in the source and destination ApsaraDB for MongoDB instances and configure sharding based on your business needs. For more information,
see Configure sharding to maximize the performance of shards.
Note
- After you configure sharding for a sharded cluster instance, the synchronized data will be distributed among different shards. This maximizes the performance of shards in the sharded cluster instance.
- The shard nodes in a sharded cluster instance must share the same account password and endpoint. For more information, see Apply for an endpoint for a shard or Configserver node.
- In this scenario, you can configure a DTS task and then purchase a DTS instance. You
do not need to specify the number of shards in the source ApsaraDB for MongoDB instance (sharded cluster architecture).
If you purchase a DTS instance before you configure a DTS task, you must specify the number of shards when you purchase the instance.
Limits
Category | Description |
---|---|
Limits on the source and destination databases |
|
Other limits |
|
Billing
Synchronization type | Task configuration fee |
---|---|
Schema synchronization and full data synchronization | Free of charge. |
Incremental data synchronization | Charged. For more information, see Pricing. |
Supported synchronization topologies
DTS supports two-way data synchronization only between two ApsaraDB for MongoDB instances (sharded cluster architecture). DTS does not support two-way data synchronization between multiple ApsaraDB for MongoDB instances.
Conflict detection
To ensure data consistency, make sure that data records with the same primary key, business primary key, or unique key are updated only on one of the synchronization nodes. If data records are updated on both nodes, DTS responds to conflicts based on the conflict resolution policy that you specify for the data synchronization task.
- Uniqueness conflicts caused by INSERT operations
INSERT operations that do not comply with the uniqueness constraint cannot be synchronized. For example, if a record with the same primary key value is inserted into the two synchronization nodes at almost the same time, one of the inserted records fails to be synchronized. The synchronization fails because a record with the same primary key value already exists in the other node.
- Inconsistent records caused by UPDATE operations
- If the records to be updated do not exist in the destination instance, DTS converts the UPDATE operation into an INSERT operation. However, uniqueness conflicts may occur.
- The primary keys or unique keys of the records to be inserted may conflict with those of existing records in the destination instance.
- Non-existent records to be deleted
The records to be deleted do not exist in the destination instance. In this case, DTS ignores the DELETE operation regardless of the conflict resolution policy that you specify.
- During two-way synchronization, the system time of the source and destination instances may be different. Synchronization latency may occur. For these reasons, DTS cannot guarantee that the conflict detection mechanism can prevent all data conflicts. To perform two-way synchronization, make sure that records with the same primary key, business primary key, or unique key are updated only on one of the synchronization nodes.
- DTS provides conflict resolution policies to prevent conflicts that may occur during data synchronization. You can select a conflict resolution policy when you configure two-way data synchronization.
Synchronization types
Synchronization type | Description |
---|---|
Schema synchronization | DTS synchronizes the schemas of the required objects from the source ApsaraDB for MongoDB instance to the destination ApsaraDB for MongoDB instance. |
Full data synchronization | DTS synchronizes historical data of the required objects from the source ApsaraDB for MongoDB instance to the destination ApsaraDB for MongoDB instance.
Note DTS supports full data synchronization for the following types of objects: database,
collection, and index.
|
Incremental data synchronization | DTS synchronizes incremental data from the source ApsaraDB for MongoDB instance to the destination ApsaraDB for MongoDB instance.
Note DTS supports the following types of incremental updates:
|
Procedure
If you purchase a DTS instance before you configure a DTS task, you must specify the number of shards when you purchase the instance.