This topic describes how to migrate data from a replica set instance to a sharded cluster instance by using Data Transmission Service (DTS). DTS supports both full data migration and incremental data migration. You can use these two methods together to migrate data to ApsaraDB for MongoDB without interruptions to your business.

Prerequisites

Make sure that the shards in the destination sharded cluster instance have sufficient storage space.

Precautions

  • We recommend that you migrate your data during off-peak hours to avoid business interruptions.
  • If the source and destination instances run different database versions or storage engines, ensure there are no compatibility issues between them before you start migration. For more information about the database versions and storage engines supported by ApsaraDB for MongoDB, see MongoDB versions and storage engines.

Billing

Migration type Link configuration fee Internet traffic fee
Full data migration Free of charge Charged only when data is migrated from Alibaba Cloud over the Internet. For more information, see Data Transmission Service pricing.
Incremental data migration Charged. For more information, see Data Transmission Service pricing.

Migration types

Migration type Description
Full data migration All data of the migration objects is migrated from the source instance to the destination instance.
Note Data migration is supported at the database, collection, and index levels.
Incremental data migration Updated data of the migration objects is synchronized from the source instance to the destination instance.
Note
  • The create and delete operations on databases, collections, and indexes can be synchronized.
  • The create, delete, and update operations on documents can be synchronized.

Required database account permissions

Data source Full data migration Incremental data migration
Source ApsaraDB for MongoDB replica set instance Read permissions on the source database Read permissions on the source database, admin database, and local database
Destination ApsaraDB for MongoDB sharded cluster instance Read/write permissions on the destination database Read/write permissions on the destination database
Note For more information about how to create and authorize a database account, see Use DMS to manage MongoDB users.

Preparation

Create databases and collections to be sharded in the destination ApsaraDB for MongoDB instance, and configure data sharding based on your business needs. For more information, see Configure sharding to maximize the performance of shards.

Note After data sharding is configured, data is no longer migrated to the same shard. This helps maximize cluster performance.

Procedure

  1. Log on to the DTS console.
  2. In the left-side navigation pane, click Data Migration.
  3. In the Migration Tasks section, select the region where the destination ApsaraDB for MongoDB instance resides.
    Select a region
  4. In the upper-right corner, click Create Migration Task.
  5. Configure both the source and destination databases.
    Configure source and destination databases
    Section Parameter Description
    N/A Task Name DTS automatically generates a task name. We recommend that you specify your own task name that helps identify the task. Task names do not need to be unique.
    Source Database Instance Type Select ApsaraDB for MongoDB.
    Instance Region Select the region where the source ApsaraDB for MongoDB instance resides.
    MongoDB Instance ID Select the ID of the source ApsaraDB for MongoDB instance.
    Database Name Enter the name of the authentication database. It is the database where the database account is created.
    Note If the database account is root, enter admin.
    Database Account Enter the username of the database account you use to manage the source database. For more information about the account permission requirements, see Required database account permissions.
    Database Password Enter the password of the database account.
    Note After you specify the source database information, click Test Connectivity next to Database Password to check whether the information is correct. If the information is correct, the Passed message is displayed. If the information is incorrect, the Failed message is displayed, and you must click Check next to the Failed message to modify the information as prompted.
    Destination Database Instance Type Select MongoDB Instance.
    Instance Region Select the region where the destination ApsaraDB for MongoDB instance resides.
    MongoDB Instance ID Select the ID of the destination ApsaraDB for MongoDB instance.
    Database Name Enter the name of the authentication database. It is the database where the database account is created.
    Note If the database account is root, enter admin.
    Database Account Enter the username of the database account you use to manage the destination database. For more information about the account permission requirements, see Required database account permissions.
    Database Password Enter the password of the database account.
    Note After you specify the destination database information, click Test Connectivity next to Database Password to check whether the information is correct. If the information is correct, the Passed message is displayed. If the information is incorrect, the Failed message is displayed, and you must click Check next to the Failed message to modify the information as prompted.
  6. In the lower-right corner, click Set Whitelist and Next.
    Note The IP addresses of DTS servers are automatically added to the whitelists of the source and destination ApsaraDB for MongoDB instances. This ensures that the DTS servers can connect to the two ApsaraDB for MongoDB instances. After the migration is complete, you can remove these IP addresses from the whitelists if you no longer need them. For more information, see Configure a whitelist for a sharded cluster instance.
  7. Configure migration types and migration objects.
    Configure migration types and migration objects
    Parameter Description
    Migration Types
    • If you want to migrate all data, select Full Data Migration.
    • If you want to migrate data without interruptions to your business, select both Full Data Migration and Incremental Data Migration.
    Note If Incremental Data Migration is not selected, do not write data into the source database during full data migration. This ensures data consistency between the source and destination databases.
    Available
    • In the Available section, select the objects you want to migrate and then click the Rightwards arrow icon to move them to the Selected section.
      Note Data in the admin database cannot be migrated even if this database is selected.
    • A migration object can be a database, collection, or function.
    • By default, the name of an object remains unchanged after migration. If you want a different object name after migration, use the object name mapping feature provided by DTS. For more information, see Object name mapping.
  8. In the lower-right corner, click Precheck.
    Note
    • A precheck is performed before the migration task starts. The migration task starts only after the precheck succeeds.
    • If the precheck fails, click the Tip icon for each failed check item to view their details. Perform a precheck again after the failures are fixed.
  9. After the precheck succeeds, click Next.
  10. On the Confirm Settings page, set Channel Specification and select Data Transmission Service (Pay-As-You-Go) Service Terms.
  11. Click Buy and Start to start the migration task.
    • Full data migration

      Do not manually end a migration task. If you do so, the system may fail to migrate all data of the database. Wait until the migration task is complete.

    • Incremental data migration

      An incremental data migration task does not automatically end. You need to manually end the task.

      Note Select an appropriate point in time to manually end a migration task. For example, you can end the migration task during off-peak hours or before you switch over your business to the destination ApsaraDB for MongoDB instance.
      1. When the task progress bar displays Incremental Data Migration and The migration task is not delayed, stop writing data to the source database for a few minutes. Wait until the progress bar displays the delay time of the incremental data migration next to Incremental Data Migration.
      2. After the status of Incremental Data Migration changes to The migration task is not delayed, manually end the migration task.Incremental data migration without delay
  12. Switch over your business to the destination ApsaraDB for MongoDB instance.