This topic describes how to incrementally migrate a MongoDB Atlas database to Alibaba Cloud through Data Transmission Service (DTS). DTS supports the full migration and incremental migration methods. You can use both methods to migrate the MongoDB Atlas database without business interruption.

Prerequisites

The storage space of ApsaraDB for MongoDB instances must be greater than that of the MongoDB Atlas database.

Precautions

  • We recommend that you migrate your data during off-peak hours to prevent negative impacts on your business.
  • Data in the admin and local database is not migrated even if the admin database is selected.
  • The config database is an internal database. Do not migrate this database unless otherwise essential to your business.
  • For more information about the versions and storage engines supported by ApsaraDB for MongoDB instances, see MongoDB versions and storage engines. If you want to migrate the database between different versions or engines, confirm compatibility of these versions and engines first.

Billing

Migration type Configuration fee Public traffic fee
Full data migration Free of charge. You are charged for migrating data from the Internet to Alibaba Cloud. For more information, see Pricing.
Incremental data migration Charged. For more information, see Pricing.

Migration types

Migration type Description
Full data migration All data of the migration objects is migrated from the source instance to the destination instance.
Note Data migration is supported at the database, collection, and index levels.
Incremental data migration Updated data of the migration objects is synchronized from the source instance to the destination instance.
Note
  • The create and delete operations on databases, collections, and indexes can be synchronized.
  • The create, delete, and update operations on documents can be synchronized.

Account permissions for the database

Database Full data migration Incremental data migration
MongoDB Atlas database Permissions to read the database to be migrated and perform the listDatabases operation.
  • Permissions to read the database to be migrated, admin database, and local database.
  • Permissions to perform the listDatabases operation.
ApsaraDB for MongoDB instance Permissions to read and write the destination database. Permissions to read and write the destination database.

How to create a database account and grant permissions to the account:

Preparations

  1. Log on to the MongoDB Atlas console.
  2. In the left-side navigation pane, click Network Access. Click ADD IP ADDRESS on the page.
    Create an Atlas whitelist
  3. In the dialog box that appears, click ALLOW ACCESS FROM ANYWHERE and then Confirm.
    Configure the Atlas whitelist
    Note This step allows all IP addresses to access the MongoDB Atlas database. Delete this rule after the migration.

Procedure

  1. Log on to the Data Transmission Service console.
  2. In the left-side navigation pane, click Data Migration.
  3. At the top of the Migration Tasks page, select the region where the ApsaraDB for MongoDB instance is located.
    Select a region
  4. In the upper-right corner, click Create Migration Task.
  5. Configure the source and destination databases of the migration task.
    Configure the source and destination databases
    Section Parameter Description
    Task Name N/A DTS automatically generates a task name. We recommend that you use an identifiable name to simplify future management. The name is not necessarily unique.
    Source Database Instance Type Select User-Created Database with Public IP Address.
    Instance Region When Instance Type is set to User-Created Database with Public IP Address, you do not need to set Instance Region.
    Database Type Select MongoDB.
    Hostname or IP Address

    Enter the endpoint of the PRIMARY node in the MongoDB Atlas database.

    You can obtain the endpoint in the MongoDB Atlas console, as shown in the following figure.

    Obtain the Atlas endpoint
    Port Number Enter the service port of the MongoDB Atlas database. The default port is 27017.
    Database Name Enter the name of the database used for account authentication.
    Database Account Enter the account of the MongoDB Atlas database. For more information about permission requirements, see Account permissions for the database.
    Database Password Enter the password of the database account.
    Note After the source database configuration is complete, you can click Test Connectivity after the Database Password field to check whether the source database configuration is correct. If the source database configuration is correct, the Test Passed message is displayed. If the Test Failed message is displayed, click Diagnose in the Test Failed message. Modify the source database configuration as prompted.
    Encryption Select SSL-encrypted.
    Destination Database Instance Type Select MongoDB Instance.
    Instance Region Select the region of the ApsaraDB for MongoDB instance.
    MongoDB Instance ID Select the ID of the ApsaraDB for MongoDB instance.
    Database Name Enter the name of the database used for account authentication.
    Note If you use the root account, the database name is admin.
    Database Account Enter the database account of the ApsaraDB for MongoDB instance. For more information about permission requirements, see Account permissions for the database.
    Database Password Enter the password of the database account.
    Note After the destination database configuration is complete, you can click Test Connectivity after the Database Password field to check whether the destination database configuration is correct. If the destination database configuration is correct, the Test Passed message is displayed. If the Test Failed message is displayed, click Diagnose in the Test Failed message. Modify the destination database configuration as prompted.
  6. After you configure the parameters, click Set Whitelist and Next in the lower-right corner.
    Note Then the IP address of the DTS server is automatically added to the whitelist of the ApsaraDB for MongoDB instance. This ensures that the DTS server can connect to the ApsaraDB for MongoDB instance.
  7. Configure the migration object and migration type.
    Item Description
    Migration type
    • If you want to migrate all data, select Full Data Migration.
    • If live migration is required, select both Full Data Migration and Incremental Data Migration.
    Note If Incremental Data Migration is not selected, do not write data into the source database during full data migration. This ensures data consistency between the source and destination databases.
    Migration object
    • Select the database to be migrated from the Available list and click Right arrow to move it to the Selected list.
      Note Data in the admin and local database is not migrated even if the admin database is selected.
    • The migration object can be a database or a collection/function.
    • By default, the object name will remain the same after it is migrated. If the object you migrate has a different name in the ApsaraDB for MongoDB instance, you must use the object name mapping feature provided by DTS. For more information, see Object name mapping.
  8. After you configure the preceding settings, click Precheck.
    Note
    • A pre-check must be performed before a migration task is formally started. The migration task can be started only after a successful pre-check.
    • If the pre-check fails, click the Tip icon after each check item to view the failure details. Perform the pre-check again after you have rectified the failed items.
  9. After the pre-check is successful, click Next.
  10. On the Confirm Settings page, select Channel Specification and Data Transmission Service (Pay-As-You-Go) Service Terms.
  11. Click Buy and Start to start the migration task.
    • Full data migration

      Do not manually stop a full data migration task. Otherwise, the system may fail to migrate the full data. Wait until the migration task ends automatically.

    • Incremental data migration

      An incremental data migration task does not end automatically. You must manually stop the migration task.

      Note Select an appropriate time to manually stop the migration task, such as during off-peak hours or when you are ready to switch your business to an ApsaraDB for MongoDB instance.
      1. When Incremental Data Migration and The migration task is not delayed. are displayed in the progress bar of the migration task, stop writing data to the source database for a few minutes. At this time, the latency time of Incremental Data Migration may be displayed in the progress bar of the migration task.
      2. When Incremental Data Migration and The migration task is not delayed. are displayed again in the progress bar of the migration task, manually stop the migration task.Incremental migration task with no latency
  12. Switch your business to the ApsaraDB for MongoDB instance.