Data Transmission Service:Migrate a standalone self-managed MongoDB database to ApsaraDB for MongoDB by using DTS

Alternatively, you can use MongoDB tools to migrate a self-managed database to a standalone instance.

For more information about other data migration or synchronization solutions, see Data migration and synchronization.

Prerequisites

• The source self-managed MongoDB database and the destination ApsaraDB for MongoDB instance must run supported versions. For more information, see Migration solutions overview.

• We recommend that the destination ApsaraDB for MongoDB instance have at least 10% more storage space than the source self-managed MongoDB database.

Notes

• By default, a standalone self-managed MongoDB database does not support oplog. To use incremental data migration, configure the standalone node as a single-node replica set to enable oplog.

• To avoid service interruptions, perform the data migration during off-peak hours.

• For the supported versions and storage engines of ApsaraDB for MongoDB instances, see Versions and storage engines. If you migrate data across versions or storage engines, verify compatibility in advance.

• The admin and local databases cannot be set as the source or destination database.

• DTS writes data concurrently, which may cause the destination database to use 5% to 10% more storage than the source database.

• Ensure that the destination database does not contain documents with the same primary key (_id) as the source database. Otherwise, data loss may occur. If such documents exist, delete them from the destination database before the migration, provided that this does not affect your services.

Billing

Migration types

• Full data migration: DTS migrates all data from the selected objects in the source MongoDB database to the destination MongoDB database.

  Note

  DTS supports the migration of databases, collections, and indexes.

• Incremental data migration: After full migration completes, DTS synchronizes incremental data changes from the source MongoDB database to the destination MongoDB database.

  Note

  • DTS can synchronize create and delete operations on databases, collections, and indexes.

  • DTS can synchronize create, update, and delete operations on documents.

Database account permissions

To create a database account and grant permissions:

• For a self-managed MongoDB database, see the official MongoDB documentation on db.createUser().

• For an ApsaraDB for MongoDB instance, see Manage database accounts by using DMS.

Procedure

1. Log on to the DTS console.

2. In the left-side navigation pane, click Data Migration.

3. At the top of the Migration Tasks page, select the region of your destination ApsaraDB for MongoDB instance.

4. In the upper-right corner of the page, click Create Data Migration Task.

5. Configure the source and destination databases.

   Category	Parameter	Description
   N/A	Task name	DTS automatically generates a task name. We recommend that you specify a descriptive name for easy identification. The name does not need to be unique.
   Source database	Instance type	Select the deployment location of the source database. This topic uses User-Created Database with Public IP Address as an example. Note If your self-managed database is of a different instance type, you may need to complete additional preparations. For more information, see Preparations for data migration.
   	Instance region	If you select User-Created Database with Public IP Address, you do not need to set the Instance Region. Note If your self-hosted database has a whitelist configured, click Obtain DTS IP Segment after the Instance Region configuration item to obtain the IP addresses of the DTS servers, and add them to the whitelist of your self-hosted database.
   	Database type	Select MongoDB.
   	Hostname or IP address	Enter the connection address of the self-managed MongoDB database. In this example, enter a public IP address.
   	Port	Enter the service port of the self-managed MongoDB database. Note In this example, the service port must be accessible over the public network.
   	Database name	Enter the name of the authentication database that contains the database account.
   	Database account	Enter the account for the self-managed MongoDB database. For the required permissions, see Database account permissions.
   	Database password	Enter the password for the database account. Note After you enter the source database information, click Test Connectivity next to Database Password to verify the information. If the information is correct, a Passed message appears. If a Failed message appears, click Diagnose next to Failed and adjust the source database information based on the prompts.
   	Connection mode	Select Non-encrypted. Note You can select SSL-encrypted only when you migrate data from a MongoDB Atlas database.
   Destination database	Instance type	Select MongoDB Instance.
   	Instance region	Select the region of the destination ApsaraDB for MongoDB instance.
   	MongoDB instance ID	Select the ID of the destination ApsaraDB for MongoDB instance.
   	Database name	Enter the name of the authentication database that contains the database account.
   	Database account	Enter the account for the ApsaraDB for MongoDB instance. For the required permissions, see Database account permissions.
   	Database password	Enter the password for the database account. Note After you enter the destination database information, click Test Connectivity next to Database Password to verify the information. If the information is correct, a Passed message is displayed. If a Failed message is displayed, click Diagnose next to Failed and adjust the destination database information as prompted.

6. After you complete the configuration, click Set Whitelist and Next.

   If the source or destination is an Alibaba Cloud database instance such as ApsaraDB RDS for MySQL or ApsaraDB for MongoDB, DTS automatically adds the CIDR blocks of its servers in the corresponding region to the instance's whitelist. If the source or destination is a self-managed database on an ECS instance, DTS automatically adds its server CIDR blocks to the security rules of the ECS instance's security group. You must also ensure that the self-managed database allows access from the ECS instance. If your database is a cluster deployed across multiple ECS instances, you must manually add the DTS CIDR blocks to the security rules for each additional ECS instance. If the source or destination is a database in an on-premises data center or on another cloud platform, you must manually add the DTS server CIDR blocks to that environment's firewall or security group. For a list of DTS server CIDR blocks, see CIDR blocks of DTS servers.

   Warning

   Adding the public CIDR blocks of DTS servers, whether automatically or manually, may introduce security risks. By using this product, you acknowledge and accept these potential risks. You are responsible for implementing basic security measures, including but not limited to using strong passwords, restricting open ports, using authentication for internal API calls, regularly reviewing and restricting unnecessary network segments, or connecting through private networks such as Express Connect, VPN Gateway, or Smart Access Gateway.

7. Select migration types and objects.

   Parameter	Description
   Migration types	To perform only a full data migration, select Full Data Migration. To migrate data with minimal downtime, select both Full Data Migration and Incremental Data Migration. Note DTS supports incremental data migration for a standalone instance only after you enable oplog for it. If you do not select incremental data migration, do not write new data to the source database during the full data migration to ensure data consistency.
   Migration objects	In the Available box, select the objects to migrate and click the icon to move them to the Selected Objects box. Note DTS does not support migrating the admin, local, or config databases. You can select databases, collections, or functions as migration objects. By default, object names remain the same in the source and destination databases. To rename an object in the destination, use the object name mapping feature. For more information, see Object name mapping.
   Rename mapped object	Use this feature to rename migrated objects in the destination database. For more information, see Object name mapping.
   Connection retry duration	By default, DTS retries connections for 12 hours. You can customize this duration. If the connection to the source or destination database is restored within the specified time, the task resumes automatically. Otherwise, the task fails. Note You are charged for the DTS instance during the retry period. We recommend that you set a reasonable retry duration, or release the DTS instance promptly after the source and destination instances are released.

8. After you complete the configuration, click Precheck and Start in the lower-right corner of the page.

   Note

   • Before the migration task starts, DTS runs a precheck. The task can start only after it passes the precheck.

   • If the precheck fails, click the icon next to the failed item to view details.

     • Fix the issues as prompted and run the precheck again.

     • If you do not need to fix the warning items, you can select Ignore and then click Ignore Warnings and Rerun Precheck to run the precheck again.

9. After the precheck passes, click Next.

10. On the Confirm Settings page, select an Instance Class and select the Data Transmission Service (Pay-As-You-Go) Service Terms checkbox.

11. Click Buy and Start to start the migration task.

    • Full data migration

      Do not manually stop the task, as this may cause incomplete data. Wait for the task to finish. It stops automatically.

    • Incremental data migration

      The task does not stop automatically. You must stop it manually.

      Note

      We recommend that you stop the task at an appropriate time, such as during off-peak hours or before you switch workloads to the destination ApsaraDB for MongoDB instance.

      1. Wait until the task status changes to Incremental Data Migration and shows Undelayed. Then, stop writing data to the source database for a few minutes. The Incremental Data Migration status may temporarily show a delay.

      2. After the Incremental Data Migration status changes back to Undelayed, manually stop the task. At this point, the task status on the Migration Tasks page is Migrating, and the Full Migration progress shows 100%.

12. Switch your workloads to the ApsaraDB for MongoDB instance.