All Products
Search
Document Center

ApsaraDB for MongoDB:Migrate data from an ECS-hosted self-managed MongoDB database that uses the standalone or replica set architecture to ApsaraDB for MongoDB

Last Updated:Jun 06, 2024

This topic describes how to use Data Transmission Service (DTS) to migrate a self-managed MongoDB database that is hosted on Elastic Compute Service (ECS) and uses the standalone or replica set architecture to ApsaraDB for MongoDB. DTS allows you to migrate historical and incremental data without business interruptions.

This section describes how to perform data migration in the DTS console of the old version. For more information about how to perform data migration in the DTS console of the new version, see the following topics:

Prerequisites

  • The version of the self-managed MongoDB database is MongoDB 3.0 or later.

  • The storage space of the ApsaraDB for MongoDB instance is larger than the size of the self-managed MongoDB database.

Usage notes

  • To prevent service interruptions, we recommend that you migrate data during off-peak hours.

  • The config database is an internal database. We recommend that you do not migrate data from this database.

  • If the source self-managed MongoDB database and the destination ApsaraDB for MongoDB instance run different database versions or storage engines, make sure that no compatibility issues occur between them before you start migration. For more information about the database versions and storage engines supported by ApsaraDB for MongoDB, see MongoDB versions and storage engines.

Billing information

Migration Types Instance configurations Internet traffic
Full data migration Free of charge Free of charge
Incremental data migration Charged, For more information, see Data Transmission Service (DTS) pricing. Free of charge

Migration type description

  • Full data migration: All data of the migration objects is migrated from the source instance to the destination instance..
    Note Data migration is supported at the database, collection, and index levels.
  • Incremental data migration: Updated data of the migration objects is synchronized from the source instance to the destination instance..
    Note
    • The create and delete operations for databases, collections, and indexes can be synchronized.
    • The create, delete, and update operations on documents can be synchronized.

Permissions required for database accounts

Database

Full data migration

Incremental data migration

Self-managed MongoDB database hosted on ECS

Read permissions on the source database

Read permissions on the source database, the admin database, and the local database

Destination ApsaraDB for MongoDB instance

Read and write permissions on the destination database

Read and write permissions on the destination database

For more information about how to create a database account and grant permissions to the database account, see the following topics:

Preparations

If the self-managed MongoDB database that is hosted on ECS uses the replica set architecture, skip this step.

If the self-managed MongoDB database that is hosted on ECS uses the standalone architecture, and you want to migrate the incremental data of the database, you must enable oplog for the database. For more information, see Migrate a self-managed standalone MongoDB database to Alibaba Cloud by using DTS.

Procedure

  1. Log on to the DTS console.
  2. Click the 返回DTS旧版控制台 icon in the lower-right corner of the page that appears to go back to the DTS console of the previous version.

  3. In the left-side navigation pane, click Data Migration.
  4. In the Migration Tasks section, select the region where the ApsaraDB for MongoDB instance is deployed.

  5. In the upper-right corner, click Create Migration Task.
  6. Configure the source and destination databases.

    Section

    Parameter

    Description

    Task Name

    N/A

    • DTS automatically generates a task name. You do not need to use an unique task name.

    • We recommend that you specify a name that can help you identify the task.

    Source Database

    Instance Type

    The type of the source database. Select User-Created Database in ECS Instance.

    Instance Region

    The region where the ECS instance is deployed.

    The ID of the ECS instance.

    The ID of the ECS instance on which the self-managed MongoDB database is deployed.

    Database Type

    The type of the destination database. Select MongoDB.

    Port Number

    Enter the service port number of the self-managed MongoDB database.

    Database Name

    The name of the database used for authentication.

    Database Account

    The account used to connect to the self-managed MongoDB database. For information about the permissions required for the account, see Permissions required for database accounts.

    Database Password

    The password of the database account used to connect to the self-managed MongoDB database.

    Note

    After you specify the information of the source self-managed database, you can click Test Connectivity next to Database Password to check whether the information is valid. If the information is valid, the Passed message appears. If the information is invalid, the Failed message appears. In this case, click Check next to the Failed message to modify the information.

    Destination Database

    Instance Type

    The type of the destination database. Select MongoDB Instance.

    Instance Region

    The region where the destination ApsaraDB for MongoDB instance resides.

    MongoDB Instance ID

    The ID of the destination ApsaraDB for MongoDB instance.

    Database Name

    The name of the database used for authentication.

    Database Account

    The account used to connect to the destination database. For information about the permissions required for the account, see Permissions required for database accounts.

    Database Password

    The password of the database account used to connect to the destination database.

    Note

    After you specify the information of the destination database, you can click Test Connectivity next to Database Password to check whether the information is valid. If the information is valid, the Passed message appears. If the information is invalid, the Failed message appears. In this case, you must click Check next to the Failed message to modify the information.

  7. In the lower-right corner of the page, click Set Whitelist and Next.

    Note
    • You do not need to modify the security settings for Alibaba Cloud database instances (such as ApsaraDB RDS for MySQL instances and ApsaraDB for MongoDB instances) and ECS-hosted self-managed databases. DTS automatically adds the CIDR blocks of DTS servers to the IP whitelists of Alibaba Cloud database instances or the security rules of ECS instances. For more information, see Add the CIDR blocks of DTS servers.

    • If the data source or destination is a self-managed database and the public CIDR blocks of DTS servers are added to the IP whitelist of the self-managed database to allow access from DTS servers, security risks may occur. We recommend that you strengthen authentication with accounts and passwords, restrict allowed ports, or connect to the database over Express Connect, VPN Gateway, or Smart Access Gateway.

    • After data migration is complete, we recommend that you remove the CIDR blocks of DTS servers from the whitelists.

  8. Select the migration types and the objects that you want to migrate.

    Parameter

    Description

    Migration Type

    • To perform only full data migration, select only Full Data Migration.

      Note

      To ensure data consistency, do not write new data to the self-managed MongoDB database during full data migration.

    • To ensure service continuity during data migration, select both Full Data Migration and Incremental Data Migration.

      Note

      Before you migrate incremental data from a standalone self-managed MongoDB database, you must enable oplog for the database. For more information, see Before you begin.

    Source Objects

    • Select one or more objects from the Source Objects section and click the icon to add the objects to the Selected Objects list.

      Note
      • DTS cannot migrate data from the admin or local database.

      • The config database is an internal database. We recommend that you do not migrate data from the config database.

    • A migration object can be a database, collection, or function.

    • By default, after an object is migrated to the destination database, the name of the object remains unchanged. You can change the names of the objects in the destination database by using the object name mapping feature provided by DTS. For more information, see Object name mapping.

  9. In the lower-right corner of the page, click Precheck.
    Note
    • A precheck is performed before the migration task starts. The migration task starts only after the precheck succeeds.
    • If the precheck fails, click the Tip icon for each failed check item to view their details. Perform a precheck again after the failures are fixed.
  10. After the precheck succeeds, click Next.
  11. In the Confirm Settings dialog box, configure the Instance Class parameter. Then, read and select the check box to agree to Data Transmission Service (Pay-As-You-Go) Service Terms.

  12. Click Buy and Start to start the migration task.
    • Full data migration

      Do not manually end a migration task. If you do so, the system may fail to migrate all data of the database. Wait until the migration task is complete.

    • Incremental data migration

      An incremental data migration task does not automatically end. You need to manually end the task.

      Note Select an appropriate point in time to manually end a migration task. For example, you can end the migration task during off-peak hours or before you switch over your business to the destination ApsaraDB for MongoDB instance.
      1. When the task progress bar displays Incremental Data Migration and The migration task is not delayed, stop writing data to the source database for a few minutes. Wait until the progress bar displays the delay time of the incremental data migration next to Incremental Data Migration.
      2. After the status of Incremental Data Migration changes to The migration task is not delayed, manually end the migration task.Incremental data migration without delay
  13. Switch your workloads to the destination ApsaraDB for MongoDB instance.