This topic describes the precautions and limits when you migrate data from a MongoDB database, such as a self-managed MongoDB database and an ApsaraDB for MongoDB database. To ensure that your data migration task runs as expected, read the precautions and limits before you configure the task.

Scenarios of migrating data from a MongoDB database

You can view the precautions and limits based on the following migration scenarios:

Migrate data from a MongoDB database (standalone architecture) to another MongoDB database (standalone, replica set, or sharded cluster architecture)

Category Description
Limits on the source database
  • Bandwidth requirements: The server to which the source database belongs must have sufficient egress bandwidth. Otherwise, the data migration speed is affected.
  • The tables to be migrated must have PRIMARY KEY or UNIQUE constraints and all fields must be unique. Otherwise, the destination database may contain duplicate data records.
  • If you select collections as the objects to be migrated and you need to edit collections (such as renaming collections), up to 1,000 collections can be migrated in a single data migration task. If you run a task to migrate more than 1,000 collections, a request error occurs. In this case, we recommend that you split the collections to be migrated, configure multiple tasks to migrate the collections, or configure a task to migrate the entire database.
  • Limits on operations:
    • During schema migration and full data migration, do not change the schemas of databases or collections. Otherwise, the data migration task fails.
    • Incremental data migration is not supported in this scenario. To ensure data consistency, we recommend that you do not write data to the source MongoDB database during full data migration.
Other limits
  • Only schema migration and full data migration are supported in this scenario. You cannot use DTS to migrate incremental data from a standalone MongoDB database because the oplog feature cannot be enabled for the database.
  • DTS cannot migrate data from the admin or local database.
  • Transaction information is not retained. When transactions are migrated to the destination database, they are converted into a single record.
  • To ensure compatibility, the version of the destination MongoDB database must be the same as or later than the version of the source MongoDB database. If the version of the destination database is earlier than the version of the source database, database compatibility issues may occur.
  • Before you migrate data, evaluate the impact of data migration on the performance of the source and destination databases. We recommend that you migrate data during off-peak hours. During full data migration, DTS uses read and write resources of the source and destination databases. This may increase the loads of the database servers.
  • During full data migration, concurrent INSERT operations cause fragmentation in the collections of the destination database. After full data migration is completed, the storage space for collections of the destination database is larger than that of the source database.
  • You must make sure that the precision settings for columns of the FLOAT or DOUBLE data type meets your business requirements. DTS uses the ROUND(COLUMN,PRECISION) function to retrieve values from columns of the FLOAT or DOUBLE data type. If you do not specify a precision, DTS sets the precision for the FLOAT data type to 38 digits and the precision for the DOUBLE data type to 308 digits.
  • DTS attempts to resume data migration tasks that failed within the last seven days. Before you switch workloads to the destination instance, stop or release the data migration task. You can also run the revoke command to revoke the write permissions from the accounts that are used by DTS to access the destination instance. Otherwise, the data in the source database will overwrite the data in the destination instance after the task is resumed.
Special cases If the source database is a self-managed MongoDB database, we recommend that you do not perform a primary/secondary switchover on the database when the data migration task is running. Otherwise, the task fails.

Migrate data from a MongoDB database (replica set architecture) to another MongoDB database (replica set architecture or sharded cluster architecture)

Category Description
Limits on the source database
  • Bandwidth requirements: The server to which the source database belongs must have sufficient egress bandwidth. Otherwise, the data migration speed is affected.
  • The tables to be migrated must have PRIMARY KEY or UNIQUE constraints and all fields must be unique. Otherwise, the destination database may contain duplicate data records.
  • If you select collections as the objects to be migrated and you need to edit collections (such as renaming collections), up to 1,000 collections can be migrated in a single data migration task. If you run a task to migrate more than 1,000 collections, a request error occurs. In this case, we recommend that you split the collections to be migrated, configure multiple tasks to migrate the collections, or configure a task to migrate the entire database.
  • If you need to migrate incremental data, you must make sure that the following requirements are met:
    • The oplog is enabled. Otherwise, error messages are returned during precheck and the data migration task cannot be started.
    • If you perform only incremental data migration, the oplogs of the source database must be stored for more than 24 hours. If you perform both full data migration and incremental data migration, the oplogs of the source database must be stored for at least seven days. After full data migration is completed, you can set the retention period to more than 24 hours. Otherwise, DTS may fail to obtain the oplogs and the task may fail. In extreme cases, data may be inconsistent or lost. Make sure that you set the retention period of oplogs based on the preceding requirements. Otherwise, the Service Level Agreement (SLA) of DTS does not guarantee service reliability and performance.

  • Limits on operations:
    • During schema migration and full data migration, do not change the schemas of databases or collections. Otherwise, the data migration task fails.
    • If you perform only full data migration, do not write data to the source database during data migration. Otherwise, data will be inconsistent between the source and destination databases. To ensure data consistency, we recommend that you select Schema Migration, Full Data Migration, and Incremental Data Migration as the migration types.
Other limits
  • To ensure compatibility, the version of the destination MongoDB database must be the same as or later than the version of the source MongoDB database. If the version of the destination database is earlier than the version of the source database, database compatibility issues may occur.
  • DTS cannot migrate data from the admin or local database.
  • Transaction information is not retained. When transactions are migrated to the destination database, they are converted into a single record.
  • Before you migrate data, evaluate the impact of data migration on the performance of the source and destination databases. We recommend that you migrate data during off-peak hours. During full data migration, DTS uses read and write resources of the source and destination databases. This may increase the loads of the database servers.
  • During full data migration, concurrent INSERT operations cause fragmentation in the collections of the destination database. After full data migration is completed, the storage space for collections of the destination database is larger than that of the source database.
  • You must make sure that the precision settings for columns of the FLOAT or DOUBLE data type meets your business requirements. DTS uses the ROUND(COLUMN,PRECISION) function to retrieve values from columns of the FLOAT or DOUBLE data type. If you do not specify a precision, DTS sets the precision for the FLOAT data type to 38 digits and the precision for the DOUBLE data type to 308 digits.
  • DTS attempts to resume data migration tasks that failed within the last seven days. Before you switch workloads to the destination instance, stop or release the data migration task. You can also run the revoke command to revoke the write permissions from the accounts that are used by DTS to access the destination instance. Otherwise, the data in the source database will overwrite the data in the destination instance after the task is resumed.
Special cases If the source database is a self-managed MongoDB database, take note of the following limits:
  • If you perform a primary/secondary switchover on the source database when the data migration task is running, the task fails.
  • DTS calculates migration latency based on the timestamp of the latest migrated data in the destination database and the current timestamp in the source database. If no update operation is performed on the source database for a long time, the migration latency may be inaccurate. If the latency of the migration task is too high, you can perform an update operation on the source database to update the latency.
Note If you select an entire database as the object to be migrated, you can create a heartbeat. The heartbeat is updated or receives data every second.