Full data migration from a standalone ApsaraDB for MongoDB instance to an ApsaraDB for MongoDB instance of any architecture - Data Transmission Service

Supported source and destination types

The source and destination databases must use the same access method. For example, if the source is an ApsaraDB for MongoDB instance, the destination must also be an ApsaraDB for MongoDB instance.

Source (standalone architecture)	Destination (any architecture)
ApsaraDB for MongoDB	ApsaraDB for MongoDB
Self-managed database on an Elastic Compute Service (ECS) instance	Self-managed database on an ECS instance
Self-managed database connected over Express Connect, VPN Gateway, or Smart Access Gateway	Self-managed database connected over Express Connect, VPN Gateway, or Smart Access Gateway
Self-managed database with a public IP address	Self-managed database with a public IP address

Prerequisites

Before you begin, make sure that:

The source ApsaraDB for MongoDB standalone instance and the destination instance are both created. The destination can be a standalone, replica set, or sharded cluster instance. See Create a standalone instance, Create a replica set instance, and Create a sharded cluster instance.
(Recommended) The destination instance has at least 10% more available storage than the total size of data in the source instance.
If the destination is a sharded cluster instance, create and shard the target databases and collections before starting the migration. See Configure sharding to maximize the performance of shards.
For supported source and destination MongoDB versions, see Overview of data migration scenarios.

Billing

Migration type	Instance configuration fee	Internet traffic fee
Schema migration and full data migration	Free	Charged only when data is migrated from Alibaba Cloud over the Internet. See Billing overview.

Migration types

Migration type	Scope
Schema migration	Migrates schemas for databases, collections, and indexes.
Full data migration	Migrates full data for databases and collections.

Limitations

Source database

The source server must have enough outbound bandwidth. Insufficient bandwidth reduces migration speed.
Collections to be migrated must have PRIMARY KEY or UNIQUE constraints, with all fields unique. Otherwise, duplicate records may appear in the destination.
If you select individual collections as migration objects and need to rename them in the destination, a single task supports up to 1,000 collections. Exceeding this limit causes a request error. In that case, split into multiple tasks or migrate the entire database instead.
Do not make schema changes to source databases or collections during schema migration or full data migration. Schema changes cause the task to fail.
Do not write data to the source during full data migration. Incremental data migration is not supported for standalone instances.
Collections with time to live (TTL) indexes cannot be migrated. TTL index behavior depends on local clocks, which can cause data inconsistency between source and destination.

Destination database (sharded cluster)

If the destination is a sharded cluster, the following additional constraints apply:

Delete all orphaned documents before migration. During migration, _id conflicts from orphaned documents can cause data inconsistency or task failure.
Add shard keys to the data in the source before starting migration. If adding shard keys is not possible, see Migrate data from a MongoDB instance without a sharding key to a MongoDB sharded cluster instance.
During migration, INSERT operations must include shard keys. UPDATE operations cannot modify shard keys.

General

DTS cannot migrate the admin or local database.
Transaction information is not preserved. Transactions are converted to individual records in the destination.
The destination MongoDB version must be the same as or later than the source version. An older destination version causes compatibility issues.
Run migrations during off-peak hours. Full data migration uses read and write resources on both the source and destination, which increases server load.
After full data migration, concurrent INSERT operations cause collection fragmentation. The destination storage usage will be 5%–10% larger than the source.
Collections with a unique index or capped: true support only single-threaded writes and do not support concurrent replay during incremental data migration. This may increase migration latency.
DTS uses ROUND(COLUMN, PRECISION) to retrieve FLOAT and DOUBLE values. Default precision: 38 digits for FLOAT, 308 digits for DOUBLE. Adjust if your data requires different precision.
DTS retries failed tasks for up to seven days. Before switching workloads to the destination, stop or release any failed tasks — or revoke DTS write permissions on the destination. Otherwise, a resumed task may overwrite data in the destination.
To run count queries on the destination, use: db.$table_name.aggregate([{ $count: "myCount" }]).
Make sure the destination does not already contain documents with the same _id as the source. If it does, delete those conflicting documents from the destination before starting migration.

Special cases

If the source database is a self-managed MongoDB database, do not perform a primary/secondary switchover on the source database while the data migration task is running. Otherwise, the task fails.

Required permissions

Database	Required permissions
Source ApsaraDB for MongoDB instance	Read on the source database and the `config` database
Destination ApsaraDB for MongoDB instance	`dbAdminAnyDatabase`, read and write on the destination database, and read on the `local` database

To create a database account and grant permissions, see Manage user permissions on MongoDB databases.

Configure and start the migration task

Step 1: Open the Data Migration page

DTS console

Log on to the DTS console.DTS console
In the left-side navigation pane, click Data Migration.
In the upper-left corner, select the region where the migration instance will reside.

DMS console

The exact navigation path may vary based on the DMS console mode and layout. See Simple mode and Customize the layout and style of the DMS console. Alternatively, go directly to the Data Migration page in the new DTS console.

Log on to the Data Management (DMS) console.DMS console
In the top navigation bar, move the pointer over DTS, then choose DTS (DTS) > Data Migration.
From the drop-down list on the right of Data Migration Tasks, select the region where the migration instance will run.

In the new DTS console, select the region in the upper-left corner.

Step 2: Configure the source and destination databases

Click Create Task.

Warning
After configuring the source and destination databases, read the Limits displayed at the top of the page. Skipping this step may cause task failure or data inconsistency.

Configure the source database:

Parameter	Description
Task Name	A name for the migration task. DTS generates a name automatically. Specify a descriptive name to make the task easier to identify.
Select a DMS database instance	Select an existing DMS database instance to auto-populate the parameters, or leave blank and configure manually.
Database Type	Select MongoDB.
Access Method	Select Alibaba Cloud Instance.
Instance Region	The region where the source instance resides.
Architecture	Select Replica Set for a standalone ApsaraDB for MongoDB instance.
Instance ID	The ID of the source ApsaraDB for MongoDB instance.
Authentication Database	The authentication database for the source instance. Default: `admin`.
Database Account	The account for the source instance. See Required permissions.
Database Password	The password for the database account.

Configure the destination database:

Parameter	Description
Select a DMS database instance	Select an existing DMS database instance to auto-populate the parameters, or leave blank and configure manually.
Database Type	Select MongoDB.
Access Method	Select Alibaba Cloud Instance.
Instance Region	The region where the destination instance resides.
Architecture	Select the architecture of the destination instance: Replica Set or Sharded Cluster.
Instance ID	The ID of the destination ApsaraDB for MongoDB instance.
Authentication Database	The authentication database for the destination instance. Default: `admin`.
Database Account	The account for the destination instance. See Required permissions.
Database Password	The password for the database account.

Step 3: Test connectivity

Click Test Connectivity and Proceed.

DTS automatically adds its server CIDR blocks to the IP address whitelist of ApsaraDB for MongoDB instances and to the security group rules of ECS-hosted databases. For self-managed databases in data centers or hosted by third-party cloud providers, manually add the DTS server CIDR blocks to the database whitelist. See Add the CIDR blocks of DTS servers.

Warning

Adding DTS server CIDR blocks to whitelists or security group rules introduces security exposure. Before proceeding, take preventive measures such as: using strong credentials, limiting exposed ports, authenticating API calls, auditing whitelist rules regularly, and removing unauthorized CIDR blocks. For stronger network security, connect the database to DTS over Express Connect, VPN Gateway, or Smart Access Gateway instead of the public internet.

Step 4: Select migration objects and configure migration settings

Parameter	Description
Migration Types	Select Schema Migration and Full Data Migration. Incremental data migration is not supported for standalone instances.
Processing Mode of Conflicting Tables	Precheck and Report Errors (default): fails the precheck if the destination contains collections with the same names as in the source. Use the object name mapping feature to rename conflicting collections. Ignore Errors and Proceed: skips the name collision check. Records with matching primary keys are not migrated, which may cause partial data loss or task failure.
Capitalization of Object Names in Destination Instance	Controls the case of database and collection names in the destination. Default: DTS default policy. See Specify the capitalization of object names.
Source Objects	Select objects from the Available section. Click the right-arrow icon to move them to Selected. You can select individual databases or collections.
Selected Objects	Right-click an object to rename it in the destination. To rename multiple objects at once, click Batch Edit. See Map object names. To filter rows with a WHERE condition, right-click an object and specify the condition. See Specify filter conditions.

Renaming an object with the object name mapping feature may cause other objects that depend on it to fail migration.

Step 5: Configure advanced settings

Click Next: Advanced Settings, then configure the following:

Parameter	Description
Monitoring and Alerting	Select Yesalert notification settings to receive alerts when the task fails or migration latency exceeds the threshold. Configure alert thresholds and notification contacts. See Configure monitoring and alerting.
Retry Time for Failed Connections	How long DTS retries failed connections after the task starts. Range: 10–1,440 minutes. Default: 720 minutes. Set to at least 30 minutes. If DTS reconnects within this window, the task resumes; otherwise, it fails.
Configure ETL	Select Yes to transform data during migration using domain-specific language (DSL) statements. See Configure ETL.

If multiple tasks share the same source or destination database and have different retry time settings, the most recently configured value takes precedence. DTS charges for the instance during retry periods. Release the DTS instance as soon as the source and destination are no longer needed.

Step 6: Run the precheck

Click Next: Save Task Settings and Precheck.

To review the API parameters for this task configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

DTS runs a precheck before starting migration. If the precheck fails:

Click View Details next to the failed item, resolve the issue, then click Precheck Again.
If the failure is an alert (not a hard error) that can be safely ignored, click Confirm Alert Details > Ignore > OK > Precheck Again. Ignoring alerts may result in data inconsistency.

Step 7: Purchase the migration instance

Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

Configure the following:

Parameter	Description
Resource Group	The resource group for the migration instance. Default: default resource group. See What is Resource Management?
Instance Class	The instance class determines migration speed. See Instance classes of data migration instances.

Step 8: Start the migration

Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms by selecting the check box.
Click Buy and Start, then click OK in the confirmation dialog.

View task progress on the Data Migration page.

After the migration completes

When the task status shows that migration is complete:

Verify the data in the destination instance — spot-check row counts, document structure, and key collections.
Stop all writes to the source database.
Update your application's connection string to point to the destination instance.
Release or stop the DTS migration task to avoid incurring additional charges.