Data Transmission Service:Synchronize data from an ApsaraDB for MongoDB replica set instance to a Function Compute function

Source database

Other limitations

• DTS cannot synchronize the admin, config, or local databases.

• Full data synchronization is not supported — incremental only.

• Cross-region synchronization is not supported.

• The object name mapping feature is not supported.

• Assign only one DTS task per destination function to avoid data errors.

• Transaction context is not preserved. Each transaction is converted to individual records.

• If a DTS task fails, DTS technical support attempts to restore it within 8 hours. The task may be restarted and task parameters (not database parameters) may be modified during restoration.

Special cases (self-managed MongoDB)

• A primary/secondary switchover while the task is running causes the task to fail.

• Synchronization latency is calculated from the latest synchronized record's timestamp versus the current source timestamp. If no writes occur on the source for an extended period, the reported latency may be inaccurate. Write a record to the source database to refresh the latency reading.

If you synchronize an entire database, create a heartbeat table. DTS updates the heartbeat table every second, which keeps the latency reading accurate.

Step 1: Open the Data Synchronization page

Use one of the following consoles:

Step 2: Configure source and destination databases

Click Create Task, then configure the parameters described in the following tables.

General

Source database

Destination database

Step 3: Test connectivity

Click Test Connectivity and Proceed at the bottom of the page.

DTS server CIDR blocks must be added to the security settings of the source and destination databases. See Add the CIDR blocks of DTS servers. If the source or destination is a self-managed database not using Alibaba Cloud Instance access, click Test Connectivity in the CIDR Blocks of DTS Servers dialog box.

Step 4: Configure objects to synchronize

In the Configure Objects step, set the following parameters.

Click Next: Advanced Settings.

Step 5: Configure advanced settings

Step 6: Run the precheck

Click Next: Save Task Settings and Precheck.

To preview the OpenAPI parameters for this task configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

DTS runs a precheck before starting the task. If the precheck fails:

• For each failed item, click View Details, resolve the issue, then click Precheck Again.

• For alert items that can be ignored: click Confirm Alert Details > Ignore > OK > Precheck Again. Ignoring alerts may result in data inconsistency.

Step 7: Purchase an instance

1. Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

2. On the purchase page, configure the following parameters.

1. Accept Data Transmission Service (Pay-as-you-go) Service Terms.

2. Click Buy and Start, then click OK in the dialog box.

The task appears in the task list. Monitor its progress from there.

DDL examples

DML examples

// Insert multiple records at once
db.runCommand({insert: "user", documents: [{"name":"jack","age":20},{"name":"lili","age":20}]})

// Insert one record at a time
db.user.insert({"name":"jack","age":20})
db.user.insert({"name":"lili","age":20})

{
  "Records": [
    {
      "data": [{"doc": "{\"_id\": {\"$oid\": \"64f9397f6e255f74d65a****\"}, \"name\": \"jack\", \"age\": 20}"}],
      "pkNames": ["_id"],
      "type": "INSERT",
      "es": 1694054783000,
      "database": "MongoDBTest",
      "id": 0,
      "isDdl": false,
      "table": "user",
      "ts": 1694054784427
    },
    {
      "data": [{"doc": "{\"_id\": {\"$oid\": \"64f9397f6e255f74d65a****\"}, \"name\": \"lili\", \"age\": 20}"}],
      "pkNames": ["_id"],
      "type": "INSERT",
      "es": 1694054783000,
      "database": "MongoDBTest",
      "id": 0,
      "isDdl": false,
      "table": "user",
      "ts": 1694054784428
    }
  ]
}

db.user.update({"name":"jack"},{$set:{"age":30}})

{
  "Records": [{
    "data": [{"doc": "{\"$set\": {\"age\": 30}}"}],
    "pkNames": ["_id"],
    "old": [{"doc": "{\"_id\": {\"$oid\": \"64f9397f6e255f74d65a****\"}}"}],
    "type": "UPDATE",
    "es": 1694054989000,
    "database": "MongoDBTest",
    "id": 0,
    "isDdl": false,
    "table": "user",
    "ts": 1694054990555
  }]
}

db.user.remove({"name":"jack"})

{
  "Records": [{
    "data": [{"doc": "{\"_id\": {\"$oid\": \"64f9397f6e255f74d65a****\"}}"}],
    "pkNames": ["_id"],
    "type": "DELETE",
    "es": 1694055452000,
    "database": "MongoDBTest",
    "id": 0,
    "isDdl": false,
    "table": "user",
    "ts": 1694055452852
  }]
}