AnalyticDB:Use zero-ETL to synchronize data

How it works

Traditional extract, transform, load (ETL) processes require separate tools per data source, add operational overhead, and introduce latency that makes near-real-time analytics impractical.

Zero-ETL removes these constraints. Select a source MongoDB instance and a destination AnalyticDB for MySQL cluster, and the feature handles continuous data extraction, transformation, and loading—so you can focus on analysis rather than infrastructure.

Supported synchronization paths

AnalyticDB for MySQL supports zero-ETL synchronization from three sources:

• ApsaraDB RDS for MySQL to AnalyticDB for MySQL. See Use zero-ETL to synchronize data.

• PolarDB for MySQL to AnalyticDB for MySQL. See Use zero-ETL to synchronize data.

• ApsaraDB for MongoDB to AnalyticDB for MySQL (this topic).

Billing

Zero-ETL tasks are free.

Limits

• Supported regions: China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Zhangjiakou), China (Qingdao), China (Guangzhou), China (Hong Kong), Singapore, US (Silicon Valley), and US (Virginia).

• Task limits per cluster: The maximum number of zero-ETL tasks you can create in the AnalyticDB for MySQL console depends on the cluster's total reserved resources, measured in AnalyticDB compute units (ACUs): If you reach the limit, create additional tasks in the Data Transmission Service (DTS) console, or delete unused tasks in the AnalyticDB for MySQL console.

  • ACUs < 24: 1 task maximum.

  • ACUs >= 24: Maximum tasks = 3 + 3 × [(Total ACUs - 24) / 50], rounding down the bracketed value. Example: A cluster with 48 ACUs gives [(48 - 24) / 50] = 0.48, rounded down to 0, so the maximum is 3 tasks.

  How total ACUs are counted varies by cluster edition: - Data Lakehouse Edition: Reserved computing ACUs + reserved storage ACUs. - Data Warehouse Edition: 1 ACU = 1 core. - Elastic mode: computing resource cores + elastic I/O resource cores. - Reserved mode: computing resource cores only.

• MongoDB architecture: Only replica set architecture is supported.

Before you begin

Make sure you have:

• An AnalyticDB for MySQL cluster and an ApsaraDB for MongoDB instance in the same region.

• Database accounts created for both the cluster and the instance.

Set up permissions

Complete these steps before creating a zero-ETL task. All steps use the RAM console.

1. Create a service-linked role for AnalyticDB for MySQL.

2. Grant a RAM user permissions to manage AnalyticDB for MySQL.

3. Grant a RAM user permissions to create zero-ETL tasks.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:dds:*:*:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:dds:*:*:dbinstance/dds-t4n8aaa4dcdb****"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Create a zero-ETL task

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner, select a region. In the left-side navigation pane, click Clusters, then click the cluster ID.

2. Go to the zero-ETL configuration page for your cluster edition:

   • Enterprise Edition, Basic Edition, or Data Lakehouse Edition: Choose Data Ingestion > Database Data Synchronization.

   • Data Warehouse Edition: Choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. On the Configure Source and Destination Databases step, configure the source instance: Then configure the destination cluster:

   For Migration Method: - Oplog pulls logs at high speed, resulting in low incremental sync latency. Oplog is enabled by default on ApsaraDB for MongoDB instances and self-managed MongoDB databases. Use Oplog unless your source requires change streams. - ChangeStream is required if the source is an Amazon DocumentDB instance (non-elastic cluster). This option is only available when change streams are enabled on the source.

   Parameter	Description
   Task Name	A name for the zero-ETL task.
   Database Type	Select MongoDB.
   Access Method	Automatically set to Alibaba Cloud Instance.
   Instance Region	The region of the ApsaraDB for MongoDB instance.
   Architecture	Only ReplicaSet is supported.
   Migration Method	Select the incremental sync method: Oplog (recommended) or ChangeStream. See the note below.
   Instance ID	The ID of the ApsaraDB for MongoDB instance.
   Authentication Database	The database the account belongs to. Defaults to admin.
   Database Account	The database account of the ApsaraDB for MongoDB instance.
   Database Password	The password for the database account.
   Encryption	Non-encrypted or SSL-encrypted. If you select SSL-encrypted, first enable SSL encryption on the instance.

   Parameter	Description
   Database Type	Automatically set to AnalyticDB for MySQL 3.0.
   Access Method	Automatically set to Alibaba Cloud Instance.
   Instance Region	The region of the AnalyticDB for MySQL cluster.
   Instance ID	The ID of the AnalyticDB for MySQL cluster.
   Database Account	The database account of the AnalyticDB for MySQL cluster.
   Database Password	The password for the database account.

4. Click Test Connectivity and Proceed. On the Configure Zero-ETL step, configure the following:

   Parameter	Description
   DDL and DML Operations to Be Synchronized	The DML operations to sync: INSERT, UPDATE, and DELETE. All are selected by default.
   Source Objects and Selected Objects	The source objects to synchronize.
   Advanced Settings (Optional)	Retry intervals for failed connections and other issues between the source and destination.

5. Click Next: Save Task Settings and Precheck.

6. After the precheck completes successfully, click Start.

The task appears on the zero-ETL configuration page, where you can track its Name, Source/Destination, and Status.

Monitor zero-ETL tasks

After starting a task, set up monitoring and alerts in CloudMonitor.