AnalyticDB:Synchronize data using seamless integration (Zero-ETL)

How it works

A Zero-ETL pipeline continuously extracts and loads data changes (DML and DDL operations) from RDS for MySQL into an AnalyticDB for MySQL cluster. There is no intermediary transformation layer. After the pipeline is active, you can run analytics directly on the synchronized data.

RDS for MySQL  →  Zero-ETL pipeline  →  AnalyticDB for MySQL
  (OLTP source)      (real-time sync)      (OLAP analytics)

Benefits

• No pipeline management: Select the source database and destination cluster, and the pipeline handles the rest. No ETL infrastructure to build or operate.

• Free of charge: Zero-ETL pipelines have no additional cost. Analyze upstream data in AnalyticDB for MySQL at no extra charge.

• Multi-source aggregation: Synchronize data from multiple RDS for MySQL instances to a single AnalyticDB for MySQL cluster for a unified analytics view.

Supported pipelines

• RDS for MySQL → AnalyticDB for MySQL (this document)

• PolarDB for MySQL → AnalyticDB for MySQL. See Synchronize data using seamless integration (Zero-ETL).

Billing

Data synchronization pipelines are free of charge.

Limitations

Regional availability

The seamless integration (Zero-ETL) feature is available in the following regions: China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Zhangjiakou), China (Qingdao), China (Guangzhou), Hong Kong (China), Singapore, US (Silicon Valley), and US (Virginia).

Pipeline quota

The number of Zero-ETL pipelines per cluster depends on the cluster's total reserved AnalyticDB Compute Units (ACUs):

For example, a cluster with 48 ACUs can have 3 + 3 × floor((48 − 24) / 50) = 3 + 0 = 3 pipelines.

If the quota is reached, delete unused pipelines or use Data Transmission Service (DTS) to create additional synchronization pipelines.

How ACUs are counted depends on the cluster edition:

Data Lakehouse Edition: Total ACUs = reserved computing resources (ACUs) + reserved storage resources (ACUs).

Data Warehouse Edition, Elastic mode: Total ACUs = cores of computing resources + elastic I/O units.

Data Warehouse Edition, Reserved mode: Total ACUs = cores of computing resources.

Prerequisites

Before you begin, ensure that you have:

• An AnalyticDB for MySQL cluster and an RDS for MySQL instance in the same region

• Database accounts for both the AnalyticDB for MySQL cluster and the RDS for MySQL instance

Prepare your account and permissions

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:rds:*:*:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:rds:*:*:dbinstance/rm-2ze6fs8ouh43****"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Create a Zero-ETL pipeline

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner, select a region. In the left navigation pane, click Clusters, then click the cluster ID.

2. Go to the Zero-ETL page. The navigation path depends on your cluster edition:

   • Enterprise Edition, Basic Edition, or Data Lakehouse Edition: Choose Data Ingestion > Database Data Synchronization.

   • Data Warehouse Edition: Choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. Configure the source and destination database settings.

   Source database settings:

   Setting	Description
   Task name	A name for the Zero-ETL task.
   Database type	Select RDS for MySQL.
   Access method	Only Alibaba Cloud Instance is supported.
   Instance region	The region where the RDS for MySQL instance resides.
   RDS instance ID	The ID of the RDS for MySQL instance.
   Database account	The database account for the RDS for MySQL instance.
   Database password	The password for the database account.
   Encryption	Unencrypted Connection or SSL Secure Connection. If you select SSL, enable SSL encryption on the RDS for MySQL instance first. See Use a cloud certificate to quickly enable SSL encryption.

   Destination database settings:

   Setting	Description
   Database type	Only AnalyticDB for MySQL 3.0 is supported.
   Access method	Only Alibaba Cloud Instance is supported.
   Instance region	The region where the AnalyticDB for MySQL cluster resides.
   Instance ID	The ID of the AnalyticDB for MySQL cluster.
   Database account	The database account for the AnalyticDB for MySQL cluster.
   Database password	The password for the database account.

4. Click Test Connectivity and Proceed. Configure the synchronization settings:

   Setting	Description
   DDL and DML operations to be synchronized	The DML and DDL operations to synchronize. All operations are selected by default.
   Source objects and Selected objects	The database objects to synchronize.
   Advanced settings (optional)	The retry time for source or destination database connection failures.

5. Click Next: Configure Database and Table Fields. Configure the target table settings:

   Setting	Description
   Database name	Select an existing database.
   Table name	Select an existing table.
   Primary key column	The field that serves as the primary key in the target table.
   Distribution key	The field that serves as the distribution key.
   Type	Partitioned Table or Replicated Table.
   Definition status	Changes from Undefined to Defined after configuration is complete.

6. Click Next: Save Task Settings and Precheck.

7. After the precheck passes, click Start.

The task appears on the Seamless Data Integration page, where you can view its ID/Name, Source/Destination, and Status.

Monitor Zero-ETL tasks

What's next

• Synchronize data using seamless integration (Zero-ETL) — Set up a pipeline from PolarDB for MySQL.