AnalyticDB:Use zero-ETL to synchronize data

How it works

Lindorm handles online transaction processing (OLTP) workloads: high-frequency writes, row-level lookups, and operational data. AnalyticDB for MySQL handles online analytical processing (OLAP) workloads: complex queries, aggregations, and business intelligence.

Traditionally, moving data between OLTP and OLAP systems requires separate ETL infrastructure to maintain, which adds cost and introduces batch delays that reduce data freshness.

Zero-ETL eliminates that middle layer. It continuously captures changes from your Lindorm HBase tables and applies them to AnalyticDB for MySQL.

Benefits

• No pipeline management: Select a source Lindorm instance and a destination AnalyticDB for MySQL cluster—zero-ETL handles the rest. No custom pipelines to build or maintain.

• No additional cost: Zero-ETL tasks are free. Analyzing synchronized data in AnalyticDB for MySQL incurs no extra charges.

• Multi-source aggregation: Synchronize data from multiple Lindorm instances into a single AnalyticDB for MySQL cluster for unified analysis.

Limitations

Supported regions

Zero-ETL for Lindorm is available in: China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Zhangjiakou), China (Qingdao), China (Guangzhou), China (Hong Kong), Singapore, US (Silicon Valley), and US (Virginia).

Data source

Only HBase tables in Lindorm are supported. SQL tables cannot be synchronized.

Task limits per cluster

The maximum number of zero-ETL tasks you can create in the AnalyticDB for MySQL console depends on the total reserved resources of your cluster:

Example: A cluster with 48 ACUs: (48 − 24) ÷ 50 = 0.48, rounded down to 0. Maximum tasks = 3 + 3 × 0 = 3.

How ACUs are counted:

• Data Lakehouse Edition: Reserved computing ACUs + reserved storage ACUs

• Data Warehouse Edition (elastic mode): Computing resource cores + elastic I/O resource cores

• Data Warehouse Edition (reserved mode): Computing resource cores only

• For Data Warehouse Edition, 1 ACU = 1 core.

If a cluster reaches its task limit, create additional tasks in the Data Transmission Service (DTS) console, or delete tasks you no longer need in the AnalyticDB for MySQL console.

Prerequisites

Before you begin, make sure you have:

• An AnalyticDB for MySQL cluster and a Lindorm instance in the same region

• LindormTable activated on the Lindorm instance

• A database account created for the AnalyticDB for MySQL cluster — see Create a database account

Billing

Zero-ETL tasks are free.

Preparations

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:lindorm:*:*:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:lindorm:*:*:dbinstance/ld-bp19yeo38q4****"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Create a zero-ETL task

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner, select a region. In the left-side navigation pane, click Clusters. Click an edition tab, find the cluster, and click its ID.

2. Go to the zero-ETL configuration page:

   • Data Lakehouse Edition: In the left-side navigation pane, choose Data Ingestion > Database Data Synchronization.

   • Data Warehouse Edition: In the left-side navigation pane, choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. In the Configure Source and Destination Databases step, configure the source instance: Then configure the destination cluster:

   Parameter	Description
   Task Name	A name for the zero-ETL task
   Database Type	Select Lindorm
   Access Method	Automatically set to Alibaba Cloud Instance
   Instance Region	The region of the Lindorm instance
   Instance ID	The ID of the Lindorm instance
   Database Account	The account name for connecting to LindormTable. To reset a forgotten password, see the "Change the password of a user" section of Manage users.
   Database Password

   Parameter	Description
   Database Type	Automatically set to AnalyticDB for MySQL 3.0
   Access Method	Automatically set to Alibaba Cloud Instance
   Instance Region	The region of the AnalyticDB for MySQL cluster
   Instance ID	The ID of the AnalyticDB for MySQL cluster
   Database Account	The database account name for the cluster
   Database Password	The password for the database account

4. Click Test Connectivity and Proceed. In the Configure Zero-ETL step, configure the synchronization options:

   Parameter	Description
   DDL and DML operations to be synchronized	The operations to synchronize. DML operations: INSERT, UPDATE, DELETE. DDL operations: CREATE, ALTER, DROP, TRUNCATE. All operations are selected by default.
   Source objects and selected objects	The tables to synchronize. You cannot synchronize an entire database or table—click Edit next to a source table in the Selected Objects section to choose which columns to include.
   Advanced settings (Optional)	Retry duration for failed connections and other errors on the source and destination.

5. Click Next: Configure Database and Table Fields. For each selected table, configure the following:

   Parameter	Description
   Database Name	The name of the selected database
   Table Name	The name of the selected table
   Primary key column	The primary key column of the table
   Distribution key	The distribution key column of the table
   Type	Partitioned table or Replicated table
   Definition Status	Changes from Undefined to Defined after you configure the fields

6. Click Next: Save Task Settings and Precheck.

7. If the precheck passes, click Start.

The task appears on the zero-ETL configuration page. You can view its Name, Source/Destination, and Status.

Monitor zero-ETL tasks

After a task starts, use CloudMonitor to track its health and set up alerts.

Troubleshooting

What's next

• Synchronize data from ApsaraDB RDS for MySQL: Use zero-ETL to synchronize data

• Synchronize data from PolarDB for MySQL: Use zero-ETL to synchronize data

• Create a Lindorm instance: Create an instance