AnalyticDB:Synchronize data with zero-ETL

Create a AnalyticDB for MySQL

1. Navigate to the Role page in the RAM console.

2. Check if a service-linked role named AliyunServiceRoleForAnalyticDBForMySQL already exists. If not, you must create the role.

3. In the left-side navigation pane, click Create Role.

4. In the Create Role dialog box, select Alibaba Cloud Service and click Next.

5. For the role type, select Service-Linked Role, and then select AnalyticDB for MySQL from the service drop-down list.

6. Click Finish. Return to the role list and verify that the service-linked role has been created.

Grant permissions to a RAM user

A RAM user needs the following two types of permissions to create and manage zero-ETL tasks.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:*",
                "acs:rds:*:*:*",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": [
                "acs:dts:*:*:*"
            ]
        }
    ]
}

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "dts:*",
            "Resource": [
                "acs:adb:*:*:dbcluster/am-2zeod8ax4b9a****",
                "acs:rds:*:*:dbinstance/rm-2ze6fs8ouh43****",
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "dts:DescribeRegions",
                "dts:DescribeConfigRelations",
                "dts:DescribeSrcLinkConfig",
                "dts:DescribeDestLinkConfig",
                "dts:DescribeLinkConfig"
            ],
            "Resource": "acs:dts:*:*:*"
        }
    ]
}

Procedure

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.

2. Navigate to the zero-ETL feature page. The entry point varies based on the cluster edition.

   • For an Enterprise Edition, Basic Edition, or Data Lakehouse Edition cluster: In the left-side navigation pane, choose Data Ingestion > Database Data Synchronization.

   • For a Data Warehouse Edition cluster: In the left-side navigation pane, choose Data Transmission Service > Zero-ETL.

3. Click Create Zero-ETL Task. On the Create Zero-ETL Task page, configure the source and destination database information.

   • Configure the source database information:

     Source database	Description
     Task Name	The name of the zero-ETL task.
     Database Type	Select RDS for MySQL.
     Access Method	Only Alibaba Cloud Instance is supported.
     Instance Region	The region where the RDS for MySQL instance is located.
     RDS Instance ID	The ID of the RDS for MySQL instance.
     Database Account	The database account for the RDS for MySQL instance.
     Database Password	The password for the database account of the RDS for MySQL instance.
     Encryption	The method to connect to the RDS for MySQL instance. Options include non-encrypted connection and SSL secure connection. Note If you select SSL-encrypted, you must first enable SSL encryption for the RDS for MySQL instance. For more information, see Use a cloud certificate to quickly enable SSL encryption.

   • Configure the destination database information:

     Destination database	Description
     Database Type	Currently, only AnalyticDB for MySQL 3.0 is supported as the destination.
     Access Method	Only Alibaba Cloud Instance is supported.
     Instance Region	The region where the AnalyticDB for MySQL cluster is located.
     Instance ID	The cluster ID of the AnalyticDB for MySQL cluster.
     Database Account	The database account for the AnalyticDB for MySQL cluster.
     Database Password	The password for the database account of the AnalyticDB for MySQL cluster.

4. After you configure the parameters, click Test Connectivity and Proceed. On the configuration page, set the following parameters:

   Parameter	Description
   DDL and DML Operations to Be Synchronized	Select the DML and DDL operations to synchronize. All operations are selected by default.
   Source Objects and Selected Objects	The source objects in the database and the objects to be synchronized.
   Advanced Settings (Optional)	Configure retry intervals for database unavailability and other potential issues.

5. After you complete the settings, click Next: Configure Database and Table Fields and configure the following information:

   Configuration	Description
   Database Name	Select an existing database.
   Table Name	Select an existing table.
   Primary Key Column	The primary key column of the selected table.
   Distribution Key	The distribution key column of the selected table.
   Type	The table type. Options include partitioned table and replicated table.
   Definition Status	After the database and table fields are configured, the status changes from Undefined to Defined.

6. After configuring all parameters, click Next: Save Task Settings and Precheck.

7. If the precheck is successful, click Start to launch the zero-ETL task.

   On the Zero-ETL page, you can view information about the task, including its ID/Name, Source/Destination, and Status.

1. Log on to the Cloud Monitor console.

2. View monitoring information.

   1. In the left-side navigation pane, choose Cloud Service Monitoring > Cloud Service Monitoring.

   2. Hover over the AnalyticDB for MySQL card and click AnalyticDB for mysql 3.0 - ZeroETL Latency.

   3. Click a cluster ID to view the monitoring information for the zero-ETL tasks in that cluster.

3. Create an alert rule.

   1. In the left-side navigation pane, click Alerts > Alert Rules.

   2. Click Create Alert Rule. In the Create Alert Rule panel, configure an alert rule for the zero-ETL task. For more information, see Create an alert rule.

      Note

      For the Product parameter, select AnalyticDB for mysql 3.0 - ZeroETL Latency.

4. Create a subscription policy.

   1. In the left-side navigation pane, click Event Center > Event Subscription.

   2. Click Create a subscription policy. On the Create a subscription policy page, configure the subscription policy for the zero-ETL task. For more information, see Create an event subscription.

      Note

      • Product: Select Analyticdb for mysql 3.0.

      • event types: Select Abnormal or Restore.

      • Event Name: Select zero_etl_job_abnormal or zero_etl_job_restore.