Before you synchronize data in a single PolarDB table in real time, you must perform operations in this topic to configure the network environment, IP address whitelist, and permissions for the data source.

Prerequisites

Before you configure a data source, make sure that the following operations are performed:
  • Prepare data sources: A PolarDB for MySQL cluster and a destination are prepared. The destination can be MaxCompute, Hologres, Elasticsearch, DataHub, or Kafka. In this topic, a PolarDB for MySQL cluster is used as the source.
  • Plan and prepare resources: An exclusive resource group for data integration is purchased and configured. For more information, see Plan and configure resources.
  • Evaluate and plan the network environment: Before you perform data integration, connect data sources to exclusive resource groups for data integration based on your business requirements. After data sources and exclusive resource groups for data integration are connected, you can refer to the operations in this topic to configure access settings such as vSwitches and whitelists.
    • If data sources and exclusive resource groups for data integration reside in the same region and virtual private cloud (VPC), they are automatically connected.
    • If data sources and exclusive resource groups for data integration reside in different network environments, you must connect data sources and resource groups by using methods such as a VPN gateway.

Background information

Before you synchronize data between data sources, make sure that the data sources and the exclusive resource group for data integration are connected. In addition, you must create an account and authorize the account to access the data sources.
  • Configure whitelists for the data sources
    If the data sources and the exclusive resource group for data integration reside in the same VPC, you must add the CIDR block of the exclusive resource group for data integration to the whitelists of the data sources. This ensures that the exclusive resource group for data integration can be used to access the data sources. Synchronize the data in a single PolarDB table in real time
  • Create an account and authorize the account

    You must create an account that can be used to access the data sources, read data from the source, and write data to the destination during the data synchronization process.

  • Enable the binary logging feature

    If the source is a PolarDB for MySQL cluster, you must enable the binary logging feature for the cluster. Alibaba Cloud PolarDB for MySQL is fully compatible with MySQL and uses high-level physical logs to replace binary logs. To facilitate the integration between PolarDB and the MySQL ecosystem, you can enable the binary logging feature for PolarDB clusters.

Limits

  • Only PolarDB for MySQL clusters can be used as source data sources. In this topic, PolarDB indicates PolarDB for MySQL data sources.
  • Only data stored on the primary node of the PolarDB cluster can be synchronized.

Procedure

  1. Configure a whitelist for the PolarDB cluster.
    Add the CIDR block of the VPC where the exclusive resource group resides to the whitelist of the PolarDB cluster.
    1. View and record the elastic IP address (EIP) and CIDR block of the exclusive resource group.
      1. Log on to the DataWorks console.
      2. In the left-side navigation pane, click Resource Groups.
      3. On the Exclusive Resource Groups tab, find the resource group whose information you want to view and click View Information in the Actions column.
      4. In the dialog box that appears, click the value of the EIPAddress parameter. The value is automatically copied. Add the value to the whitelist of the data store. Use the same method to add the value of the CIDR Blocks parameter to the whitelist.EIP
      5. On the Exclusive Resource Groups tab, find the resource group whose information you want to view and click Add VPC Binding in the Actions column. On the page that appears, obtain the value of the VSwitch CIDR Block parameter and add the value to the whitelist of the data store.VSwitch CIDR Block
    2. Add the EIP and CIDR block of the exclusive resource group to the whitelist of the PolarDB cluster.
      Whitelist of the PolarDB clusterFor more information, see Configure a whitelist for a cluster.
  2. Create an account and grant the required permissions to the account.
    You must create an account to log on to the database of the PolarDB cluster. You must grant the SELECT, REPLICATION SLAVE, and REPLICATION CLIENT permissions to the account.
    1. Create an account.
      For more information, see Create a database account.
    2. Grant permissions to the account.
      You can run the following command to grant permissions to the account, or you can directly grant the SUPER permission to the account.
      -- CREATE USER 'Account for data synchronization'@'%' IDENTIFIED BY 'Account for data synchronization';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *. * TO 'Account for data synchronization'@'%';
  3. Enable the binary logging feature for the PolarDB cluster.
    For more information, see Enable binary logging.

What to do next

After the data sources are configured, the source, destination, and exclusive resource group for data integration are connected, and you can use the authorized account to access the data sources. You can add both the source and destination to DataWorks, and associate them with a data synchronization solution when you create the solution. For more information about how to add a data source, see Add a data source.