Before you synchronize data from a PolarDB data source to Kafka, you can refer to the operations in this topic to configure the network, whitelists, and permissions for the data sources to implement data synchronization.
Prerequisites
- Prepare data sources: A PolarDB cluster and a Kafka data source are prepared. In this topic, a PolarDB for MySQL cluster is used as the source data source.
- Plan and prepare resources: An exclusive resource group for Data Integration is purchased and configured. For more information, see Plan and configure resources.
- Evaluate and plan the network environment: Before you perform data integration, connect
data sources to exclusive resource groups for Data Integration based on your business
requirements. After data sources and exclusive resource groups for Data Integration
are connected, you can refer to the operations in this topic to configure access settings
such as vSwitches and whitelists.
- If data sources and exclusive resource groups for Data Integration reside in the same region and virtual private cloud (VPC), they are automatically connected.
- If data sources and exclusive resource groups for Data Integration reside in different network environments, you must connect data sources and resource groups by using methods such as a VPN gateway.
Background information
- Configure whitelists for data sources
If the source and destination data sources and the exclusive resource group for Data Integration reside in the same VPC, you must add the CIDR block of the exclusive resource group for Data Integration to the whitelists of the data sources. This ensures that the exclusive resource group for Data Integration can be used to access the data sources.
- Create an account and grant permissions the account
You must create an account that can be used to access the data sources, read data from the source, and write data to the destination during the data synchronization process.
- Enable the binary logging feature
If the source data source is a PolarDB for MySQL cluster, you must enable the binary logging feature for the cluster. PolarDB for MySQL is fully compatible with MySQL and uses high-level physical logs to replace binary logs. To facilitate the integration between PolarDB and the MySQL ecosystem, you can enable the binary logging feature for PolarDB clusters.
Limits
- Only PolarDB for MySQL clusters can be used as source data sources. In this topic, PolarDB indicates PolarDB for MySQL data sources.
- Only data stored on the primary node of the PolarDB cluster can be synchronized.
Procedure
What to do next
After data sources are configured, the source data source, destination data source, and exclusive resource group for Data Integration are connected. Then, the exclusive resource group for Data Integration can be used to access the data sources. You can add the source data source and destination data source to DataWorks, and associate them with a data synchronization solution when you create the solution.
For more information about how to add the source and destination data sources, see Add data sources.