DataWorks provides MySQL Reader and MySQL Writer for you to read data from and write data to MySQL data sources. You can use the codeless user interface (UI) or code editor to configure synchronization nodes for MySQL data sources. This topic describes how to add a MySQL data source. This topic also describes the network environment and permissions that you must prepare before you add a data source.
Prerequisites
- Prepare a data source: A MySQL data source is created.
- Plan and prepare resources: An exclusive resource group for Data Integration is purchased and configured. For more information, see Plan and configure resources.
- Evaluate and plan the network environment: Before you add the data source to DataWorks,
connect the data source to an exclusive resource group for Data Integration based
on your business requirements. After you connect the data source to the exclusive
resource group for Data Integration, configure network access settings such as a vSwitch
and a whitelist.
- If data sources and exclusive resource groups for Data Integration reside in the same region and virtual private cloud (VPC), they are automatically connected.
- If data sources and exclusive resource groups for Data Integration reside in different network environments, you must connect data sources and resource groups by using methods such as a VPN gateway.
- Execute the following statement to check the version of your MySQL database. You can
use a
MySQL 5.X
orMySQL 8.X
database.select version();
Note Real-time data synchronization from or to MySQL is performed based on real-time subscription to MySQL binary logs. For a real-time data synchronization node that uses a MySQL data source, you can use an ApsaraDB RDS for MySQL data source whose MySQL version isV5.X
orV8.X
. PolarDB for MySQL is not supported. If the MySQL version of your ApsaraDB RDS for MySQL database is notV5.X
orV8.X
, use an ApsaraDB RDS for MySQL database whose MySQL version isV5.X
orV8.X
. Otherwise, the data synchronization node fails to run. - Prepare an account and grant permissions to the account.
You must create an account that can be used to access the data sources, read data from the source, and write data to the destination during the data synchronization process.
For more information, see Create and authorize an account. - Enable the MySQL binary logging feature. This feature is required only for real-time
data synchronization. For more information about real-time data synchronization, see
Overview.
If the source data source is a MySQL database, you must enable the binary logging feature. Binary logs record changes to all table schemas and modifications on table data. You can execute statements such as CREATE and ALTER to perform operations on table schemas. You can execute statements such as INSERT, UPDATE, and DELETE to perform operations on table data. You can use binary logs to view the change history of the database, back up incremental data and restore data in the database, and replicate data from the primary database to secondary databases.
Formats of binary logs:For more information about how to enable the binary logging feature, see Enable the binary logging feature.- Statement: SQL statement-based replication. Binary logs in this format record the SQL statements that are executed to modify data entries.
- Row: row-based replication. Binary logs in this format record only modification details about data entries in rows.
- Mixed: replication in mixed mode. This mode combines the statement and row formats. In most cases, binary logs in the statement format are used to record the SQL statements that are executed to modify data entries, such as functions. If data replication from the primary database to secondary databases cannot be implemented by using binary logs in this format, switch to the row format. MySQL determines which format to use based on each SQL statement that is executed.
Background information
Limits
Real-time data synchronization from or to MySQL is performed based on real-time subscription to MySQL binary logs. For a real-time data synchronization node that uses a MySQL data source, you can use an ApsaraDB RDS for MySQL data source whose MySQL version isV5.X
or V8.X
. PolarDB for MySQL is not supported.