Before you use the MySQL Change Data Capture (CDC) connector, make sure that the upstream MySQL storage meets the configuration requirements, such as version, network connection, and account permissions. You must also enable the binary logging feature. This topic describes the configuration requirements and related operations for ApsaraDB RDS for MySQL, PolarDB for MySQL, and self-managed MySQL databases.

ApsaraDB RDS for MySQL database

  • Network connection configurations
    The network connection between ApsaraDB RDS for MySQL and Ververica Platform (VVP) is established.
  • MySQL server configurations
    • Supported versions

      Only MySQL V5.7 and MySQL V8.0.X are supported. You can run the select version() command to query the MySQL version.

    • Binary logging feature
      The binary logging feature must be enabled. You can run the show variables like "log_bin" command to check whether this feature is enabled.
    • Format of binary logs

      The binary logs must be in the ROW format. You can run the show variables like "binlog_format" command to query the format of binary logs. The default format of the binary logs of ApsaraDB RDS for MySQL is ROW.

    • Setting of the binlog_row_image parameter

      The binlog_row_image parameter must be set to FULL. You can run the show global variables like "binlog_row_image" command to query the value of the binlog_row_image parameter. The default value of the binlog_row_image parameter for ApsaraDB RDS for MySQL is FULL.

  • Session parameter configurations
    When you use the MySQL CDC connector to read full snapshots from a large-sized database, the established connection may time out. To prevent this issue, you can configure the interactive_timeout and wait_timeout parameters in the MySQL configuration file. Parameter description:
  • Account permission configurations

    Create a database account and grant the SELECT, REPLICATION SLAVE, and REPLICATION CLIENT permissions to the account. These permissions are required to synchronize data from ApsaraDB RDS for MySQL by using the MySQL CDC connector. For more information, see Create an account on an ApsaraDB RDS for MySQL instance.

ApsaraDB RDS for MySQL database

  • Network connection configurations
    The network connection between PolarDB for MySQL and VVP is established.
  • MySQL server configurations
    • Supported versions

      Only MySQL V5.7 and MySQL V8.0.X are supported. You can run the select version() command to query the MySQL version.

    • Binary logging feature
      The binary logging feature must be enabled. You can run the show variables like "log_bin" command to check whether this feature is enabled.
      • If the return value is ON, this feature is enabled.
      • If the return value is OFF, this feature is disabled.
        Note For more information about how to enable the binary logging feature, see Enable binary logging.
    • Format of binary logs

      The binary logs must be in the ROW format. You can run the show variables like "binlog_format" command to query the format of binary logs. The default format of binary logs of PolarDB for MySQL is ROW.

    • Setting of the binlog_row_image parameter

      The binlog_row_image parameter must be set to FULL. You can run the show global variables like "binlog_row_image" command to query the value of the binlog_row_image parameter. The default value of the binlog_row_image parameter for PolarDB for MySQL is FULL.

  • Session parameter configurations
    When you use the MySQL CDC connector to read full snapshots from a large-sized database, the established connection may time out. To prevent this issue, you can configure the interactive_timeout and wait_timeout parameters in the MySQL configuration file. Parameter description:
    • interactive_timeout: specifies the number of seconds the server waits for an activity before the server closes an interactive connection. For more information, see Specify cluster parameters.
    • wait_timeout: specifies the number of seconds the server waits for an activity before the server closes a non-interactive connection. For more information, see Specify cluster parameters.
  • Account permission configurations
    Create a database account and grant the SELECT, REPLICATION SLAVE, and REPLICATION CLIENT permissions to the account. These permissions are required to synchronize data from PolarDB for MySQL by using the MySQL CDC connector. For more information about how to create an account and grant permissions to the account, see Create a database account.
    Note By default, an account that is created in the PolarDB for MySQL console does not have the SELECT permission. You must specify the preceding permissions when you create an account. Then, you must go to the Database Management Service (DMS) console to create an account and grant the preceding permissions to the account. For more information, see Manage user permissions on MySQL databases.

Self-managed MySQL database

  • Network connection configurations
    The network connection between the self-managed MySQL database and VVP is established.
    • If your MySQL database is deployed on Alibaba Cloud and resides in the same VPC as VVP, the network between the MySQL database and VVP is connected.
    • If your MySQL database is not deployed on Alibaba Cloud, you must enable VVP to access the public IP address of the MySQL database. In this case, you need to connect the self-managed MySQL database and VVP by using VPN Gateway. For more information, see How does a fully managed Flink cluster access the Internet?.
  • MySQL server configurations
    • Supported versions

      Only MySQL V5.7 and MySQL V8.0.X are supported. You can run the select version() command to query the MySQL version.

    • Binary logging feature
      The binary logging feature must be enabled. You can run the show variables like "log_bin" command to check whether this feature is enabled.
      • If the return value is ON, this feature is enabled.
      • If the return value is OFF, this feature is disabled.
        Note For more information about how to enable the binary logging feature, see Enable binary logging.
    • Format of binary logs

      The binary logs must be in the ROW format. You can run the show variables like "binlog_format" command to query the format of binary logs. If the binary logs are not in the ROW format, you must change the format to ROW. For more information, see Configure the binary log format.

    • Setting of the binlog_row_image parameter

      The binlog_row_image parameter must be set to FULL. You can run the show global variables like "binlog_row_image" command to query the value of the binlog_row_image parameter. If the value of the binlog_row_image parameter is not FULL, you must change the value to FULL. For more information, see Setting of binlog_row_image.

  • Session parameter configurations
    When you use the MySQL CDC connector to read full snapshots from a large-sized database, the established connection may time out. To prevent this issue, you can configure the interactive_timeout and wait_timeout parameters in the MySQL configuration file. Parameter description:
  • Account permission configurations
    Create a database account and grant the SELECT, REPLICATION SLAVE, and REPLICATION CLIENT permissions to the account. To configure account permissions, perform the following steps:
    1. Create an account and configure a password for the account.
      CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';
      For more information, see Create a user.
    2. Grant the required permissions to the account.
       GRANT SELECT, REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'user' IDENTIFIED BY 'password'; 

      *.* indicates that the account is granted the preceding permissions on all tables in all databases. You can also use mydb.* to grant the preceding permissions on a database to an account.

    3. Make sure that the permissions granted to the account are valid.
      FLUSH PRIVILEGES;