Realtime Compute for Apache Flink:PolarDB-X CDC

Connector overview

Server-side binary log filtering

The PolarDB-X CDC connector filters binary logs on the server before sending them to the client. Only the change events you subscribe to are transmitted, which reduces network I/O and improves throughput.

This differs from connectors that pull all binary logs from an instance and filter them on the client side.

To subscribe to specific tables, set polardbx.binlog.include.tables in your CREATE TABLE statement:

CREATE TABLE polardbx_table_foo (
  -- Define the table schema here
) WITH (
  'connector' = 'polardbx-cdc',
  'database-name' = 'db',
  'table-name' = '.*',
  -- Other required parameters
  'polardbx.binlog.include.tables' = 'db.table1,db.table2'
);

Server-side binary log filtering requires PolarDB-X server version 2.5.0 or later and Simple Log Service component version 5.4.20 or later.

Limitations

• The connector can only be used as a source table. For sink or dimension table access, use the MySQL connector.

• Server-side binary log filtering requires PolarDB-X 2.5.0 or later and Simple Log Service 5.4.20 or later.

SQL syntax

CREATE TABLE polardbx_customer_table (
  `id` STRING,
  [columnName dataType,]*
  PRIMARY KEY (`id`) NOT ENFORCED
) WITH (
  'connector' = 'polardbx-cdc',
  'hostname' = '<cluster-endpoint>',
  'username' = '<username>',
  'password' = '<password>',
  'database-name' = '<database-name>',
  'table-name' = '<table-name>'
);

Replace the placeholders with your actual values:

WITH parameters

Startup modes

The scan.startup.mode parameter controls where the connector begins reading when the job starts for the first time.

Monitoring metrics

The following metrics are available for the PolarDB-X CDC connector. Metrics that depend on binary log position return 0 during the snapshot phase.

Type mapping