Flink CDC source and sink modules - Realtime Compute for Apache Flink

The source and sink modules define the input and output ends of a Flink Change Data Capture (CDC) data ingestion job. Configure each module with a connector type and its parameters in YAML.

Connector parameter configuration

A Flink CDC job includes a source block and a sink block. Set type to the connector identifier, then add connection parameters in key: value format.

# Source module
source:
  type: mysql
  name: MySQL Source
  hostname: rm-bp1xxxxxxx.mysql.rds.aliyuncs.com
  port: 3306
  username: root
  password: <your-password>
  tables: mydb\.orders, mydb\.products
  server-id: 5401-5404

# Sink module
sink:
  type: paimon
  name: Paimon Sink
  warehouse: oss://my-bucket/paimon-warehouse
  catalog-name: my-paimon-catalog

Replace the placeholder values with your actual connection parameters. For the full parameter list for each connector, see the links in the Supported connectors table.

Supported connectors

The following connectors are available as sources and sinks in Flink CDC jobs. Connectors that require a specific minimum Ververica Runtime (VVR) version are noted in the Minimum version column.

Connector	Source	Sink	Minimum version
MySQL	✓	—	—
Streaming data lakehouse Paimon	—	✓	—
Message Queue for Kafka	✓ (VVR 8.0.10+)	✓	—
Upsert Kafka	—	✓	—
StarRocks	—	✓	—
Hologres	—	✓	—
Simple Log Service (SLS)	✓	—	VVR 11.1
MongoDB	✓	—	VVR 11.2
MaxCompute	—	✓	VVR 11.1
SelectDB	—	✓	VVR 11.1
Postgres CDC (Public Preview)	✓	—	VVR 11.4
Print	—	✓	—
Iceberg	—	✓	VVR 11.6

The MySQL connector supports connections to RDS for MySQL, PolarDB for MySQL, and self-managed MySQL.

Reuse an existing catalog for connection information

Starting from VVR 11.5, you can reference a built-in catalog from the Data Management page in a Flink CDC job. This pulls connection properties — such as URL, username, and password — directly from the catalog, so you don't need to enter them again in the YAML job.

Syntax

Use the using.built-in-catalog key in the source or sink block:

source:
  type: mysql
  using.built-in-catalog: mysql_rds_catalog

sink:
  type: paimon
  using.built-in-catalog: paimon_dlf_catalog

In this example, mysql_rds_catalog already stores hostname, username, and password. Those parameters are applied automatically and do not need to appear in the YAML job.

Supported connectors

The following connectors support catalog reuse:

Module	Connectors
Source	MySQL, Message Queue for Kafka, Simple Log Service (SLS)
Sink	Upsert Kafka, StarRocks, Hologres, Streaming data lakehouse Paimon, Iceberg

Catalog parameters that are incompatible with CDC YAML are ignored. See the parameter list for each connector for details.