Flink CDC Route module - Realtime Compute for Apache Flink

The route module maps source tables to sink tables in a Flink CDC pipeline. Configure routing rules when you need to rename tables, merge multiple source tables into one sink table, or replicate a single source table to multiple destinations.

Parameters

Parameter	Description	Required
`source-table`	The source table to match. Supports regular expressions.	Required
`sink-table`	The sink table to write to. Supports `replace-symbol` placeholders and capturing group references (`$1`, `$2`, ...).	Required
`replace-symbol`	A string in the `sink-table` value that is replaced by the matched source table name. For example, set `replace-symbol` to `<>` and `sink-table` to `sinkdb.<>`. If the source table is `table1`, data is written to `sinkdb.table1`.	Optional
`description`	A description of the routing rule.	Optional

Important

After modifying the route module, perform a stateless startup to apply the new routing rules.

Examples

Single table sync

Route data from mysql_db.web_order to sink_db.ods_web_order:

route:
  - source-table: mysql_db.web_order
    sink-table: sink_db.ods_web_order
    description: sync data from a source table to a sink table

One-to-many mapping

To replicate a source table to multiple sink tables, define multiple routing rules that share the same source-table. The following rules replicate mydb.orders to both sink_db.orders and backup_sink_db.orders simultaneously:

route:
  - source-table: mydb.orders
    sink-table: sink_db.orders
  - source-table: mydb.orders
    sink-table: backup_sink_db.orders

Multiple one-to-one mappings

Define multiple independent rules using the YAML list indicator (-). All rules are applied concurrently:

route:
  - source-table: mydb.orders
    sink-table: ods_db.ods_orders
    description: sync orders table to ods_orders
  - source-table: mydb.shipments
    sink-table: ods_db.ods_shipments
    description: sync shipments table to ods_shipments
  - source-table: mydb.products
    sink-table: ods_db.ods_products
    description: sync products table to ods_products

Sharded table merging

Merge all tables from source_db into the single sink table sink_db.merged:

route:
  - source-table: source_db.\.*
    sink-table: sink_db.merged
    description: merge sharded tables to a unified sink table

Database sync

Sync all tables from source_db to corresponding tables in sink_db, keeping table names unchanged. The <> placeholder in sink-table is replaced by each matched source table name, so source_db.XXX routes to sink_db.XXX:

route:
  - source-table: source_db.\.*
    sink-table: sink_db.<>
    replace-symbol: <>
    description: route all tables in source_db to sink_db

Advanced routing with capturing groups

Create capturing groups in the source-table regular expression using parentheses, then reference the captured values in sink-table using $1, $2, and so on.

Add a prefix to database names, keep table names unchanged:

route:
  - source-table: (\.*).(\.*)
    sink-table: ods_$1.$2

$1 captures the full database name and $2 captures the full table name.

Add a prefix to both database and table names:

route:
  - source-table: (\.*).(\.*)
    sink-table: ods_$1.upstream_$2

Flatten database and table names into a single sink table:

route:
  - source-table: db_no_(\.*).table_no_(\.*)
    sink-table: sink_db.table_$1_$2

For example, db_no_100.table_no_300 matches capturing groups 100 and 300, and routes to sink_db.table_100_300.