Realtime Compute for Apache Flink:Flink CDC pipeline module

Parameter

Description

Required

Data type

Default value

Note

name

The name of the Flink CDC data ingestion job.

No

STRING

Flink CDC Pipeline Job

-

schema.change.behavior

The strategy to handle schema changes.

No

STRING

LENIENT

Valid values are listed below. For more information, see Schema evolution configurations.

• LENIENT (default)

• EXCEPTION

• EVOLVE

• TRY_EVOLVE

• IGNORE

schema-operator.rpc-timeout

The timeout period for SchemaOperator to wait for the completion of a downstream schema change event application.

No

DURATION

3min

The timeout period for the schema change coordination process.

local-time-zone

The local time zone for job execution.

No

STRING

System time zone

This setting affects time-related calculations in the transform module.

user-defined-function

The user-defined functions (UDFs) to register for the job.

No

LIST

-

For more information, see User-defined functions.

transform.allow.trimming.pk-columns

Allows transform rules to reduce upstream primary key constraints

No

BOOLEAN

false

Removing upstream PK constraints can cause multiple data entries to merge, potentially leading to data losses. Before using this setting, ensure the uniqueness of the new PK (if defined).

transform.decimal.precision.mode

Specifies the maximum Decimal precision used by the Transform module.

No

STRING

UP_TO_19

Valid values:

• UP_TO_19: The maximum precision of the Decimal type returned by expressions is 19. This is the default behavior in VVR 11.6 and earlier.

• UP_TO_38: The maximum precision of the Decimal type returned by expressions is 38.

sink.partitioning.strategy

Specifies the partitioning strategy that distributes data change events to the downstream subtasks of the sink.

No

STRING

SINK_DEFINED

Valid values:

• SINK_DEFINED: Uses the partitioning strategy defined by the sink. This is the default value. The Paimon, Fluss, and MaxCompute sinks support only SINK_DEFINED.

• PRIMARY_KEY: Partitions events by table ID and primary key value. Events with the same primary key are routed to the same subtask.

• TABLE_ID: Partitions events by table ID. All events of the same table are routed to the same subtask.

transform.async-execution.enabled

Specifies whether to enable ordered asynchronous execution for Post-Transform (public preview).

No

BOOLEAN

false

Experimental feature.

If this parameter is enabled, Transform calculations run asynchronously in a thread pool while preserving event output order. This option is suitable for Transforms that include high-latency UDF or AI model calls.

transform.async-execution.timeout

The timeout period for asynchronously processing a single Post-Transform event (public preview).

No

DURATION

5min

Experimental feature.

This parameter takes effect only when asynchronous Transform is enabled. The value must be greater than 0.

transform.async-execution.capacity

The maximum number of in-flight events that each asynchronous Post-Transform subtask can process concurrently (public preview).

No

INTEGER

100

Experimental feature.

This parameter takes effect only when asynchronous Transform is enabled. The value must be greater than 0. A larger value typically provides higher concurrency but consumes more memory.

transform.async-execution.worker-threads

The number of worker threads used by each asynchronous Post-Transform subtask (public preview).

No

INTEGER

16

Experimental feature.

This parameter takes effect only when asynchronous Transform is enabled. The value must be greater than 0. The total number of threads increases with Transform parallelism.