This topic describes whether a job remains compatible with its state data after you modify a regular join, interval join, temporal join, or lookup join query.
Regular join
A regular join holds both input streams in state indefinitely. Its operator state cannot be migrated between job versions.
Incompatible modifications
The following modifications make a job incompatible with its state data:
|
Modification |
Effect |
|
Change the join type |
Incompatible |
|
Change the join condition |
Incompatible |
|
Change the input schema |
Incompatible |
Compatible exception
If you change equality or non-equality conditions and the change can be pushed down to an input stream to generate a Calc node with conditions, add table.optimizer.state-compatibility.ignore-filter: true to the Parameters section. The job remains fully compatible with the state data.
Interval join
An interval join bounds state by the interval window, so Flink discards old values after the interval elapses. Its operator state cannot be migrated between job versions.
Incompatible modifications
The following modifications make a job incompatible with its state data:
|
Modification |
Effect |
|
Change the join type |
Incompatible |
|
Change the join condition |
Incompatible |
|
Change the interval |
Incompatible |
|
Change the input schema |
Incompatible |
Compatible modifications
The following modifications keep the job fully compatible with its state data:
|
Modification |
Condition |
|
Change the time unit |
The final value of the interval must remain the same. For example, changing |
|
Change equality or non-equality conditions |
The change must be pushable down to an input stream to generate a Calc node with conditions, and |
Temporal join
A temporal join correlates a stream with a versioned table at a specific point in time. Its operator state cannot be migrated between job versions.
Incompatible modifications
The following modifications make a job incompatible with its state data:
|
Modification |
Effect |
|
Change the join type |
Incompatible |
|
Change the join condition |
Incompatible |
|
Change the input schema |
Incompatible |
Lookup join
By default, a lookup join runs as a non-stateful operator. Modifying a lookup join does not affect the state compatibility of its own operator. However, if you change the output of a lookup join, the state compatibility of downstream operators may be affected.
When a lookup join becomes stateful
A lookup join runs as a stateful operator under either of the following conditions:
-
table.optimizer.non-deterministic-update.strategy: TRY_RESOLVEis configured for the job and the system detects non-deterministic updates from the lookup join. -
Asynchronous lookup is configured for the lookup join.
When a lookup join runs as a stateful operator, the system may fail to detect compatibility issues and incorrectly report that modifications are fully compatible. If this happens, the job fails to start because the state data cannot be recovered.