Modifying pattern recognition attributes in a Flink SQL job breaks compatibility between the job and its existing state data.
How it works
Realtime Compute for Apache Flink implements pattern recognition through the MATCH_RECOGNIZE clause, which uses complex event processing (CEP) internally. The CEP engine builds a finite state machine from the pattern definition and stores in-flight event sequences as state data tied to that state machine structure.
CEP state data is a snapshot of the running state machine and is tightly coupled to the pattern that generated it. When the pattern changes, the new state machine has a different structure, and the existing state data no longer maps to it. As a result, Flink cannot migrate CEP operator state across pattern changes.
Attributes that break state compatibility
Modifying any of the following attributes in a pattern recognition SQL statement makes the job incompatible with its existing state data:
| Attribute | Role in pattern recognition |
|---|---|
pattern |
Defines the event sequence to match |
measures |
Defines the output columns extracted from a match |
after |
Controls the start position of the next match |
PARTITION BY |
Partitions the input stream for independent per-key matching |
ORDER BY |
Defines the order in which events are evaluated |