When you modify a WHERE clause in a running deployment, the change does not break the WHERE clause operator's own state — it has none. However, it does break state compatibility for downstream operators, such as GROUP BY aggregations, because those operators receive a different set of input rows. To bypass this downstream impact and keep your deployment running from existing state, set table.optimizer.state-compatibility.ignore-filter=true.
Why WHERE changes affect downstream state
A WHERE clause filters rows without accumulating state — each output row depends on exactly one input row. This makes the WHERE operator itself stateless, so changing its condition does not affect its own state restoration.
Downstream operators that consume the filtered output — for example, a GROUP BY aggregation — do maintain state. When the WHERE condition changes, those operators receive a different data set, making their existing state incompatible with the new input.
The following SQL examples show the compatibility difference based on the table.optimizer.state-compatibility.ignore-filter setting:
-- Original: no WHERE clause
SELECT a, sum(b), max(c) FROM MyTable GROUP BY a;
-- After adding WHERE, with ignore-filter=false (default):
-- The deployment becomes incompatible with existing state data.
SELECT a, sum(b), max(c) FROM (SELECT * FROM MyTable WHERE a > 10) GROUP BY a;
-- After adding WHERE, with ignore-filter=true:
-- The deployment remains fully compatible with existing state data.
SELECT a, sum(b), max(c) FROM (SELECT * FROM MyTable WHERE a > 10) GROUP BY a;Ignore the downstream state compatibility impact
If you need to modify a WHERE condition — for example, to filter out dirty data — set the following property to make the system ignore the state compatibility impact on downstream operators.
Add the following configuration to the Other Configuration field in the Parameters section of the Configuration tab for the deployment. For more information, see How do I configure parameters for deployment running?
table.optimizer.state-compatibility.ignore-filter=trueThe default value of table.optimizer.state-compatibility.ignore-filter is false.
With this property set to true, the system ignores the impact of the WHERE clause modification on the state data compatibility of downstream operators.