Row filters in Data Authorization let you control the data range returned by queries. This topic covers configuration specifications, supported expressions, and limitations.
Limitations
When configuring row filters, note the following limitations:
Engine version: Only compute engines integrated with DLF Row Filter are supported. Currently, only Serverless Spark is supported, and you must use one of the following versions:
esr-5.2.0 (Spark 4.0.1, Scala 2.13) or later
esr-4.8.0 (Spark 3.5.2, Scala 2.12) or later
esr-3.7.0 (Spark 3.4.4, Scala 2.12) or later
Single rule limit: For the same table, a user can configure only one composite expression.
Schema change impact: If you rename or delete a column, the original filter expression becomes invalid and you must reconfigure it.
Multi-policy merging: When filters based on role and user are configured simultaneously, the system merges them using an AND logical operation.
Row filter expressions
Row filter expressions restrict the data range of query results. They fall into two categories: simple expressions and composite expressions.
Simple expressions
A simple expression defines basic comparison logic for a single column.
Syntax:
<Column Name> <Comparison Operator> <Value>
Parameters:
Column Name: The column to filter. Supported data types:
Boolean: BOOLEAN
String: CHAR, VARCHAR
Numeric: DECIMAL, TINYINT, SMALLINT, INT, BIGINT, FLOAT, DOUBLE
Temporal: TIME, TIMESTAMP
Comparison Operator: The comparison method. For supported operators, see the comparison table below.
Value: A constant for comparison. The type must match the column data type.
Composite expressions
Composite expressions combine multiple simple expressions with logical operators for more complex filtering.
Syntax:
(<Simple Expression>) <AND/OR> (<Simple Expression>)
Composite expressions support nesting and can be further combined using the logical operators AND or OR.
Operator comparison table
The following table lists the supported operators and their corresponding predicate implementations in Apache Paimon.
Operator | Description | Paimon Predicate | Example |
AND | Logical AND | PredicateBuilder.and | year > 2010 AND country <> 'US' |
OR | Logical OR | PredicateBuilder.or | (year > 2010) OR (month < 8) |
IS NULL | Is null | PredicateBuilder.isNull | country IS NULL |
IS NOT NULL | Is not null | PredicateBuilder.isNotNull | country IS NOT NULL |
IN | Set inclusion | PredicateBuilder.in | id IN (1, 2, 3) |
NOT IN | Set exclusion | PredicateBuilder.notIn | id NOT IN (1, 2, 3) |
= | Equals | PredicateBuilder.equal | year = 2010 |
<> | Does not equal | PredicateBuilder.notEqual | year <> 2010 |
< | Less than | PredicateBuilder.lessThan | year < 2010 |
<= | Less than or equal to | PredicateBuilder.lessOrEqual | year <= 2010 |
> | Greater than | PredicateBuilder.greaterThan | year > 2010 |
>= | Greater than or equal to | PredicateBuilder.greaterOrEqual | year >= 2010 |
BETWEEN | Range | PredicateBuilder.between | year BETWEEN 2010 AND 2025 |