how to use filter conditions for RDS/PolarDB data sources - OpenSearch

A filter condition restricts which rows are pulled from your RDS or PolarDB database table into OpenSearch — for both full and incremental data sync. Write the condition as a comma-separated expression using database table field names (not OpenSearch application schema field names).

Supported field types

Category	Types
Integer	TINYINT, SMALLINT, INTEGER, BIGINT
Floating point	FLOAT, REAL, DOUBLE, NUMERIC, DECIMAL
Temporal	TIME, TIMESTAMP

Operators

Numeric and temporal comparison

Use <, >, <=, >= with numeric types (TINYINT through DECIMAL) and temporal types (TIME, TIMESTAMP):

price > 100
createtime > '2021-02-05 00:00:00'

Equality and inequality

Use =, !=, <> with both numeric and character types:

status = 1
field_string = 'active'
field_string != 'deleted'
field_string <> 'deleted'

For character values, single quotes ('), double quotes ("), or no quotes are all valid.

NULL check

Use =, !=, or <> to check for NULL values — not IS NULL or IS NOT NULL. This applies to both numeric and character fields:

field = null
field = NULL
field != null
field != NULL
field <> null
field <> NULL

Character type examples

field = ''
field != ''
field <> ''
field = 'null'
field = 'NULL'
field != 'null'
field != 'NULL'
field <> 'null'
field <> 'NULL'

Combine multiple conditions

Use a comma (,) to represent AND logic. The OR operator is not supported.

status=1,literal_id='abcd',createtime>'2021-02-05 00:00:00'

The equivalent SQL WHERE clause is:

WHERE status = 1 AND literal_id = 'abcd' AND createtime > '2021-02-05 00:00:00'

Do not use AND or the WHERE keyword directly in the filter condition:

WHERE status=1 AND type=2   -- Incorrect
status=1 AND type=2         -- Incorrect
status=1,type=2             -- Correct

Usage notes

Scope

The data source filtering feature is available for RDS and PolarDB data sources.

Effect on deletions

When a filter condition is active, OpenSearch removes any document that no longer matches the condition — including during incremental sync.

For example, if the filter is status=1 and a document's status changes from 1 to 2 in RDS, that document is deleted from OpenSearch.

If your data source has a primary table and a secondary table, apply filter conditions to both tables to prevent unnecessary storage consumption on the secondary table.

Field naming

Use field names from the database table, not from the OpenSearch application schema.

DATE and DATETIME fields

For DATE or DATETIME fields, the time value must be in the required format. Example:

createtime>'2018-03-01 00:00:00'

Unsupported operations

Unsupported	Incorrect example	Correct alternative
`IS NULL` / `IS NOT NULL`	`field IS NULL`	`field = null`
`LIKE` / `NOT LIKE`	`LIKE '%aaa'`	—
Functions and expressions	`in(type,1,2,3)`, `length(title) > 10`	—
Identity equations	`1 = 1`	—
OR logic	`status=1 OR type=2`	Use comma for AND: `status=1,type=2`