This topic describes the index schema, search syntax, and search features of OpenSearch.
Schema
OpenSearch supports various data types and analyzers to meet the requirements in most scenarios. For more information about analyzers, see Text analyzers. Take note of the following items:
Supported data types: INT, FLOAT, DOUBLE, LITERAL, TEXT, SHORT_TEXT, and their corresponding ARRAY types.
DynamicField: Not supported. As a workaround, you can dynamically modify the application schema in OpenSearch.
CopyField: Copying fields are not supported. Therefore, you can merge table fields in advance.
A maximum of 256 fields are allowed. If more than 256 fields need to be added for an OpenSearch application, you can merge the fields that are used for non-range queries into a single field of the ARRAY type. This can reduce the total number of fields that are added for the application.
patternTokenizer: OpenSearch supports custom analyzers for analysis. However, the default delimiter is \t. In this case, you must convert the original delimiter to \t.
LOCATION: This type of field is converted to two fields of the FLOAT or DOUBLE type, which are used to store the longitude and latitude values.
BOOLEAN: This type of field is converted to a field of the INT type, whose value is 0 or 1.
date: Convert this field type to an INT field. Data from a database source is automatically converted to a millisecond timestamp. Data pushed using an API must be converted manually.
Payload analyzer: This type of analyzer is not supported.
Bitwise analyzer: This type of analyzer is not supported.
Paoding analyzer: This type of analyzer uses Chinese basic analyzers of OpenSearch.
Search syntax
OpenSearch supports features such as query, filtering, statistics, aggregation, and sorting.
q: required. This parameter is equivalent to a query in OpenSearch. The following table describes the specific conversion rules.
q conversion rules |
The colon character (:) is not supported. |
For a range index, use a filter range. |
+A ==> A |
-A ==> This type of conversion is not supported. |
A AND B ==> A AND B |
A AND -B ==> A ANDNOT B |
A OR B ==> A OR B |
A OR +B ==> A RANK B |
A AND B OR C ==> A AND B RANK C. Example: Hongfushi AND Apple OR Shandong. |
A OR B AND C ==> B AND C RANK A. Example: Hongfushi OR Apple AND Shandong. |
A AND B OR +C ==> A AND B AND C. Example: Hongfushi AND Apple OR +Shandong. |
A OR +B AND C ==> B AND C RANK A. Example: Hongfushi OR +Apple AND Shandong. |
+A OR B AND C ==> A AND B AND C. Example: +Hongfushi OR Apple AND Shandong. |
A AND B OR -C ==> (A AND B) ANDNOT C. Example: Hongfushi AND Apple OR -Shandong. |
A AND -B OR C ==> A ANDNOT B RANK C. Example: Apple AND -Hongfushi OR Shandong. |
-A AND B OR C ==> B ANDNOT A RANK C. Example: -Hongfushi AND Apple OR Shandong. |
A OR B AND -C ==> B ANDNOT C RANK A. Example: Hongfushi OR Apple AND -Shandong. |
A OR -B AND C ==> C ANDNOT B RANK A. Example: Hongfushi OR -Shandong AND Apple. |
-A OR B AND C ==> (B AND C) ANDNOT A. Example: -Hongfushi OR Shandong AND Apple. |
A OR B OR -C == A OR -C OR B == -C OR A OR B ==> (A OR B) ANDNOT C |
A AND B OR C AND D ==> A AND B AND C AND D |
fq: Used for filtering. This parameter affects retrieval but does not affect score calculation. Use the filter clause for exact queries and the query clause for fuzzy queries. Do not use this field for sorting.
fl: Uses the OpenSearch fetch_fields parameter to define the return value.
hl: To configure search result summaries and highlighting, use the console.
start, rows: These parameters are equivalent to the start and hit parameters in the config clause.
wt: The format of the config clause.
df: This parameter specifies the default field for the query.
sort: field desc is equivalent to -field. field asc is equivalent to +field. score is equivalent to sort=RANK.
facet: the index attribute that must be configured for the field.
Statistics conversion rules |
facet.field: This parameter is equivalent to the group_key parameter in the OpenSearch aggregate clause. |
facet.limit: This parameter is equivalent to the max_group parameter in the OpenSearch aggregate clause. The default value is 1,000. |
facet.mincount: Not supported. You must retrieve all results and process them manually. |
facet.offset: Not supported. You must retrieve all results and handle pagination manually. |
facet.sort: Not supported. You must retrieve all results and sort them manually. |
facet=true&facet.field=price&facet.limit=200 ==> aggregate=group_key:price,agg_fun:count(),max_group:200 |
group: not supported. You can use the distinct clause with the sort clause to sort data for some simple scenarios.
stats: Some of its features are equivalent to the features of the aggregate clause in OpenSearch. However, agg_func supports only min, max, count, and avg. It does not support missing, sumOfSquares, mean, stddev, distinctValue, or countDistinct.
Search features
Deep pagination: OpenSearch provides two query interfaces: search and scroll. Search is a common query scenario. In this scenario, a maximum of 5,000 results can be returned. Page turning is supported. A maximum of 500 results can be displayed on each page. Scroll is a data export scenario. In this scenario, tens of millions of data records can be exported. Sorting is not supported. The results that are returned can be further analyzed.
Accuracy of statistical results: To ensure better retrieval performance, OpenSearch performs sampling and estimation in many cases, which may lead to inaccurate statistical results.
Total number of search results: To ensure the search performance, OpenSearch estimates the total number of results to be returned for queries regardless of the total amount of data.
Multiple OR operators in a query: The length of a query string can be up to 1 KB after encoding. If excessive OR operators are used, an error occurs and no results are returned. In this case, we recommend that you increase the upper limit of the query string length or concurrently perform multiple queries and merge the results that are returned.