TairSearch is a full-text search data structure developed in-house based on Redis modules instead of Lucene or other open source search engine software libraries. TairSearch uses query syntax that is similar to that of Elasticsearch.
Overview
- Low latency and high performance: provides millisecond-level write and full-text search capabilities based on the ultra-high computing power of Tair.
- Incremental and partial updates: supports partial updates of indexes and incremental updates of documents, including adding, updating, removing, and auto-incrementing fields.
- Flexible syntax: provides custom sorting order and supports the JSON syntax that allows bool, match, term, and paging queries. This syntax is similar to that of Elasticsearch.
- Aggregate query: supports terms, metrics, and filter aggregations. For more information, see Aggregations.
- Auto-complete suggestion: supports fuzzy match with prefixes and automatic completion for search operations.
- Custom tokenizer: provides tokenizers for popular languages such as Chinese and English. You can also use a custom tokenizer.
- On March 11, 2022, TairSearch was released with Tair V1.7.27.
- On May 24, 2022, Tair V1.8.5 was released to add support for the TairSearch aggregation feature.
- On September 6, 2022, Tair V5.0.15 was released to add support for the TFT.MSEARCH command.
Best practices
Prerequisites
Precautions
- The TairSearch data that you want to manage is stored on the Tair instance.
- To reduce memory usage, we recommend that you perform the following operations:
- When you create indexes, set index to true for document fields that you want to specify as inverted fields. For other document fields, set index to false.
- Specify the includes and excludes parameters of the _source field to filter out document fields that you do not need and save fields that you need.
- Do not add a large number of documents to a single index. Add the documents to multiple indexes. We recommend that you keep the number of documents per index within 500 million to prevent data skew in cluster instances, balance read and write traffic, and reduce the number of large keys and hotkeys.
Supported commands
Command | Syntax | Description |
---|---|---|
TFT.CREATEINDEX | TFT.CREATEINDEX index mappings |
Creates an index and a mapping for the index. The syntax used to create a mapping is similar to that used to create an explicit mapping in Elasticsearch. For more information, see Explicit mapping. You must create an index before you can add documents to the index. |
TFT.UPDATEINDEX | TFT.UPDATEINDEX index mappings |
Adds a properties field to the specified index. The field that you want to add cannot have the same name as an existing one. Otherwise, the field cannot be added. |
TFT.GETINDEX | TFT.GETINDEX index |
Obtains the mapping content of an index. |
TFT.ADDDOC | TFT.ADDDOC index document [WITH_ID doc_id] |
Adds a document to an index. You can specify a unique ID for the document in the index by using WITH_ID doc_id. If the document ID already exists, the existing ID is overwritten. If you do not specify WITH_ID doc_id, a document ID is automatically generated. By default, WITH_ID doc_id is not specified. |
TFT.MADDDOC | TFT.MADDDOC index document doc_id [document1 doc_id1] ... |
Adds multiple documents to an index. Each document must have a document ID that is specified by doc_id. |
TFT.UPDATEDOCFIELD | TFT.UPDATEDOCFIELD index doc_id document |
Updates the document specified by doc_id in an index. If the document fields that
you want to update are mapped and indexed fields, the fields must be of the same type
as those used by the mapping. If the fields are not indexed fields, the fields to
be updated can be of a random type.
Note If the fields already exist, the document is updated. If the fields do not exist,
the fields are added. If the document does not exist, the document is automatically
created. In this case, the command is equivalent to TFT.ADDDOC.
|
TFT.DELDOCFIELD | TFT.DELDOCFIELD index doc_id field [field1 field2 ...] |
Deletes the specified field from the document specified by doc_id in an index. If
the field is an indexed field, the information of the field is also deleted from the
index.
Note If the field does not exist, the operation cannot be performed. For example, the field
may not exist because it is filtered out by using _source.
|
TFT.INCRLONGDOCFIELD | TFT.INCRLONGDOCFIELD index doc_id field increment |
Adds an increment to the specified field in the document specified by doc_id in an
index. The increment can be a positive or negative integer. The data type of the field
can only be LONG or INTEGER.
Note If the document does not exist, the document is automatically created. In this case,
the existing value of the field is 0, and the updated field value is obtained by adding
the increment value to the existing field value. If the field does not exist, the
operation cannot be performed. For example, the field may not exist because it is
filtered out by using _source.
|
TFT.INCRFLOATDOCFIELD | TFT.INCRFLOATDOCFIELD index doc_id field increment |
Adds an increment to the specified field in the document specified by doc_id in an
index. The increment can be a positive or negative floating-point number. The data
type of the field can be INTEGER, LONG, or DOUBLE.
Note If the document does not exist, the document is automatically created. In this case,
the existing value of the field is 0, and the updated field value is obtained by adding
the increment value to the existing field value. If the field does not exist, the
operation cannot be performed. For example, the field may not exist because it is
filtered out by using _source.
|
TFT.GETDOC | TFT.GETDOC index doc_id |
Obtains the content of the document specified by doc_id in an index. |
TFT.EXISTS | TFT.EXISTS index doc_id |
Checks whether the document specified by doc_id exists in an index. |
TFT.DOCNUM | TFT.DOCNUM index |
Obtains the number of documents in an index. |
TFT.SCANDOCID | TFT.SCANDOCID index cursor [MATCH *value*] [COUNT count] |
Obtains the IDs of all documents in an index. |
TFT.DELDOC | TFT.DELDOC index doc_id [doc_id] ... |
Deletes the document specified by doc_id from an index. Multiple document IDs can be specified. |
TFT.DELALL | TFT.DELALL index |
Deletes all documents from an index but retains the index. |
TFT.SEARCH | TFT.SEARCH index query |
Queries the documents in an index. The query syntax is similar to the Query Domain Specific Language (DSL) syntax used in Elasticsearch. For more information, see Query DSL. |
TFT.MSEARCH | TFT.MSEARCH index_count index [index1] ... query |
Queries documents in multiple indexes that have mappings set to the same value by using the query clause and gathers the results from these indexes. Then, the results are rated, sorted, aggregated, and returned. |
DEL | DEL key [key ...] |
Deletes one or more TairSearch keys. |
Command | Syntax | Description |
---|---|---|
TFT.ADDSUG | TFT.ADDSUG index text weight [text weight] ... |
Adds one or more auto-complete texts and their weights to the specified index. |
TFT.DELSUG | TFT.DELSUG index text [text] ... |
Deletes one or more auto-complete texts from the specified index. |
TFT.SUGNUM | TFT.SUGNUM index |
Obtains the number of auto-complete texts in the specified index. |
TFT.GETSUG | TFT.GETSUG index prefix [MAX_COUNT count] [FUZZY] |
Obtains the auto-complete texts that can be matched based on the specified prefix. Texts are returned in descending order of weights. |
TFT.GETALLSUGS | TFT.GETALLSUGS index |
Obtains the full auto-complete texts in the specified index. |
Uppercase keyword
: the command keyword.Italic
: Words in italic indicate variable information that you supply.[options]
: optional parameters. Parameters that are not included in brackets are required.AB
: specifies that these parameters are mutually exclusive. Select one of two or more parameters....
: specifies to repeat the preceding content.
TFT.CREATEINDEX
Item | Description |
---|---|
Syntax | TFT.CREATEINDEX index mappings |
Command description |
Creates an index and a mapping for the index. The syntax used to create a mapping is similar to that used to create an explicit mapping in Elasticsearch. For more information, see Explicit mapping. You must create an index before you can add documents to the index. Note To prevent large keys from being generated, you can split large indexes into small
indexes and devise load distribution rules to write data to different indexes. When
you create these indexes, make sure that they have the mappings parameter set to the
same value.After these indexes are created, you can query them by using TFT.MSEARCH.
|
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.UPDATEINDEX
Item | Description |
---|---|
Syntax | TFT.UPDATEINDEX index mappings |
Command description |
Adds a properties field to the specified index. The field that you want to add cannot have the same name as an existing one. Otherwise, the field cannot be added. |
Parameter |
Note For more information about the mappings parameter, see the "TFT.CREATEINDEX" section of this topic.
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.GETINDEX
Item | Description |
---|---|
Syntax | TFT.GETINDEX index |
Command description |
Obtains the mapping content of an index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.ADDDOC
Item | Description |
---|---|
Syntax | TFT.ADDDOC index document [WITH_ID doc_id] |
Command description |
Adds a document to an index. You can specify a unique ID for the document in the index by using WITH_ID doc_id. If the document ID already exists, the existing ID is overwritten. If you do not specify WITH_ID doc_id, a document ID is automatically generated. By default, WITH_ID doc_id is not specified. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.MADDDOC
Item | Description |
---|---|
Syntax | TFT.MADDDOC index document doc_id [document1 doc_id1] ... |
Command description |
Adds multiple documents to an index. Each document must have a document ID that is specified by doc_id. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.UPDATEDOCFIELD
Item | Description |
---|---|
Syntax | TFT.UPDATEDOCFIELD index doc_id document |
Command description |
Updates the document specified by doc_id in an index. If the document fields that
you want to update are mapped and indexed fields, the fields must be of the same type
as those used by the mapping. If the fields are not indexed fields, the fields to
be updated can be of a random type.
Note If the fields already exist, the document is updated. If the fields do not exist,
the fields are added. If the document does not exist, the document is automatically
created. In this case, the command is equivalent to TFT.ADDDOC.
|
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.DELDOCFIELD
Item | Description |
---|---|
Syntax | TFT.DELDOCFIELD index doc_id field [field1 field2 ...] |
Command description |
Deletes the specified field from the document specified by doc_id in an index. If
the field is an indexed field, the information of the field is also deleted from the
index.
Note If the field does not exist, the operation cannot be performed. For example, the field
may not exist because it is filtered out by using _source.
|
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.INCRLONGDOCFIELD
Item | Description |
---|---|
Syntax | TFT.INCRLONGDOCFIELD index doc_id field increment |
Command description |
Adds an increment to the specified field in the document specified by doc_id in an
index. The increment can be a positive or negative integer. The data type of the field
can only be LONG or INTEGER.
Note If the document does not exist, the document is automatically created. In this case,
the existing value of the field is 0, and the updated field value is obtained by adding
the increment value to the existing field value. If the field does not exist, the
operation cannot be performed. For example, the field may not exist because it is
filtered out by using _source.
|
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.INCRFLOATDOCFIELD
Item | Description |
---|---|
Syntax | TFT.INCRFLOATDOCFIELD index doc_id field increment |
Command description |
Adds an increment to the specified field in the document specified by doc_id in an
index. The increment can be a positive or negative floating-point number. The data
type of the field can be INTEGER, LONG, or DOUBLE.
Note If the document does not exist, the document is automatically created. In this case,
the existing value of the field is 0, and the updated field value is obtained by adding
the increment value to the existing field value. If the field does not exist, the
operation cannot be performed. For example, the field may not exist because it is
filtered out by using _source.
|
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.GETDOC
Item | Description |
---|---|
Syntax | TFT.GETDOC index doc_id |
Command description |
Obtains the content of the document specified by doc_id in an index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.EXISTS
Item | Description |
---|---|
Syntax | TFT.EXISTS index doc_id |
Command description |
Checks whether the document specified by doc_id exists in an index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.DOCNUM
Item | Description |
---|---|
Syntax | TFT.DOCNUM index |
Command description |
Obtains the number of documents in an index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.SCANDOCID
Item | Description |
---|---|
Syntax | TFT.SCANDOCID index cursor [MATCH *value*] [COUNT count] |
Command description |
Obtains the IDs of all documents in an index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.DELDOC
Item | Description |
---|---|
Syntax | TFT.DELDOC index doc_id [doc_id] ... |
Command description |
Deletes the document specified by doc_id from an index. Multiple document IDs can be specified. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.DELALL
Item | Description |
---|---|
Syntax | TFT.DELALL index |
Command description |
Deletes all documents from an index but retains the index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.SEARCH
Item | Description |
---|---|
Syntax | TFT.SEARCH index query |
Command description |
Queries the documents in an index. The query syntax is similar to the Query Domain Specific Language (DSL) syntax used in Elasticsearch. For more information, see Query DSL. |
Parameter |
index: the name of the index that you want to manage. query: the Query DSL statement that is similar to the syntax used in Elasticsearch. The following fields are supported:
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.MSEARCH
Item | Description |
---|---|
Syntax | TFT.MSEARCH index_count index [index1] ... query |
Command description |
Queries documents in multiple indexes that have mappings set to the same value by using the query clause and gathers the results from these indexes. Then, the results are rated, sorted, aggregated, and returned. Note The output of the TFT.MSEARCH command is a result of rating, sorting, and aggregating
query results from these indexes. The output is different from results generated by
directly rating, sorting, and aggregating datasets in multiple indexes. TFT.MSEARCH
policy:
|
Parameter |
Note Unlike the query statement of the TFT.SEARCH command, the query statement of the TFT.MSEARCH command does not support the from parameter, but supports paged query by using the size, reply_with_keys_cursor, and keys_cursor parameters. For more information about the syntax of other parameters, see the "TFT.SEARCH" section of this topic.
|
Output |
|
Example |
Run the following commands in advance:
Sample command:
Sample output:
Sample command for querying the second page:
Sample output:
|
TFT.ADDSUG
Item | Description |
---|---|
Syntax | TFT.ADDSUG index text weight [text weight] ... |
Command description |
Adds one or more auto-complete texts and their weights to the specified index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.DELSUG
Item | Description |
---|---|
Syntax | TFT.DELSUG index text [text] ... |
Command description |
Deletes one or more auto-complete texts from the specified index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.SUGNUM
Item | Description |
---|---|
Syntax | TFT.SUGNUM index |
Command description |
Obtains the number of auto-complete texts in the specified index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.GETSUG
Item | Description |
---|---|
Syntax | TFT.GETSUG index prefix [MAX_COUNT count] [FUZZY] |
Command description |
Obtains the auto-complete texts that can be matched based on the specified prefix. Texts are returned in descending order of weights. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
TFT.GETALLSUGS
Item | Description |
---|---|
Syntax | TFT.GETALLSUGS index |
Command description |
Obtains the full auto-complete texts in the specified index. |
Parameter |
|
Output |
|
Example |
Sample command:
Sample output:
|
Aggregations
You can add the aggs (aggregations) parameter to TFT.SEARCH commands to aggregate results obtained by using query clauses.
Usage
TFT.SEARCH shares '{"query":{"term":{"investor":"Jay"}},"aggs":{"Jay_Sum":{"sum":{"field":"purchase_price"}}}}'
# Specify Jay_Sum as the aggregation name, sum as the aggregation type, and purchase_price as the aggregation field.
Query and aggregation results are returned. Example:"{"hits":{"hits":[{"_id":"16581351808123930","_index":"today_shares0718","_score":1.0,"_source":{"shares_name":"XAX","logictime":14300210,"purchase_type":1,"purchase_price":101.1,"purchase_count":100,"investor":"Jay"}},{"_id":"16581351809626430","_index":"today_shares0718","_score":1.0,"_source":{"shares_name":"XAX","logictime":14300310,"purchase_type":1,"purchase_price":111.1,"purchase_count":100,"investor":"Jay"}}],"max_score":1.0,"total":{"relation":"eq","value":2}},"aggregations":{"Jay_Sum":{"value":212.2}}}"
"size":0
to the query syntax so that only aggregation results are returned.
Aggregation types supported by aggs
Item | Description |
---|---|
Metric aggregation | Computes metrics based on values extracted from the fields of documents that are being
aggregated. These fields are typically of a numeric type, such as INTEGER or DOUBLE.
Nested aggregations are not supported. Valid values:
Note All parameters except for value_count only support numeric fields.
|
Terms aggregation | Calculates the deduplicated number of values. Only fields of the keyword type are supported. Nested aggregations are supported. Valid values:
Example:
Output: a JSON object obtained by using the key aggregation. In the object, the key
aggregation uses buckets to display statistics. Each bucket contains a key (aggregation
field) and a doc_count (number of documents associated with the aggregation field).
The following code provides an example:
|
Filter aggregation | Filters query results of a query statement. Nested aggregations are supported.
Output: the number of documents (doc_count) that match the filter conditions. |