Tair (Redis® OSS-Compatible):Vector global index

Category

Description

Syntax

TVS.CREATEINDEXGLOBAL global_index dims algorithm distance_method [algo_param_key algo_param_value] ...

Time complexity

O(1)

Command description

Creates a global index and specifies the algorithm and distance function for index building and querying.

A global index can be deleted only with the TVS.DELINDEXGLOBAL command.

Options

• global_index: The name of the global index.

• dims: The vector dimension, an integer from 1 to 32,768. All vectors in the index must have this dimension.

• algorithm: The algorithm used to build and query the index. Valid values:

  • FLAT: Performs a brute-force search without building a separate index. This algorithm is suitable for small datasets with fewer than 10,000 entries.

  • HNSW: Builds and queries the index using an HNSW graph structure. This algorithm is suitable for large datasets.

• distance_method: The distance function used to calculate vector distances. Valid values:

  • L2: The squared Euclidean distance.

  • IP: The inner product. The distance is calculated as 1 - inner product.

  • COSINE: The cosine distance. The distance is calculated as 1 - cosine similarity. Using COSINE normalizes inserted vectors into unit vectors (L2 normalization). As a result, vectors returned by a query may differ from the originals.

  • JACCARD: The Jaccard distance. The distance is calculated as 1 - Jaccard coefficient. This distance function requires the data_type to be set to BINARY.

• algo_param_key and algo_param_value:

  • data_type: The vector data type. Valid values:

    • FLOAT32 (default): A 4-byte single-precision floating-point number.

    • FLOAT16: A 2-byte half-precision floating-point number (IEEE 754-2008 standard). This data type reduces vector storage space at the cost of some precision. The value range for FLOAT16 is [-65519, 65519].

    • BINARY: A binary vector containing only 0s and 1s. This data type supports only the Jaccard distance function.

  • Parameters specific to HNSW indexes:

    • ef_construct: The length of the dynamic list used when building the index with the HNSW algorithm. The default value is 100, and the valid range is [1, 1000]. A larger value improves the accuracy of Approximate Nearest Neighbor (ANN) queries but increases performance overhead.

    • M: The maximum number of outgoing edges for each node on each layer of the graph structure. The default value is 16, and the valid range is [1, 100]. A larger value improves the accuracy of Approximate Nearest Neighbor (ANN) queries but increases performance overhead.

    • auto_gc: Specifies whether to automatically reclaim index space. Valid values are false (default, disabled) and true (enabled). This feature requires the minor version of the instance to be 6.2.8.2 or later. When enabled, this feature automatically reclaims space from vectors removed via marked deletion. This reduces memory usage but may impact the performance of nearest-neighbor queries. This feature is recommended for memory-sensitive scenarios.

  • Parameters specific to HybridIndex (hybrid index):

    • To create an inverted index for a specific tag field, specify the field name and its corresponding data type in advance.

      The syntax is inverted_index_<field_name> int|long|float|double|string. This creates an inverted index on the field_name field. Supported data types are int, long, float, double, and string. The field_name and the data type keyword must be in lowercase.

      For example, to create an inverted index on a productname field of the String type, use inverted_index_productname string.

    • lexical_algorithm: The full-text search algorithm.

      • bm25: The BM25 (Okapi BM25) algorithm, suitable for full-text search.

        • When you write data using TVS.HSET, if attribute_key is set to the TEXT keyword (uppercase required), a full-text index is automatically built for the corresponding attribute_value.

        • The Tair vector service builds this index from the provided raw text.

      • vector: The vector algorithm. You need to encode the original text (embedding) and pass in a sparse vector. The data format is "[[2,0.221],[42,09688],...]", where the key is Indices (type uint32_t), and the value is the term frequency overview for that index (type Float).

    • If lexical_algorithm is set to bm25, you can also configure the analyzer and the weights for k1 and b.

      • analyzer: The analyzer. Currently, jieba (default) and ik_smart are supported.

      • k1: Controls the term frequency saturation in the BM25 algorithm. The default value is 1.2, and the value must be greater than 0.

      • b: Controls the influence of document length in the BM25 algorithm. The default value is 0.75, and the valid range is [0, 1].

    • hybrid_ratio: The default weight for vector search during a query. The default value is 0.5, and the valid range is [0, 1]. The value must be a float. The weight for full-text search is 1-hybrid_ratio.

Return value

• Returns OK on success.

• Otherwise, an error message is returned.

Examples

Command examples:

# Creates an HNSW index with 2 dimensions, Jaccard distance, and the BINARY data type.
TVS.CREATEINDEXGLOBAL index_name0 2 HNSW JACCARD data_type BINARY

# Creates a vector index with a vector dimension of 2, the HNSW algorithm, the L2 distance function, and the FLOAT32 data type.
TVS.CREATEINDEXGLOBAL index_name1 2 HNSW L2 

# Creates a vector index with a vector dimension of 2, the FLAT algorithm, the IP distance function, and the FLOAT32 data type.
TVS.CREATEINDEXGLOBAL index_name2 2 FLAT IP 

# Creates a vector index with a vector dimension of 2, the FLAT algorithm, the Jaccard distance function, and the BINARY data type.
TVS.CREATEINDEXGLOBAL index_name3 2 FLAT JACCARD data_type BINARY

# Creates a hybrid HNSW index with 2 dimensions and IP distance, and adds a bm25 full-text search on the 'productname' string field.
TVS.CREATEINDEXGLOBAL index_name4 2 HNSW IP lexical_algorithm bm25 inverted_index_productname string

All examples return the following:

OK

Item

Description

Syntax

TVS.EXPANDINDEXGLOBAL global_index num

Time complexity

O(1)

Command description

Creates data indexes in a global index.

When you specify the number of new data indexes to create, Tair automatically creates and names them, distributing them evenly across shards. The metadata of a data index is identical to that of the global index. Data indexes are associated with the global index but are stored independently in Tair.

Parameter

• global_index: The name of the global index.

• num: The number of data indexes to create.

Return value

• Returns OK on success.

• Otherwise, returns an error message.

Example

Example command:

TVS.EXPANDINDEXGLOBAL index_name0 3

Example output:

OK

Item

Description

Syntax

TVS.IMPORTINDEXGLOBAL global_index sub_index

Time complexity

O(1)

Command description

Binds an existing data index to a global index. The metadata of the existing data index must match that of the global index.

Parameter

• global_index: The name of the global index.

• sub_index: The name of the data index.

Output

• If the command is successful, it returns OK.

• If the metadata of the data index does not match the global index, the command returns (error)ERR two indices are not suitable.

• Otherwise, the command returns an error message.

Example

Example command:

TVS.IMPORTINDEXGLOBAL index_name0 testvector

Example output:

OK

Category

Description

Syntax

TVS.LISTINDEXGLOBAL num [global_index] [global_index] ...

Time complexity

O(1)

Command description

Lists information about one or more global indexes.

Parameter

• num: The number of global indexes to list. Specify 0 to list all global indexes.

• global_index: The name of a global index.

Return value

• An array of strings, where each string is a JSON object that describes a global index.

• An error message if the command fails.

Example

Command:

TVS.LISTINDEXGLOBAL 2 index_name0 index_name4

Output:

1) "{\"dimension\":\"2\",\"algorithm\":\"HNSW\",\"distance_method\":\"JACCARD\",\"parameters\":[\"data_type\",\"BINARY\"],\"sub_index_list\":[\"index_name0_sub_1719912517_2\",\"index_name0_sub_1719912517_3\",\"index_name0_sub_1719912517_4\"]}"
2) "{\"dimension\":\"2\",\"algorithm\":\"HNSW\",\"distance_method\":\"IP\",\"parameters\":[\"lexical_algorithm\",\"bm25\",\"inverted_index_productname\",\"string\"],\"sub_index_list\":[\"index_name4_sub_1719986971_2\",\"index_name4_sub_1719986971_0\",\"index_name4_sub_1719986971_1\"]}"

Item

Description

Syntax

TVS.DELINDEXGLOBAL global_index [sub_index] ...

Time complexity

O(1)

Command description

Disassociates a global index from one or more data indexes. If you do not specify any data indexes, this command deletes the global index and disassociates all related data indexes.

Parameter

• global_index: The name of the global index.

• sub_index: The name of the data index.

Return value

• If the command succeeds, it returns an integer:

  • 0: The specified data indexes were disassociated from the global index.

  • 1: The global index was deleted.

• If the command fails, it returns an error message.

Example

Command example:

TVS.DELINDEXGLOBAL index_name0 index_name0_sub_1720000607_0 index_name0_sub_1720000607_1

Example output:

(integer) 0

Category

Description

Syntax

TVS.HSETGLOBAL global_index key attribute_key attribute_value [attribute_key attribute_value] ...

Time complexity

The time complexity is O(1) if no vector is created or updated. Otherwise, the time complexity is O(log(N)), where N is the number of keys in the vector index.

Command description

Inserts a data record (identified by a key) into a global index. Tair automatically writes the key to an appropriate data index to ensure data balancing.

If the record already exists, it is overwritten.

Parameters

• global_index: The name of the global index.

• key: The primary key of the record. The record can be deleted with the TVS.DELGLOBAL command.

• attribute_key and attribute_value: the vector that you want to insert in the key-value format.

  • To insert a vector, set attribute_key to VECTOR and attribute_value to a vector string that has the same dimension as that of the vector index specified by the dims parameter. Example: VECTOR [1,2]. You can write only one vector to a key. If you write a vector more than once, the existing data is overwritten.

  • To insert text data into a hybrid index that has required parameters specified, set attribute_key to TEXT and attribute_value to text data such as "TairVector is an in-house vector database service of Tair" or vectorized data such as "[[2,0.221],[42,09688],...]".

  • To insert other attributes, specify custom parameters related to these attributes. Examples: create_time 1663637425 and location Hangzhou.

Output

• On success, returns the number of added attributes. If only existing attributes are updated, it returns 0.

• Otherwise, the command returns an error message.

Example

Example command:

TVS.HSETGLOBAL my_index key0 VECTOR [5,6] creation_time 1750 productname "Aliyun"

Example output:

(integer) 3

Item

Description

Syntax

TVS.HGETALLGLOBAL global_index key

Time complexity

O(1) * M, where M is the number of data indexes.

Command description

Returns all attributes and values of the record for a specified key in a global index.

Parameter

• global_index: The name of the global index.

• key: The primary key of the record.

Output

• If successful, the command returns an array of attributes and values for the specified key.

• If the specified global index or key does not exist, the command returns an (empty array).

• Otherwise, the command returns an error message.

Example

Example command:

TVS.HGETALLGLOBAL my_index key0

Example output:

1) "VECTOR"
2) "[5,6]"
3) "productname"
4) "Aliyun"
5) "creation_time"
6) "1750"

Item

Description

Syntax

TVS.HMGETGLOBAL global_index key attribute_key [attribute_key ...]

Time complexity

O(1) * M, where M is the number of attribute keys.

Command description

Returns the values of one or more specified attribute_keys for a key in a global index.

Parameters

• global_index: The name of the global index.

• key: The primary key of the record.

• attribute_key: One or more attribute keys to retrieve. To retrieve vector data, specify the VECTOR keyword (must be uppercase). To retrieve raw text data from the full-text index, specify the TEXT keyword (must be uppercase).

Output

• A successful command returns an array of values for the specified attribute_keys, in the order requested.

• Otherwise, it returns an error message.

Example

Sample command:

TVS.HMGETGLOBAL index_name0 key0 create_time VECTOR

Sample output:

1) "1800"
2) "[7,8]"

Item

Description

Syntax

TVS.DELGLOBAL global_index key [key ...]

Time complexity

O(1)

Command description

Deletes the data record specified by the key from the full index.

Parameters

• global_index: The name of the global index.

• key: The primary key of the data record to delete. You can specify one or more keys.

Return value

• Successful execution: Deletes the specified data record (key), and returns the number of deleted keys.

• Returns 0 if the specified index does not exist.

• Returns an error message on other failures.

Example

Example command:

TVS.DELGLOBAL my_index key0 key1

Example output:

 (integer) 2

Category

Description

Syntax

TVS.HDELGLOBAL global_index key attribute_key [attribute_key ...]

Time complexity

O(1)

Command description

Deletes the specified attribute_key and its value from the data record (key) in the full index.

Parameters

• global_index: The name of the global index.

• key: The primary key of the record.

• attribute_key: The attribute key to delete. You can specify multiple attribute keys. To delete vector data, specify the VECTOR keyword (must be uppercase). To delete full-text data, specify the TEXT keyword (must be uppercase).

Output

• On success, returns the number of attribute_keys deleted.

• Returns 0 if the specified index does not exist.

• Otherwise, an error is returned.

Example

Example command:

TVS.HDELGLOBAL index_name0 keyc VECTOR

Example output:

 (integer) 1

Item

Description

Syntax

TVS.SCANGLOBAL global_index cursor [MATCH pattern] [COUNT count] [FILTER filter_string] [VECTOR vector] [MAX_DIST max_distance]

Time complexity

O(N), where N is the number of keys in the global index.

Command description

In the global index, scan for data records that meet the specified condition (key).

In a single scan, Tair returns results from only one data shard.

Parameter

• global_index: The name of the global index.

• cursor: The cursor for the current scan. The initial cursor is 0.

• pattern: A glob-style pattern to match against key names.

• count: Specifies a hint for the number of keys to return per scan. The default value is 10. The command does not guarantee that the exact number of keys specified by count is returned in each iteration.

• filter_string: The filter conditions.

  • Supported operators include +, -, *, /, <, >, !=, (), &&, and ||. String values must be enclosed in escaped double quotes (for example, create_time > 1663637425 && location == \"Hangzhou\"). Lexicographical string comparison is not supported.

  • Operators must be separated from operands by spaces, for example, "creation_time > 1735".

  • Boolean comparisons such as flag == true are not supported. Instead, pass the boolean value as a string, for example, flag == \"true\".

• vector: The query vector. This parameter must be used with the max_distance parameter.

• max_distance: The maximum distance threshold. This parameter must be used with the vector parameter. When both are specified, the command returns only results whose distance to the query vector is less than max_distance.

Return value

• On success, the command returns an array of two elements:

  • The first element is the cursor for the next scan. A returned cursor of 0 indicates that the scan is complete.

  • The second element: The name of the data record (key) for this query.

• Otherwise, an error message is returned.

Example

Command example:

TVS.SCANGLOBAL index_name0 0

Return example:

1) "72057594037927936"
2) 1) "key2"
   2) "key1"

Category

Description

Syntax

TVS.KNNSEARCHGLOBAL global_index topN vector [filter_string] [param_key param_value]

Time complexity

• HNSW algorithm: O(log(N))*M

• FLAT algorithm: O(N)*M

N is the number of keys in each data index, and M is the number of data indexes.

Command description

Performs a nearest neighbor query for a specified vector in a global index and returns up to topN results.

For example, if you want to query for the topN results, Tair first retrieves topN results from each data index. At this point, there can be a maximum of topN × number of data indexes results. Tair then aggregates and sorts the results, and finally returns the topN results.

Options

• global_index: The name of the global index.

• topN: The number of results to return. The value must be in the range of [1,10000].

• vector: The query vector. To perform only a full-text search on a hybrid index, you can pass an empty string ("").

• filter_string: The filter string.

  • Supports operators such as +-*/<>!=()&&||. String comparison is not supported. To use a string value in the filter, use an escape character (\). For example: "create_time > 1663637425 && location == \"Hangzhou\"".

  • Separate operators from their operands with spaces. For example: "creation_time > 1735".

  • Comparisons with boolean values, such as flag == true, are not supported. Instead, pass the value as a string, for example, flag == \"true\".

• param_key and param_value: The runtime parameters for the query.

  • ef_search: The length of the dynamic list during a query. The default value is 100, and the valid range is [1,1000]. A larger value increases query accuracy but also increases performance overhead. This parameter is specific to the HNSW algorithm.

  • MAX_DIST: The maximum distance threshold, specified as a float. If a key's distance from the query vector exceeds this value, it is not returned.

  • TEXT: The text for the query in a hybrid search. You can pass text or vector data. If you do not specify this parameter or pass an empty string (""), the command performs only a vector search and not a full-text search.

  • hybrid_ratio: The weight for vector search in the current query. The default value is the hybrid_ratio value set by TVS.CREATEINDEX. The value must be a float in the range of [0,1]. The weight for full-text search is 1-hybrid_ratio.

  • By default, the system uses a PostFilter policy, which performs K-nearest neighbors (KNN) vector search before scalar filtering.

    • vector_filter_count: The maximum number of records to filter during vector search. Default value: 10000.

      In the PostFilter policy, the system terminates the vector search traversal if the number of records filtered out exceeds vector_filter_count before enough results are found.

    • fulltext_filter_count: The maximum number of records to filter during a full-text search. Default value: 10000.

      In the PostFilter policy, the system terminates the full-text search traversal if the number of records filtered out exceeds fulltext_filter_count before enough results are found.

    • To use the PreFilter policy, which performs scalar filtering on an inverted index before the KNN vector search, add the search_policy scala parameter to your query.

      You can also provide the ivf_filter_count parameter to specify the maximum number of records to filter from the inverted index. The default value is 10000. In the PreFilter policy, if the number of records that pass the inverted index filter exceeds ivf_filter_count, the system automatically falls back to the PostFilter policy.

    Note

    In most cases, the default parameters ensure low latency while maintaining accuracy. You can adjust these parameters for specific requests, but be aware that larger values may increase query latency.

Return value

• If the command is successful, it returns an array of nearest neighbor keys and their distances to the query vector, ordered from nearest to farthest.

• If the specified vector index does not exist, it returns an (empty array).

• Otherwise, it returns an error message.

Example

TVS.CREATEINDEXGLOBAL my_global_index 2 HNSW L2 inverted_index_productname string
TVS.EXPANDINDEXGLOBAL my_global_index 3
TVS.HSETGLOBAL my_global_index key0 VECTOR [1,2] creation_time 1730 productname "Aliyun"
TVS.HSETGLOBAL my_global_index key1 VECTOR [3,4] creation_time 1740 productname "other"
TVS.HSETGLOBAL my_global_index key2 VECTOR [5,6] creation_time 1750 productname "Aliyun"

Sample command 1:

TVS.KNNSEARCHGLOBAL my_global_index 2 [3,3.1] "creation_time > 1735"

Sample output 1:

1) "key1"
2) "0.81000018119812012"
3) "key2"
4) "12.410000801086426"

Sample command 2:

TVS.KNNSEARCHGLOBAL my_global_index 2 [3,3.1]  "creation_time > 1735 && productname ==  \"Aliyun\"" search_policy scala ivf_filter_count 15000

Sample output 2:

1) "key2"
2) "12.410000801086426"

Item

Description

Syntax

TVS.KNNSEARCHFIELDGLOBAL global_index topN vector field_count field_name [field_name ...] [filter_string] [param_key param_value]

Time complexity

• HNSW algorithm: O(log(N))*M

• FLAT algorithm: O(N)*M

N is the number of keys in a data index, and M is the number of data indexes.

Command description

Searches for the nearest neighbors of a specified vector in a global index. The search logic is the same as for TVS.KNNSEARCHGLOBAL, but this command can also return tag attributes.

Options

• global_index: The name of the global index.

• topN: The number of results to return. The value must be an integer from 1 to 10,000.

• vector: The vector for the nearest neighbor query. To perform only a full-text search (for a hybrid index), pass an empty string ("") for this parameter.

• field_count: The number of tag attributes to return. To return all tag attributes, set field_count to 0.

• field_name: The name of a tag attribute. The number of field_name parameters must match the value of field_count.

• filter_string: The filter string.

  • Supports operators such as +-*/<>!=()&&||. String comparison is not supported. To use a string value in the filter, use an escape character (\). For example: "create_time > 1663637425 && location == \"Hangzhou\"".

  • Separate operators from their operands with spaces. For example: "creation_time > 1735".

  • Comparisons with boolean values, such as flag == true, are not supported. Instead, pass the value as a string, for example, flag == \"true\".

• param_key and param_value: The runtime parameters for the query.

  • ef_search: The length of the dynamic list during a query. The default value is 100, and the valid range is [1,1000]. A larger value increases query accuracy but also increases performance overhead. This parameter is specific to the HNSW algorithm.

  • MAX_DIST: The maximum distance threshold, specified as a float. If a key's distance from the query vector exceeds this value, it is not returned.

  • TEXT: The text for the query in a hybrid search. You can pass text or vector data. If you do not specify this parameter or pass an empty string (""), the command performs only a vector search and not a full-text search.

  • hybrid_ratio: The weight for vector search in the current query. The default value is the hybrid_ratio value set by TVS.CREATEINDEX. The value must be a float in the range of [0,1]. The weight for full-text search is 1-hybrid_ratio.

  • By default, the system uses a PostFilter policy, which performs K-nearest neighbors (KNN) vector search before scalar filtering.

    • vector_filter_count: The maximum number of records to filter during vector search. Default value: 10000.

      In the PostFilter policy, the system terminates the vector search traversal if the number of records filtered out exceeds vector_filter_count before enough results are found.

    • fulltext_filter_count: The maximum number of records to filter during a full-text search. Default value: 10000.

      In the PostFilter policy, the system terminates the full-text search traversal if the number of records filtered out exceeds fulltext_filter_count before enough results are found.

    • To use the PreFilter policy, which performs scalar filtering on an inverted index before the KNN vector search, add the search_policy scala parameter to your query.

      You can also provide the ivf_filter_count parameter to specify the maximum number of records to filter from the inverted index. The default value is 10000. In the PreFilter policy, if the number of records that pass the inverted index filter exceeds ivf_filter_count, the system automatically falls back to the PostFilter policy.

    Note

    In most cases, the default parameters ensure low latency while maintaining accuracy. You can adjust these parameters for specific requests, but be aware that larger values may increase query latency.

Return value

• If the command is successful, it returns the nearest neighbor keys, their distances to the query vector, and the corresponding tag attribute key-value pairs, sorted in ascending order by distance.

• If the specified global index does not exist, the command returns an (empty array).

• Otherwise, it returns an error message.

Example

TVS.CREATEINDEXGLOBAL my_global_index 2 HNSW L2 inverted_index_productname string
TVS.EXPANDINDEXGLOBAL my_global_index 3
TVS.HSETGLOBAL my_global_index key0 VECTOR [1,2] creation_time 1730 productname "Aliyun"
TVS.HSETGLOBAL my_global_index key1 VECTOR [3,4] creation_time 1740 productname "other"
TVS.HSETGLOBAL my_global_index key2 VECTOR [5,6] creation_time 1750 productname "Aliyun"

Example command:

TVS.KNNSEARCHFIELDGLOBAL my_global_index 2 [3,3.1] 1 VECTOR "creation_time > 1735"

Example output:

1) 1) "key1"
   2) "0.81000018119812012"
   3) "VECTOR"
   4) "[3,4]"
2) 1) "key2"
   2) "12.410000801086426"
   3) "VECTOR"
   4) "[5,6]"