Tablestore:Vector search

Prerequisites

• A Tablestore client is initialized.

• A search index with a vector field is created on the data table.

Usage notes

• Vector search requires Tablestore SDK for Java 5.17.0 or later. Make sure the correct SDK version is installed.

  Note

  For the version history of Tablestore SDK for Java, see Java SDK version history.

• Limits apply to the number of vector field types, dimensions, and other properties. For more information, see Search index limits.

• The search index is partitioned on the server side. Each partition returns its own TopK nearest neighbors, and the results are then aggregated at the client node. Therefore, if a token is used to paginate through all data, the total number of rows returned depends on the number of server-side partitions.

Parameters

Example

The following example retrieves the 10 nearest neighbors to a given vector, with a minimum similarity score of 0.1. The results are further filtered to rows where Col_Keyword is 'hangzhou' and Col_Long is less than 4.

private static void knnVectorQuery(SyncClient client) {
    SearchQuery searchQuery = new SearchQuery();
    KnnVectorQuery query = new KnnVectorQuery();
    query.setFieldName("Col_Vector");
    query.setTopK(10); // Return the top K nearest neighbors.
    query.setMinScore(0.1f); // Return only rows with a score greater than 0.1.
    query.setFloat32QueryVector(new float[]{0.1f, 0.2f, 0.3f, 0.4f});
    // The nearest neighbors must also satisfy the conditions: Col_Keyword = 'hangzhou' AND Col_Long < 4.
    query.setFilter(QueryBuilders.bool()
            .must(QueryBuilders.term("Col_Keyword", "hangzhou"))
            .must(QueryBuilders.range("Col_Long").lessThan(4))
    );
    searchQuery.setQuery(query);
    searchQuery.setLimit(10);
    // Sort the results by score.
    searchQuery.setSort(new Sort(Collections.singletonList(new ScoreSort())));
    SearchRequest searchRequest = new SearchRequest("<TABLE_NAME>", "<SEARCH_INDEX_NAME>", searchQuery);
    SearchRequest.ColumnsToGet columnsToGet = new SearchRequest.ColumnsToGet();
    columnsToGet.setColumns(Arrays.asList("Col_Keyword", "Col_Long"));
    searchRequest.setColumnsToGet(columnsToGet);
    // Call the search operation.
    SearchResponse resp = client.search(searchRequest);
    for (SearchHit hit : resp.getSearchHits()) {
        // Print the score.
        System.out.println(hit.getScore());
        // Print the row data.
        System.out.println(hit.getRow());
    }
}

FAQ

Optimize vector search performance in Tablestore

Related topics

• When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, geo query, KNN vector query, Boolean query, nested query, and exists query. After you create a search index, you can use the query methods provided by the search index to query data from multiple dimensions based on your business requirements.

  You can sort or paginate rows that meet the query conditions by using the sorting and paging features. For more information, see Perform sorting and paging.

  You can use the collapse (distinct) feature to collapse the result set based on a specific column. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).

• If you want to analyze data in a table, you can call the Search operation to use the aggregation feature or use the SQL query feature. For example, you can query the maximum and minimum values, the sum of the values, and the number of rows. For more information, see Aggregation and SQL query.

• If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.