Tablestore:Vector search

Prerequisites

• Initialize a Tablestore client.

• Create a search index for the data table and specify a vector field.

Usage notes

• Install the latest version of Tablestore SDK for Go. For more information, see Install Tablestore SDK for Go.

• Limits apply to the number of vector field types, dimensions, and other properties. For more information, see Search index limits.

• The search index is partitioned on the server side. Each partition returns its own TopK nearest neighbors, and the results are then aggregated at the client node. Therefore, if a token is used to paginate through all data, the total number of rows returned depends on the number of server-side partitions.

Parameters

Examples

The following example retrieves the 10 nearest neighbors of a given vector, filtering for results with a score greater than 0.1 and a col_keyword value of "hangzhou".

func query(client *tablestore.TableStoreClient) {
    searchQuery := search.NewSearchQuery()
    searchQuery.Query = &search.KnnVectorQuery{
        FieldName:          "col_vector",
        TopK:               proto.Int32(10),
        MinScore:           proto.Float32(0.1),
        Float32QueryVector: []float32{-1.4, 1, 1, 1.2},
        Filter: &search.BoolQuery{
            ShouldQueries: []search.Query{
                &search.TermQuery{
                    FieldName: "col_keyword",
                    Term:      "hangzhou",
                },
            },
        },
    }
    searchQuery.Sort = &search.Sort{
        Sorters: []search.Sorter{
            search.NewScoreSort(), // Sort by score.
        },
    }
    searchRequest := &tablestore.SearchRequest{
        SearchQuery: searchQuery,
        TableName:   "<TABLE_NAME>",
        IndexName:   "<SEARCH_INDEX_NAME>",
        ColumnsToGet: &tablestore.ColumnsToGet{Columns: []string{
            "col_keyword",
            "col_long",
        }},
    }

    if resp, err := client.Search(searchRequest); err != nil {
        fmt.Println("float32 vector query failed: ", err)
    } else {
        for _, hit := range resp.SearchHits {
            fmt.Println("score:", *hit.Score)
            jsonBody, err := json.Marshal(hit.Row)
            if err != nil {
                panic(err)
            }
            fmt.Println("row: ", string(jsonBody))
        }  
    }
}

FAQ

How to optimize AISearch performance

References

• When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, geo query, Boolean query, KNN vector query, nested query, and exists query. You can use the query methods provided by the search index to query data from multiple dimensions based on your business requirements.

  You can sort or paginate rows that meet the query conditions by using the sorting and paging features. For more information, see Sorting and paging.

  You can use the collapse (distinct) feature to collapse the result set based on a specific column. This way, data of the specified type appears only once in the query results. For more information, see Collapse (deduplicate).

• If you want to analyze data in a data table, you can use the aggregation feature of the Search operation or execute SQL statements. For example, you can obtain the minimum and maximum values, sum, and total number of rows. For more information, see Aggregation and SQL query.

• If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Perform a parallel scan.