Create a search index with the Tablestore Go SDK - Tablestore

Prerequisites

A Tablestore client is initialized. For more information, see Initialize the Tablestore client.
A data table is created and meets the following requirements. For more information, see Create a data table.
- The maximum number of versions is set to 1.
- The time to live (TTL) is set to -1, or updates are disabled for the data table.

Parameters

To create a search index, specify the table name (TableName), search index name (IndexName), and index schema (IndexSchema). The IndexSchema parameter includes FieldSchemas (field settings), IndexSetting (index settings), and IndexSort (pre-sorting settings). The following table describes these parameters.

Parameter	Description
TableName	The name of the data table.
IndexName	The name of the search index.
FieldSchemas	A list of FieldSchema objects. Each FieldSchema object contains the following parameters: FieldName (Required): The name of the field in the search index. The value is a column name. Type: String. A field in the search index can be a primary key column or an attribute column. FieldType (Required): The data type of the field. Type: tablestore.FieldType_XXX. Array (Optional): Specifies whether the field is an array. Type: Boolean. If you set this parameter to true, the column stores array data. Data must be written in JSON array format, such as ["a","b","c"]. Nested fields are inherently arrays. You do not need to set this parameter when FieldType is Nested. Index (Optional): Specifies whether to enable indexing for the field. Type: Boolean. Default value: true. An inverted index or a spatial index is created for the column. If you set this parameter to false, no index is created for the column. Analyzer (Optional): The tokenizer type. Set this parameter when the field type is Text. Default value: single-word tokenization. EnableSortAndAgg (Optional): Specifies whether to enable sorting and aggregation. Type: Boolean. Only fields with EnableSortAndAgg set to true support sorting. Important Fields of the Nested type do not support sorting and aggregation. However, sub-columns within a Nested type field support these features. DateFormats (Optional): The date format. Type: String. Set this parameter when the field type is Date. For more information, see Date and time types. EnableHighlighting (Optional): Specifies whether to enable the summary and highlighting feature. Type: Boolean. Default value: false. Set this parameter to true to enable the summary and highlighting feature. Only Text fields support this feature. VectorOptions (Optional): The property parameters for a vector field. Set this parameter when the field type is Vector. VectorOptions includes the following parameters: DataType: The data type of the vector. Currently, only float32 is supported. To use other types, submit a ticket to contact us. Dimension: The number of dimensions for the vector. Maximum value: 4096. MetricType: The distance measure algorithm for vectors. Valid values: Euclidean distance (euclidean), cosine similarity (cosine), and dot product (dot_product). Euclidean distance (euclidean): Measures the straight-line distance between two vectors in a multidimensional space. For performance reasons, the Euclidean distance algorithm in Tablestore does not perform the final square root calculation. A larger Euclidean distance score indicates higher similarity between the two vectors. Cosine similarity (cosine): Measures the cosine of the angle between two vectors. A higher cosine similarity score indicates higher similarity between the two vectors. This algorithm is commonly used for text data. Dot product (dot_product): Calculates the sum of products of corresponding coordinates of two vectors with the same dimension. A higher dot product score indicates higher similarity between the two vectors. For more information about how to select a distance measure algorithm, see Distance measure algorithms. JsonType (Optional): The index type for JSON data. Valid values: OBJECT and NESTED. Set this parameter when the field type is JSON.
IndexSetting	Index settings. Includes the RoutingFields setting. RoutingFields (Optional): Custom routing fields. You can select some primary key columns as routing fields. Typically, you only need to set one. If you set multiple routing keys, the system concatenates their values into a single value. When index data is written, the system determines the data partition based on the values of the routing fields. Records with the same routing field values are stored in the same data partition.
IndexSort	Index pre-sorting settings. Includes the Sorters setting. Default: data is sorted by primary key. Note Search indexes that contain Nested type fields do not support IndexSort and have no pre-sorting. Sorters (Required): The pre-sorting method for the index. Supports sorting by primary key or by field value. For more information, see Sorting and pagination. PrimaryKeySort: Sorts by primary key. Includes the following setting: Order: The sort order. Valid values: ascending and descending. Default value: ascending. FieldSort: Sorts by field value. Includes the following settings: Pre-sorting requires the field to be indexed and have sorting and aggregation enabled. FieldName: The name of the field to sort by. Order: The sort order. Valid values: ascending and descending. Default value: ascending. Mode: The sort mode when a field contains multiple values.
TimeToLive	Optional. The time-to-live (TTL) of data in the search index. Unit: seconds. Default value: -1, which means data never expires. Minimum value: 86400 (one day). Set the value to -1 if you do not want data to expire. Tablestore automatically deletes expired data when the retention period exceeds the TTL. For more information about the search index lifecycle, see Lifecycle management.

Examples

Create a search index with default settings

The following example creates a search index with three columns: col_keyword (Keyword type), col_long (Long type), and col_vector (Vector type).

func createSearchIndex(client *tablestore.TableStoreClient) {
    request := &tablestore.CreateSearchIndexRequest{}
    request.TableName = "<TABLE_NAME>"
    request.IndexName = "<SEARCH_INDEX_NAME>"
    request.IndexSchema = &tablestore.IndexSchema{
        FieldSchemas: []*tablestore.FieldSchema{
            {
                FieldName:        proto.String("col_keyword"),
                FieldType:        tablestore.FieldType_KEYWORD, // String type
                Index:            proto.Bool(true),
                EnableSortAndAgg: proto.Bool(true),
            },
            {
                FieldName:        proto.String("col_long"),
                FieldType:        tablestore.FieldType_LONG, // Numeric type
                Index:            proto.Bool(true),
                EnableSortAndAgg: proto.Bool(true),
            },
            {
                FieldName: proto.String("col_vector"),
                FieldType: tablestore.FieldType_VECTOR, // Vector type
                Index:     proto.Bool(true),
                VectorOptions: &tablestore.VectorOptions{
                    VectorDataType:   tablestore.VectorDataType_FLOAT_32.Enum(),
                    Dimension:        proto.Int32(4), // The vector dimension is 4, and the similarity algorithm is dot product.
                    VectorMetricType: tablestore.VectorMetricType_DOT_PRODUCT.Enum(),
                },
            },
        },
    }
    _, err := client.CreateSearchIndex(request)
    if err != nil {
        fmt.Println("Failed to create searchIndex with error:", err)
        return
    }
}

Create a search index with IndexSort

The following example creates a search index with pre-sorting configured. The search index contains two columns: col1 (Keyword type) and col2 (Long type).

func createSearchIndex_withIndexSort(client *tablestore.TableStoreClient){
    request := &tablestore.CreateSearchIndexRequest{}
    request.TableName = "<TABLE_NAME>" // Set the table name.
    request.IndexName = "<SEARCH_INDEX_NAME>" // Set the search index name.

    schemas := []*tablestore.FieldSchema{}
    field1 := &tablestore.FieldSchema{
        FieldName: proto.String("col1"), // Set the field name. Use proto.String to get the string pointer.
        FieldType: tablestore.FieldType_KEYWORD, // Set the field type.
        Index:     proto.Bool(true), // Enable indexing.
        EnableSortAndAgg: proto.Bool(true), // Enable sorting and statistical aggregation.
    }
    field2 := &tablestore.FieldSchema{
        FieldName: proto.String("col2"),
        FieldType: tablestore.FieldType_LONG,
        Index:     proto.Bool(true),
        EnableSortAndAgg: proto.Bool(true),
    }

    schemas = append(schemas, field1, field2)
    request.IndexSchema = &tablestore.IndexSchema{
        FieldSchemas: schemas, // Set the fields to include in the search index.
        IndexSort: &search.Sort{ // Specify index pre-sorting. Sort by col2 in ascending order, then by col1 in descending order.
            Sorters: []search.Sorter{
                &search.FieldSort{
                    FieldName: "col2",
                    Order:     search.SortOrder_ASC.Enum(),
                },
                &search.FieldSort{
                    FieldName: "col1",
                    Order:     search.SortOrder_DESC.Enum(),
                },
            },
        },
    }
    resp, err := client.CreateSearchIndex(request) // Call the client to create the search index.
    if err != nil {
        fmt.Println("error :", err)
        return
    }
    fmt.Println("CreateSearchIndex finished, requestId:", resp.ResponseInfo.RequestId)
}

Create a search index with a TTL

Important

Ensure that updates are disabled for the data table.

func createIndexWithTTL(client *tablestore.TableStoreClient) {
    request := &tablestore.CreateSearchIndexRequest{}
    request.TableName = "<TABLE_NAME>"
    request.IndexName = "<SEARCH_INDEX_NAME>"
    schemas := []*tablestore.FieldSchema{}
    field1 := &tablestore.FieldSchema{
        FieldName:        proto.String("col1"),         // Set the field name. Use proto.String to get the string pointer.
        FieldType:        tablestore.FieldType_KEYWORD, // Set the field type.
        Index:            proto.Bool(true),             // Enable indexing.
        EnableSortAndAgg: proto.Bool(true),             // Enable sorting and statistical aggregation.
    }
    field2 := &tablestore.FieldSchema{
        FieldName:        proto.String("col2"),
        FieldType:        tablestore.FieldType_LONG,
        Index:            proto.Bool(true),
        EnableSortAndAgg: proto.Bool(true),
    }
    schemas = append(schemas, field1, field2)
    request.IndexSchema = &tablestore.IndexSchema{
        FieldSchemas: schemas, // Set the fields to include in the search index.
    }
    request.TimeToLive = proto.Int32(3600 * 24 * 7) // Set the TTL of the search index to 7 days.
    resp, err := client.CreateSearchIndex(request)
    if err != nil {
       fmt.Println("error :", err)
       return
   }
    fmt.Println("createIndexWithTTL finished, requestId:", resp.ResponseInfo.RequestId)
}

Create a search index with query highlighting

The following example creates a search index with four columns: col_keyword (Keyword type), col_long (Long type), col_text (Text type), and col_nested (Nested type). The col_nested column contains two sub-columns: level1_text (Text type) and level1_nested (Nested type). The level1_nested sub-column contains one sub-column: level2_text (Text type). Query highlighting is enabled for col_text, the level1_text sub-column of col_nested, and the level2_text sub-column of col_nested.level1_nested.

func createSearchIndexwithHighlighting(client *tablestore.TableStoreClient) {
    request := &tablestore.CreateSearchIndexRequest{}
    request.TableName = "<TABLE_NAME>"
    request.IndexName = "<SEARCH_INDEX_NAME>"
    request.IndexSchema = &tablestore.IndexSchema{
        FieldSchemas: []*tablestore.FieldSchema{
            {
                FieldName:        proto.String("col_keyword"),
                FieldType:        tablestore.FieldType_KEYWORD, // String type.
                Index:            proto.Bool(true),
                EnableSortAndAgg: proto.Bool(true),
            },
            {
                FieldName:        proto.String("col_long"),
                FieldType:        tablestore.FieldType_LONG, // Numeric type.
                Index:            proto.Bool(true),
                EnableSortAndAgg: proto.Bool(true),
            },
            {// Enable query highlighting for a non-nested type.
                FieldName: proto.String("col_text"),
                FieldType: tablestore.FieldType_TEXT, // Tokenizable string type.
                Index:     proto.Bool(true),
                EnableSortAndAgg: proto.Bool(true),
                EnableHighlighting: proto.Bool(true),
            },
            {// Enable query highlighting for sub-columns in a nested type field.
                FieldName: proto.String("col_nested"),
                FieldType: tablestore.FieldType_NESTED,
                FieldSchemas: []*tablestore.FieldSchema{
                    {
                        FieldName:          proto.String("level1_text"),
                        FieldType:          tablestore.FieldType_TEXT,
                        Index:              proto.Bool(true),
                        EnableHighlighting: proto.Bool(true),
                    },
                    {
                        FieldName: proto.String("level1_nested"),
                        FieldType: tablestore.FieldType_NESTED,
                        FieldSchemas: []*tablestore.FieldSchema{
                            {
                                FieldName:          proto.String("level2_text"),
                                FieldType:          tablestore.FieldType_TEXT,
                                Index:              proto.Bool(true),
                                EnableHighlighting: proto.Bool(true),
                            },
                        },
                    },
                },
            },
        },
    }
    _, err := client.CreateSearchIndex(request)
    if err != nil {
        fmt.Println("Failed to create searchIndex with error:", err)
        return
    }
}

FAQ

References

After you create a search index, use various query types for multi-dimensional data queries, including term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, geo query, boolean query, AISearch, nested query, and column existence query.
When you query data, you can apply sort and paginate, highlight, or collapse (deduplicate) to the result set.
After you create a search index, manage it by dynamically modifying the schema, managing the lifecycle, listing search indexes, querying search index descriptions, and deleting a search index.
For data analytics such as finding the maximum or minimum value, calculating a sum, or counting rows, use the aggregation feature or the SQL query feature.
To quickly export data without requiring a specific order for the result set, use the parallel scan feature.

Tablestore:Create a search index