Dynamically modify a search index schema - Tablestore SDK for Go - Tablestore

Use the Tablestore SDK for Go to create a reindex index from a source search index and modify the schema without interrupting table reads or writes.

Prerequisites

Install the Tablestore Go SDK and initialize a client.

Description

Dynamic schema modification creates a reindex index, synchronizes data, validates the new index with traffic, and switches indexes. Table reads and writes continue during the process, and query code that uses the source index name does not change. The workflow is as follows:

Create a reindex index that contains the complete new schema based on the source index.
Wait until the reindex index enters incremental synchronization and catches up with the source index.
Allocate query traffic as needed and validate results from the new schema.
Switch indexes after the results meet expectations.
Observe the index for a period of time and delete the reindex index that points to the old schema after you confirm normal operation.

Important
The reindex index name must end with _reindex. Index targets and traffic weights change during the workflow. Do not use the reindex index name as a long-term query endpoint. You can rebuild up to five indexes at the same time. Before starting the next batch, wait until the current batch catches up with the source indexes or completes index switching.

1. Create a reindex index

sourceIndexName := "example_index"
reindexName := "example_index_reindex"

newSchema := []*tablestore.FieldSchema{
    {
        FieldName: proto.String("category"),
        FieldType: tablestore.FieldType_KEYWORD,
        Index:     proto.Bool(true),
    },
    {
        FieldName: proto.String("new_field"),
        FieldType: tablestore.FieldType_KEYWORD,
        Index:     proto.Bool(true),
    },
}

_, err := client.CreateSearchIndex(&tablestore.CreateSearchIndexRequest{
    TableName:       "example_table",
    IndexName:       reindexName,
    SourceIndexName: &sourceIndexName,
    IndexSchema: &tablestore.IndexSchema{
        FieldSchemas: newSchema,
    },
})
if err != nil {
    log.Fatal(err)
}

After the request succeeds, use Query search index information to check the schema and synchronization status. A SyncPhase value of INCR indicates incremental synchronization. Before allocating traffic, also confirm that synchronization has caught up with the source index.

2. Configure query traffic

weights := []*tablestore.QueryFlowWeight{
    {IndexName: "example_index", Weight: 50},
    {IndexName: "example_index_reindex", Weight: 50},
}

_, err := client.UpdateSearchIndex(&tablestore.UpdateSearchIndexRequest{
    TableName:        "example_table",
    IndexName:        "example_index",
    QueryFlowWeights: weights,
})
if err != nil {
    log.Fatal(err)
}

Start with a small amount of query traffic to validate the new schema, and then gradually increase the reindex index weight. The Weight values of the two indexes must add up to 100.

3. Switch indexes

reindexName := "example_index_reindex"
_, err := client.UpdateSearchIndex(&tablestore.UpdateSearchIndexRequest{
    TableName:       "example_table",
    IndexName:       "example_index",
    SwitchIndexName: &reindexName,
})
if err != nil {
    log.Fatal(err)
}

After the switch, the source index name points to the new schema, and the reindex index name points to the old schema. Observe the new index for a period of time and delete the reindex index only after you confirm normal operation.

Parameters

Create the reindex index

Name	Type	Description
TableName (required)	string	The name of the data table.
IndexName (required)	string	The reindex index name, which must end with _reindex.
SourceIndexName (required)	*string	The source index name.
IndexSchema (required)	*tablestore.IndexSchema	The complete updated index schema. For field, routing, and presorting parameters, see Create a search index.
TimeToLive (optional)	*int32	The TTL of the reindex index. For valid values and constraints, see Configure the TTL of a search index.

Configure traffic and switch indexes

Name	Type	Description
TableName (required)	string	The name of the data table.
IndexName (required)	string	The source index name.
QueryFlowWeights (optional)	[]*tablestore.QueryFlowWeight	The traffic weights of the source and reindex indexes.
SwitchIndexName (optional)	*string	The reindex index whose target is switched with the source index.