All Products
Search
Document Center

Lindorm:CREATE INDEX

Last Updated:Mar 28, 2024

Lindorm provides two types of efficient and easy-to-use indexes: secondary indexes and search indexes. These indexes are individually applicable to queries in which the primary key is not used for matching, multi-dimensional queries, and queries performed on columnar data. When you use the CREATE INDEX syntax to create an index, you can specify the index type and add attributes to the index.

Applicable engines

The CREATE INDEX syntax is applicable only to LindormTable.

Syntax

create_index_statement   ::=  CREATE INDEX [IF NOT EXISTS] [ index_identifier ]
                              [ USING index_method_definition ]
                              ON table_identifier '(' index_key_expression ')'
                              [ INCLUDE include_expression]
                              [ PARTITION BY partition_definition ]
                              [ { ASYNC | SYNC} ]
                              [ WITH '(' index_options  ')' ]
index_method_definition  ::=  { KV | SEARCH | COLUMNAR }
index_key_expression     ::=  '('
                                  index_key_definition 
                                  |wildcard_string_literal                                 
                              ')'

index_key_definition     ::= {
                                  column_identifier [ DESC ]
                                  | column_identifier'(' column_options ')'
                                  | function_expression
                             }
                             [ (',' index_key_definition)* ]
column_options    ::=  '(' 
                           option_definition (',' option_definition )* 
                       ')'
function_expression      ::=  function_identifier 
                              '(' 
                                  column_identifer ( ',' column_identifer )* 
                              ')'
option_definition        ::=  option_identifer '=' string_literal
include_expression       ::=  '(' 
                                  column_identifier( ',' column_identifier )*
                              ')'
partition_definition     ::= {
                                {RANGE TIME} 
                                     '(' 
                                          column_identifer 
                                     ')' [ PARTITIONS number_literal ]
                                |
                                HASH '(' 
                                          column_identifer 
                                          ( ',' column_identifer )*
                                     ')' [ PARTITIONS number_literal ]
                                |
                                ENUMERABLE 
                                     '(' 
                                          column_identifer 
                                          ( ',' column_identifer )*
index_options            ::=  '(' 
                                  option_definition (',' option_definition )* 
                              ')'

Supported parameters

LindormTable allows you to use the CREATE INDEX syntax to create two types of indexes: secondary indexes and search indexes.

The following table describes the parameters that are supported when you create each type of indexes.

Parameter

Secondary index

Search index

Index type (index_method_definition)

Index key expression (index_key_expression)

Included column (include_expression)

✖️

Index partitioning (partition_definition)

✖️

Index creation method (ASYNC|SYNC)

Important

Only LindormTable 2.6.3 and later versions support the SYNC creation method.

Index attributes (index_options)

Parameters

You can create up to three secondary indexes, one search index for a wide table.

Index type (index_method_definition)

You can specify the type of an index by using the USING keyword when you create the index. The following table describes the values that you can configure for this parameter when you create the three types of indexes supported by LindormTable.

Parameter value

Index type

Description

KV

Secondary index

By default, if you do not explicitly specify an index type in the CREATE INDEX statement, a secondary index is created. For more information about secondary indexes, see Secondary indexes.

Important

Up to eight secondary indexes can be created at the same time in an instance. If eight secondary indexes are being created when you try to create a secondary index, the index fails to be created.

SEARCH

Search index

Search indexes are used for full-text queries based on LindormSearch. Search indexes are suitable for complex multi-dimensional query scenarios, such as word segmentation, fuzzy queries, aggregate analysis, and sorting and paging. For more information, see Overview.

Search indexes support all basic data types except for DATE, TIME, and TIMESTAMP. For more information about the supported data types, see Basic data types.

Important
  • Before you create a search index, you must enable search indexes for the instance. After you enable search indexes, you are charged fees for LindormSearch nodes and LTS nodes. For more information, see Enable the search index feature.

  • The key of a search index must contain a column that is not contained in the primary key.

  • The number of search indexes that you can create in a Lindorm instance is not limited.

Index key expression (index_key_expression)

You can define one column or multiple columns as an index key. An index that consists of multiple index keys is also referred as a federated index.

Index key definition (index_key_definition)

You can use this parameter to add attributes for the index key of a search index. You can also use this parameter to specify an index key as a function expression.

Index key attributes for search indexes (option_definition)

You can specify attributes for index keys when you use the ALTER INDEX syntax to add index columns. For example, you can specify c3(type=text,analyzer=ik) in the ALTER INDEX statement to create an index for the c3 column and specify that the IK analyzer is used for the c3 column.

The following table describes the index key attributes that are supported by search indexes.

Attribute

Data type

Description

indexed

STRING

Specify whether to create an inverted index for the specified column in the index key.

  • true (default): creates an inverted index.

  • false: does not create an inverted index.

rowStored

STRING

Specifies whether to store raw data. Valid values:

  • true: stores raw data.

  • false (default): does not store raw data.

columnStored

STRING

Specifies whether to use the columnar structure to accelerate data sorting and analysis. Valid values:

  • true (default): uses the columnar structure.

  • false: does not use the columnar structure.

type

STRING

The data type. When analyzers are used, set this attribute to text for the related fields. In other scenarios, set the value of this attribute to the data type in your wide table by default.

Important

The type attribute must be configured together with the analyzer attribute.

analyzer

STRING

The analyzer that you want to use. Valid values:

  • standard

  • english

  • ik

  • whitespace

  • comma

Important

The analyzer attribute must be configured together with the type attribute.

Secondary index function expression (function_expression)

  • When you create a secondary index, you can specify the index key as a function expression. The following two functions are supported: Z-ORDER and CAST.

    • Z-ORDER: creates spatio-temporal secondary indexes for one or more spatio-temporal columns. Syntax:

      Z-ORDER '(' column_identifier ( ',' column_identifer )*  ')'

      The column specified by column_identifer must be a spatio-temporal column. For more information about spatio-temporal indexes, see Spatio-temporal indexes.

    • CAST: creates indexes for a column whose data type is converted. Syntax:

      CAST(column_identifier AS type)

      In the preceding syntax, type specifies the data type to which the data in the column is converted. For more information, see Basic data types.

Wildcard constant (wildcard_string_literal)

Only search indexes support the wildcard constant (*).

The wildcard constant (*) can be used to create an index for all columns including columns that are added to the table after the index is created. Example: CREATE INDEX IF NOT EXISTS idx5 USING SEARCH ON test(*);

Important

Dynamic columns are not contained in indexes. For more information, see Dynamic columns.

Redundancy column (include_expression)

You can use the INCLUDE keyword to configure included columns in secondary indexes and search indexes. In this case, the included columns cannot be dynamic columns in the base table.

Included columns are index columns that are the same as those in the base table. Indexes with Included columns are also referred as covering indexes. A covering index can provide all data required for a query that hits the index without the need to access the base table. This way, the query performance can be ensured.

Note

You can use the WITH keyword to specify dynamic columns as included columns in secondary indexes by adding table attributes. For more information, see Secondary indexes.

Index partitioning (partition_definition)

Only search indexes support index partitioning.

Index partitioning is an index management policy based on which the server automatically splits and stores data into different partitions and prunes partitions when the data is queried.

Search indexes support range partitioning and hash partitioning. For more information, see Partition index.

Index creation method (ASYNC|SYNC)

When you use the CREATE INDEX syntax to create an index, you can specify the ASYNC or SYNC keyword to specify whether to create the index synchronously or asynchronously.

  • ASYNC: creates the index asynchronously. If you use this method, the index starts to be created immediately after the CREATE INDEX statement is executed. The response to the CREATE INDEX statement is immediately returned regardless of whether the index is successfully created.

  • SYNC: creates the index synchronously. If you use this method, the index starts to be created immediately after the CREATE INDEX statement is executed. No results are returned for the CREATE INDEX statement until the index is successfully created.

The following table shows the creation methods supported by secondary indexes, search indexes, and columnar indexes.

Creation method

Secondary indexes

Search indexes

ASYNC

Important

In LindormTable 2.6.1 and later versions, the default index creation method for the CREATE INDEX syntax is ASYNC.

SYNC

Important

The SYNC method is supported only in LindormTable 2.6.3 and later versions.

Index attributes (index_options)

When you use the CREATE INDEX syntax to create an index, you can use the WITH keyword to specify attributes for the index. The following tables describe the attributes supported by different types of indexes.

Secondary index

Attribute

Type

Description

COMPRESSION

STRING

The compression algorithm you want to use for the index table. Valid values:

  • SNAPPY

  • ZSTD

  • LZ4

INDEX_COVERED_TYPE

STRING

The included columns in the index. Valid values:

  • COVERED_ALL_COLUMNS_IN_SCHEMA: All columns in the base table are included in the index.

  • COVERED_DYNAMIC_COLUMNS: Dynamic columns are included in the index.

    Note

    Before you include dynamic columns in the index, make sure that dynamic columns are enabled. For more information, see Dynamic columns.

You can use the INCLUDE keyword to include specified columns in the index.

STARTKEY

STRING

The start key of the index.

Important

Timestamp columns and spatio-temporal columns cannot be set to the start key of the index.

ENDKEY

STRING

The end key of the index.

Important

Timestamp columns and spatio-temporal columns cannot be set to the end key of the index.

NUMREGIONS

INTEGER

The estimated number of partitions of the index table.

Important

This attribute is not supported by Timestamp columns and spatio-temporal columns.

Search index

Attribute

Type

Description

indexState

STRING

The status of the search index. Valid values:

  • ACTIVE

  • INACTIVE

  • DISABLED

numShards

INTEGER

The number of shards. The default value is equal to twice the number of search nodes.

RANGE_TIME_PARTITION_START

INTEGER

The number of days before you create the index. The first partition is created based on the point in time n days before the point in time when you create the index. The number of days is the value of n. This attribute can be specified in scenarios in which your table contains historical data. If the timestamp of a row of historical data is earlier than the point in time based on which the first partition is created, an error message is returned.

Note

This attribute is required if you create a partitioned index.

RANGE_TIME_PARTITION_INTERVAL

INTEGER

The interval at which a new partition is created. For example, If you set RANGE_TIME_PARTITION_INTERVAL to '7', a partition is created at an interval of seven days.

Note

This attribute is required if you create a partitioned index.

RANGE_TIME_PARTITION_TTL

INTEGER

The retention period of your data in a partition. For example, if you set RANGE_TIME_PARTITION_TTL to '180', the retention period is 180 days and the data that is retained for more than 180 days in a partition is automatically removed. If you do not specify this attribute, partitions are not automatically deleted.

Note

This attribute is required if you create a partitioned index.

RANGE_TIME_PARTITION_MAX_OVERLAP

INTEGER

If the timestamp of the data that you write indicates a time in the future, this option specifies the maximum time period between the point in time when data is written and the point in time when you create the index. Unit: days. If you do not specify this attribute, the timestamp of the data that you want to write is not limited.

Note

This attribute is required if you create a partitioned index.

RANGE_TIME_PARTITION_FIELD_TIMEUNIT

LONG

The unit of values in the partition key column. Default value: ms. The value ms means milliseconds.

  • If you set the value of this attribute to s, each value in the partition key column is 10 digits in length. The value s means seconds.

  • If you set the value of this attribute to ms, each value in the partition key column is 13 digits in length.

Note

This attribute is required if you create a partitioned index.

Examples

In the following examples, a base table named test is created by executing the following statement:

CREATE TABLE test (
  p1 VARCHAR NOT NULL,
  p2 INTEGER NOT NULL,
  c1 BIGINT,
  c2 DOUBLE,
  c3 VARCHAR,
  c4 TIMESTAMP,
  c5 GEOMETRY(POINT),
  PRIMARY KEY(p1, p2)
) WITH (CONSISTENCY = 'strong', MUTABILITY='MUTABLE_LATEST');

Secondary indexes

Create a secondary index asynchronously

By default, if you do not specify the creation method, the index is created asynchronously.

CREATE INDEX idx1 ON test(c1 desc) include(c3,c4) WITH (COMPRESSION='ZSTD');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Create a federated index

Create a federated index for the c1, c2, and c3 columns.

CREATE INDEX idx1 ON test(c1, c2, c3) include(c4) SYNC WITH ( COMPRESSION ='ZSTD');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Include all columns in the index

CREATE INDEX idx1 ON test(c4 desc) WITH (INDEX_COVERED_TYPE ='COVERED_ALL_COLUMNS_IN_SCHEMA');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Include all dynamic columns

CREATE INDEX idx1 ON test(c4 desc) WITH (INDEX_COVERED_TYPE='COVERED_DYNAMIC_COLUMNS');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Configure the estimated number of partitions of the index table

Set the estimated number of partitions of the index table to 32.

CREATE INDEX idx1 ON test(c4 desc) include(c5,c6)  WITH (NUMREGIONS ='32');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Specify the start and end keys of the index table

Specify that the index table is split into 32 partition in the range from the start key 11111111 to the end key9999999.

CREATE INDEX idx1 ON test(c3 desc) include(c5,c6) WITH (NUMREGIONS ='32', STARTKEY ='11111111', ENDKEY = '9999999');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Create a spatio-temporal secondary index

Create a spatio-temporal secondary index for the c5 column of the spatial data type. ​

CREATE INDEX idx1 ON test(Z-ORDER(c5));

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Convert the data type of the specified column

Create a secondary index for the c3 column after converting its data type to INTEGER. ​

CREATE INDEX idx1 ON test(CAST(c3 AS INTEGER));

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Search indexes

Create a search index asynchronously

CREATE INDEX IF NOT EXISTS idx2 USING SEARCH ON test(p1, p2, c1, c2, c3);

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Create a search index on all columns

Create a search index for all columns without specifying column attributes. Note: Dynamic columns are used to create the index. For more information, see Use dynamic columns in search indexes.

CREATE INDEX IF NOT EXISTS idx2 USING SEARCH ON test('*');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Add attributes for index keys

Create a search index for all columns. If the c3 column is included in the index, set the following attributes for all index columns: type=text, analyzer=ik, indexed=true.

CREATE INDEX IF NOT EXISTS idx2 USING SEARCH ON test('*',c3(type=text,analyzer=ik,indexed=true));

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Configure the index status

Create a search index on the specified columns, specify index column attributes, and set the status of the search index to ACTIVE.

CREATE INDEX IF NOT EXISTS idx2 USING SEARCH ON test(c1, c3(type=text,analyzer=ik)) WITH (indexState=ACTIVE,numShards=4);

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.

Configure time-based partitioning

Perform partitioning based on the time column c4. Partitioning is automatically performed every seven days from 30 days ago. By default, partitions are retained for 90 days.

CREATE INDEX IF NOT EXISTS idx2 USING SEARCH ON test (c1, c2, c3, c4)
PARTITION BY RANGE TIME(c4) PARTITIONS 16
WITH (indexState=ACTIVE, RANGE_TIME_PARTITION_START='30', RANGE_TIME_PARTITION_INTERVAL='7', RANGE_TIME_PARTITION_TTL='90', RANGE_TIME_PARTITION_MAX_OVERLAP='90');

Verify the result

You can execute the SHOW INDEX FROM test; statement to check whether the index is created.