Java SDK search index data types - Tablestore - Alibaba Cloud Documentation Center

Before you use a search index, you must understand the data types that it supports and the mapping between these types and the data types in your data table.

Data type descriptions

Search index provides primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, IP, Geopoint, Vector, and JSON. It also provides special types, such as array types, nested types, and virtual columns.

Important

The search index Date type is supported in Tablestore Java software development kit (SDK) 5.13.9 and later. To use the search index Date type, make sure that you have the correct Java SDK version. For more information about the release history of the Java SDK, see Java SDK release history.

Primitive data types

Search index provides primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, IP, Geopoint, Vector, and JSON. The following table describes these data types.

Primitive data type	Description
Long	A 64-bit long integer.
Double	A 64-bit double-precision floating-point number.
Boolean	A Boolean value.
Keyword	A non-tokenized string.
FuzzyKeyword	A string that supports high-performance fuzzy queries.
Text	A tokenized string or text.
Date	A date and time type that supports custom date formats.
IP	An IP data type that supports IP addresses in IPv4 and IPv6 formats.
Geopoint	Coordinate information for a location point. The format is `latitude,longitude`. The latitude comes first, followed by the longitude. The latitude must be in the range of `[-90,+90]`, and the longitude must be in the range of `[-180,+180]`. Example: `35.8,-45.91`.
Vector	A vector type. The format is a string of a Float32 array. The length of the array is equal to the dimension of the field. For example, the vector string `[1, 5.1, 4.7, 0.08 ]` has four dimensions.
JSON type	A JSON type that supports the OBJECT and NESTED index types.

Array and nested types

In addition to primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, Geopoint, and Vector, search index also provides two special types: array and nested. The array type is suitable for storing a series of data of the same type. The nested type is suitable for storing data with a hierarchical structure, similar to JSON. For more information, see Array and nested types.

Array type

Important

The Array data type can be used only in search indexes. Data tables do not support the Array data type.
You can use the Array data type in the same manner you use a non-Array data type. When you use an Array field to query data, a row of data is returned if at least one of the values in the array meets the condition.
The vector data type cannot be used in arrays.

The Array data type is a composite data type and can be combined with primitive data types, such as Long, Double, Boolean, Keyword, Text, Date, and Geopoint, to construct complex data structures. For example, the combination of the Long data type and the Array data type is used to construct long arrays. A long array can contain multiple long integers. The Array data type is suitable for storing a collection of the same type of data.

If the data type of a field in a search index is a combination of the Array data type and a primitive data type, such as Long or Double, the field in the data table for which the search index is created must be of the String type and the field in the search index must be of the corresponding primitive data type.

The following table describes the combinations of the Array data type and primitive data types in search indexes.

Combination	Description
Long Array	An array of long integers. Example: `"[1000, 4, 5555]"`.
Double Array	An array of floating-point numbers. Example: `"[3.1415926, 0.99]"`.
Boolean Array	An array of Boolean values. Example: `"[true, false]"`.
Keyword Array	An array of strings. A keyword array is a JSON array. Example: `"[\"Hangzhou\", \"Xi'an\"]"`.
Text Array	An array of text. A text array is a JSON array. Example: `"[\"Hangzhou\", \"Xi'an\"]"`. Text arrays are not commonly used.
Date Array	An array of date data. Example of date data of the Integer type: `"[1218197720123, 1712850436000]"`. Example of date data of the String type: `"[\"2024-04-11 23:47:16.854775807\", \"2024-06-11 23:47:16.854775807\"]"`.
Geopoint Array	An array of latitude and longitude coordinate pairs. Example: `"[\"34.2, 43.0\", \"21.4, 45.2\"]"`.

Nested type

Data of the Nested type is nested documents. Nested documents are used when a row of data (document) contains multiple child rows (child documents). Multiple child rows are stored in a nested field. The Nested data type is suitable for storing data that has a hierarchical structure.

You must specify the schema of child rows in a nested field. The schema must include the fields of the child rows and the property of each field. The Nested data type can be used to store multiple values, which is similar to the JSON data type.

If a field in a search index is of the Nested type, the field in the data table for which the search index is created must be of the String type, and the field in the search index must be of the Nested type. You must perform nested queries to query fields of the Nested type.

When you write data to a field in a data table and the field corresponds to a Nested field in the search index created for the data table, make sure that the field in the data table is of the JSON Array type. Example: [{"tagName":"tag1", "score":0.8,"time": 1730690237000 }, {"tagName":"tag2", "score":0.2,"time": 1730691557000}].

Important

You must write strings of the JSON Array type to a nested field regardless of whether the field contains only one child row.

Nested fields are classified into single-level and multi-level nested fields. The following table describes the two types.

Type

Description

Single-level nested field

A single-level nested field has a simple data structure that contains only one level. Single-level nested fields are suitable for scenarios in which data structures of multiple levels are not required but hierarchical structures are required. Example:

[
    {
        "tagName": "tag1",
        "score": 0.8
    },
    {
        "tagName": "tag2",
        "score": 0.2
    }
]

Multi-level nested field

A multi-level nested field has a complex data structure that contains multiple levels. Multi-level nested fields are suitable for scenarios in which complex data structures are required to store organized data of various levels in centralized modules. Example:

[
    {
        name:"John",
        "age": 20,
        "phone": "1390000****",
        "address": [
            {
                "province": "Zhejiang",
                "city": "Hangzhou",
                "street": "1201 Xingfu Community, Sunshine Avenue"
            }
        ]
    }
]

Virtual columns

To query new fields with new data types without changing the storage structure and data in Tablestore, use virtual columns in a search index. For more information, see Virtual columns.

The virtual column feature lets you map a column from a table to a virtual column in a search index when you create the index. The type of the new virtual column can be different from the original column type in the table. This lets you create a new column without modifying the table schema and data. The new column can be used to accelerate queries or to use a different tokenizer.

You can configure different tokenization methods for Text fields that are mapped to the same field in a table.
A single String column can be mapped to multiple Text columns of a search index. Different Text columns use different tokenization methods to meet various business requirements.
Query acceleration
You do not need to cleanse data or re-create a table schema. You need to only map required columns of a table to the columns in a search index. The column types can be different between the table and the search index. For example, map the numeric type to the Keyword type to improve the performance of a term query, and map the String type to the numeric type to improve the performance of a range query.

Data type mapping

The value of a field in a search index is derived from the value of the corresponding field in the data table. The data types of these fields must match. The following table describes the mappings between field data types in search indexes and data tables.

Data type of the search index field	Data type of the data table field
Long	Integer
Long Array	String
Double	Double
Double Array	String
Boolean	Boolean
Boolean Array	String
Keyword	String
Keyword Array	String
Text	String
Date	Integer, String
Date Array	String
IP	String
IP Array	String
Geopoint	String
Geopoint Array	String
Vector	String
Nested	String
JSON	String