Background information
The data that you upload to OpenSearch is first stored in offline data tables. To facilitate data upload, OpenSearch allows you to create data tables that have multiple fields based on your business requirements and provides data processing plug-ins. If you create multiple data tables, you must specify the fields that are used to join the tables. After the data in multiple data tables is processed, the tables are joined to form an index table. The index table defines search attributes and can be used by the search engine to build indexes and search for data.
Fields in a data table
You can use data tables to import data to OpenSearch. Each data processing plug-in can process only specific types of fields. For more information about the limits on field values, see the "Limits on fields" section of the Limits topic. If a field value is not in the specified value range, an overflow error occurs or the value is truncated. You must select correct field types.
Field type | Description |
INT | 64-bit integer. |
INT_ARRAY | 64-bit integer array. |
FLOAT | Floating-point number. |
FLOAT_ARRAY | Floating-point number array. |
DOUBLE | Floating-point number. |
DOUBLE_ARRAY | Floating-point number array. |
LITERAL | String constant that supports only exact match. |
LITERAL_ARRAY | String constant array. A single element in a string constant array supports only exact match. |
SHORT_TEXT | Short text. A value of this type cannot exceed 100 bytes in length. This type of field supports multiple analysis methods. |
TEXT | Long text. This type of field supports multiple analysis methods. |
TIMESTAMP | 64-bit unsigned integer. This type of field indicates a timestamp. |
GEO_POINT | String constant. This type of field indicates a pair of latitude and longitude in the "Latitude value Longitude value" format. |
Notes on reserved fields:
The following field names are reserved and cannot be used as the names of your own fields: service_id, ops_app_name, inter_timestamp, index_name, pk, ops_version, ha_reserved_timestamp, and summary.
Notes on fields of array types:
After you create a field of an array type in an application, you can map the field to a field of the VARCHAR or STRING type in a data source. In addition, you can use a data processing plug-in to process the field. For more information, see Use data processing plug-ins.
If you use the OpenSearch API or OpenSearch SDKs to upload a field of an array type, upload the field as an array instead of a string. Example: String[] literal_array = {"Alibaba Cloud","OpenSearch"};
Notes on fields of timestamp types:
Fields of the INT and TIMESTAMP types can be mapped to a field of the DATETIME or TIMESTAMP type in a data source. The field values are automatically converted to the number of milliseconds. You can use the range function to retrieve search results by time range. For more information, see Range search.
Supported field types in a data source
Data source | Supported field type |
ApsaraDB RDS | TINYINT, SMALLINT, INTEGER, BIGINT, FLOAT, REAL, DOUBLE, NUMERIC, DECIMAL, TIME, DATE, TIMESTAMP, and VARCHAR |
PolarDB | TINYINT, SMALLINT, INTEGER, BIGINT, FLOAT, REAL, DOUBLE, NUMERIC, DECIMAL, TIME, DATE, TIMESTAMP, and VARCHAR |
MaxCompute | BIGINT, DOUBLE, BOOLEAN, DATETIME, STRING, DECIMAL, MAP, ARRAY, TINYINT, SMALLINT, INT, FLOAT, CHAR, VARCHAR, DATE, TIMESTAMP, BINARY, INTERVAL_DAY_TIME, INTERVAL_YEAR_MONTH, and STRUCT |
Mappings between field types in OpenSearch and field types in a data source
Field type in OpenSearch | Field type in ApsaraDB RDS | Field type in PolarDB | Field type in MaxCompute |
INT | BIGINT, TINYINT, SMALLINT, and INTEGER | BIGINT, TINYINT, SMALLINT, and INTEGER | BIGINT, TINYINT, SMALLINT, and INT |
INT_ARRAY | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. |
FLOAT | FLOAT, NUMERIC, and DECIMAL | FLOAT, NUMERIC, and DECIMAL | FLOAT and DECIMAL |
FLOAT_ARRAY | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. |
DOUBLE | DOUBLE, NUMERIC, and DECIMAL | DOUBLE, NUMERIC, and DECIMAL | DOUBLE and DECIMAL |
DOUBLE_ARRAY | String types such as VARCHAR. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. |
LITERAL | String types such as VARCHAR. | String types such as VARCHAR. | String types such as VARCHAR and STRING. |
LITERAL_ARRAY | String types such as VARCHAR. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR. This field type must be converted by using the data processing plug-in MultiValueSpliter. | String types such as VARCHAR and STRING. This field type must be converted by using the data processing plug-in MultiValueSpliter. |
SHORT_TEXT | String types such as VARCHAR. | String types such as VARCHAR. | String types such as VARCHAR and STRING. |
TEXT | String types such as VARCHAR. | String types such as VARCHAR. | String types such as VARCHAR and STRING. |
TIMESTAMP | DATETIME and TIMESTAMP | DATETIME and TIMESTAMP | DATETIME and TIMESTAMP |
GEO_POINT | String types such as VARCHAR. | String types such as VARCHAR. | Strings such as VARCHAR and STRING in the format of lon lat. lon specifies the longitude, and lat specifies the latitude. Both the longitude and latitude values must be of the DOUBLE type and are separated by a space character. Valid values of the lon parameter: [-180, 180]. Valid values of the lat parameter: [-90,90]. |
Usage notes
If fields in a data source are of the FLOAT or DOUBLE type, we recommend that you change the field types to DECIMAL. Otherwise, data precision may be compromised.
Create an application schema
OpenSearch allows you to create an application schema by using one of the following methods:
Configure a data source. For more information, see Configure an ApsaraDB RDS for MySQL data source, Configure a MaxCompute data source, and Configure a PolarDB for MySQL data source.
Manually create an application schema.
Use a template. For more information, see Migrate an application.
Upload a file. For more information, see Create an application schema by uploading a file.