If you use open source clients to use search indexes, you must synchronize data from wide tables to index tables. To synchronize data, create column mappings between wide tables and index tables. This topic describes the parameters of column mappings.

Configuration example

Column mappings between a wide table and an index table are stored in JSON files. In the following example, data is synchronized from the wide table testTable to the index table democollection. The f:name column in the wide table is mapped to the f_name_s column in the index table. The f:age column in the wide table is mapped to the f_age_i column in the index table.
{
  "sourceNamespace": "default",
  "sourceTable": "testTable",
  "targetIndexName": "democollection",
  "indexType": "SOLR",
  "rowkeyFormatterType": "STRING",
  "fields": [
    {
      "source": "f:name",
      "targetField": "name_s",
      "type": "STRING"
    },
     {
      "source": "f:age",
      "targetField": "age_i",
      "type": "INT"
    }
  ]
}

Parameters

The following table describes the parameters of column mappings between the wide table and the index table.

Parameter Description
sourceNamespace The namespace name of the wide table. If the wide table does not have a namespace, the default namespace name is default.
sourceTable The name of the wide table. This parameter is required.
targetIndexName The index table name of the search engine LindormSearch. This parameter is required.
indexType The default value is SOLR.
rowkeyFormatterType Specifies the method that you want to use to map the primary key of the wide table to the ID of the primary key of the index table. The data type of the primary key of the index table is STRING. The following two methods are supported:
  • STRING: This method is suitable for scenarios in which the primary key of the wide table is of the STRING type. For example, the primary key is row1,order0001. This method uses the Bytes.toString(byte[]) function to convert the primary key of the wide table to the ID of the primary key of the index table. After you query the ID of the primary key of the index table from the index based on the specified conditions, you can use the Bytes.toBytes(String) function to convert the ID of the primary key of the index table to the primary key of the wide table.
  • HEX: This method is suitable for scenarios in which the primary key of the wide table is not of the STRING type. For example, the primary key is 12345. This method uses the encodeAsString(byte[]) function in the org.apache.commons.codec.binary.Hex package to convert the primary key of the wide table to the ID of the primary key of the index table. After you query the ID of the primary key of the index table from the index based on the specified conditions, you can use the Hex.decodeHex(String.toCharArray()) function to convert the ID of the primary key of the index table to the primary key of the wide table.
fields Specifies one or more columns that you want to map and the types of the columns. The parameter value consists of the following three parameters. Separate multiple columns with commas (,).
  • source: the name of the column that you want to map in the wide table. Separate the column family name and the column name with a colon (:).
  • targetField: the name of the column to which you want to map in the index table. We recommend that you use the dynamic column feature provided by LindormSearch. LindormSearch automatically identifies the data type of the dynamic column by using the suffix of the dynamic column name. For example, the dynamic column name_s indicates that the data type of the dynamic column in the index table is STRING. For information about the formats of dynamic columns, see targetField parameter.
  • type: the data type of the column that you want to map in the wide table. The parameter value is case-sensitive. The following data types are supported:
    • INT
    • LONG
    • STRING
    • BOOLEAN
    • FLOAT
    • DOUBLE
    Note Make sure that the data type of the wide table is the same as the data type that is specified. If the data type of the wide table is not the same as the data type that is specified, the index data fails to be generated. In LindormTable, you must call the Bytes.toBytes(string/long/int/...) method to convert the corresponding data type to a byte array and store the byte array in the wide table.

targetField parameter

The targetField parameter specifies the column in the index table to which you want to map the column in the wide table. We recommend that you use the dynamic column feature provided by LindormSearch. LindormSearch automatically identifies the data type of the dynamic column by using the suffix of the dynamic column name. The format of the dynamic column name is Column name:Data type of the column. In the following example, the column name is name. The following table describes the data types that are supported by dynamic columns.
Data type Example value of the targetField parameter
INT name_i
LONG name_l
STRING name_s
STRING (data type of Chinese character strings that are tokenized) name_ik
BOOLEAN name_b
FLOAT name_f
DOUBLE name_d