OpenSearch:Implement table-based conversational search

Features

As the content of conversational searches becomes more diverse, the documents that are used for conversational searches are not limited to regular text or images and the requests for accurate conversational searches based on structured data continuously increase. To provide accurate answers to questions asked by users and improve the conversational search performance in complex structured data scenarios, OpenSearch LLM-Based Conversational Search Edition provides the table-based conversational search feature. You can use this feature to accurately query structured data in tables after you configure simple table schemas. This feature allows you to synchronize structured data from data sources such as MaxCompute in a convenient and easy way. This reduces development costs.

Manually create a secondary table in the console

Create a secondary table

1. On the Instance Management page, find the instance that you want to manage and click Manage in the Actions column to go to the Instance Details page. In the left-side pane, choose Configuration Center > Data Configuration. On the page that appears, click Change Configurations. In the Data Processing Configuration step, use the default parameter settings and click Next to go to the Data Structure step. In the Data Structure step, click Add Table and select Manually Create.

   

2. If you want to implement text-based conversational search based on specific fields, you can turn on Text Q&A and add tags to the fields based on which you want to implement text-based conversational search.

   

   Note

   • A table name can contain letters, digits, and underscores (_). The name must be 1 to 20 characters in length.

   • Field Name: A field name must start with a letter and can contain letters, digits, and underscores (_). The field name can be up to 30 characters in length.

   • Field Description: A field description is strongly related to the effects of conversational search. We recommend that you specify a description for each field. Otherwise, the effects of conversational search may be affected. You can modify the description at any time based on the effects of conversational search. The description of each field can be up to 30 characters in length.

   • Make sure that each primary key value is unique. The primary key ID can be of the INT or TEXT type.

   • Type: Specify the field type based on the field type in the table whose data is to be uploaded. If the field types are inconsistent, the table data fails to be uploaded. A field with the Content tag must be of the TEXT type.

   • Field Tag: If you do not turn on Text Q&A, you do not need to specify this parameter. If you turn on Text Q&A, you must select Content or Primary Key from the drop-down list in the Field Tag column. A field with the Content tag is synchronized to the main table. You can add the Content tag to only one field in the custom secondary table.

   • Text Q&A: If you turn on Text Q&A, a table-based conversational search is performed on the custom secondary table during the conversational search test. If no answer is found, a table-based conversational search is performed on the field to which the Content tag is added. This increases the response rate.

3. Click Completed. Wait until the secondary table is created. You can upload data by using the OpenSearch console. You can also click Add Data Source to add a data source to synchronize data.

Upload data by using the OpenSearch console

Click Import File. Refer to the sample data before you import data by importing files.

(Optional) Add a data source to synchronize data

1. In the Add Data Source panel, click MaxCompute. Then, click Connect to Database. In the Connect to Database dialog box, configure the Project Name, AccessKey ID, and AccessKey Secret parameters as prompted, and click Connect.

   

   

   Note

   • Project Name: the name of the MaxCompute project.

   • AccessKey ID: the AccessKey ID of the account to which the MaxCompute project belongs.

   • AccessKey Secret: the AccessKey secret of the account to which the MaxCompute project belongs.

   • For more information about how to view the AccessKey pair of an account, see View the information about AccessKey pairs of a RAM user.

2. Select the table that you want to use for conversational searches and click OK.

   

3. Map fields in the MaxCompute data table to the fields of the secondary table in OpenSearch. After the fields are mapped, click OK.

   

4. Specify the conditions based on which partition data is pulled and click Completed. Wait until the data is synchronized from the data source. If you do not specify the conditions based on which partition data is pulled, data is pulled from all partitions.

   

Create a secondary table by using a data source

1. On the Instance Management page, find the instance that you want to manage and click Manage in the Actions column to go to the Instance Details page. In the left-side pane, choose Configuration Center > Data Configuration. On the page that appears, click Change Configurations. In the Data Processing Configuration step, use the default parameter settings and click Next to go to the Data Structure step. In the Data Structure step, click Add Table and select Use Data Source.

2. In the Select Data Source panel, click MaxCompute. Then, click Connect to Database. In the Connect to Database dialog box, configure the Project Name, AccessKey ID, and AccessKey Secret parameters as prompted, and click Connect.

   

3. Select the table that you want to use for conversational searches and click OK.

   

4. If you want to implement text-based conversational search based on specific fields, you can turn on Text Q&A and add tags to the fields based on which you want to implement text-based conversational search.

   

   Note

   • A table name can contain letters, digits, and underscores (_). The name must be 1 to 20 characters in length.

   • Field Name: A field name must start with a letter and can contain letters, digits, and underscores (_). The field name can be up to 30 characters in length.

   • Field Description: A field description is strongly related to the effects of conversational search. We recommend that you specify a description for each field. Otherwise, the effects of conversational search may be affected. You can modify the description at any time based on the effects of conversational search. The description of each field can be up to 30 characters in length.

   • Make sure that each primary key value is unique. The primary key ID can be of the INT or TEXT type.

   • Type: Specify the field type based on the field type in the table whose data is to be uploaded. If the field types are inconsistent, the table data fails to be uploaded. A field with the Content tag must be of the TEXT type.

   • Field Tag: If you do not turn on Text Q&A, you do not need to specify this parameter. If you turn on Text Q&A, you must select Content or Primary Key from the drop-down list in the Field Tag column. A field with the Content tag is synchronized to the main table. You can add the Content tag to only one field in the custom secondary table. For more information about other tag settings, see Import data.

   • Text Q&A: If you turn on Text Q&A, a table-based conversational search is performed on the custom secondary table during the conversational search test. If no answer is found, a table-based conversational search is performed on the field to which the Content tag is added. This increases the response rate.

5. Specify the conditions based on which partition data is pulled and click Completed. Wait until the MaxCompute data source is added and the secondary table is created. If you do not specify the conditions based on which partition data is pulled, data is pulled from all partitions.

Perform a test

After data is uploaded, you can perform a conversational search test on the Q&A Test page. You can view the total number of documents used for conversational searches in the Data Query section of the Data Configuration page.

Usage notes

• An instance can contain up to five secondary tables.

• You can write data to and query up to 30 fields in a secondary table.