OpenSearch:Table-based Q&A

Overview

OpenSearch LLM-Based Conversational Search Edition provides table-based conversational search for precise Q&A over structured data. Define a simple table schema, ingest structured data, and get accurate answers from tabular content. The feature also supports data synchronization from sources like MaxCompute, reducing development costs.

Method 1: Create a table in the console

1. Create a table

By default, a secondary table supports only table-based conversational search. To have a secondary table participate in Text Q&A, turn on the Text Q&A option and configure field tags for the table, as described in the following steps.

1. In the console, navigate to Data Configuration > Change Configurations. Keep the default data processing settings and click Next to go to the Data Structure step. Click Add Table and select Manually Create.

2. To enable Text Q&A for certain fields, turn on the Text Q&A option and assign the appropriate field tags.

   For example, to create a table named company, you can define seven fields: company_id (INT, primary key), company_name (TEXT), company_type (TEXT), company_address (TEXT), company_size (INT), establishment_date (INT), and contact_info (INT). After defining the fields, click Next.

   Note

   • A table name must contain only letters, digits, and underscores, and be no more than 20 characters long.

   • Field Name: Must start with a letter, consist of letters, numbers, and underscores, and cannot exceed 30 characters.

   • Field Description: Field descriptions are critical to Q&A performance. We recommend that you provide a description for each field. Otherwise, the Q&A performance will be affected. You can adjust the descriptions at any time based on the Q&A performance. The description for each field cannot exceed 30 characters.

   • Ensure that primary key values are unique. The primary key supports only the INT and TEXT types.

   • Field Type: Configure this setting based on the data in the table to be uploaded. A type mismatch will cause the table upload to fail. For fields configured as Content, set the field type to TEXT.

   • Field Tag: This parameter is required only when the Text Q&A feature is enabled. In this case, you must configure the Content and Primary Key field tags. The field that is tagged as Content is synchronized to the main table. Currently, you can tag only one field as Content.

   • Text Q&A: When enabled, the system first performs a table-based conversational search on custom secondary tables during a Q&A test. If no answer is found, Text Q&A is performed on fields that are set to Content to increase the answer rate.

   • If secondary table content is still not retrieved after you turn on Text Q&A, check the following. Ensure that the field tags are set to Content and Primary Key, the data structure update progress has reached 100%, and the configuration changes have been saved.

3. You can then add data by uploading a file in the console or by syncing from a configured data source.

2. Upload data from the console

Use the Import File feature to upload test data. Before you do, refer to the Sample Data.

File import supports JSON and Excel formats. Files must be UTF-8 encoded and cannot exceed 128 MB. Download a template by clicking the sample data link in the import panel.

3. (Optional) Sync data from a data source

1. Select the MaxCompute data source, click Connect to Database, enter the project name, AccessKey ID, and AccessKey Secret as prompted, and then click Connect.

   Note

   • Project name: The name of your MaxCompute project.

   • AccessKey ID: The AccessKey ID for the account that owns the MaxCompute project.

   • AccessKey Secret: The AccessKey Secret for the account that owns the MaxCompute project.

     You can use an existing AccessKey pair or create a new one by following the instructions in Create an AccessKey pair. To reduce the risk of disclosure, the AccessKey Secret is shown only once when you create it and cannot be retrieved later. Store it securely.

2. In the Add Data Source dialog box, on the Connect Data tab, select MaxCompute as the data source type. Then, select the target database and move the table you want to use for Q&A to the Selected list. Click OK.

3. From the drop-down lists, map the fields from your MaxCompute source table to the fields in the OpenSearch table. After mapping is complete, click OK.

4. Specify partition filters. If you leave this field blank, the system pulls data from all partitions by default. Click Finish and wait for the new version to build.

Method 2: Create a table from a data source

1. In the console, navigate to Data Configuration > Change Configurations. Keep the default data processing settings and click Next to go to the Data Structure step. Click Add Table and select From Data Source.

2. Select the MaxCompute data source, click Connect to Database, enter the project name, AccessKey ID, and AccessKey Secret as prompted, and then click Connect.

3. Select the data tables for the Q&A and click Confirm.

4. To enable Text Q&A for certain fields, turn on the Text Q&A option and assign the appropriate field tags.

   Note

   • A table name must contain only letters, digits, and underscores, and be no more than 20 characters long.

   • Field Name: Must start with a letter, consist of letters, digits, and underscores, and cannot exceed 30 characters.

   • Field Description: Field descriptions are strongly related to the performance of the Q&A feature. We recommend that you provide a description for each field. Otherwise, the Q&A performance will be affected. You can adjust the descriptions at any time based on the Q&A performance. The description for each field cannot exceed 30 characters.

   • Ensure that primary key values are unique. The primary key supports only the INT and TEXT types.

   • Field Type: Set the field type based on the data in the table that you want to upload. A type mismatch will cause the table upload to fail. For a field that is configured as Content, the field type must be set to TEXT.

   • Field Tag: You can leave this parameter empty if the Text Q&A feature is disabled. If you enable the Text Q&A feature, you must configure the Content and Primary Key field tags. The field that is tagged as Content is synchronized to the main table. Currently, you can tag only one field as Content. For other tag settings, see Data Configuration.

   • Text Q&A: When this feature is enabled, the system first performs a table-based conversational search on custom secondary tables during a Q&A test. If no answer is found, the system then performs a Text Q&A on fields that are set to Content to increase the answer rate.

5. Specify partition filters. If you leave this field blank, the system pulls data from all partitions by default. Click Finish and wait for the build to complete.

Feature test

After the data is uploaded, test the feature on the Q&A test page. You can view the total document count under Data Query.

Limitations

• You can add a maximum of 5 secondary tables.

• Each secondary table supports data writes and queries for a maximum of 30 fields.