All Products
Search
Document Center

Intervention dictionaries for NER

Last Updated: Sep 09, 2021

OpenSearch allows intervention in named entity recognition (NER) results and entity priorities. If NER results are not as expected, intervention can be implemented.

Concepts

Intervention in NER results

To intervene in NER results, you can create an NER intervention dictionary and associate the NER intervention dictionary with a query analysis rule. Semantic entities are closely related to an analyzer. Therefore, when you create an NER intervention dictionary, an analyzer must be used. An intervention dictionary can be associated with a query analysis rule only when the intervention dictionary uses the same analyzer as that used for indexes.

Usage notes:

  • You can enable full match or partial match for queries in an intervention dictionary.

  • The query to be added in an intervention entry must be normalized. Otherwise, the intervention entry can be added but cannot be used for intervention. The normalization process involves converting upper-case letters to lower-case letters and full-width characters to half-width characters.

  • When you associate an NER intervention dictionary with a query analysis rule, you can specify whether to ignore spaces in search queries.
  • In some cases, the analysis results of specific queries may vary. For example, after the analyzer is updated, the analysis results that are obtained when a search query is entered may be different from the analysis results that are obtained when an intervention entry is created. In such cases, the intervention entry is invalid. An error code 6612 is returned for the query. When this error code is returned, modify this intervention entry in the OpenSearch console.

Intervention in entity priorities

For ease of use, OpenSearch provides default priority configurations for entities. You can modify the default configurations as required. The modifications to priority configurations may affect the rules for rewriting queries.

Usage notes:

  • When you configure entity priorities, make sure that both high and medium priorities are specified. Otherwise, the NER-based query rewriting feature does not take effect.

  • Make sure that only one priority type is configured for each entity.

Intervention procedure

OpenSearch is embedded with dictionaries to intervene in NER. To intervene in NER, perform the following steps:

  1. Create an NER intervention dictionary. Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create in the upper-right corner. Specify a name for the intervention dictionary, select a dictionary type, and then click Save. After the intervention dictionary is created, it is displayed in the dictionary list.

  2. Add and manage intervention entries in the intervention dictionary. To go to the entry management page for the created dictionary, find the dictionary in the dictionary list and click Manage Entries in the Actions column. On the page that appears, add and manage intervention entries as needed.

  3. Use the intervention dictionary. After you add intervention entries to the intervention dictionary, you can use the intervention dictionary for query analysis on an application as needed.

  4. Test and publish the intervention dictionary. After the intervention dictionary is associated with a query analysis rule, perform a search test before you publish the rule. This ensures expected search performance.

Examples

Scenario: You are the operator of an e-commerce shopping guide service. You have configured query analysis rules with NER capabilities for your application instance in OpenSearch. However, error recognition results are returned when the rules are published. To resolve this issue, you decide to use NER intervention.

Error recognition example: A user enters the following search query: Chanel Foundation. By default, the following recognition results are returned: Chanel is classified as a regular term and Foundation is classified as a type of material. After the query is rewritten, the following query clause is obtained: title: 'Foundation' RANK title: 'Chanel'. However, Chanel is not a regular term in business. Diagnosis: Chanel is a special term in business. In this case, NER intervention is required.

Solution: Create an NER intervention dictionary and associate the intervention dictionary with an online query analysis rule.

Procedure:

1.Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create in the upper-right corner to create an intervention dictionary.

1

Specify a name for the intervention dictionary. Set the Dictionary Type parameter to Entity Recognition and the Associated Analyzer parameter to E-commerce Analyzer. If the Dictionary Type parameter is set to Entity Recognition, only E-commerce Analyzer can be selected.

2

2.In the Manage Entries page of the created intervention dictionary, click Add Intervention Entry. In the Add Intervention Entry panel, enter a query in the Search Query field, click Obtain Recognition Results, and then modify the recognition results.

The following figure shows the error recognition results.

3

The following figure shows the recognition results after intervention.

4

Note: The Intervention Type parameter can be set to Full Match or Partial Match. A value of Full Match indicates that intervention is implemented when the specified search query fully matches the intervention query. A value of Partial Match indicates that intervention is implemented when the specified search query contains the intervention query.

3.On the Query Analysis Rule Management page, associate the created NER intervention dictionary with an unpublished query analysis rule to perform a search test.

5

Note: You can specify whether to ignore spaces in search queries during NER intervention. By default, spaces are not ignored.

4.Test whether the search performance meets the requirements. The documents related to Chanel Foundation are retrieved.