All Products
Search
Document Center

Intervention dictionaries for spelling correction

Last Updated: Sep 09, 2021

An intervention dictionary is a collection of custom intervention entries. Intervention entries are used to correct spelling errors that OpenSearch cannot identify in search queries. After you create an intervention dictionary for spelling correction, you can select the dictionary when you create or modify a query analysis rule. This way, you can intervene in the spelling correction of search queries.

Overview

OpenSearch is built in with a dictionary to implement automatic spelling correction. You can perform the following steps to intervene in spelling correction:

  1. Create an intervention dictionary for spelling correction. To create an intervention dictionary, log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create in the upper-right corner. Specify a name for the intervention dictionary, select a dictionary type, and then click Save. After the intervention dictionary is created, it is displayed in the dictionary list.

  2. Create and manage intervention entries in the intervention dictionary. To go to the entry management page for the created dictionary, find the dictionary in the dictionary list and click Manage Entries in the Actions column. On this page, create and manage intervention entries as needed. You can intervene in spelling correction in the following ways:

    1. Add a correcting query and enable it for the spelling correction of a search query. After you perform this operation, OpenSearch corrects the search query and retrieves documents based on the correcting query.

    2. Add a correcting query and block it in the spelling correction of a search query. After you perform this operation, OpenSearch no longer corrects the search query or retrieves documents based on the correcting query.

  3. Use the intervention dictionary. After you create intervention entries in the intervention dictionary, you can use the intervention dictionary in a query analysis rule on an application as needed.

  4. Test and publish the intervention dictionary. After you use the intervention dictionary in the query analysis rule, we recommend that you test the search performance before you apply the rule to your online application. This ensures expected search performance.

Rules on the matching sequence of intervention entries

1.The intervention entry that is created earlier takes precedence.

For example, a custom search query is specified as china constructing back and the following intervention entries are created: china constructing -> china construction and construction back -> construction bank. In this case, the predefined search queries in the two intervention entries can both be matched by consecutive semantic terms after analysis. However, an overlap exists between the predefined search queries in the two intervention entries and only the intervention entry that is created earlier takes effect. Therefore, the result of spelling correction is china construction back.

2.For intervention entries whose predefined search queries share the same beginning, the intervention entry that has a longer predefined search query takes precedence. The length of a predefined search query indicates the number of semantic terms after analysis.

For example, a custom search query is specified as opansearching is excellent and the following intervention entries are created: opan -> open and opansearching -> opensearch. In this case, the result of spelling correction is opensearch is excellent.

3.Multiple intervention entries can take effect on a single custom search query in inclusive match mode.

Notes:

  • A spelling correction operation with human interventions has high credibility.

  • OpenSearch uses the built-in dictionary for spelling correction together with the intervention entries that you create. For a custom search query, after its spelling is corrected based on an intervention entry, OpenSearch retains the changes and processes the unchanged part by using the built-in dictionary.

Sample intervention entry:

mobile phone changer -> mobile phone charger

Sample custom search query:

which mobile phone changer brad is good

Result of spelling correction: The custom search query is first changed to which mobile phone charger brad is good after spelling correction based on the intervention entry mobile phone changer -> mobile phone charger. Then, OpenSearch retains this change and processes the unchanged part of the search query by using the built-in dictionary for spelling correction. The final result is which mobile phone charger brand is good.

Example

Scenario: You have created query analysis rules with spelling correction capabilities for the OpenSearch application of your e-commerce shopping guide service. After you apply these rules to the online application, invalid spelling correction results exist. To resolve the issue, intervention in spelling correction is implemented.

Invalid spelling correction results: A customer enters the search query "hujian" and only a few documents that contain the word "fujian" are retrieved. Most documents that contain the word "fujian" fail to be retrieved.

Problem description: The search query "hujian" entered by the customer has a spelling error and the built-in dictionary for spelling correction fails to identify the spelling error. The corresponding accurate word is "fujian".

Solution: Create an intervention dictionary for spelling correction and create an intervention entry "hujian -> fujian" for the search query "hujian". Then, use the intervention dictionary in a query analysis rule on the online application.

Procedure:

1.Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create in the upper-right corner to create an intervention dictionary.

1

In the Create Query Analysis Dictionary panel, specify a name for the intervention dictionary and set the Dictionary Type parameter to Spelling Correction.

2

2.Find the created intervention dictionary and click Manage Entries in the Actions column. On the page that appears, click Add Intervention Entry. In the Add Intervention Entry panel, enter hujian in the Search Query field and fujian in the Corrected Word field, set the Intervention Type parameter to Add, and then click Save.

3

3.Go to the Query Analysis Rule Management page and click Create in the upper-right corner. In the Add Rule panel, associate the created intervention dictionary for spelling correction with the query analysis rule.

4

4.Perform a search test. OpenSearch returns a large number of documents that contain the search query "fujian" after spelling correction. The search performance is as expected.

Usage notes

  • You cannot change the name and type of an intervention dictionary after it is created.

  • You must specify different search queries for different intervention entries.

  • An intervention dictionary can be used by multiple query analysis rules.

  • OpenSearch uses the built-in dictionaries together with the intervention entries that you create. If you enable spelling correction when you create a query analysis rule, the built-in dictionary of this intervention type is automatically selected.

  • The intervention dictionaries for spelling correction takes effect only when one or more consecutive complete semantic terms (up to 5) in a query match the query in the intervention entry. Substring matches that are not completely equivalent to strings.

  • If an intervention dictionary is used by a query analysis rule, no matter whether the rule is published or not, you cannot delete the dictionary. You must disassociate the dictionary from the rule before you delete the dictionary.

Limits

  • You can create a maximum of 20 intervention dictionaries for spelling correction.

  • You can specify only one search query and one correcting query for an intervention entry.

  • You can create a maximum of 1,000 intervention entries in an intervention dictionary for spelling correction.

  • An intervention entry takes effect when one of the terms in the custom search query after analysis matches the predefined search query in the entry. For example, you create an intervention entry "changer -> charger". If you specify a custom search query "mobile phone changer", OpenSearch corrects the spelling based on the intervention entry "changer -> charger". In this case, OpenSearch does not need to use an intervention entry for the entire search query "mobile phone changer".

  • OpenSearch normalizes the search queries and correcting queries that you specify for intervention entries. OpenSearch converts uppercase letters into lowercase letters and full-width characters into half-width characters.