OpenSearch:Spelling correction intervention dictionary

An intervention dictionary provides a way for you to define custom correction behavior when the system fails to correct a query. You can create an intervention dictionary and, when configuring spelling check in query analysis, select the corresponding intervention dictionary to intervene in the correction.

Manual intervention is currently supported for the system's built-in spelling correction dictionary. The intervention process typically involves the following four steps:

1. Create a spelling correction intervention dictionary. Go to Search Algorithm Center--->Recall Configuration--->Dictionary Management to open the query analysis intervention dictionary page, then click "Create" in the upper-right corner of the page. After you select the dictionary type and name the dictionary, the intervention dictionary is created and appears in the dictionary list on the page.

2. Add and manage intervention entries in the intervention dictionary. After the dictionary is created, click the dictionary name in the list or click "Manage" for the dictionary to open its details page. On the details page, you can add and manage intervention entries. You can apply two types of intervention to a query:

   1. Add a correction: Add a correction to a query. When the query is searched, the system rewrites it to the added correction to retrieve results.

   2. Block a correction: Block a correction for a query. When the query is searched, the system no longer rewrites it to the blocked correction to retrieve results.

3. Use the intervention dictionary. After you create and populate the spelling correction intervention dictionary, you can select it for use in the query rules of any application.

4. Test the intervention dictionary and go live. After a query analysis rule uses an intervention dictionary, we recommend that you run a search effect test before applying it to production to evaluate whether the results meet your intervention expectations.

1. Earlier entry takes precedence:

For example, intervention entries 中国建社->中国建设 and 建社很行->建设银行, query: 中国建社很行. Both entries have consecutive semantic term matches in the query, but because the entries overlap in the query, the entry positioned earlier takes effect. The final correction result is 中国建设很行.

2. If the positions are the same, the entry with the greater length (number of semantic terms) takes precedence:

For example, intervention entries 开法->开发 and 开法搜嗦->开放搜索, query: 开法搜嗦真优秀. The final correction result is 开放搜索真优秀.

3. Multiple inclusive-match interventions can take effect for a single query.

Note:

• As long as there is user intervention, the credibility of this spelling check is high.

• For entries that are successfully matched by intervention, the system passes the query with the intervention entry applied through the correction model. While retaining the user's correction intervention, it corrects the parts that were not intervened. For example:

Intervention entry:

手机冲电器->手机充电器

Query:

query：手机冲电器哪个排子好

Result: After intervention, the result is 手机充电器哪个排子好. This query is then passed through the system model for correction. While ensuring that the intervention 手机冲电器->手机充电器 takes effect, the system performs further correction and produces the result 手机充电器哪个牌子好.

Business scenario: An e-commerce shopping guide business configured and used a query analysis rule in its OpenSearch application instance. The rule includes the spelling correction feature, but a bad case was found in production, so the business decided to use the intervention feature.

Bad case: When a user searches the query "福健", only a few results are returned, and they all contain the keyword "福建" in full. In fact, many relevant results for this query were not retrieved.

Problem diagnosis: The user's input query is incorrect; the correct spelling should be "福建". The system's spelling correction failed to detect the incorrect spelling.

Solution: Create a spelling correction intervention dictionary, add the correction "福建" for the query "福健", and then apply the spelling correction dictionary to the query analysis rule used in production.

Procedure:

1. Console > Search Algorithm Center > Recall Configuration > Dictionary Management, and create a spelling correction dictionary:

Enter the "Name" and select "Spelling Correction" for the dictionary type:

For Name, enter dic_error. For Dictionary Type, select Spelling Correction.

2. In the newly created spelling correction intervention dictionary, add an intervention entry. Enter "福健" in the Query field, "福建" in the correction field, and select "Add" for the intervention type.

3. On the query analysis page, first apply the newly created "spelling correction intervention dictionary" to a query analysis rule that is not yet live, so that you can run a search effect test:

In the Add Rule dialog, configure the following parameters:

• Rule Name: Enter test_error

• Index Range: Select Default

• Industry Type: Select General

• Feature Selection: Select Spelling Correction

• Intervention Dictionary: Select dic_error

After the configuration is complete, click OK.

4. Test whether the search meets the expected results. You find that the search retrieves results rewritten to "福建", which meets expectations.

• The dictionary type and name cannot be modified after creation.

• When you add an intervention entry, the query you enter must not duplicate a query that has already been intervened in the intervention list.

• The same intervention dictionary can be used by multiple query analysis rules.

• Because the current intervention dictionary is a patch-style intervention based on the system's built-in dictionary, the system's built-in dictionary is selected by default when the dictionary is used.

• Currently, the rule for a spelling correction intervention dictionary to take effect is inclusive match. It takes effect only when one or more (up to five) consecutive, complete semantic terms in the query match the query in the intervention entry. Inclusive match is not exactly equivalent to substring match.

• An intervention dictionary that is used by any query rule (whether live or not) cannot be deleted. To delete it, you must first disassociate it from use.

• You can create a total of 20 spelling correction intervention dictionaries.

• Each time you add a spelling correction intervention entry, you can enter only one query, and only one correction can be entered for the same query.

• Each spelling correction intervention dictionary can contain up to 1,000 intervention entries.

• A spelling correction intervention entry takes effect only when the intervened query exactly matches the search query. For example, if you add the spelling correction "武汉" for the query "武汗", searching "武汗" is rewritten to "武汉" for retrieval. However, searching "去武汗" is not rewritten and is retrieved based on the original query.

• Added intervention content is normalized for case and full-width/half-width characters. Uppercase letters are normalized to lowercase, and full-width characters are normalized to half-width.