OpenSearch:Synonym intervention dictionary

The appropriate synonyms for a term can change depending on the business scenario. The built-in dictionary may lack certain synonyms or even suggest incorrect ones. To address this, OpenSearch lets you customize synonyms with an intervention feature.

You can create a synonym intervention dictionary and apply it to a query analysis rule to customize synonym behavior.

Introduction

A synonym intervention dictionary adds custom rules to the default synonym feature in query analysis. The process involves creating the dictionary, adding entries, applying it to a rule, and then testing the result.

1. Create a synonym intervention dictionary. Navigate to Search Algorithm Center > Retrieval Configuration > Dictionary Management. On the intervention dictionary page for query analysis, click Create in the upper-right corner. After you select the dictionary type and provide a name, the intervention dictionary is created and appears in the dictionary list.

2. Add and manage intervention entries. After the dictionary is created, find it in the list and click its name or click Manage Entries to open the details page.

You can use two types of interventions for a search query:

• Add a synonym: For a given search query, you can add one or more synonyms. The system then returns results that contain either the original search query or the added synonyms.

• Block a synonym: For a given search query, you can block one or more existing synonyms. The system then no longer returns results that contain the blocked synonyms.

  1. Use the intervention dictionary. After you create and populate the synonym intervention dictionary, you can select it in the query analysis rule of any application.

  2. Test and deploy the intervention dictionary. After you apply the intervention dictionary to a query analysis rule, perform a search test to ensure the results meet your expectations before deploying to production.

How entries take effect

• An intervention entry takes effect if one or more consecutive semantic terms (up to five) in the search query, excluding any stop words, match the intervention entry's query.

• Example of a synonym intervention taking effect: The search query is dior aj co-branded low-top shoes and you have configured the intervention entries dior->dior, aj->air jordan, and co-branded->collaboration. All three interventions take effect. This can match results for up to eight query combinations, such as "dior aj co-branded low-top shoes", "dior air jordan co-branded low-top shoes", "dior aj collaboration low-top shoes", and "dior air jordan collaboration low-top shoes".

• The intervention entry for the term with the highest match weight takes priority. For example, consider the following intervention entries: pharyngitis -> sore throat, tonsillitis, inflammation, red throat; low-grade fever -> fever, high temperature; and medicine -> traditional Chinese medicine, Western medicine, patent Chinese medicine. The search query is "what medicine is good for pharyngitis and a low-grade fever". Assume the weights for "pharyngitis" and "medicine" are 7, and the weight for "low-grade fever" is 4. In this case, the synonyms for "pharyngitis" and "medicine" are applied, but the synonyms for "low-grade fever" are not.

• If match weights are identical, the intervention entry for the term that appears first in the search query takes priority. For example, if you have the intervention entries apple phone->iphone and phone charger->power bank, and the search query is apple phone charger, the intervention apple phone->iphone takes effect.

• If the match positions are the same, the entry with more semantic terms takes priority. For example, if you have the intervention entries phone charger->power bank and phone->mobile phone, and the search query is which brand of phone charger is good, the intervention phone charger->power bank takes effect.

• User-defined interventions take priority over synonyms from the built-in dictionary. A conflict occurs when the term range of a user intervention overlaps with or is contained within the term range of a system-defined synonym. Examples:

  • The user intervention's range contains the system's range. Assume the system has the synonym cabbage -> round cabbage. A search for fried cabbage expands to fried AND (cabbage OR round cabbage). If you add a user intervention fried cabbage -> stir-fried kale, the query expands to fried cabbage OR stir-fried kale. The system synonym round cabbage is ignored.

  • The user intervention's range is contained within the system's range. Assume the system has the synonym apple phone -> iphone. A search for apple phone expands to apple phone OR iphone. If you add a user intervention phone -> mobile phone, the search for apple phone expands to apple AND (phone OR mobile phone).

Note:

• The number of generated query groups limits the number of synonyms. The current limit for query groups is 36.

Walkthrough

Scenario: An e-commerce shopping guide service uses a query analysis rule with the synonym feature in its OpenSearch application. You discover a poor search result in the production environment that requires an intervention dictionary.

Failure case: A user searches for "apple", but no relevant products are returned. This happens because the products exist in the database, but their descriptions use the synonym "pinguo" (the romanized form of the Chinese word for apple).

Diagnosis: The system's synonym dictionary fails to recognize "pinguo" as a synonym for "apple".

Solution: Create a new synonym intervention dictionary, add an intervention for the query "apple" with the synonym "pinguo", and apply the dictionary to the query analysis rule that is used in production.

Procedure:

1. In the console, go to Search Algorithm Center > Retrieval Configuration > Dictionary Management and click Create.

In the Create Query Analysis Dictionary dialog box, enter dic_tong for Name and select Synonym for Dictionary Type.

2. In the new synonym intervention dictionary, Add Intervention Entry. In the Query field, enter "apple", and in the Add Synonym field, enter "Apple Inc.":

In the Block Synonym field, enter pinguo.

3. On the query analysis page, apply the synonym intervention dictionary that you created to a non-production query analysis rule to perform a search test.

In the Add Rule dialog box, select the Synonym checkbox. The built-in dictionary is selected by default. From the Intervention Dictionary drop-down list, select the dictionary that you created, such as dic_tong, and then click OK.

4. Test the search feature. As expected, the search returns results that contain 'pinguo' or 'apple':

On the Search Test page, the actual search query is displayed as (((default:"apple") OR (default:"fuji apple"))). This confirms that the synonym intervention has successfully expanded apple to fuji apple, and the search results return product records where cate_name=fuji apple.

Notes

• You cannot change the dictionary type or name after the dictionary is created.

• When you add an intervention entry, the query must not exist in the intervention list. For existing queries, you can directly add, delete, or modify their synonyms.

• To add or block multiple synonyms for a single search query, separate the synonyms with semicolons (;).

• If the sync status of a new or modified intervention entry remains Syncing, click the Refresh button to get the latest sync status.

• A single intervention dictionary can be used by multiple query analysis rules.

• Intervention dictionaries supplement the built-in dictionary. Therefore, when you use an intervention dictionary, the option to use the built-in dictionary is selected by default.

• You cannot delete an intervention dictionary that is in use by any query analysis rule, regardless of whether the rule is online. To delete the dictionary, first remove it from all associated rules.

Limitations

• You can create a maximum of 20 synonym intervention dictionaries.

• When you add an intervention entry, you can specify only one query. The total number of synonyms you can add and block for that query cannot exceed five.

• Each synonym intervention dictionary can contain a maximum of 1,000 intervention entries.

• Synonym intervention entries take effect by matching terms after analysis. For example, if you add "capital" as a synonym for the query "Beijing", a search for "Beijing" returns results that contain "Beijing" or "capital". A search for "welcome to Beijing" returns results that contain "welcome to Beijing" or "welcome to capital".

• When you add an intervention entry, any English letters in the query must be lowercase.

• For information about the related SDK, see Receive intervention entry changes.

• To bulk add synonym intervention entries, use the JSON format in the following example.

[{"cmd":"add","word":"hey","alias":["hei","hei2"],"antiAlias":["hi"]}]

If you are an existing user on the old console, use the following format:

[{"cmd":"add","word":"hey","alias":["hei","hei2"],"anti_alias":["hi"]}]