Usage
You can manually fine-tune the results of a trained category prediction model. The process is similar to query analysis intervention and typically involves the following three steps:
1. Create an intervention dictionary. In the console, navigate to Search Algorithm Center > Retrieval Configuration > Dictionary Management. Click Create in the upper-right corner, select a dictionary type, and enter a name. The new dictionary is then added to the list.
2. Add and manage intervention entries. After you create the dictionary, click its name or click Manage in the dictionary list to open its details page. On this page, you can add and manage intervention entries. You can adjust the relevance of categories for a specific query to correct inaccurate predictions from the model or add categories that the model failed to identify.
3. Use the intervention dictionary. After you create and populate an intervention dictionary for category prediction, you can apply it to any category prediction model. If both the intervention dictionary and the category prediction model define a relevance score for the same category and query, the system merges the two results.
-
For example, for the query "A Smile is Beautiful", the category prediction model calculates a relevance of 0 (no relevance) for category 20 and 2 (strong relevance) for category 10. In the intervention dictionary, the relevance for category 20 is changed to 2 (strong relevance), and a relevance of 1 (weak relevance) is added for category 30. The final, merged relevance scores are: the relevance for category 20 is 2, the relevance for category 30 is 1, and the relevance for category 10 is 2.
Use case
Scenario: After an e-commerce business notices incorrect search results in their live environment, they use the intervention feature of their category prediction model in the OpenSearch application instance.
Problem: When a user enters the query "milk", products like "milk cups" appear higher in the search results than actual milk.
Diagnosis: The category prediction model has incorrectly calculated the relevance of categories for the query "milk".
Solution: Create an intervention dictionary for category prediction. In the dictionary, add an intervention entry for the query "milk". Set the relevance of the "household supplies" category (ID 20), which contains "milk cups", to weak relevance. Set the relevance of the "food" category (ID 15), which contains "milk", to strong relevance.
Procedure:
1. In the console, navigate to Search Algorithm Center > Retrieval Configuration > Dictionary Management. Set the dictionary type to Category Prediction and click Create to create an intervention dictionary for category prediction.
2. In the "test_2" dictionary, click Add Intervention Entry. In the Query field, enter "milk". In the Category ID-Relevance field, enter "20-1;15-2" (2 indicates strong relevance, 1 indicates weak relevance, and 0 indicates no relevance).
3. Apply the intervention dictionary to the production category prediction model:
In the Edit Rule dialog box, select the Category Prediction checkbox. From the Intervention Dictionary drop-down list, select test_2. For Category Prediction Model, select cate_test_lsh_1. Click OK to complete the configuration.
Usage notes
-
The dictionary name cannot be changed after creation.
-
When adding a new intervention entry, the query must be unique. To modify an existing intervention, find the query in the list and directly edit its Category ID-Relevance settings.
-
To add multiple category ID-relevance pairs for a single query, separate each pair with a semicolon (;).
-
After you add or modify an intervention entry, if the status remains "Applying", click the Refresh button to view the latest status.
-
You can apply a single intervention dictionary for category prediction to multiple category prediction models.
-
Intervention dictionaries do not affect model training.
-
An intervention dictionary that is applied to any model, whether in production or not, cannot be deleted. You must first disassociate the dictionary from all models before you can delete it.
Limitations
-
You can create a maximum of 10 intervention dictionaries for category prediction.
-
Each intervention entry can contain only one query. A single query can have a maximum of five category ID-relevance pairs.
-
Each intervention dictionary for category prediction can contain a maximum of 500 intervention entries.
-
An intervention for category prediction takes effect only if the search query is an exact match. For example, if you create an intervention for the query "dress" to set the relevance of category 23 to strong relevance, it will apply when a user searches for "dress". However, it will not apply if the user searches for "2018 dress".
-
All content added to an intervention entry is normalized. Uppercase letters are converted to lowercase, and full-width characters are converted to half-width characters.