All Products
Search
Document Center

Stop word filtering

Last Updated: Nov 22, 2021

Overview

Meaningless words in search queries are filtered out based on the built-in stop word dictionary. Meaningless words are the words that appear at a high frequency but do not affect the search results, such as punctuations and modal particles. For example, if you set a search query to Running!Man, the exclamation point (!) is filtered out during data retrieval. For another example, if you set a search query to Did you eat, the modal particle Did is filtered out during data retrieval.

Procedure

1.Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Query Analysis Rule Management in the left-side pane.

1

2.On the Query Analysis Rule Management page, select an application name and the online or offline version of the application, and click Create.

2

3.In the Add Rule panel, set the Rule Name, Index Range, and Industry Type parameters, select Stop Word, and then click OK.

4

Note: If no intervention dictionary for stop word filtering is specified, stop words are filtered out based on the built-in stop word dictionary. If identified stop words are invalid or specific stop words are not identified based on the built-in stop word dictionary, specify an intervention dictionary. For more information, see the "Intervention dictionaries for stop word filtering" section of this topic.

4.After the rule is created, run a search test.

5

View the process of query analysis:

6

5.After you confirm that the process of query analysis is correct, switch to Index Orientation on the Query Analysis Rule Management page. Then, set the created query analysis rule as the default query analysis rule.

7

6.Check the default query analysis rule.

8

Intervention dictionaries for stop word filtering

Stop words vary with business scenarios. Specific stop words may not exist in the built-in stop word dictionary, or the built-in stop word dictionary may contain invalid stop words. To resolve the issue, OpenSearch allows you to customize stop words. After you create an intervention dictionary for stop word filtering, you can specify the intervention dictionary when you create or modify a query analysis rule. This way, you can intervene in stop word filtering. For more information, see Intervention dictionaries for stop word filtering.