OpenSearch:Drop-down suggestion

The drop-down suggestion feature extracts queries from your document content. It can generate candidate queries based on various matching methods for Chinese, including prefix matching, full Pinyin spelling, Pinyin initial acronyms, a mix of Chinese characters and Pinyin, tokenized prefixes, and homophones.

For example, you can search for the query long dress as follows:

• Chinese prefix: 连, 连衣, …

• Pinyin prefix (full spelling): l, li, lian, lianyi, lianyiqun, …

• Pinyin prefix (initials): l, ly, lyq, …

• Chinese characters plus Pinyin: 连yi, 连衣qun, …

• Prefixes after tokenization: long style, long style dress, dress long, …

• Chinese homophones and similar-sounding incorrect characters: 连衣群, 联谊群, …

You can also use manual intervention to influence suggestion results. To view key performance metrics for your drop-down suggestions, see the Drop-down suggestion report.

Data sources

Drop-down suggestions use data from your application documents and end-user queries. You can apply filters to both documents and queries.

Candidate queries from application documents

You can select up to three fields from an application as the data source for each drop-down suggestion model. During processing, the system selects a sample of documents (up to one million) and processes the selected fields according to specified rules to generate candidate queries. The system then retains a subset as final suggestions. Two generation rules are supported: Extract and Retain original value.

• Extract: This method uses a proprietary Alibaba NLP analyzer trained on vast amounts of natural language data. It applies tokenization to the field content, extracts meaningful terms, and combines them to form candidate queries. This approach ensures that generated suggestions can retrieve corresponding documents.

• Retain original value: This method uses the raw content of a field as a candidate query without tokenization. If the content exceeds 30 characters, it is truncated to the first 30 characters. This method is suitable for fields that do not require tokenization, such as store names, usernames, or song titles, or when you provide your own pre-generated candidate queries. We recommend using fields with short and clear content.

When you use the Extract rule, you can also specify an extraction model for the drop-down suggestion model. After an extraction model is specified, the system uses a large language model to perform an additional round of filtering on the extracted candidate queries, removing meaningless candidates and improving the quality of suggestions. The extraction model is an optional configuration and takes effect only when a field uses the Extract rule. After the extraction model takes effect, you are charged based on the number of tokens actually consumed by the model. For more information about billing, see Billing.

Candidate queries from end-user queries

The system analyzes user search histories over the past N days (default is 7) and considers metrics such as search frequency, average number of results, historical term weights, and recent query success rates. Based on this analysis, it selects representative queries as candidate suggestions.

Historical search query feature: The drop-down suggestion feature also supports the historical search query option, which prioritizes suggestions based on the current user's past queries (requires the raw_query and user_id parameters in requests). The historical search query feature is an extension that incurs training fees based on compute hours consumed per training job. The historical search query feature is being deprecated. The option is no longer displayed when you create a drop-down suggestion model. The following description applies only to existing models that have this feature enabled.

Manual intervention

You can manually influence drop-down suggestions in the following ways:

• Manage candidate query results using a blocklist and an allowlist.

• Set a filter condition on the source application documents. When you apply a filter, the system uses only documents that meet the condition to generate candidate suggestions.

On the Create Drop-down Suggestion Model page in the console, you can configure the following settings:

• Target Application: Select the application that contains the source data for candidate suggestions.

• Model Name: Enter a name that is 1 to 30 characters long. It must start with a letter and can contain uppercase letters, lowercase letters, digits, and underscores (_). The name must be unique among all your models.

• Training Fields: Select source fields from the target application and specify a processing method for each field: Extract or Retain original value.

• Extraction Model: This is an optional configuration. The currently available extraction model is deepseek-v4-flash. After you select an extraction model, the system uses a large language model to filter the extracted candidate queries and remove meaningless candidates. This configuration takes effect only when a field uses the Extract rule. After the extraction model takes effect, you are charged based on the number of tokens actually consumed by the model.

• Doc Filter Condition and Query Filter Condition: These are filter conditions that apply to the application document data source and the end-user query data source, respectively (for example, status=1 or biz_type=phone). For syntax rules, see the filter condition description above.

Controlling suggestion results

Blocklist: The blocklist supports contains-matching. Any query that contains a blocklisted keyword is excluded from the drop-down suggestion results. If you see undesirable suggestions, add relevant keywords to the blocklist to block them.

Allowlist: If a query in the allowlist meets the recommendation criteria, it is prioritized in the drop-down suggestion results. If high-quality queries do not appear or are ranked too low, add them to the allowlist to boost their visibility. For details on how to configure blocklists and allowlists, click here.

If your drop-down suggestion model training fails, follow these steps to troubleshoot:

1. Check the anomaly report: On the model details page, in the Basic Information area, check the Latest Version Status. If the status is Training Failed or Abnormal Data, the area displays an Anomaly Report tooltip. Hover over it to view the Exception Details and Solution, and use this information to identify the cause of the training failure.

2. Check the completeness level and upgrade conditions: In the Data Verification area of the model details page, check the Completeness Level and the corresponding Upgrade Conditions. Adjust your data based on the upgrade conditions to improve the completeness level, and then retrain the model.

3. T-1 data logic: Model training uses statistical data from the previous day (T-1). When a newly created model runs its first training job, the system may report a "Training field does not exist" error because no data from the previous day is available yet. Wait until the following day after data becomes ready, and then retrain the model.

4. Data volume requirements (L1 conditions): Model training requires the following minimum data thresholds:

   • The total data volume (the combined total of raw_query data and application document data) must be at least 1,000.

   • The number of unique raw_query values must exceed 100.

5. Filter condition format check: Incorrect filter condition format is a common cause of training failure. Field names must match fields defined in the application schema that have indexes created. Supported operators are <, >, <=, >=, =, and !=. Multiple conditions have an AND relationship and must be separated by commas (,). For the full filter condition syntax, see the Manual intervention section above.

• To improve the effectiveness of drop-down suggestions (for example, to increase suggestion-guided searches and click-through rates), associate suggestion requests with search requests. For instructions, see the "Associate a drop-down suggestion request with a search request" section at the end of this topic.

• Choose fields with concise content that is relevant to the document's main topic.

• Use the Extract and Retain original value rules appropriately based on your needs.

• In the response, suggestions contains the search results, while errors indicates whether an error occurred. A non-empty errors field does not necessarily mean that the suggestions field is empty. Therefore, when parsing the response, check if suggestions is empty to determine whether to display data.

1. In the console, navigate to Search Algorithm Center > Search Guidance > Drop-down Suggestions, and click Create.

2. Enter a Model Name, select Training Fields and an extraction method, select an Extraction Model (optional), enter Filter Conditions (optional), and click Submit.

3. On the Drop-down Suggestion list page, find the model and click Train to start training.

4. Training typically takes 20 to 30 minutes to complete.

5. After the model is trained, you can test the suggestions. The following shows an example of the Retain original value method:

Retain original value:

Click the Effect Preview button in the upper-right corner of the model details page. In the pop-up search box, enter a query (for example, the letter E). Matching suggestions appear below (such as "Eifini" and "ELF SACK"). On the right side, you can set the user id and display count parameters.

6. To query for candidate queries online, see the demo below. For detailed API information, see Drop-down suggestion development guide.

Drop-down Suggestion list page

In the OpenSearch console, navigate to Search Algorithm Center > Search Guidance > Drop-down Suggestions to access the list page.

The list page displays information for each drop-down suggestion model, including Model Name, Creation Time, Model Status, Last Training Start Time, and Latest Version Status (which can be Pending Training, Scheduling, Training, Trained and Passed, Training Failed, or Abnormal Data). The Actions column allows you to view details, train, or perform other operations on the model.

Drop-down suggestion model details page

The top of the model details page shows the Basic Information area, displaying the creation time, model status, and latest version status. The upper-right corner has an Effect Preview button. The page includes three tabs: Drop-down Suggestion, Top Search & Hint, and Blocklist/Allowlist.

The Basic Information area shows the model creation time, model status, last training start time, and latest version status. If the status is Training Failed or Abnormal Data, an Anomaly Report tooltip is displayed in this area.

The Configuration Information section shows the configured training fields and their processing methods, extraction model, Doc filter condition, Query filter condition, blocklist/allowlist, and scheduled tasks. Click the Configure button in the upper-right corner to modify these settings.

The Data Verification section shows the data completeness and completeness level for model training.

The Training History section shows the training records for the model.

Core metrics data

You can select different time ranges to view core metrics for the drop-down suggestion model, displayed in a table and a line chart.

Note: For definitions of specific metrics, see the Drop-down suggestion report.

API:

GET v3/openapi/suggestions/{suggestion_name}/actions/search?hit=10&query={your_query}&re_search=homonym&user_id=xxx

Java SDK Maven dependency:

<dependency>
    <groupId>com.aliyun.opensearch</groupId>
    <artifactId>aliyun-sdk-opensearch</artifactId>
    <version>4.0.0</version>
</dependency>

Related link: Release Notes

Code sample:

package com.example.opensearch;

import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.SuggestionClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchClientException;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.nio.charset.Charset;

public class SuggestDemo {
    static private final String accesskey = "YOUR_ACCESSKEY_ID";
    static private final String secret = "YOUR_ACCESSKEY_SECRET";
    static private final String host = "THE_ENDPOINT_OF_THE_REGION_WHERE_YOUR_APP_IS_LOCATED";
    OpenSearch openSearch;
    OpenSearchClient openSearchClient;
    static private final byte hits = 8; // The maximum number of suggestions to return.
    static private final String suggestionName = "YOUR_SUGGESTION_MODEL_NAME"; // The name of your drop-down suggestion model.

    @Before
    public void setUp() {
        // Initialize the OpenSearch object.
        openSearch = new OpenSearch(accesskey, secret, host);
        openSearchClient = new OpenSearchClient(openSearch);
    }

    @Test
    public void TestEnv() {
        // View the file and default encoding formats.
        System.out.println(String.format("file.encoding: %s", System.getProperty("file.encoding")));
        System.out.println(String.format("defaultCharset: %s", Charset.defaultCharset().name()));

        // Create a SuggestionClient object.
        SuggestionClient suggestionClient = new SuggestionClient("YOUR_APP_NAME", suggestionName, openSearchClient);
        String query = "YOUR_SEARCH_QUERY";
try {
            SuggestParams suggestParams = new SuggestParams();
            suggestParams.setQuery(query); // Set the query.
            suggestParams.setHits(10); // Set the maximum number of suggestions to return.
            suggestParams.setUserId("12345678"); // Set the user ID.
            // The Chinese homophone matching feature is enabled by default. You can adjust this by using the re_search parameter.
            // To disable this feature, use ReSearch.findByValue(1). It is enabled by default (equivalent to ReSearch.findByValue(0) or omitting the parameter).
            suggestParams.setReSearch(ReSearch.findByValue(1));
            SearchResult result = suggestionClient.execute(suggestParams); 
            System.out.println(result); // Print the results.
        } catch (OpenSearchException e) {
            e.printStackTrace();
        } catch (OpenSearchClientException e) {
            e.printStackTrace();
        }
    }

    @After
    public void clean() {
        openSearch.clear();
    }
}

For more details about the drop-down suggestion Java SDK, see the drop-down suggestion demo.

Sample response:

{
  "request_id": "159851481919726888064081",
  "searchtime": 0.006246,
  "suggestions": [
    {
      "suggestion": "trendy skirts"
    },
    {
      "suggestion": "dresses for petite women"
    },
    {
      "suggestion": "polka dot dresses"
    },
    {
      "suggestion": "youthful skirts"
    },
    {
      "suggestion": "polka dot skirt"
    },
    {
      "suggestion": "skirts for petite women"
    },
    {
      "suggestion": "polka dot skirts for petite women"
    }
  ]
}

Note: The request_id returned in the response can be used to associate the suggestion with a subsequent search request.

Associating suggestion requests with search requests provides the following benefits:

1. It allows you to collect metrics to measure the impact of drop-down suggestions on search performance. These metrics include suggestion-guided search PVs, click-through rates, and zero/low result rates. For more details, see the Drop-down suggestion report.

2. The associated request data provides insights, such as suggestion click data, which can be used to optimize the suggestion ranking model and improve the effectiveness of suggestion-guided searches.

   Association method:

   If a user starts a search by selecting a drop-down suggestion, include the from_request_id={from_request_id} parameter in the search request. The from_request_id parameter indicates the source of the search. If the current query comes from a recommendation list, such as drop-down suggestion, top search model, or hint model, you can assign the request_id of that recommendation request to this parameter. By associating these events, you can calculate key metrics for upstream features, measure their effectiveness, and gather data for optimization. This parameter is also described in the search processing documentation.

Example:

Assume a drop-down suggestion API call returns a request_id of 159851481919726888064081. You can associate it with a search request as shown below:

SearchParams searchParams = new SearchParams(config);
searchParams.setQuery("title:'skirts for petite women'"); // Query guided by a drop-down suggestion.

// Add the from_request_id parameter.
Map<String, String> customParam =new HashMap<>();
customParam.put("from_request_id","159851481919726888064081");
searchParams.setCustomParam(customParam);

// Execute the query and return a SearchResult object.
SearchResult execute = searcherClient.execute(searchParams);
// Return the query data as a string.
String result = execute.getResult();
System.out.println(result);