All Products
Search
Document Center

Drop-down suggestions

Last Updated: Sep 09, 2021

Overview

The drop-down suggestions feature is a basic feature of OpenSearch. This feature displays recommended search queries when you enter a search query. This feature improves search efficiency and helps you obtain the expected search results as soon as possible.

OpenSearch intelligently extracts search queries from documents to implement the drop-down suggestions feature. When you enter a search query, OpenSearch provides suggestions based on the first few characters in Chinese phrases, complete PinYin spelling, PinYin abbreviations, Chinese characters and Pinyin, first few characters after segmentation, and Chinese homophones.

For example, the search query 连衣裙长款 is recommended when you enter the following content:

  • First few characters in Chinese phrases: 连 and 连衣

  • First few letters in complete PinYin spelling: l, li, lian, lianyi, and lianyiqun

  • First few letters in PinYin abbreviations: l, ly, and lyq

  • Chinese characters and PinYin: 连yi, and 连衣qun

  • First few characters after segmentation: 长款, 长款连衣, and 连衣长

  • Chinese homophones: 连衣群 and 联谊群

You can also use intervention entries to intervene in drop-down suggestions. OpenSearch provides multiple metrics for drop-down suggestions. For more information, see Drop-down suggestion report.

Usage notes

Data source

Data that is used to provide drop-down suggestions is from application documents. You can select a maximum of three fields from an application as the data source for each drop-down suggestions model. When OpenSearch provides drop-down suggestions, OpenSearch selects millions of documents from an application and processes the values of the selected fields in these documents based on specified rules to generate recommend search queries. Then, OpenSearch displays only a number of recommended search queries in drop-down suggestions based on specified rules.

Rules for generating recommended search queries

OpenSearch generates recommended search queries based on the historical search queries in the last N days based on specified conditions. Conditions include the term weights, number of retrieved documents, and number of search times of a historical search query, and whether search results are returned for the historical search query within the last day. By default, historical search queries in the last seven days are used to generate recommended search queries. To generate recommended search queries, OpenSearch can extract terms from historical search queries or use original historical search queries:

  • Extract terms from historical search queries: OpenSearch uses the analyzer that is trained by the natural language processing (NLP) team of Alibaba Cloud to segment search queries. This analyzer is developed based on a large amount of natural language data. After segmentation, OpenSearch combines terms that have actual meanings to generate recommended search queries. This ensures that the required documents can be retrieved based on the recommended search queries.

  • Use original historical search queries: OpenSearch selects original historical search queries without changes as recommended search queries in drop-down suggestions. If a historical search query contains more than 60 bytes, OpenSearch truncates the search query by keeping only the first 60 bytes and uses the truncated search query as the recommended search query. This method applies to application fields that require no segmentation, such as the store name, username, and song name. This method also applies to custom search queries that you want to use as recommended search queries without changes. We recommend that you select short, concise, and explicit fields.

Industry templates

The drop-down suggestions feature provides industry templates based on the data characteristics of different industries. The following industry templates are provided, including the general-purpose, e-commerce, and content industry templates:

  • General-purpose industry template: uses the general analyzer for Chinese to segment search queries and generates recommended search queries based on the general-purpose analyzer template.

  • E-commerce industry template: uses the e-commerce analyzer to segment search queries. Alibaba Cloud develops the e-commerce analyzer based on its years of experience in the e-commerce industry. This analyzer can recognize tags such as product names, brand names, product qualifiers, style names, and material names and obtain their term weights from e-commerce text that is tagged with query or title. Then, OpenSearch combines the recognized tags into common phrases in e-commerce to generate recommended search queries. In addition, OpenSearch calculates scores for the recommended search queries based on the term weight, number of retrieved documents, and historical query statistics. These scores are used to sort the recommended search queries in drop-down suggestions.

  • Content industry template: uses the IT-content analyzer to segment search queries and combine recognized tags into common phrases in the content industry to generate recommended search queries.

Note:

  • We recommend that you set the raw_query parameter in the request so that OpenSearch can recognize original search queries that are entered by your users. For more information, see Search and processing.

  • The content industry template is still being optimized. If you have questions in use, contact the OpenSearch team in time.

Manual intervention

You can intervene in drop-down suggestions in the following ways:

  • Filter application documents that are used as the data source by configuring conditions.

  • Intervene in recommend search queries by using whitelists and blacklists.

    Filter source documents

    After you configure filter conditions, OpenSearch generates recommended search queries based on only application documents that meet the conditions.

Parameter

Description

Filter Conditions

You must configure conditions for schema fields of your OpenSearch application. The configured filter conditions are used to filter all the documents that are used in the current service. The following format is supported: OpenSearch application field (<,>, <=,>=,=, or!=) value. For example, you can use the status field. If you configure multiple filter conditions, separate the conditions with a comma (,). In this case, all the conditions take effect at the same time based on the AND logical operation. The OR logical operation is not supported. For example, if the filter condition is status=1,level=1, only documents that meet this condition can be used as the data source.

The following figure shows you how to configure filter conditions in the OpenSearch console.1

Intervene in recommend search queries

Blacklists: If a recommended search query contains a keyword in a blacklist, the recommended search query does not appear in drop-down suggestions. If a recommended search query is not applicable to your business, you can add related keywords to a blacklist. This way, all recommended search queries that contain the keywords do not appear in drop-down suggestions.

Whitelists: When a search query in a whitelist meets the specified recommendation conditions, the search query ranks top in drop-down suggestions. If a recommended search query that is applicable to your business ranks low or is not included in drop-down suggestions, you can add the search query to a whitelist. This improves the quality of drop-down suggestions. **For more information about how to configure whitelists and blacklists, see Blacklists and whitelists.

Quotas

The drop-down suggestions feature is free. OpenSearch provides you with fixed quotas of computing resources and storage resources. For each drop-down suggestions model, OpenSearch provides computing resources to support a maximum of 100 QPS and storage resources to support a maximum of 2 million recommended search queries.

Considerations

  • Standard applications do not support drop-down suggestions. This feature is supported by only advanced applications.

  • You can create a maximum of 10 drop-down suggestions models for each application.

  • The names of models, including drop-down suggestions models, popularity models, category prediction models, top searches models, and hints models, that belong to the same Alibaba Cloud account must be unique.

  • You can use only the fields that are of the TEXT, SHORT_TEXT, LITERAL, and INT types and have been created as indexed in an application as the data source.

  • You can select a maximum of three fields to train a model.

  • When you modify the application schema, you cannot modify application fields that are used as the data source for drop-down suggestions.

  • When you delete an application, the drop-down suggestions models of the application are also deleted.

  • In the request for a search based on drop-down suggestions, the value of the query parameter cannot exceed 18 bytes in length. Otherwise, an error occurs and no data is returned.

  • In the request for a search based on drop-down suggestions, the value range of the hit parameter must be an integer that is greater than 0 and smaller than or equal to 10. Otherwise, the value of this parameter is corrected to 10 and an error occurs. For example, if the hit parameter is set to 0, -1, or 11, the request is processed in the same way as the hit parameter is set to 10 except that an error is reported.

  • A blacklist can contain a maixmum of 500 keywords.

  • A whitelist can contain a maximum of 500 search queries.

  • If a search query in a whitelist contains a keyword in a blacklist, the blacklist prevails and the search query does not appear in drop-down suggestions.

  • Modifications on blacklists and whitelists take effect in real time.

  • After a drop-down suggestions model is created, the system enables scheduled training for the model by default. The data of the model is updated on a regular basis after training.

  • The training duration of a drop-down suggestions model varies with the size of the application data and the system load. If the training duration exceeds half an hour, contact the OpenSearch team.

  • By default, OpenSearch also retrieves documents by relacing terms in search queries with Chinese homophones. To disable this feature, set the re_search parameter to disable in the request.

Usage suggestions

  • To optimize drop-down suggestions, for example, to increase the number of recommended search queries and the click-through rate, we recommend that you associate a drop-down suggestions request with a search request. For more information, see the Associate a drop-down suggestions request with a search request section of this topic.

  • Select concise fields that are related to their document topics.

  • Extract terms from historical search queries or use original historical search queries without changes based on your requirements.

  • In the response of a drop-down suggestions request, the suggestions parameter contains the recommended search entries. The errors parameter indicates whether an error occurs. The suggestions parameter may contain data even if the errors parameter is returned. If recommended search entries are returned in the suggestions parameter, these entries appear in drop-down suggestions.

Procedure

1.In the OpenSearch console, choose Search Algorithm Center > Search Guide. On the Drop-down Suggestions page, click Create.2

2.On the Create a drop-down prompt model, set the Model Name, Industry Type, and Training field parameters. Select Extract and generate or Original value retained for each training field. Configure filter conditions. Then, click Completed.3

3.On the Drop-down Suggestions page, click Start training in the Actions column of the created drop-down suggestions model to train the model.4

4.Wait until the training is complete. The training will last for 20 to 30 minutes.5

5.Test the drop-down suggestions model. The following figures show the search effects that are achieved by using original historical search queries and by extracting terms from historical search queries.

Use original historical search queries6

Extract terms from historical search queries7

6.Obtain recommended search queries in the OpenSearch console, as described in the demo in the following part of this topic. For more information about the operations, see Drop-down suggestions.

Drop-down suggestions pages

Drop-down Suggestions

To go to the Drop-down Suggestions page, choose Search Algorithm Center > Search Guide in the OpenSearch console. Then, the Drop-down Suggestions page appears, as shown in the following figure.8On the Drop-down Suggestions page, you can view the name, the latest training time, the latest training status, and the number of page views (PVs) of your drop-down suggestions models, and an available model is generated after the latest training. The status of the latest training includes Pending Training, Training, Trained and Passed, and Data Exception. In the Actions column, you can view the details of a drop-down suggestions model, train and retrain a model, view the metrics of a model, and delete a model.

Details page of a drop-down suggestions model

Basic Information and Configuration information9

Basic Information: In this section, you can view the creation time, the latest training time, and the latest training status of the drop-down suggestions model, and whether an available model is generated after the latest training. If the model is in the Data Exception state, you can view the exception report on this page. Configuration Information: In this section, you can view the industry type, the period of scheduled training, training fields, filter conditions, whitelists, and blacklists of the drop-down suggestions model.

extensions

Extended features:

  • High-frequency search queries: This feature can be used to sort recommended search queries based on the frequency search queries are searched. High-frequency search queries that meet the recommendation conditions rank top in drop-down suggestions. This feature can be in the Opened or Unopened state.

  • Historical search queries: Search queries that are searched before rank top in drop-down suggestions. This feature can be in the Opened or Unopened state.

  • Intelligent sorting: Recommended search queries are intelligently sorted based on user behavior data such as the click and purchase data. This feature can be in the Opened or Unopened state.

Enabling methods:

  • High-frequency search queries: You must set the raw_query parameter in the search request.

  • Historical search queries: You must set the raw_query parameter in the search request and the user_id parameter in the drop-down suggestions request.

  • Intelligent sorting: You must set the raw_query, user_id, and from_request_id parameters in the search request, and enable data collection.

Note:

  • For more information about the raw_query, user_id, and the from_request_id parameters, see Search and processing.

  • Intelligent sorting depends on a large amount of click data. We recommend that you enable this feature when the number of clicks per day exceeds 10,000.

  • The high-frequency search queries and low-frequency search queries features are automatically enabled after you set the required parameters. To enable the intelligent sorting feature, you must submit a ticket after you set the required parameters.

Core indicator data

OpenSearch displays the key metrics of drop-down suggestions models in a table and a line chart. You can filter the metrics by time range.10Note: For more information about the metrics, see Drop-down suggestion report.

Demo of the drop-down suggestions SDK

API:

GET v3/openapi/suggestions/{suggestion_name}/actions/search?hit=10&query={your_query}&re_search=homonym&user_id=xxx

Maven dependencies for the SDK for Java:

<dependency>
    <groupId>com.aliyun.opensearch</groupId>
    <artifactId>aliyun-sdk-opensearch</artifactId>
    <version>3.2.0</version>
</dependency>

Download URL: aliyun-sdk-opensearch 3.2.0

Demo code:

package com.example.opensearch;

import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.SuggestionClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchClientException;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchException;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.nio.charset.Charset;

public class SuggestDemo {
    static private final String accesskey = "AccessKey information";
    static private final String secret = "Secret information";
    static private final String host = "the host of region where the application to which the drop-down suggestions model is associated resides";
    OpenSearch openSearch;
    OpenSearchClient openSearchClient;
    static private final byte hits = 8; //The maximum number of recommended search queries in drop-down suggestions.
    static private final String suggestionName = "Name of the drop-down suggestions model"; //The name of the drop-down suggestions model.

    @Before
    public void setUp() {
        // Initialize the OpenSearch object.
        openSearch = new OpenSearch(accesskey, secret, host);
        openSearchClient = new OpenSearchClient(openSearch);
    }

    @Test
    public void TestEnv() {
        // Obtain the file encoding format and default encoding format.
        System.out.println(String.format("file.encoding: %s", System.getProperty("file.encoding")));
        System.out.println(String.format("defaultCharset: %s", Charset.defaultCharset().name()));

        // Create a drop-down suggestions object.
        SuggestionClient suggestionClient = new SuggestionClient(suggestionName, openSearchClient);
        String query = "search query";
try {
            SuggestParams suggestParams = new SuggestParams();
            suggestParams.setQuery(query); //Specify search queries.
            suggestParams.setHits(10); //Specify the maximum number of recommended search queries in drop-down suggestions.
            suggestParams.setUserId("12345678"); //Set the user ID.
            SearchResult result = suggestionClient.execute(suggestParams); 
            System.out.println(result); //Print the retrieved data.
        } catch (OpenSearchException e) {
            e.printStackTrace();
        } catch (OpenSearchClientException e) {
            e.printStackTrace();
        }
    }

    @After
    public void clean() {
        openSearch.clear();
    }
}

For more information about the drop-down suggestions SDK for Java, see Demo of the drop-down suggestions SDK.

Sample retrieved data:

{
  "request_id": "159851481919726888064081",
  "searchtime": 0.006246,
  "suggestions": [
    {
      "suggestion": "skirt fashion"
    },
    {
      "suggestion": "skirt shorty dress"
    },
    {
      "suggestion": "skirt dotted dress"
    },
    {
      "suggestion": "skirt young"
    },
    {
      "suggestion": "skirt dotted"
    },
    {
      "suggestion": "skirt shorty"
    },
    {
      "suggestion": "skirt shorty dotted"
    }
  ]
}

Note: The request ID that is returned with the retrieved data can be used to associate search requests.

Associate a drop-down suggestions request with a search request

Association of a drop-down suggestion request and a search request has the following benefits:

  1. You can obtain the data of metrics to evaluate how much the drop-down suggestions feature optimizes search results. For example, you can obtain the data of metrics, such as the PVs and click-through rate of search results that are intervened by drop-down suggestions, and the ratio of search engine results pages (SERPs) that are returned with few or zero results. For more information, see Drop-down suggestion report.

  2. After you associate a drop-down suggestions request with a search request, you can obtain the data of metrics such as the click data. You can optimize the related drop-down suggestions model based on such data to improve the search effect.

    Association method:

    If a search request of an OpenSearch application is based on drop-down suggestions, you must include from_request_id={from_request_id} in the request. The from_request_id variable specifies the source of the current search query. If the search query is recommended by the drop-down suggestions, top searches, or hints feature, pass the ID of the related feature request to the from_request_id parameter. After you associate the related feature request with the current search request, you can obtain the metrics of the feature to evaluate and improve the search effect. For more information about the from_request_id parameter, see Search and processing.

Example:

The request ID that is returned by a drop-down suggestions operation is 159851481919726888064081. The following code provides an example on how to associate the request ID with a search request:

SearchParams searchParams = new SearchParams(config);
searchParams.setQuery("title:'skirt shorty'"); //The search query is recommended by the drop-down suggestions feature.

// Add the from_request_id parameter.
Map<String, String> customParam =new HashMap<>();
customParam.put("from_request_id","159851481919726888064081");
searchParams.setCustomParam(customParam);

// Run the query statement to return the data object.
SearchResult execute = searcherClient.execute(searchParams);
// Return data in strings.
String result = execute.getResult();
System.out.println(result);