OpenSearch:Industry-specific Enhanced Edition for E-commerce: Multimodal search

The multimodal search feature of OpenSearch combines text searches and vector searches. Multimodal searches have lower search latency, consume fewer computing resources, and provide higher accuracy than the searches that use the OR logical operator. These benefits of multimodal searches have been verified in question-answering search scenarios. The multimodal retrieval architecture that is used by the multimodal search feature can also be used in scenarios such as vector-based image retrieval, expression-based retrieval, and personalized retrieval.

Why do you need multimodal searches?

• The multimodal search feature retrieves documents that are not retrieved after query analysis features such as synonym configuration and spelling correction are configured.

• The multimodal search feature retrieves more documents for long search queries.

Example:

After analysis, the search entry "beefjerky" is segmented into two terms: beef and jerky.

If you want to retrieve a document that contains "Tone River natural beefjerky", it cannot be retrieved based on the search entry "beefjerky" in a text search. This is because the content of the document is segmented into the following terms after analysis: Tone, River, natural, beef, and jerky. A text search retrieves documents based on the terms that are obtained after analysis. In this case, you can use a vector search to retrieve the document based on semantics.

1. Create an application of Industry-specific Enhanced Edition for E-commerce. After the application is created, check whether a vector index is configured. In this example, the vector index uses the Vector - General E-commerce Vector analyzer.

2. Create a query analysis rule. Select Text Vectorization to enable the word embedding feature and select the vector index that is configured in Step 1.

3. Configure sort policies. By default, the following two sort policies are created for the application: the rough sort policy sys_first_default and the fine sort policy sys_second_default.

4. Create a multimodal search policy by performing the following operations: Set the policy name, configure the vector search and text search features, and then configure the merge sort feature. To configure the merge sort feature, select a sort policy for merge sort and specify the number of documents to be sorted.

5. Perform a search test in the OpenSearch console.

In this example, the search query is "Search test", the multimodal search policy is sys_strategy, and the query analysis rule is sys_default.Note: All three parameters shown in the preceding figure must be specified. If the default query analysis rule is used, you can ignore the qp parameter. Search request in this example:

query=default:'Search test'&search_strategy=sys_strategy&raw_query=Search test&qp=sys_default

Perform a search test by using OpenSearch SDK. In this example, OpenSearch SDK for Java is used.

...
   
// Create a Config object that is used to specify the paging-related parameters or the data format of returned results in the config clause.
Config config = new Config(Lists.<String>newArrayList(appName));
config.setStart(0);
config.setHits(10);
 
// Set the data format of returned results to JSON.
config.setSearchFormat(SearchFormat.FULLJSON);
 
 
// Create a SearchParams object.
SearchParams searchParams = new SearchParams(config);
 
// Specify the input parameters in the search request.
HashMap<String,String> paraMap=new HashMap<String,String>();
// Specify the raw_query parameter.
paraMap.put("raw_query","Search test");
// Specify the multimodal search policy.
paraMap.put("search_strategy","sys_strategy");
searchParams.setCustomParam(paraMap);
 
// Specify the query analysis rule.
List<String> qpName = new ArrayList<String>();
qpName.add("sys_default");  // The name of the query analysis rule.
searchParams.setQueryProcessorNames(qpName);
 
...

Usage notes

• Only exclusive applications can use the multimodal search feature.

• The multimodal search feature does not support the aggregate or distinct clause.

• A maximum of 10 multimodal search policies can be created for each application.

• Both the text search and vector search features must be enabled for multimodal searches. When you specify the proportions of retrieval results for the text search and vector search features, the sum of the proportions must be 100%.

• When you configure the text search and vector search features, select rough and fine sort policies that are configured in Step 3.

• When you configure the merge sort feature, you can select a fine sort policy for merge sort. By default, this sort policy is not specified. The Number of Sort Documents parameter is required. You must specify a value in the range of 1 to 5000.

• A vector index that is not generated by the system cannot be selected when you configure the word embedding feature.

• If you want to use a custom vector index for multimodal searches, submit a ticket.