OpenSearch:Best practices for vectorized image retrieval

Constraints

Review these constraints before you start:

For low-latency retrieval, consider the mmap index loading strategy.

Choose an architecture

Three architecture patterns are available:

This guide uses the OSS + MaxCompute + OpenSearch pattern.

Prerequisites

Before you begin, make sure you have:

• An Alibaba Cloud account with an AccessKey ID and AccessKey Secret. For RAM user access, see RAM user creation and authorization.

• An Object Storage Service (OSS) bucket with images uploaded: This guide uses 1,000 images uploaded to /test/images/ in a bucket named test-image-vector.

  1. Activate the OSS service

  2. Create a bucket

  3. Upload objects

• An OpenSearch Retrieval Engine Edition instance. See Purchase an OpenSearch Retrieval Engine Edition instance.

Step 1: Configure tables

A newly purchased instance shows a status of Pending Configuration. An empty cluster matching your purchased node count and specifications is automatically deployed. Complete the following configuration to enable search.

{
  "content_type": "oss",
  "oss_endpoint": "oss-cn-hangzhou-internal.aliyuncs.com",
  "oss_bucket": "test-image-vector",
  "crop": "true",
  "oss_use_slr": "true",
  "uid": "<your-alibaba-cloud-uid>"
}

{
  "vector_model": "clip",
  "vector_modal": "image",
  "vector_source_field": "vector_source_image"
}

"fields": [
  {
    "field_name": "id",
    "field_type": "INT64",
    "compress_type": "equal"
  },
  {
    "user_defined_param": {
      "oss_endpoint": "oss-cn-hangzhou-internal.aliyuncs.com",
      "oss_bucket": "/opensearch",
      "crop": "true",
      "content_type": "oss",
      "oss_use_slr": "true",
      "uid": "xxx"
    },
    "field_name": "source_image",
    "field_type": "STRING",
    "compress_type": "uniq"
  },
  {
    "field_name": "cate_id",
    "field_type": "INT64",
    "compress_type": "equal"
  },
  {
    "user_defined_param": {
      "vector_model": "clip",
      "vector_modal": "image",
      "vector_source_field": "vector_source_image"
    },
    "field_name": "vector",
    "field_type": "FLOAT",
    "multi_value": true
  }
]

Step 2: Rebuild the index

Click Confirm to create the configuration. Monitor progress in Function Extension > Change History. Once complete, the instance is ready for queries.

Step 3: Run search queries

query=image_index:'<search-content>&modal=<text|image>&n=<top-n>&search_params={}'&&kvpairs=formula:proxima_score(vector)&&sort=+RANK

vector:'motorcycle helmet&modal=text&n=10&search_params={}'&&kvpairs=formula:proxima_score(vector)&&sort=+RANK

vector:'<base64-encoded-image>&modal=image&n=10&search_params={}'&&kvpairs=formula:proxima_score(vector)&&sort=+RANK

import base64

def image_to_base64(file_path):
    with open(file_path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

base64_image = image_to_base64("/path/to/your/image.png")

Step 4: Search with the SDK

Install the SDK:

pip install alibabacloud-ha3engine

The following example submits a text-based image search request:

# -*- coding: utf-8 -*-

from alibabacloud_ha3engine import models, client
from alibabacloud_tea_util import models as util_models
from Tea.exceptions import TeaException, RetryError

def search():
    config = models.Config(
        endpoint="<api-endpoint>",        # From the API entry section on the instance details page
        instance_id="",
        protocol="http",
        access_user_name="<username>",    # Set when purchasing the instance
        access_pass_word="<password>"     # Set when purchasing the instance
    )

    # Increase timeout values for long-running requests (in milliseconds)
    runtime = util_models.RuntimeOptions(
        connect_timeout=5000,
        read_timeout=10000,
        autoretry=False,
        ignore_ssl=False,
        max_idle_conns=50
    )

    ha3_client = client.Client(config)

    try:
        query_str = (
            "config=hit:4,format:json,fetch_summary_type:pk,qrs_chain:search"
            "&&query=image_index:'motorcycle helmet&modal=text&n=10&search_params={}'"
            "&&cluster=general"
        )
        search_query = models.SearchQuery(query=query_str)
        request = models.SearchRequestModel({}, search_query)
        response = ha3_client.search(request)
        print(response)
    except TeaException as e:
        print(f"Request failed with TeaException: {e}")
    except RetryError as e:
        print(f"Request failed with connection error: {e}")

Replace the following placeholders:

For more SDK examples, see the Developer Guide.

What's next

• MaxCompute data source configuration

• Index table loading strategies (mmap)

• Developer Guide