OpenSearch:Quick start: Text vector search

Purchase an instance

To purchase an instance, see Purchase an OpenSearch Vector Search Edition instance.

Configure the cluster

On the instance list page, the status of a newly purchased instance is Pending Configuration. In the Actions column, click Configure to go to the instance details page. Configure the cluster by completing the following steps: Basic Table Information > Data Synchronization > Field Configuration > Index Schema > Confirm Creation. After the index is rebuilt, you can start searching.

1. Basic table information

In Table Management, click →Add Table to open the Basic Table Information page. Enter the Table Name, set the Number of Data Shards and Number of Data Update Resources, and set the scenario template to Vector: Text Semantic Search - Dense Vector Retrieval. Check the settings, then click Next.

Notes on configuration:

• Table name: You can specify a custom name.

• Data shards: Enter a positive integer up to 256. Sharding improves full indexing speed and single-query performance. For some existing instances, ensure all tables have the same number of data shards. Alternatively, one table can have a single data shard, while all other tables must have an identical number of shards.

• Number of data update resources: The number of resources used for data updates. By default, each index provides two free resources, each with 4 vCPUs and 8 GB of memory. Resources exceeding the free quota incur charges. For more information, see Billing overview of OpenSearch Vector Search Edition

• Scenario template: OpenSearch Vector Search Edition provides three built-in templates: General, Vector - Image Search, and Vector - Text Semantic Template.

2. Data synchronization

Select a Full data source. Supported data sources include MaxCompute+API, Object Storage Service (OSS) + API, Synchronize vector data from OpenLake-DLF to Alibaba Cloud OpenSearch, and API data source. This topic uses MaxCompute + API as an example. Configure your AccessKey, AccessKey Secret, Project, Table, Partition. Enable Automatic reindexing if needed. After configuration, click Validate in the Data Source Validation section. You can proceed only after the validation is successful.

3. Field configuration

OpenSearch Vector Search Edition pre-populates fields based on the selected scenario template and automatically imports fields from the full data source to the field list. You can also manually add fields as needed.

Preset fields for the text semantic search template:

• id (primary key)

• source_text (source text)

• namespace (namespace)

• source_text_vector (the text field to be vectorized)

Notes on field configuration:

• Required fields: The primary key field and the vector field. The primary key field must be of the int or string type and marked as the primary key. The vector field must be of the float type and marked as a vector field.

• By default, the vector field is a multi-value FLOAT type whose values are separated by commas. You can also specify a custom multi-value separator.

• Select "Embedding Required" for the text field to vectorize.

• Define the fields for vector search in the following order: primary key field, namespace field (optional), and vector field.

• Click Configure for preprocessing to configure data preprocessing for source_text.

Data preprocessing settings for source_text:

• Preprocessing template: Available preprocessing templates depend on the selected scenario template and data type. Because the Text Semantic Search - Dense Vector Retrieval scenario is selected, the available templates are Dense Vectorization and Dense + Sparse Vectorization.

• Service list: After you select a preprocessing template, the Service list for that template is displayed automatically.

• Model:

  • The text vectorization service offers two types of models: Built-in and AI Platform. Built-in models are free to use. AI Platform models from the AI Search Open Platform are a paid service, and their charges are billed separately from OpenSearch Vector Search Edition. For billing details, see Billing methods and billable items of AI Search Open Platform.

  • To use a model from the AI Search Open Platform, you must first create an AI Search Open Platform workspace and an API key.

• Generated field:

  • For services that involve embedding, the generated field is created by default and cannot be deselected or deleted.

• API call configuration: Because you are using a model service from the AI Search Open Platform, you must configure the service API call. Click Configure in the API Call Configuration section. On the configuration page that appears, set the Workspace, API Key, Endpoint. These settings need to be configured only once per table.

4. Index schema

OpenSearch automatically builds indexes on the primary key and vector fields. The index names are the same as the field names. You only need to configure the vector index in the console.

Set the Vector dimension (768 in this example), Distance type (for example, SquaredEuclidean), Vector index algorithm (for example, Qc), and Real-time index (for example, true).

Advanced configuration: Configure as needed. For details, see General vector index configuration.

5. Confirm creation

Click Confirm Creation to create the table. In the left-side navigation pane, choose Logs > Change History to view creation progress. After the status becomes Completed, you can test queries.

After you confirm, the system automatically starts building the index. This process takes about 2 minutes. Once complete, the index is automatically enabled in the current instance.

6. Query test

In the left-side navigation pane, choose Vector Management>Query Test to query data, Query Test supports both Form Mode and Developer Mode.

6.1 Form mode

Select Form mode>Table name, Query type, you can choose Vector or Text. This example uses the Text type. Enter your query content in the Text field and click Search. The results are displayed in the Search result section.

6.2 Developer mode

Select Developer mode>Table name. Enter the query parameters in the search box below. Click Search to view the search results in the Results pane on the right.

For detailed query syntax, see the syntax reference below.

Syntax reference

• Text prediction query

• Primary key-based query

• Filter expressions

Vector search with an SDK

• Query data (Perform vector and primary key queries using an SDK)

• Update data (Add or delete records using an SDK)