OpenSearch:Single-node quick start

Purchase an instance

1. Go to the OpenSearch console. In the upper-left corner, select OpenSearch Vector Search Edition. On the instance list page, click Create Instance.

2. Set Product Version to Vector Search Edition. Select a region. Configure the Number of Data Nodes, Data Node Specifications, and Total Storage Space per Data Node. Set the Virtual Private Cloud and Virtual Switch. Set a username and password as prompted. These credentials are used for permission verification during queries and are not the same as your Alibaba Cloud account password. Click Purchase Now.

   Note

   • The number and specifications of data nodes to purchase depend on your business needs. After you determine the specifications, the actual cost is automatically calculated on the product page.

   • For a single-node solution, you do not need to purchase QRS workers. Set the number of QRS workers to 0. The QRS worker specification family parameter will not take effect.

   • The configurations of the virtual private cloud (VPC) and virtual switch must be the same as those of the ECS instance that accesses the Vector Search Edition instance. Otherwise, a `{'errors':{'code':'403','message':'Forbidden'}}` error occurs when you try to access the instance.

   • The SSD-based data node family includes a free quota of 50 GB. Additional storage is billed in 50 GB increments. The cloud disk type has no free quota and is also billed in 50 GB increments.

3. On the Confirm Order page, review the Terms of Service and click Enable Now.

4. After the purchase is successful, click Management Console to view the purchased Vector Search Edition instance on the Instance List page.

Configure the instance

For a new instance, its status is Pending Configuration on the details page. An empty instance with the number and specifications of data nodes that you purchased is automatically deployed. You must then configure the basic table information, data synchronization, field configuration, and index schema for the instance. You can perform searches after the index is built.

1. Basic table information

In Table Management, click Add Table. Enter a Table Name. Set the Number of Data Shards and the Number of Data Update Resources. Select a Scenario Template and click Next.

Configuration description:

• Table Name: You can customize the name.

• Number of Data Shards: For a single-node instance, the value is fixed at 1 and cannot be changed. This instance supports only a single-shard solution. You can scale out the instance to a multi-node deployment. For more information, see the scaling document.

• Number of Data Update Resources: Specifies the number of resources used for data updates. Each index includes two free update resources of 4-core 8 GB. Resources that exceed the free quota are charged. For more information, see Billing overview for Vector Search Edition.

• Scenario Template: Vector Search Edition provides three built-in templates: General, Vector - Image Search, and Vector - Text Semantics.

2. Data synchronization

Select a source for full data. Supported data sources include MaxCompute+API, Object Storage Service (OSS)+API, OpenLake-DLF Vector Data Synchronization to Alibaba Cloud OpenSearch, and API. This topic uses MaxCompute+API as an example. After you configure the parameters and the configuration passes verification, click Next.

After selecting MaxCompute+API, fill in the following parameters:

• AccessKey and AccessKey Secret

• Project Name (Project): Must start with a letter, contain only letters, digits, and underscores, and be 3–28 characters long.

• Table Name (Table): 1–128 characters long.

• Table Partition (Partition): If the data source table has multiple partition levels, separate the partition values with commas.

• Auto Index Rebuild: Select Yes. Ensure that a done table has been created in MaxCompute; otherwise, auto index rebuild will not take effect.

After filling in the parameters, click Verify to validate the data source.

3. Field configuration

OpenSearch pre-configures relevant fields based on the scenario template that you select. OpenSearch also automatically imports fields from the full data source into the field list.

Field configuration description:

• Required fields: A primary key field and a vector field. The primary key field must be of the int or string type, and you must select the Primary Key checkbox. The vector field must be of the float type, and you must select the Vector Field checkbox.

• By default, the vector field is a multi-value float type, and its values are split by a comma (,), which is the system's default separator. You can also specify a custom multi-value separator.

• If a field is missing or empty in the data, the system automatically populates it with a default value. The default value is 0 for numeric types and an empty string for string types. You can also customize the default value.

4. Index schema

4.1. Vector index

In the Vector Index tab, enter an Index Name (for example, message_vector) and choose whether to enable Hybrid Search. A note at the top of the page indicates that OpenSearch has automatically built an index with the same name for all fields except the vector field.

• The primary key field and vector field are required. The namespace field is optional and can be left empty.

• You can only select from these three fixed fields. You cannot add new ones.

• Vector Dimensions: Select the dimension based on the vectors generated by the model.

• Distance Type: Supports InnerProduct (the higher the score, the more relevant) and SquareEuclidean (the lower the score, the more relevant).

• Vector Index Algorithm: Supports Linear, HNSW, QGraph, and QC. GPU instance types also support the CAGRA algorithm. For more information about the algorithms, see Vector introduction.

• Real-time Index: Specifies whether to build a vector index in real-time for incremental data from the API. The default value is true.

• For more advanced configurations, click to expand the section. For parameter descriptions, see Vector index general configurations.

  The advanced configuration includes the following parameters:

  • Real-time Index Parameters: Default value {"proxima.oswg.streamer.segment_size":2048}

  • Index Search Parameters: Default value {"proxima.qc.searcher.scan_ratio":0.01}

  • Index Build Parameters: Default value {"proxima.qc.builder.quantizer_class":"Int8Quant"...}

  • Minimum Recall Candidate Set Size: Default value 20000

  • Linear Build Threshold: Default value 5000

  • Ignore Problematic Vector Data: Default value true

  • Recall Rate Metric Reporting: Default value true

4.2. Other index types

OpenSearch automatically creates an index with the same name for each field except the vector field. You can manage these indexes here. You must configure the vector index manually.

In the Other Index Types tab, the index configuration table contains the following columns: Index Name, Included Fields, Build Index (toggle switch), and Index Type (primary key index or regular index). You can enable or disable index building for each field and select the corresponding index type based on your business needs.

4.3. Global index configuration

This feature is disabled by default. If you enable it, you can set a time-to-live (TTL) for integer fields. If the time elapsed since the document was created exceeds the TTL, the document is automatically deleted.

The configuration options include Document Time Field (select an integer-type field that represents the document time and specify the time unit) and Time-to-Live (value range: 1–49710 days).

5. Confirm creation

After you finish the index configuration, click Confirm Creation. The table status changes to Creating.

6. Change history

In Instance Management > Change History, you can view change records from the last 3, 7, or 30 days. These records include all processes, such as creating tables, adding indexes, and scaling. After all steps are complete and the DPI engine setup is finished, you can start query testing.

7. Query test

After you create a table and add data, you can run queries on the Vector Management > Query Test page. The query test supports both form mode and developer mode.

7.1. Form mode

Select Form mode, a Table Name, and a Query type in that order. For the Query type, you can select Vector or Text. In this example, Text is selected. Enter the query text in the text field. Click Search to view the results in the Search Result section.

7.2. Developer mode

Select Developer mode and a Table Name. Then, enter the query parameters in the Search Box. Click Search to view the results in the Search Result section.

• vector: The specific vector to query.

• topK: The number of top K results to return.

• includeVector: Specifies whether to return the vector information in the document.

For detailed query syntax, see the syntax descriptions below.

Syntax description

• Vector query

• Primary key query

• filter expression

Use vector search in an SDK

• Query data (query vectors and primary keys)

• Update data (add and delete records)

Upgrade from a single node to a multi-node deployment

To upgrade a single-node instance to a multi-node deployment, follow these steps:

1. On the Instance List page, find the target instance. In the Operation column, click Upgrade/Downgrade.

2. On the Upgrade/Downgrade page, select the number and specifications of QRS workers for scaling. Select the Terms of Service checkbox and click Purchase Now.