How to quickly integrate OpenSearch Vector Search Edition with PAI-RAG -

To improve the accuracy and richness of answers generated by a large language model (LLM), you can use Retrieval-Augmented Generation (RAG) to generate responses based on your private data. OpenSearch Vector Search Edition supports multiple vector retrieval algorithms and delivers excellent performance with high precision. This article describes how to use PAI-EAS and OpenSearch Vector Search Edition to quickly build a complete RAG conversational system.

Notes

This solution is limited by server resource size and the LLM service's default token limit, which restricts the conversation length. Its purpose is to help you experience the basic retrieval functions of the rag-based llm chatbot.

Create an OpenSearch Vector Search Edition instance

Create instance

Go to the OpenSearch consoleand switch to OpenSearch-Vector Search Edition in the upper-left corner.

On the instance list page, create an OpenSearch Vector Search Edition instance. The key parameters are described in the following table. For more information about other parameters, see Purchase an OpenSearch Vector Search Edition instance.

Parameter	Description
Edition	Select Vector Search Edition.
VPC	Select an existing VPC and vSwitch.
vSwitch	Select an existing VPC and vSwitch.
username	The username for the OpenSearch Vector Search Edition instance.
password	The password for the OpenSearch Vector Search Edition instance.

Configure index table

After you create the instance, it enters the Pending Configuration state. Follow the instructions in Configure an instance to create an index, after which you can begin searching. Key parameters are as follows:

Select the general-purpose template and import the following configuration file.

Field configuration file

{
	"schema": {
		"summarys": {
			"parameter": {
				"file_compressor": "zstd"
			},
			"summary_fields": [
				"id",
				"embedding",
				"file_path",
				"file_name",
				"file_type",
				"node_content",
				"node_type",
				"doc_id",
				"text",
				"source_type"
			]
		},
		"file_compress": [
			{
				"name": "file_compressor",
				"type": "zstd"
			},
			{
				"name": "no_compressor",
				"type": ""
			}
		],
		"indexs": [
			{
				"index_fields": [
					{
						"boost": 1,
						"field_name": "id"
					},
					{
						"boost": 1,
						"field_name": "embedding"
					}
				],
				"indexer": "aitheta2_indexer",
				"index_name": "embedding",
				"parameters": {
					"enable_rt_build": "true",
					"min_scan_doc_cnt": "20000",
					"vector_index_type": "Qc",
					"major_order": "col",
					"builder_name": "QcBuilder",
					"distance_type": "SquaredEuclidean",
					"embedding_delimiter": ",",
					"enable_recall_report": "true",
					"ignore_invalid_doc": "true",
					"is_embedding_saved": "false",
					"linear_build_threshold": "5000",
					"dimension": "1536",
					"rt_index_params": "{\"proxima.oswg.streamer.segment_size\":2048}",
					"search_index_params": "{\"proxima.qc.searcher.scan_ratio\":0.01}",
					"searcher_name": "QcSearcher",
					"build_index_params": "{\"proxima.qc.builder.quantizer_class\":\"Int8QuantizerConverter\",\"proxima.qc.builder.quantize_by_centroid\":true,\"proxima.qc.builder.optimizer_class\":\"BruteForceBuilder\",\"proxima.qc.builder.thread_count\":10,\"proxima.qc.builder.optimizer_params\":{\"proxima.linear.builder.column_major_order\":true},\"proxima.qc.builder.store_original_features\":false,\"proxima.qc.builder.train_sample_count\":3000000,\"proxima.qc.builder.train_sample_ratio\":0.5}"
				},
				"index_type": "CUSTOMIZED"
			},
			{
				"has_primary_key_attribute": true,
				"index_fields": "id",
				"is_primary_key_sorted": false,
				"index_name": "id",
				"index_type": "PRIMARYKEY64"
			},
			{
				"index_fields": "file_path",
				"index_name": "file_path",
				"index_type": "STRING"
			},
			{
				"index_fields": "file_name",
				"index_name": "file_name",
				"index_type": "STRING"
			},
			{
				"index_fields": "file_type",
				"index_name": "file_type",
				"index_type": "STRING"
			},
			{
				"index_fields": "node_content",
				"index_name": "node_content",
				"index_type": "STRING"
			},
			{
				"index_fields": "node_type",
				"index_name": "node_type",
				"index_type": "STRING"
			},
			{
				"index_fields": "doc_id",
				"index_name": "doc_id",
				"index_type": "STRING"
			},
			{
				"index_fields": "text",
				"index_name": "text",
				"index_type": "STRING"
			},
			{
				"index_fields": "source_type",
				"index_name": "source_type",
				"index_type": "STRING"
			}
		],
		"attributes": [
			{
				"file_compress": "no_compressor",
				"field_name": "id"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "embedding"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "file_path"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "file_name"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "file_type"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "node_content"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "node_type"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "doc_id"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "text"
			},
			{
				"file_compress": "no_compressor",
				"field_name": "source_type"
			}
		],
		"fields": [
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "id"
			},
			{
				"user_defined_param": {
					"multi_value_sep": ","
				},
				"multi_value": true,
				"compress_type": "uniq",
				"field_type": "FLOAT",
				"field_name": "embedding"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "file_path"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "file_name"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "file_type"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "node_content"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "node_type"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "doc_id"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "text"
			},
			{
				"compress_type": "uniq",
				"field_type": "STRING",
				"field_name": "source_type"
			}
		],
		"table_name": "abc"
	},
	"extend": {
		"description": [],
		"vector": [
			"embedding"
		],
		"embeding": []
	}
}

In the Index schema, ensure the vector dimension is consistent with the vector dimension of the model used for your knowledge base. For Distance type, we recommend selecting InnerProduct.

Enable public access

EAS can only access OpenSearch over the public network. This requires a VPC for your EAS service with an associated NAT Gateway and Elastic IP (EIP). You must then enable public access for the OpenSearch instance and add the EIP address to its public access whitelist. The following steps detail this process. The EAS service can use the same VPC as the OpenSearch instance or a different one.

Configure public network access for the VPC that you will use for the RAG service. For more information, see Use the SNAT feature of a NAT Gateway to access the internet.
View the associated EIP address.
1. Log on to the VPC console. Click the VPC instance ID, and then go to the Resource Management tab.
2. Click the instance ID of the associated NAT Gateway to open its details page. In the Internet access service section, find Internet NAT Gateway.
3. Click the NAT Gateway instance ID to go to the Basic information page.
4. Click Associated EIP and save the EIP address. The Associated EIP tab shows details such as the allocation status, bandwidth, and entry association status of the EIP.
On the OpenSearch Vector Search Edition instance list, click the target instance's name to open its instance details page.
In the Network Information section, turn on the Public Access switch. In the Modify Public Access Whitelist panel, follow the on-screen instructions to add your saved EIP address to the public access whitelist.
In the Network Information section, copy the address next to Public Endpoint and save it locally.

Deploy and connect the RAG service

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).
On the Inference Service tab, click Deploy Service. In the Scenario-based Model Deployment section, click RAG-based Smart Dialogue Deployment.
Configure the following key parameters. For information about other parameters, see Step 1: Deploy the RAG service.
- Basic Information:
  - Version selection: Select LLM-Integrated Deployment.
  - RAG Version: Select pai-rag:0.3.4.
  - Model Type: Select qwen1.5-1.8b.
- Resource Information: The system automatically recommends a resource specification based on the selected model. To prevent service startup failures, we recommend that you use the suggested specification.
- Vector Database Settings:
  - Vector Database Type: Select OpenSearch.
  - Endpoint: Enter the public endpoint of the OpenSearch Vector Search Edition instance. Do not include http:// or https://. For example: ha-cn-****.public.ha.aliyuncs.com.
  - Instance ID: Enter the ID of the OpenSearch Vector Search Edition instance.
  - Username and Password: Enter the username and password for the OpenSearch Vector Search Edition instance.
  - Table Name: Enter the name of the index table that you created in Configure index table.
  - OSS Path: Select an existing OSS storage directory in the current region. You can manage the knowledge base by mounting an OSS path.
  - VPC: Ensure the selected VPC has public network access and that its EIP is in the OpenSearch instance's public access whitelist. For more information, see Use the SNAT feature of a NAT Gateway to access the internet and Configure a public access whitelist.
After you confirm the parameters, click Deploy. The deployment is successful when the service status changes to Running, which takes about 3 minutes.

Use the RAG conversational system

The basic usage of the RAG conversational system is as follows. For more information, see Large Model RAG Conversational System (v0.3.x).

Check vector database configuration

Click the name of the target RAG service, and then click View Web App in the upper-right corner of the page.
Verify the OpenSearch vector database configuration.
The system automatically configures the default knowledge base and applies the vector database settings from the RAG service deployment. In the Vector database configuration section, verify the OpenSearch configuration. If necessary, correct the settings and click Update knowledge base.

Upload knowledge base files

On the Knowledge Base tab, click the File Management tab to upload knowledge base files.

After you upload the knowledge base, the system automatically stores the files in the vector database in the PAI-RAG format. If you upload a knowledge base file with the same name as an existing file, the original file is overwritten in all vector databases except Faiss. Supported file types include .html, .htm, .txt, .pdf, .pptx, .md, Excel (.xlsx or .xls), .jsonl, .jpeg, .jpg, .png, .csv, and Word (.docx), for example, rag_chatbot_test_doc.txt.

Knowledge-based Q&A

On the Chat tab, select a knowledge base and an intent. To use more tools, select Query Knowledge Base. You can then perform knowledge-based Q&A.

OpenSearch feature support

Alibaba Cloud OpenSearch Vector Search Edition provides a user-friendly graphical user interface (GUI) to efficiently manage index tables and indexes. This section describes how to use the OpenSearch Vector Search Edition console to view index information and perform basic data management.

Index table management

Go to the instance details page of your OpenSearch Vector Search Edition instance.
1. Log on to the Alibaba Cloud OpenSearch Vector Search Edition console.
2. Click the ID of the instance that you created to go to its instance details page.
Go to the table management page to manage the index table.
1. In the left-side navigation pane, click Table management.
  This page lists all tables created in the current instance. At the top of the page is an Add table button. The table list below shows information for each table, including its name, status, full data source, table size, active full version, full version effective time, incremental version effective time, and the number of data update resources. The Actions column provides links to View, Edit, and Rebuild index.
2. On the table management page, perform management operations on the index table, such as viewing its structure, and editing, rebuilding, or deleting indexes. For more information, see Table management.

Data management

Go to the instance details page of your OpenSearch Vector Search Edition instance.
1. Log on to the Alibaba Cloud OpenSearch Vector Search Edition console.
2. Click the ID of the instance that you created to go to its instance details page.
Add data.
1. In the left-side navigation pane, choose Vector Management > Insert Data.
2. From the drop-down list in the upper-right corner of the page, select Form Mode or Developer mode. Form mode lets you add a single entry, while developer mode lets you add multiple entries at once.
3. Select the name of the target index table to which you want to add data.
4. Enter data for each field or provide a data write statement, and then click Add. For more information, see Add data.
  If the execution result shows "message": "success", the data has been successfully added.
View table metric data. For more information, see Table metrics.
Delete data. For more information, see Delete data.