AI RAG retrieval proxy - API Gateway - Alibaba Cloud Documentation Center

This topic describes the features of the AI RAG retrieval proxy plug-in and how to use it to connect AI applications with external Retrieval-Augmented Generation (RAG) engines.

Features

The AI RAG retrieval proxy plug-in converts retrieval protocols and transforms retrieval results. This allows AI applications such as Dify to connect to and retrieve from external RAG engines such as RAGFlow and Alibaba Cloud Model Studio knowledge bases.

Runtime properties

Plug-in execution stage: default stage.
Plug-in execution priority: 150.

Configuration

Basic configurations

Name	Data type	Required	Default value	Description
`rag_proxy.provider.name`	string	Required	-	The type of the external knowledge base to call. Supported enumeration types are `ragflow` and `bailian`.
`rag_proxy.source.name`	string	Optional	dify	The type of the engine that calls the external knowledge base. The supported enumeration type is `dify`.
`rag_proxy.source.enable_to_proxy_api_key`	boolean	Optional	false	Specifies whether to enable API key authentication between the AI Gateway and the external knowledge base caller. The default value is false. If you enable this feature, the API keys must match. Otherwise, the service will not function correctly.
`rag_proxy.source.to_proxy_api_key`	string	Optional	-	The API key that you enter when you create the external knowledge base API in the caller engine. If you specify this parameter, AI Gateway verifies that this key matches the API key entered during API creation. Otherwise, no verification is performed.

RAGFlow configurations

If you select RAGFlow as the external knowledge base to use (rag_proxy.provider.name="ragflow"), use the following configurations.

Name	Data type	Required	Default value	Description
`rag_proxy.provider.api_key`	string	Required	-	The API key for calling the RAGFlow API. You can obtain the API key in the RAGFlow console by clicking your profile picture in the upper-right corner and choosing API > RAGFlow API.
`rag_proxy.provider.dataset_ids`	list[string]	Optional	-	The ID of the dataset to retrieve from RAGFlow. If you leave this blank, specify it when you configure the external knowledge base in Dify. The ID configured in Dify takes precedence.
`rag_proxy.provider.document_ids`	list[string]	Optional	-	The ID of the document to retrieve from RAGFlow. You can limit the retrieval to specific documents.
`rag_proxy.provider.similarity_threshold`	float	Optional	0.0	The similarity threshold. Segments with a similarity score below this threshold are filtered out. You can configure this parameter on the retrieval page in Dify.
`rag_proxy.provider.top_n`	integer	Optional	30	The number of segments with the highest similarity scores to return. Other segments are filtered out. You can configure this parameter on the retrieval page in Dify.
`rag_proxy.provider.vector_similarity_weight`	float	Optional	0.3	The weight of the vector cosine similarity. If x represents the vector cosine similarity, then `(1-x)` represents the semantic similarity weight.
`rag_proxy.provider.keyword`	boolean	Optional	false	Specifies whether to enable keyword-based matching. `true`: Enables keyword-based matching. `false`: Disables keyword-based matching (default).
`rag_proxy.provider.rerank_id`	integer	Optional	-	The ID of the rerank model configured in the RAG engine. After configuration, this model is used to enable the rerank feature. For model selection, see the rerank models supported by RAGFlow. This feature is disabled by default.

For instructions on how to use RAGFlow knowledge bases, see Configure knowledge base.

For more information about RAGFlow retrieval parameters, see Retrieve chunks.

The following is a basic configuration example:

rag_proxy:
  provider:
    name: 'ragflow'
    api_key: "xxxxxxxx"

Model Studio Knowledge Base Configuration

If you select an Alibaba Cloud Model Studio knowledge base as the external knowledge base to use (rag_proxy.provider.name="bailian"), use the following configurations.

Name	Data type	Requirements	Default value	Description
`rag_proxy.provider.api_key`	string	Required	-	The API key for the Alibaba Cloud Model Studio platform. To obtain the key, go to the Model Studio API-KEY page.
`rag_proxy.provider.index_id`	string	Optional	-	The ID of the Alibaba Cloud Model Studio knowledge base. For information about how to obtain the ID, see Knowledge Base API Guide.
`rag_proxy.provider.enable_reranking`	bool	Optional	true	Specifies whether to enable the rerank feature.
`rag_proxy.provider.rerank_min_score`	float	Optional	The similarity threshold configured for the current knowledge base	This parameter takes effect only when reranking is enabled. It is the similarity threshold after reranking. Segments with a score below this threshold are filtered out. The value must be in the range of `[0.01-1.00]`.
`rag_proxy.provider.rerank_top_n`	integer	Optional	5	This parameter takes effect only when reranking is enabled. It specifies the number of top segments to return after reranking. The value must be in the range of `[1-20]`.
`rag_proxy.provider.save_retriever_history`	bool	Optional	false	Specifies whether to save historical retrieval data.
`rag_proxy.provider.dense_similarity_top_k`	integer	Optional	100	The number of top results for vector retrieval (TopK). This feature generates a vector for the input text and retrieves the K most similar text segments from the knowledge base. The value must be in the range of `[0-100]`. The sum of `dense_similarity_top_k` and `sparse_similarity_top_k` must be 200 or less.
`rag_proxy.provider.sparse_similarity_top_k`	integer	Optional	100	The number of top results for keyword retrieval (TopK). This feature finds segments in the knowledge base that exactly match the keywords in the input text. It helps filter out irrelevant text segments and provides more accurate results. The value must be in the range of `[0-100]`. The sum of `dense_similarity_top_k` and `sparse_similarity_top_k` must be 200 or less.

For instructions on how to use Alibaba Cloud Model Studio knowledge bases, see Alibaba Cloud Model Studio knowledge base operations and usage guide.

For more information about Model Studio retrieval parameters, see Retrieve - Retrieve from a knowledge base.

The following is a basic configuration example:

rag_proxy:
  provider:
    name: 'bailian'
    api_key: 'sk-xxxxxx'

Procedure

This section uses the connection of a Dify external knowledge base to RAGFlow and an Alibaba Cloud Model Studio knowledge base as an example to describe the operations and configuration steps for the plug-in.

Connect Dify to RAGFlow using the AI RAG retrieval proxy plug-in

Create an RAGFlow retrieval service in AI Gateway.
1. Go to the Instances page of the AI Gateway console. In the top menu bar, select the region where your target instance is located, and then click the target instance ID.
2. In the navigation pane on the left, choose Services, and then click Create Service.
3. This example uses an RAGFlow instance with a fixed address. Create an RAGFlow retrieval service, set the service source to a fixed address as shown in the following figure, and set the port to 80. If your RAGFlow instance is deployed in a container within the same VPC, you can also use Container Service as the service source.
Create an Agent API route.
1. Create a custom Agent API.
  In the console for the gateway instance, choose Agent API, and then click Create Agent API. Set Protocol to Custom.
2. Create an Agent API route.
  Go to the Agent API that you created and click Create Route. Make sure the path ends with `/retrieval`. For Service, select the RAGFlow retrieval service that you created in Step 1.
Obtain the API key and configure the plug-in in AI Gateway.
1. Go to RAGFlow. In the upper-right corner, click your profile picture, and then choose API > API KEY to obtain the API key.
2. In the gateway instance console, choose Plug-ins and click Install Plug-in. Set Plug-in Type to AI, select the AI RAG proxy plug-in, and then click Install and Configure to open the rule configuration page. Configure the plug-in as shown in the following figure, enable it, and then click Save to apply the changes.
Create an external knowledge base API in Dify.
Go to the Dify console. Click Knowledge Base > External Knowledge Base API > Add External Knowledge Base API. For API Endpoint, use a combination of the gateway endpoint and the Agent API route path from the previous steps (http(s)://{endpoint}/{agent api path}). Make sure to remove the `/retrieval` suffix. You can define the API Key as needed.
Important
If you configured `rag_proxy.source.enable_to_proxy_api_key` and `rag_proxy.source.to_proxy_api_key` in the plug-in, make sure the API Key matches the value of `rag_proxy.source.to_proxy_api_key`. Otherwise, Dify will fail to authenticate when accessing AI Gateway, and subsequent routing will fail.
Connect to the external knowledge base in Dify.
1. Obtain the knowledge base namespace ID.
  Go to the RAGFlow knowledge base page and select the knowledge base to retrieve from. The ID of the external knowledge base is part of the page URL, as shown in the following figure.
2. Configure the knowledge base information.
  Go to the Dify console and click Knowledge Base > Connect External Knowledge Base. You can customize the knowledge base name and description. For External Knowledge Base API, select the API that you created in Step 4. The following figure shows the configuration. Click Connect to complete the configuration.
Verify the retrieval connection.
Go to the Dify knowledge base page and select the knowledge base that you created. In the retrieval test, enter the source text to verify the connection. If text segments are returned based on the retrieval settings, the connection is successful.

Connect Dify to an Alibaba Cloud Model Studio knowledge base using the AI RAG retrieval proxy plug-in

Create a Model Studio retrieval service in AI Gateway.
1. Go to the Instances page of the AI Gateway console. In the top menu bar, select the region where your target instance is located, and then click the target instance ID.
2. In the navigation pane on the left, choose Services, and then click Create Service. Set Service Source to DNS Domain Name. The following figure shows the configuration.
Create a custom Agent API route in AI Gateway.
1. Create a custom Agent API.
  In the navigation pane on the left, choose Agent API, and then click Create Agent API. The following figure shows the configuration. You can customize the domain name and Base Path as needed. Set Protocol to Custom.
2. Create an Agent API route.
  Go to the Agent API that you created and click Create Route. Make sure the path ends with `/retrieval`. For Service, select the Model Studio retrieval service that you created in Step 1.
Obtain the API key and configure the plug-in in AI Gateway.
1. Log on to the Alibaba Cloud Model Studio platform, go to the API Key page, and obtain an API key.
2. In the gateway instance console, choose Plug-ins and click Install Plug-in. Set Plug-in Type to AI, select the AI RAG proxy plug-in, and then click Install and Configure to open the rule configuration page. Configure the plug-in as shown in the following figure, enable it, and then click Save to apply the changes.
Connecting Dify to an external knowledge base.
Go to the Dify console. Click Knowledge Base > External Knowledge Base API > Add External Knowledge Base API. For API Endpoint, use a combination of the gateway endpoint and the Agent API route path from the previous steps (http(s)://{endpoint}/{agent api path}). Make sure to remove the `/retrieval` suffix. You can define the API Key as needed.
Important
If you configured `rag_proxy.source.enable_to_proxy_api_key` and `rag_proxy.source.to_proxy_api_key` in the plug-in, make sure the API Key matches the value of `rag_proxy.source.to_proxy_api_key`. Otherwise, Dify will fail to authenticate when accessing AI Gateway, and subsequent routing will fail.
You can create an external knowledge base in Dify.
1. Obtain the Model Studio knowledge base ID.
  Go to the Alibaba Cloud Model Studio Knowledge Base page. Select the knowledge base to retrieve from and obtain its ID, as shown in the following figure.
2. Configure the knowledge base information.
  Go to the Dify console and click Knowledge Base > Connect External Knowledge Base. You can customize the knowledge base name and description. For External Knowledge Base API, select the API that you created in Step 4. The format for the external knowledge base ID is `{Knowledge Base ID}`. The following figure shows the configuration. Click Connect to complete the connection.
Verify the retrieval connection.
Go to the Dify knowledge base page and select the knowledge base that you created. In the retrieval test, enter the source text to verify the connection. If text segments are returned based on the retrieval settings, the connection is successful.