Retrieval-Augmented Generation (RAG) enhances a large language model's (LLM) ability to answer questions about private, domain-specific knowledge. It retrieves relevant information from an external knowledge base and provides it as context to the LLM with the user's query. EAS provides a scenario-based deployment option to quickly build and deploy a rag-based llm chatbot, letting you select your preferred large language model and vector database. This topic shows you how to deploy a rag-based llm chatbot service and verify its inference.
This topic applies to RAG versions 0.3.4 and 0.3.5. We recommend that you use the latest version.
Step 1: Deploy the RAG service
-
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Service tab, click Deploy Service. In the Scenario-based Model Deployment section, click RAG-based Smart Dialogue Deployment.
On the RAG-based LLM Chatbot Deployment page, configure these parameters:
Version: The RAG service supports two deployment modes. Select the mode that fits your use case and resources.
LLM-Integrated Deployment: Deploys the RAG service and a large language model (such as Qwen) in a single EAS service instance. This mode is simple to configure and ideal for quick validation and prototyping.
LLM-Separated Deployment: Deploys only the RAG service, while the large language model runs as a separate service. This mode allows the RAG service to connect to different LLM services, such as other LLMs deployed on EAS or Alibaba Cloud Model Studio services. It facilitates resource reuse and independent scaling and is suitable for production environments or when an LLM service already exists.
Deployment: Allocate appropriate compute resources for the service.
LLM-Integrated Deployment: The system automatically recommends a resource specification based on the selected Model Category. A specification lower than the recommendation may prevent the service from starting.
LLM-Separated Deployment: The RAG service itself consumes minimal resources. We recommend a specification with at least an 8-core CPU and 16 GB of memory, such as
ecs.g6.2xlargeorecs.g6.4xlarge.
Vector database settings:
Vector Database Type: Select FAISS to build a local vector database for quick practice. For production environments, we recommend using a production-grade vector database. For details, see Use an Alibaba Cloud vector database.
OSS Path: Select an existing Object Storage Service (OSS) directory in the current region to store uploaded knowledge base files. If no storage paths are available, see Console Quick Start to create one.
NoteIf you deploy a service with a self-hosted fine-tuned model, ensure that the selected OSS storage path does not conflict with the model's storage path.
VPC: Configure EAS to access public or private resources to ensure that the RAG service can communicate properly with the LLM service, vector database, and other cloud services.
Access public services: To access a vector database, an LLM service (such as Alibaba Cloud Model Studio), or web search over the internet, you must configure public network access for the EAS service.
Access private services:
To access a vector database by using a private endpoint, the RAG service and the vector database must be in the same VPC.
To access an LLM service by using a private endpoint, the RAG service and the LLM service must be in the same VPC.
NoteHologres, Elasticsearch, and RDS PostgreSQL support access over both private and public networks. We recommend using private network access.
The FAISS vector database does not require network access. OpenSearch supports access only over the public network.
After you configure the parameters, click Deploy. The service is successfully deployed when the Service Status changes to Running. The deployment typically takes about 5 minutes, but the duration may vary depending on the model size and other factors.
Step 2: Call the API
The RAG service provides APIs for tasks like knowledge base management and conversation, allowing for easy integration into your applications. For details, see RAG API reference.
Costs and risks
Cost components
The costs for this solution primarily include:
EAS resource fees: The computing resources (such as vCPUs, memory, and GPUs) consumed by the RAG service are billed on an hourly basis. Billing for these resources stops after you stop the service.
Vector database fees: If you use Elasticsearch, Hologres, OpenSearch, or RDS PostgreSQL, you are charged instance fees for the corresponding products.
Object Storage Service (OSS) fees: These fees are incurred for storing the original knowledge base files.
Large language model call fees: If you use commercial model services such as Alibaba Cloud Model Studio, API call fees apply.
Internet NAT Gateway fees: If an EAS service needs to access the internet, it incurs NAT Gateway fees.
Web search service fees: If you enable the web search feature (such as Bing), fees are incurred for the corresponding search services.
Stopping an EAS service only stops billing for EAS resources. To stop billing for other products, refer to the documentation for those products and follow the instructions to stop or delete the relevant instances.
Key risks and limitations
Conversation length limit: The amount of text processed in a single conversation is limited by the context window size (token limit) of the selected LLM service.
File overwriting: When you use a vector database other than FAISS, uploading a file with the same name overwrites existing data. Proceed with caution.
API parameter limitations: Only a subset of feature parameters can be configured through API calls. Other advanced configurations, such as most retrieval tuning parameters, must be configured in the WebUI.
Usage notes
This solution is limited by server resource size and the LLM service's default token limit, which restricts the conversation length. Its purpose is to help you experience the basic retrieval functions of the rag-based llm chatbot.
FAQ
Ineffective API parameters
Currently, the PAI-RAG service only supports configuring the parameters listed in the API reference document through the API. Other parameters must be configured in the WebUI.