This topic describes how to use LangStudio to deploy a "Retrieval-Augmented Generation (RAG)- and web search-based chatbot" application flow. The application flow integrates real-time web search and RAG capabilities, providing models with additional web search and domain-specific knowledge base capabilities. This enables the models to deliver more accurate answers to user questions. Developers can perform flexible extension and secondary development by using the application flow template to meet specific scenario requirements.
Background information
In the rapidly evolving context of modern information technology, the "RAG- and web search-based chatbot" application flow combines the advantages of real-time web search and generative artificial intelligence to provide users with more comprehensive and accurate answers. By integrating real-time web search and RAG capabilities, models can obtain specialized information from domain-specific knowledge bases and utilize the latest web data for real-time updates. The combination of dual information sources makes an application particularly effective in fields such as finance and healthcare, where highly accurate information is required. Developers can flexibly extend and customize this application flow template to better meet the needs of different application scenarios, thereby enhancing user experience and decision support capabilities.
Prerequisites
You have registered an account at the SerpApi official website and obtained an API key (the free version provides 100 searches per month).
LangStudio supports Faiss or Milvus vector databases. If you want to use a Milvus database, you must create the Milvus database first.
NoteFaiss is usually used in test environments and can be used without creating an additional database. In production environments, we recommend that you use Milvus databases, which support processing larger volumes of data.
You have uploaded the RAG knowledge base corpus to Object Storage Service (OSS).
1. (Optional) Deploy an LLM and an embedding model
The "RAG- and web search-based chatbot" application flow requires large language model (LLM) and embedding services. This section describes how to quickly deploy the required models in Model Gallery. If you already have model services that meet the requirements and support the OpenAI API, skip this step and use your existing services directly.
Choose QuickStart > Model Gallery and deploy models for the following two scenarios. For more information about deployment, see Deploy and train models.
Make sure that you use LLMs that support instruction fine-tuning. Base models cannot correctly follow user instructions to answer questions.
Select large-language-model for Scenarios. In this example, DeepSeek-R1 is used. For more information, see One-click deployment of DeepSeek-V3 and DeepSeek-R1.
Select embedding for Scenarios. In this example, bge-m3 embedding model is used.
2. Create connections
In this topic, the Elastic Algorithm Service (EAS) services deployed by using the LLM and embedding model in QuickStart > Model Gallery are used to create connections. For information about other types of connections and details, see Create a connection.
2.1 Create an LLM service connection
Go to LangStudio, select a workspace, and then click Enter LangStudio. Then, click the Connection tab. On the Model Service tab of the Connection tab, click New Connection.
The following table describes the key parameters.
Parameter | Description |
Model Name | If you deploy a model in Model Gallery, click the model to enter the model details page and then view the method for obtaining the model name. For more information, see Create a connection. |
Service Provider |
|
2.2 Create an embedding model service connection
The method for creating an embedding model service connection is similar to that for creating an LLM service connection. This section describes how to create a general embedding model service connection.
2.3 Create a SerpApi connection
On the Model Service tab of the Connection tab, click the Custom Connection tab. Then, click New Connection. On the New Connection page, set the Custom Key parameter to api_key and the Custom Value parameter to the API key that you obtained in Prerequisites.
3. Create a knowledge base index
Create a knowledge base index to store the analyzed, chunked, and vectorized corpus to a vector database, thereby building a knowledge base. The following table describes the key parameters. For information about other parameters, see Manage knowledge base indexes.
Parameter | Description |
Basic Configurations | |
Data Source OSS Path | The OSS path of the RAG knowledge base corpus that you uploaded in Prerequisites. |
Output OSS Path | Configure the path for storing intermediate results and index information generated by document parsing. Important When you use FAISS as a vector database, the application flow saves the generated index file to OSS. If you use the default PAI role, the Instance RAM Role value configured when you start the runtime for application flow development), the application flow accesses the default OSS bucket of the user workspace. Therefore, we recommend that you configure this parameter to a directory in the OSS bucket where the default storage path of the current workspace resides. If you use a custom role, you must grant OSS access permissions to it. We recommend that you attach the AliyunOSSFullAccess policy to the custom role. For more information, see Grant permissions to a RAM role. |
Embedding Model and Databases | |
Embedding Type | Select General Embedding Model. |
Embedding Connection | Select the embedding model service connection created in 2.2 Create an embedding model service connection. |
Vector Database Type | Select FAISS. This topic uses FAISS an an example. |
4. Create and run an application flow
Go to LangStudio, select a workspace, and then click Enter LangStudio. On the Application Flow tab, click Create Application Flow to create a "RAG- and web search-based chatbot" application flow.
On the application flow details page, click Create Runtime in the upper-right corner. In the Create Runtime panel, configure the parameters. Note: To parse Python nodes or view more tools, you must ensure that the runtime is started.
Take note of the following parameter:
VPC: If you set the vector database type to Milvus in 3. Create a knowledge base index, you must select the VPC in which the Milvus instance resides, or ensure that the selected VPC and the VPC where the Milvus instance is located are interconnected. If you set the vector database type to Faiss, the VPC does not need to be configured.
Develop the application flow.
In the application flow, configure the key nodes as follows. Keep the default configurations for other nodes or configure the nodes based on your business requirements.
Knowledge Retrieval: Retrieve the text related to the user question from the knowledge base.
Index Name: Select the knowledge base index that you created in 3. Create a knowledge base index.
Top K: Return the top K relevant results.
Serp Search: Use SerpApi for web search and return a specific number of search results.
SerpApi Connection: Select the SerpApi connection that you created in 2.3 Create a SerpApi connection.
Engine: Support common search engines such as Bing, Google, Baidu, Yahoo, and also support custom inputs. For more information about search engines and other parameter descriptions, see the SerApi official website.
LLM: Use the retrieved documents and Serp search results as context, and send them to the LLM along with the user question to generate an answer.
Model Configuration: Select the connection that you created in 2.1 Create an LLM service connection.
Chat History: Specify whether to enable the chat history feature to use the chat history as input variables.
For more information about each node, see Appendix: Description of built-in components.
Click Run in the upper-right corner to execute the application flow. For information about common issues related to the application flow runtime, see FAQ.
Click View Logs in the generated answer section to view trace details or the topology.
5. Deploy the application flow
On the application flow details page, click Deploy in the upper-right corner to deploy the application flow as an EAS service. The following section describes the key parameters. Keep the default configurations for the remaining deployment parameters or configure them based on your business requirements.
Resource Information > Instances: Configure the number of service instances. The deployment in this topic is for the testing purpose only, so the number of instances is set to 1. In a production environment, we recommend that you configure multiple service instances to reduce the risk of a single point of failure.
VPC > VPC (VPC): The SerpApi service needs to access the Internet, so you must configure a VPC that can be accessed over the Internet (EAS services cannot access the Internet by default). For more information, see Configure Internet access. Additionally, if the vector database type in 3. Create a knowledge base index is set to Milvus, you must ensure that the VPC that you select is the one in which the Milvus instance resides, or the selected VPC and the VPC in which the Milvus instance resides are interconnected.
For more information, see Deploy an application flow.
6. Call the service
After successful deployment, the Overview tab of the EAS service details page appears. Then, click the Online Debugging tab to configure and send a request. The Key value in the request body must be the same as the value of the Chat Input parameter in the Start Node of the application flow. In this topic, the default field question is used.
For more information about the calling methods (such as API calling) and detailed instructions, see Call a service.