This document uses Dify as an example to demonstrate how to integrate OpenSearch LLM Intelligent Q&A Edition with large-scale model application development platforms to establish a Retrieval-Augmented Generation (RAG) system.
Background
With the rapid development of AIGC technology, LLM applications are also continuously iterating. Based on LLM, agent frameworks, workflow orchestration capabilities, etc., various application services can be built for different scenarios. Among them, the Retrieval-Augmented Generation (RAG) system has become a core link in scenarios such as enterprise knowledge bases, intelligent customer service, and E-commerce guidance.
OpenSearch LLM Intelligent Q&A Edition integrates data parsing and processing, slicing, vectorization, text & vector retrieval, multimodal LLM, and other models and functions. This document will describe how to use OpenSearch LLM Intelligent Q&A Edition to dock big model application development platforms and build a RAG system.
Big model application platform
In the big model industry, there are more and more application frameworks and development platforms, such as Alibaba Cloud BaiLian and Dify. Developers can quickly build business applications based on these frameworks and platforms, and the RAG system is also a common link among them. Therefore, big model application development platforms usually come with built-in RAG systems.
However, the accuracy and search effectiveness of the RAG system are closely related, and application development platforms often face the following issues:
1. Poor usability: Various restrictions on the format, quantity, and size of knowledge base documents require developers to perform complex data preprocessing or fail to meet actual application needs.
2. Poor professionalism: Mostly black-box systems, making it difficult for developers to customize and optimize core links, resulting in poor overall application performance.
3. Weak enterprise-level capabilities: Lack of support for enterprise-level permission isolation, fast data import/update capabilities, making it difficult to implement in actual production applications.
To address the above issues, you can use OpenSearch LLM Intelligent Q&A Edition as a RAG system, apply it to workflows, and quickly build enterprise-level applications. Currently, OpenSearch supports docking with models in BaiLian, enriching LLM selection and achieving customized RAG effects. For specific usage, refer to: LLM management.
In addition, OpenSearch supports rich invocation and authentication mechanisms, flexibly docking with application development platforms such as BaiLian and Dify.
The following will use Dify as an example to introduce best practices for docking open source application platforms.
Overall architecture
Developers pre-import the knowledge base into OpenSearch and use workflow-processed dialog requests to access the RAG system in OpenSearch. OpenSearch will return dialog results, reference links, and reference images based on the knowledge base and LLM. Developers then process the results through workflows according to business needs and finally output them to end users.
Building a RAG system in OpenSearch LLM Intelligent Q&A Edition
1. Building a RAG system
OpenSearch LLM Intelligent Q&A Edition is a one-stop RAG product that can build a RAG system in minutes and allows for visual model selection, prompt customization, and effect optimization in the console. For details, see Enterprise knowledge base Q&A through the console.
2. Create and get an API key
Create and get a public API domain name and API key, and keep them safe.
Building business applications on the application platform
Step 1: Create a workflow on the big model application platform
The basic RAG workflow includes four steps:
Start: Get the dialog content input by the user.
OpenSearch LLM Intelligent Q&A Edition (HTTP request): Input the dialog content into OpenSearch and return the output results based on the RAG system.
Parse output results (code execution): Parse the dialog content in the results.
Return the answer: Return the final result to the user.
Step 2: Access the OpenSearch RAG system using an HTTP request
Authentication API-Key:
Authentication type: API-Key
API authentication type: Bearer
API key: The API key obtained from OpenSearch LLM Intelligent Q&A Edition.
URL address: The public API domain name obtained from OpenSearch LLM Intelligent Q&A Edition + OpenSearch interface address (reference URL: v3/openapi/apps/[app_name]/actions/knowledge-search)
BODY: Choose JSON format. For specific content and parameters, refer to: SearchKnowledge-Q&A document query.
Step 3: Parse the output results
The interface return results of OpenSearch are in JSON format, including dialog results, reference links, reference images, etc. Developers can use code execution to parse the output results and get the return results as needed.
Reference code to get only the output results:
def main(body: str) -> str:
import json
dat = json.loads(body)
return {
'result': [ans['answer'] for ans in dat['result']['data']][0]
}
Effect preview
On this basis, you can build other workflows to create rich business application flows.
For example, the following is an intelligent dialog assistant built based on OpenSearch and the Qwen model.
First, it will determine the user's dialog intent and classify it into after-sales issues, product usage issues, and casual chat.
For after-sales issues and product usage issues, access the corresponding knowledge base in OpenSearch and use the RAG system to reply.
For casual chat issues, access the Qwen model to chat with the user and solve general issues.
In addition to using the one-stop RAG product to build workflows, you can also use the Alibaba Cloud AI Search Development Workbench. Through the atomic capabilities provided by the workbench, such as document parsing, vectorization, search, and reordering, you can customize the Dify tool to quickly optimize various links of the RAG system.
For more information about OpenSearch LLM Intelligent Q&A Edition, see Introduction to LLM-Based Conversational Search Edition.
If you have any questions about the RAG system, feel free to join the OpenSearch LLM Intelligent Q&A Edition DingTalk support group to learn more technical details and usage information.