Build a RAG application with Dify and Tablestore - Tablestore

Private enterprise Q&A systems often provide inaccurate answers. You can build a retrieval-augmented generation (RAG) application by integrating Dify with the Alibaba Cloud Tablestore vector database. This solution provides accurate knowledge retrieval and intelligent Q&A capabilities.

Solution overview

Tablestore is a high-performance vector database. It provides millisecond-level query responses and supports a hybrid retrieval mode that combines vector search with full-text search. A single table can store tens of billions of vectors. Dify handles application orchestration and the user interface. This greatly simplifies the process of building and deploying RAG applications.

The RAG application uses the following core process for intelligent Q&A:

Knowledge vectorization: Dify automatically splits enterprise documents into knowledge segments, converts them into vector representations, and stores them in the Tablestore vector database.
Similarity search: When a user asks a question, the system quickly retrieves the most relevant knowledge segments from Tablestore.
Augmented generation: The retrieved knowledge is combined with the user's question and provided to a large language model to generate an accurate and well-founded answer.

Prepare the deployment environment

Create a Tablestore instance

Tablestore serves as the vector database for the RAG application. It stores the vectorized representations of documents and provides efficient similarity search capabilities.

Log on to the Tablestore console. At the top of the page, select a region for the instance, such as China (Hangzhou).
Click Create Instance and select CU Mode (formerly On-demand Mode). Set the instance parameters as described in the following list. Keep the default settings for the other configuration items.
- Instance Name: Enter a name for the instance.
- Instance Type: Select High-performance.
Click OK to create the instance.

Create an ECS instance

The ECS instance serves as the runtime environment for the Dify application. It requires sufficient computing resources to support the operation of the Docker container cluster.

Log on to the ECS console and click Create Instance.
Create an ECS instance with the parameters listed below. You can keep the default settings for other configuration items.
- For Billing Method, select Pay-as-you-go.
- Region: Select the region where the instance is located. Due to network considerations, this topic uses the China (Hong Kong) region.
- Network and Zone: Select the default virtual private cloud (VPC) and zone.
- Instance: Click All Instance Types to search for and select ecs.e-c1m2.large.
  Note
  If this instance type is sold out, select another one.
- Image: In the Public Image section, select Alibaba Cloud Linux (Alibaba Cloud Linux 3.2104 LTS 64-bit).
- System Disk: Set the ESSD Entry Disk size to 40 GiB.
- Public IP Address: Select Assign Public IPv4 Address.
- Bandwidth Billing Method: Select Pay-by-traffic to save costs.
- Maximum Bandwidth: Select 5 Mbps or higher.
- Security Group: Select Existing Security Group.
- Logon Credential: Select Custom Password. Set the logon username to root and set the Logon Password. Store the password in a secure location.

Deploy and access Dify

Step 1: Install the Docker environment

Dify is deployed in containers. Therefore, you must first install Docker and Docker Compose on the ECS instance to support multi-service container orchestration.

Install the dnf-plugins-core plug-in.
```
dnf -y install dnf-plugins-core
```

Configure the official Docker repository.

dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Install the Docker Engine and Docker Compose plug-in.

dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Start Docker and enable it to start automatically on system startup.
```
systemctl enable --now docker
```

Step 2: Deploy the Dify service

Next, clone the Dify source code and configure the environment variables, especially the connection parameters for Tablestore, which serves as the vector database.

Install the Git version control tool.
```
yum -y install git
```

Clone the official Dify source code repository.

git clone https://github.com/langgenius/dify.git

Go to the Docker deployment directory.
```
cd dify/docker
```
Copy the environment configuration file template.
```
cp .env.example .env
```

Edit the environment configuration file.

vi .env

Modify the following configuration items:

Configuration Item	Description
VECTOR_STORE	The type of vector database. Set it to `tablestore`.
TABLESTORE_ENDPOINT	Important New Tablestore instances have public network access disabled by default. To use a public endpoint, go to the Tablestore console. On the Instance Management page, click the Network Management tab. In the Allowed Network Types section, select Internet and click Settings to save the configuration. Go to the Tablestore console. In the instance list, click the instance alias to go to the Instance Management page. Copy the instance name and endpoint (Instance Access URL). Select an endpoint based on your deployment: If the ECS instance and Tablestore instance are in the same region, use the public endpoint (Internet) or the VPC endpoint. If the ECS instance and Tablestore instance are in different regions, you must use the public endpoint (Internet).
TABLESTORE_INSTANCE_NAME
TABLESTORE_ACCESS_KEY_ID	Go to the AccessKey Management page to create an AccessKey for your Alibaba Cloud account. Get and save the ACCESS_KEY_ID and ACCESS_KEY_SECRET.
TABLESTORE_ACCESS_KEY_SECRET

Start the Dify container cluster.

docker compose up -d

After the startup is complete, the following output is displayed:

 ✔ Network docker_default             Created                                                                                                                                                            0.1s
 ✔ Network docker_ssrf_proxy_network  Created                                                                                                                                                            0.1s
 ✔ Container docker-sandbox-1         Started                                                                                                                                                            0.8s
 ✔ Container docker-redis-1           Started                                                                                                                                                            1.0s
 ✔ Container docker-ssrf_proxy-1      Started                                                                                                                                                            1.3s
 ✔ Container docker-web-1             Started                                                                                                                                                            1.0s
 ✔ Container docker-db-1              Started                                                                                                                                                            0.9s
 ✔ Container docker-plugin_daemon-1   Started                                                                                                                                                            2.4s
 ✔ Container docker-api-1             Started                                                                                                                                                            2.4s
 ✔ Container docker-worker-1          Started                                                                                                                                                            2.3s
 ✔ Container docker-nginx-1           Started                                                                                                                                                            3.8s

Step 3: Configure security group access rules

To access the Dify management interface from the internet, you must open the corresponding port in the ECS security group.

In the ECS console, enter the security group details page of the target instance, and click the Inbound tab.
Configure the security group rule by setting Action to Allow and Protocol Type to Web HTTP Access. In the Destination (Current Instance) field, enter the Dify service port (80 by default) and add the port, as shown in the following figure.
Click OK to save the security group rule.

Step 4: Access the Dify management interface

In a browser, navigate to http://server_ip, where server_ip is the public IP address of your ECS instance. The first time you access this page, you are redirected to an initialization page. Follow the prompts to set up an administrator account. After the setup is complete, the system automatically logs you in.

Build the RAG application and verify the results

Step 1: Configure model services and API keys

The RAG application requires a large language model to understand questions and generate answers. It also needs an embedding model to convert text into vector representations.

Install a model provider and configure its API key.
1. On the Dify homepage, click your profile picture, and then click Settings from the drop-down menu.
2. On the Settings page, click Model Provider. Select TONGYI and click Install.
3. After the installation is complete, click Setup.
4. Follow the instructions on the page to obtain the API Key for Alibaba Cloud Model Studio and complete the configuration. Then, click Save.
Configure the default system models.
1. On the Model Provider page, to the right of model list, click System Model Settings.
2. Set the models according to the following recommendations. You can also choose other models as needed.
  - For System Reasoning Model, select qwen3-max-preview.
  - For Embedding Model, select text-embedding-v4.
  - For Rerank Model, select gte-rerank-v2.
  - For Speech-to-Text Model, select paraformer-realtime-v2.
  - For Text-to-Speech Model, select tts-1.

After you configure the model, click ESC to return to the Dify homepage.

Step 2: Create a knowledge base

The knowledge base is a core component of the RAG application. It stores enterprise documents and provides intelligent retrieval capabilities. Documents are automatically chunked, vectorized, and stored in Tablestore.

On the Dify homepage, click Knowledge, and then click Create Knowledge.
Click I want to create an empty Knowledge. In the Create an empty Knowledge dialog box, enter a name for the knowledge base, such as Tablestore, and click Create.
On the knowledge base details page, click Add file, select the example file Model_Studio_Series_Mobile_Phone_Product_Introduction.docx, and click Next.
To configure the knowledge processing parameters, set Index Method to High Quality and Retrieval Setting to Hybrid Search. Then, click Save & Process. You can click Go to document to view the processing status of the uploaded document.
You can navigate to the Tablestore console to view the knowledge data saved in the vector database.

Step 3: Create an AI chat assistant and verify the RAG effect

Finally, create a chat assistant application. Compare its responses before and after adding the knowledge base to verify the retrieval-augmented capabilities of the RAG application.

On the Dify home page, click Studio and then Create from Blank.
Click MORE BASIC APP TYPES and select Chatbot, enter an application name, and click Create.
Test the chat assistant by asking a question, such as What are the Model Studio phone models. The assistant's answer may not be accurate or detailed at this point.
On the Application Orchestrate page, click Add next to Knowledge, select your knowledge base, and then click Restart when prompted to reset the debug area.
Ask the chat assistant the same question again. With the content from the knowledge base, the assistant can now provide an accurate and detailed answer.

Use in a production environment

Best practices

To ensure high availability for your Dify application in a production environment, we recommend deploying it using ACK. For more information about the implementation plan, see the solution Quickly deploy Dify to efficiently build AI applications and choose the production environment deployment option that uses ACK.

Risk prevention

Permission control: Use Resource Access Management (RAM) users to manage resource access. Strictly control permission scopes and grant only the necessary operation permissions for Tablestore and ECS.
Operation audit: Enable ActionTrail to record complete operation logs for access and modification of key resources. This ensures that all actions are traceable.
Data backup: Regularly perform data backup for Tablestore. This effectively prevents data loss risks caused by accidental failures or human error.

Clean up resources

After you finish the tutorial, release the related cloud resources promptly to avoid unnecessary charges if you do not plan to use them in a production environment.

Release the ECS instance
In the ECS console, find the target instance, and click Actions > Release. Follow the on-screen instructions to release the instance.
Delete the Tablestore data table
Go to the Tablestore console. In the instance list, click the alias of the target instance. On the Instance Management page, click Indexes for the target data table. On the Indexes page, delete the search index. Then, return to the data table list. For the target data table, click ... > Delete.
Release the Tablestore instance (CU mode)
1. Go to the Tablestore console. In the instance list, find the target instance and click Release in the Actions column.
2. Click OK to release the instance. If you are prompted for identity verification, follow the on-screen instructions.

Tablestore:Build a RAG application with Dify and Tablestore