Container Compute Service:Customize an AI Q&A assistant with Dify in ACS

Dify, AI assistant, ACS, Kubernetes, LLM, knowledge base, RAG, Qwen, ack-dify, chatbot, NAS storage, generative AI, LLMOps, prompt orchestration

Overview

Dify is an open source platform for building large language model (LLM) applications. Combined with Alibaba Cloud Container Service for Kubernetes (ACS), you can:

• Deploy Dify in an ACS cluster for automatic scaling

• Create an AI Q&A assistant backed by LLMs like Qwen

• Attach a dedicated knowledge base to answer specialized questions

• Embed the assistant in your website or call it via API

High-level steps:

1. Install ack-dify: Create an ACS cluster and install the ack-dify component.

2. Create an AI Q&A assistant: Access Dify and create an assistant.

3. Customize the assistant: Prepare and attach a knowledge base.

Introduction to Dify

Dify combines backend-as-a-service (BaaS) and LLMOps to help you quickly build production-grade generative AI applications. Key components include:

• Core technical components: Multi-model compatibility, visual prompt design, retrieval-augmented generation (RAG) system, and customizable Agent architecture.

• Visual orchestration: Intuitive interface for prompt orchestration, O&M, and dataset management.

• Application templates: Out-of-the-box templates and orchestration framework for rapid development.

Step 1: Install ack-dify

Prerequisites

Before you begin:

1. Create an ACS cluster (Kubernetes version 1.26 or later). See Create an ACS cluster.

2. Configure kubectl to connect to the cluster. See Connect to your cluster.

3. Configure dynamic NAS volumes. See Use an existing NAS file system as a persistent volume.

Create a StorageClass

Create a file named nas-sc.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alibabacloud-cnfs-nas
mountOptions:
- nolock,tcp,noresvport
- vers=3
parameters:
  volumeAs: filesystem
  fileSystemType: standard
  storageType: Performance
  regionId: cn-hangzhou
  zoneId: cn-hangzhou-g
  vpcId: "vpc-2ze2fxn6popm8c2mzm****"
  vSwitchId: "vsw-2zwdg25a2b4y5juy****"
  accessGroupName: DEFAULT_VPC_GROUP_NAME
  deleteVolume: "false"
provisioner: nasplugin.csi.alibabacloud.com
reclaimPolicy: Retain

Run:

kubectl create -f nas-sc.yaml
kubectl get sc

Create a PVC

Create a file named nas-pvc-fs.yaml:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nas-pvc-fs
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: alibabacloud-cnfs-nas
  resources:
    requests:
      storage: 20Gi

Run:

kubectl create -f nas-pvc-fs.yaml
kubectl get pvc

Deploy ack-dify

1. Log on to the ACS console.

2. In the left navigation pane, click Clusters.

3. Click the name of the cluster you created to go to the cluster details page.

4. In the left navigation pane, choose Applications > Helm.

5. Click Deploy.

6. In the Deploy panel, configure the following settings:

   • Application Name: Use the default value.

   • Namespace: Use the default value.

   • Source: Select App Catalog.

   • Chart: Click the AIGC/LLM tab, then select ack-dify.

7. Click Next.

8. In the Confirm dialog box, click Yes to use the default namespace (dify-system) and application name (ack-dify) .

9. Select Chart Version 1.1.5, then click OK.

Verify the installation:

kubectl get pod -n dify-system

All pods should be in Running state. If pods are Pending, verify the NAS StorageClass and PVC. See Pod troubleshooting.

Step 2: Create an AI Q&A assistant

Access the Dify service

1. In the ACS console, go to Network > Service. Set namespace to dify-system.

2. Find the ack-dify service and copy its External IP.

3. Open the External IP in a browser.

4. Create an administrator account (email, username, password).

Configure the AI model

1. Log on to Dify using the External IP.

2. Click your username > Settings > Model Provider.

3. Install the Qwen plugin and click Get your API key From AliCloud.

4. Enter the API key and click Save.

Create the assistant

1. Click Studio > Create from Blank.

2. Enter a Name and Description.

3. Keep default values for other parameters.

Verify the assistant

Enter a question to test the assistant. A general-purpose chatbot can conduct simple conversations but cannot answer specialized questions.

Step 3: Customize the AI Q&A assistant

Create a knowledge base

To enable the assistant to answer domain-specific questions, attach a dedicated knowledge base.

1. Download the sample corpus file: dify_doc.md.

2. Click Knowledge > Create Knowledge > Import from file.

3. Select the file and click Next.

4. Keep default settings for text segmentation and cleaning.

5. Click Save.

Orchestrate and publish the assistant

1. Configure the prompt:

   You will act as Dify's AI assistant, dedicated to answering customers' questions about Dify products and their features. Your responses should be based on the existing knowledge base to ensure accuracy. If a question is beyond your knowledge, please honestly inform them that you do not know the answer, to maintain the integrity of the information. Please communicate in a friendly and warm tone, and feel free to use emoticons appropriately to enhance the interactive experience.

2. Add a contextual knowledge base: Click Add in the Context section and select the knowledge base you created.

3. Click Publish > Update.

Verify the results

The assistant with a dedicated knowledge base can provide more professional and accurate answers.

Conclusion

Deploy in production

To deploy the AI Q&A assistant in a production environment:

1. Publicly shared website: Publish as a web application accessible on the Internet. See Publish as a Single-page Web App.

2. API-based calls: Access LLM capabilities directly from frontend applications. See Develop with APIs.

3. Redevelopment based on frontend components: Use Dify to quickly publish an AI site. See Re-develop Based on Frontend Templates.

4. Embed into your website: Embed the AI application into your business website. See Embedding In Websites.