Container Service for Kubernetes:Build a general-purpose web chatbot with Flowise

1.1 Prerequisites

1. An ACK Pro cluster of version 1.22 or later is created. See Create an ACK managed cluster and Upgrade a cluster.

2. kubectl is connected to the cluster. See Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

1.2 Deploy the component

2.1 Access the Flowise service

1. Enable Internet access for the Flowise component.

   In production, enable Access Control to secure your data.

2. After configuration, find the External IP of the Flowise component. Enter this IP in your browser to access the Flowise service.

   On the Service list page, confirm that the flowise service is of the LoadBalancer type and the port mapping is 3000:30864/TCP.

   

3. Access the Flowise service.

   Open the External IP address and enter your Username and Password.

   

   If no custom username or password was set during installation, use the defaults: Username=flowiseuser and Password=flowisepassword.

2.2 Create Chatflows

Flowise can turn flows into tools. A main flow coordinates task-specific tool flows, each designed for a specific task.

Each child flow runs independently with its own memory. A final agent summarizes all outputs for higher-quality results.

To create a Chatflow:

1. Log on to the Flowise service.

2. Create a new Chatflow.

   Click ChatFlows > Add New.

   

3.1 Configure Chat Models

Flowise orchestrates LLM applications by connecting nodes. These nodes, abstracted from frameworks such as LangChain and LlamaIndex, include types such as Agents, Chains, and Chat Models.

1. In the Chat Models Panel, drag a ChatOpenAI Custom node. Click Connect Credential > Create New and enter the API key for Tongyi Qianwen.

   

2. In Connect Credential, select Tongyi Qianwen. Set Model Name to qwen-turbo, set Temperature to 0.9, and on the Additional Parameters page, set BasePath to https://dashscope.aliyuncs.com/compatible-mode/v1.

   

3.2 Set up a prompt template

In the Prompts panel, drag a Prompt Template node and configure the Template as follows.

Question: {question}
Answer: Let's think step by step.

3.3 Set up an LLM Chain

1. In the Chains panel, drag an LLM Chain node.

   The created LLM Chain node contains the following input fields: Language Model (required), Prompt (required), Output Parser, Input Moderation, and Chain Name (used to name the chain). The output field is LLM Chain.

   

2. Connect the nodes to complete the chatbot orchestration.

   The orchestrated workflow contains a ChatOpenAI Custom node (with the model set to qwen-turbo and Connect Credential configured), a Prompt Template node, and an LLM Chain node. The three nodes are connected in sequence.

   

3.4 Verify the result

After orchestrating the Chatflow, click Save in the upper-right corner. Use the Message button to debug the chatbot.

In the canvas, the ChatOpenAI Custom node (Connect Credential: Tongyi Qianwen, Model Name: qwen-turbo, Temperature: 0.9), the Prompt Template node (template: Question: {question} Answer: Let's think step by step.), and the LLM Chain node are connected in sequence. In the chat panel on the right, send a message such as "Please introduce yourself". The model returns a self-introduction about its capabilities as an AI assistant, which verifies that the conversation feature works as expected.

Apply to a production environment

Two ways to deploy the chatbot to production:

Example: Embed an AI application into a website

Follow these steps to embed an LLM application into your website.

This example is for demonstration only. Enable Access Control in production to secure your data.

1. Enable Internet access for the Flowise component and open the external IP in your browser. See Access the Flowise service.

   

2. Build a simple web application in an ACK cluster to debug the chatbot.

   Deploy a web application in the ACK cluster to host the Flowise LLM application.

   1. Obtain the Flowise service code.

      Select an embed method and copy the code into your website.

      

   2. In the ACK cluster, create a deployment and a service to run and expose the web application.

      The following YAML deploys an Nginx server with static HTML.

      1. Log on to the ACK console. On the Configurations > ConfigMaps page, select the default namespace and click Create from YAML. Paste the YAML below and replace window.difyChatbotConfig, scr, and id with your Flowise service code.

         See the example below.

         View YAML content

         apiVersion: apps/v1
         kind: Deployment
         metadata:
           name: web-deployment
         spec:
           replicas: 2
           selector:
             matchLabels:
               app: web
           template:
             metadata:
               labels:
                 app: web
             spec:
               containers:
               - name: web
                 image: registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
                 ports:
                 - containerPort: 80
                 volumeMounts:
                 - name: web-content
                   mountPath: /usr/share/nginx/html
               volumes:
               - name: web-content
                 configMap:
                   name: web-config
           
         
         ---
         apiVersion: v1
         kind: Service
         metadata:
           name: web-service
         spec:
           selector:
             app: web
           ports:
             - protocol: TCP
               port: 80
               targetPort: 80
           type: LoadBalancer
         
         
         ---
         apiVersion: v1
         kind: ConfigMap
         metadata:
           name: web-config
         data:
           index.html: |
             <!DOCTYPE html>
             <html lang="en">
             <head>
                 <meta charset="UTF-8">
                 <meta name="viewport" content="width=device-width, initial-scale=1.0">
                 <title>The simplest website service</title>
               
             </head>
             <body>
                 <h1>Welcome to my website!</h1>
             
             
                 <script type="module">
             import Chatbot from "https://cdn.jsdelivr.net/npm/flowise-embed/dist/web.js"
             Chatbot.init({
                 chatflowid: "86c03xxx-de3a-4xx3-9xx2-f5d56xxxx0a4",
                 apiHost: "http://127.xx.xx.1:8080",
             })
                 </script>
         
         
         
             </body>
             </html>

      2. Successful deployment:

         

         In the left-side navigation pane of the ACK console, choose Stateless and view the details of the web-deployment workload. The namespace is default, the images of the two pods are registry.openanolis.cn/openanolis/nginx:1.14.1-8.6, both pods are in the Running state, and the ready count is 2/2. This indicates that the deployment is successful.

      3. Enable Internet access for the deployed service.

         In production, enable Access Control to secure your data.

         In the left-side navigation pane, choose Network > Services. Find web-service and click Update in the Actions column. In the Update Service dialog box, select the LoadBalancer tab, set the load balancer type to Classic Load Balancer (CLB), set Select Resources to Create New, set Access Method to Internet Access, and set Billing Method to Pay-as-you-go. Then, click OK.

         

         After configuration, find the External IP of web-service. Enter this IP in your browser to access the web service.

         

         Important

         • For external access, ensure the cluster's security group allows traffic on port 80. See Add a security group rule.

         • Ensure your code is secure against cross-site scripting (XSS) and code injection. This is a basic example; extend and modify as needed.

      4. View the result.

         

Continuous improvement

See Build a customized AI Q&A assistant for a website using Dify.

With Dify, integrate knowledge bases into LLM applications to build customized AI Q&A solutions for your business.