AnalyticDB:Build a custom chatbot with Compute Nest

Billing

When you create a One-stop Enterprise-level Chatbot (Community Edition) (LLM + Vector Database) service, the system automatically creates an ECS instance and an AnalyticDB for PostgreSQL instance in elastic storage mode. You are charged for these resources. For more information about billing, see the following topics:

• Billing of Compute Nest

• Billing of ECS

• Billing of AnalyticDB for PostgreSQL

Benefits

• Support for multiple models: The service supports various models, such as Qwen-7b, ChatGLM-6b, Llama2-7b, Llama2-13b, . You can switch between models after the service is created.

• GPU cluster management: You can use GPU instances with low resource usage during testing. As your business grows, you can configure auto scaling for your GPU cluster based on resource usage to minimize GPU overhead.

• Fine-grained permission design based on AnalyticDB for PostgreSQL: You can adjust permission queries based on open-source code. You can also use the APIs provided for knowledge base management in AnalyticDB for PostgreSQL for more flexible integration.

• API and web UI availability: You can quickly integrate the AIGC backend with your applications.

• Data security: All data, algorithms, and GPU resources reside within your service domain. This eliminates the risk of data exfiltration and ensures the security of your core enterprise data.

RAM user authorization

If you use a RAM user to perform the following operations, you must first grant the necessary permissions to the RAM user. For more information about the RAM permissions required by Compute Nest and how to grant them, see Grant permissions to a RAM user.

Video tutorial

Create a service instance

This topic uses GenAI-LLM-RAG as an example.

1. Go to the Create Service Instance page. In the Quick Trial area, click GenAI-LLM-RAG.

2. On the Create Service Instance page, configure the following parameters.

   Type	Parameter	Description
   Service Instance Name		Enter a name for the service instance. We recommend using a descriptive name for easy identification, as the system generates a random name by default.
   Region		The region where the service instance, ECS instance, and AnalyticDB for PostgreSQL instance are located.
   Billing Method Configuration	Billing Method	Select Pay-as-you-go or Subscription based on your business requirements. This topic uses the pay-as-you-go billing method as an example.
   ECS Instance Specifications	Instance Type	Select the Specifications for the ECS instance.
   	Password	The logon password for the ECS instance.
   	Whitelist Settings	The allowlist of the ECS instance. We recommend adding the IP addresses of the servers that need to access the LLM to the allowlist.
   PAI-EAS Model Configuration	Select Large Model	Select a preconfigured large language model (LLM). This topic uses llama2-7b as an example.
   	PAI Instance Specifications	The GPU specifications for the PAI service. You cannot select a specification if it is out of stock.
   AnalyticDB for PostgreSQL	Instance specifications	The compute node specifications of the AnalyticDB for PostgreSQL instance.
   	Segment Storage Size	The storage size of compute nodes for the AnalyticDB for PostgreSQL instance, in GB.
   	Database Account Name	The initial account name for the AnalyticDB for PostgreSQL instance.
   	Database Password	The password for the initial account of the AnalyticDB for PostgreSQL instance.
   Application Configurations	Software Logon Name	The username to log on to the LangChain web service.
   	Software Logon Password	The logon password for the LLM software.
   Zone Configuration	vSwitch Zone	Select the availability zone where the service instance is located.
   Network Configuration	Whether to Create VPC	You can create a new VPC or use an existing one. This topic uses a new VPC as an example.
   	VPC IPv4 CIDR Block	Enter the CIDR block for the VPC.
   	vSwitch CIDR Block	Enter the CIDR block for the vSwitch.
   Tags and Resource Groups	Tag	Select a tag to add to the service instance.
   	Resource Group	Select the resource group to which the service instance belongs. For more information, see What is Resource Management?.

3. Click Next: Confirm Order.

4. Review the information in the Dependency Check, Service Instance Information, and Price Preview sections. Confirm the details before you proceed.

   Note

   If the Dependency Check section indicates that some role permissions are not granted, click Create Now on the right to grant them. After authorization is complete, click the refresh icon in this section.

5. Click Create Now.

6. Click View Service.

The service instance takes about 10 minutes to create. You can check the status of the service instance on the Service Instance page. When the status changes to Deployed, the instance is ready.

Use the chatbot

Before using the chatbot, you must upload files to the knowledge base. The following steps describe how to upload files and use the chatbot.

1. On the Service Instance page of the Compute Nest console, click the ID of the target service instance to go to its details page.

2. On the service instance details page, in the Use Now section, click the link to the right of Endpoint.

3. In the Log On dialog box, enter the Software Logon Name and Software Logon Password that you set during instance creation, and then click Log On.

4. In the Select a mode area in the upper-right corner of the page, select Knowledge Base Q&A.

5. In the Configure Knowledge Base section on the right, select New Knowledge Base from the Select a knowledge base to load drop-down list. Enter a name for the new knowledge base and click Add to Knowledge Base Options.

6. Set the Sentence length limit for text ingestion based on your requirements. We recommend setting this parameter to 500.

7. Add files to the new knowledge base.

   • Upload files by using the Upload File, Upload File and URL, and Upload Directory options.

   • Supported file formats include PDF, Markdown, TXT, and Word.

   • To delete a file, go to the Delete Object tab.

8. After the upload is complete, you can enter your question in the lower-left corner of the page and click Submit to start a Q&A session.

Resource management

FAQ

• Q: How do I use the vector search APIs?

  A: See Java.

• Q: How do I check the deployment progress of a service instance?

  A: After you create a service instance, the Compute Nest service, including the initialization of the ECS instance and the AnalyticDB for PostgreSQL vector database, takes about 10 minutes to complete. The large language model (LLM) is downloaded asynchronously, which can take 30 to 60 minutes. To check the model download progress, log on to the ECS instance and check the download logs. After the LLM is downloaded, you can log on to the web interface to view the chatbot application.

• Q: After I create a Compute Nest service, how do I log on to the ECS instance?

  A: On the Resources tab of the service instance, find the resource where the Resource Type is Security group securitygroup, and click its Resource ID. On the instance details page, click Remote Connection. For more connection methods, see Connect to an instance.

• Q: How do I restart the LangChain service?

  A: Log on to the ECS instance and run the following command to restart the service:

  systemctl restart langchain-chatglm

• Q: How do I query LangChain logs?

  A: Log on to the ECS instance and run the following command to view the logs:

  journalctl -ef -u langchain-chatglm

• Q: What should I do if the model fails to load after the service is deployed?

  A: After the service is activated, the system downloads the LLM from Hugging Face to the ECS instance. In the Chinese mainland, the download may take 30 to 60 minutes. Wait for the download to complete before you log on to the interface and try to reload the model.

• Q: How can I view detailed information about the deployment code?

  A: See the langchain-ChatGLM documentation.

• Q: How do I request backend support from the product team?

  A: You can subscribe to the One-stop Enterprise-level Chatbot O&M Service for support.

• Q: Why do I see a blank page when accessing the service?

  A: This is a Compute Nest service for the Alibaba Cloud China site. Access issues may occur if you are using an overseas proxy. Disable the proxy and try accessing the service again.

• Q: Where is LangChain deployed on the ECS instance?

  A: LangChain is deployed in the /home/admin/langchain-ChatGLM directory.

• Q: How do I enable the LangChain API?

  A: Run the following commands on the ECS instance to enable the API.

  # Create a systemd file for langchain-chatglm-api.
  cp /lib/systemd/system/langchain-chatglm.service /lib/systemd/system/langchain-chatglm-api.service
  # Modify ExecStart in /lib/systemd/system/langchain-chatglm-api.service.
  # For PAI-EAS version
  ExecStart=/usr/bin/python3.9 /home/langchain/langchain-ChatGLM/api.py
  # For single-GPU machine version
  ExecStart=/usr/bin/python3.9 /home/admin/langchain-ChatGLM/api.py
  # Reload systemd.
  systemctl daemon-reload
  # Restart the API.
  systemctl restart langchain-chatglm-api
  # The following log message indicates a successful start:
  INFO: Uvicorn running on http://0.0.0.0:7861 (Press CTRL+C to quit)
  # View all APIs:
  curl http://0.0.0.0:7861/openapi.json