Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Solution 1B: How to Use ECS + PAI + AnalyticDB for PostgreSQL to Build a Llama2 Solution

Solution 1B: How to Use ECS + PAI + AnalyticDB for PostgreSQL to Build a Llama2 Solution

Explore the integration of AnalyticDB for PostgreSQL with large language models on Alibaba Cloud's PAI, empowering businesses with efficiency.

By Haoran

Step 1. Use AnalyticDB for PostgreSQL to Prepare Vector Store

Check the doc of Solution 1.

Step 2. Use PAI-EAS to Provision LLM Inference Service

Check the doc of Solution 1.

Step 3. Provision a ECS on Alibaba Cloud

  1. Go to ECS console
  2. Click Instance and click Create Instance.

1

Choose the spec, eg. 4c 16g, ecs.g7.xlarge, and click pay-as-you-go.

2

Choose Public Image and Ubuntu with 22.04 64-bit.

3

Specify VPC and VSwitch and Security Group. If you have created, click go to the VPC console to create it and refresh it to configure. For network, click assign Public IPV4 address, do you choose ENI this time, otherwise coupon cannot be used in public network part.

4

Choose Key Pair and create a key pair and configure it. It is your key.pem. then you can change your instance name and host name, and click sequential suffix.

5

Here is the screenshot to create key pairs

6

Then you can configure it into same resource group for better management.

7

After final preview, you click create instance. Then it will create successfully. Click console to check the instance. You don’t need to click savings plan in this time.

8

3.  configure the security group

Click the instance to check the details

9

Click Add Rules

10

Click Add Rule to open port 22 for SSH, with address 0.0.0.0/0

11

4.  Test your connection in web workbench

Click Connect in Instance Details

12

Click Sign in now

13

You can use SSH Key Authentication to logon it.

14

Then you can successfully logon as is shown in the following picture.

15

Step 4. Use SSH to Prepare Docker Image

1.  Install docker.io on ECS instance.

apt-get -y update
apt-get -y install docker.io

2.  Start the docker server.

systemctl start docker

3.  Run the following command to mount the project to the Docker container and start it.

sudo docker run -t -d --network host --name llm_demo mybigpai-registry.cn-beijing.cr.aliyuncs.com/mybigpai/chatglm_webui_test:2.2

16

It takes around 15-25 minutes to download, depending on network speed. This image will be available only during the Generative AI Hackathon event period. If you want to use this image, please, feel free to contact us.

4.  Enter into the docker

docker exec -it llm_demo bash

5.  Open configuration file (config.json)

vim config.json

6.  Change the file as below and save it.

  "embedding": {
    "model_dir": "/code/",
    "embedding_model": "SGPT-125M-weightedmean-nli-bitfit",
    "embedding_dimension": 768
  },
  "EASCfg": {
    "url": "[http://xxxxx.pai-eas.aliyuncs.com/api/predict/llama2_model](http://xxxxx.pai-eas.aliyuncs.com/api/predict/llama2_model)",
    "token": "xxxxxx"
  },
  "ADBCfg": {
    "PG_HOST":"gp-xxx.gpdbmaster.ap-southeast-5.rds.aliyuncs.com",
"PG_DATABASE": "postgres",
    "PG_USER": "pg_user",
    "PG_PASSWORD": "password"
  },
  "create_docs":{
    "chunk_size": 200,
    "chunk_overlap": 0,
    "docs_dir": "docs/",
    "glob": "**/*"
  },
  "query_topk": 4,
  "prompt_template": "Answer user questions concisely and professionally based on the following known information. If you cannot get an answer from it, please say \\"This question cannot be answered based on known information\\" or \\"Insufficient relevant information has been provided\\", no fabrication is allowed in the answer, please use English for the answer. \\n=====\\nKnown information:\\n{context}\\n=====\\nUser question:\\n{question}"
} |

7.  Run the following code to upload the docs

python main.py --config config.json --upload true

8.  Run the following code to make a query

python main.py --config config.json --query "what is PAI?"
ECS AI Tutorials PAI AnalyticDB for PostgreSQL Large Language Model LLama2 LangChain
0 2 0
Share on

Read previous post:

Solution 1: Build Your Llama2 LLM Solution with PAI-EAS and AnalyticDB for PostgreSQL

Read next post:

Compute Nest: Enabling Cutting-Edge Generative AI Integration and Knowledge Base Systems in Collaboration with Alibaba Cloud

Farruh

29 posts | 17 followers

You may also like

Comments

Farruh

29 posts | 17 followers

Related Products

More Posts by Farruh

See All