In this blog post, we will explore how to set up and run Alibaba Cloud Open Source Qwen 0.5 model using Ollama and Open WebUI on an ECS instance. Once completed, we will be able to use the capabilities of the Qwen model through a user-friendly web interface. Here is the video:
For this simple demo, I am using ecs.g7.large (2 Cores, 8 GB RAM). However, when selecting an ECS instance, consider these key factors for AI workloads:
• CPU/GPU: Look for instances with higher vCPU counts and use GPU for efficient model computations.
• Memory: Models like Qwen can be memory-intensive; aim for more than what I am using here.
• Storage: SSD-based storage improves read/write speeds crucial for model loading.
• Bandwidth: Ensure sufficient bandwidth for smooth interactions via Open-WebUI.
Follow these steps to create your instance:
Use SSH to connect:
ssh root@<your-instance-public-ip>
Update and install Docker:
apt update
apt install docker.io
apt install docker-compose
Create the docker-compose.yml file using nano or vi editor:
version: '3.8'
services:
ollama:
image: ollama/ollama
container_name: ollama
volumes:
- ollama:/root/.ollama
ports:
- "11434:11434"
restart: always
open-webui:
image: ghcr.io/open-webui/open-webui:ollama
container_name: open-webui
volumes:
- ollama:/root/.ollama
- open-webui:/app/backend/data
ports:
- "3000:8080"
restart: always
volumes:
ollama:
open-webui:
Save the above content into a file named docker-compose.yml in your desired directory and services using:
docker-compose up -d
To access the Ollama Container and pull the Qwen 0.5 model, first enter the Ollama container by typing the following:
docker exec -it ollama /bin/bash
Once Inside the container, execute the following command to pull the Qwen 0.5 model:
ollama pull qwen:0.5
Once done, make sure that the Qwen model is available, use ollama list.
• ECS Public Access: With the public IP and security group rules in place, anyone can access your Open-WebUI by navigating to http://<your-instance-public-ip>:3000
. Use sign up by providing Name, Email and Password as shown below:
• Model Usage: After creating account, interact with the Alibaba Cloud Qwen:0.5b model:
Now we can provide different prompts to the model to get response:
Here is the output of the model:
You have now successfully set up Ollama and Open WebUI on your Alibaba Cloud ECS instance, pulled the Qwen 0.5 model, and accessed a web interface for easy interaction. This setup not only simplifies model deployment but also enhances user experience with an intuitive interface. Enjoy experimenting with the Qwen model and exploring its capabilities!
Lastly, It is pertinent to mention that if you need more accurate response, then better use Alibaba Cloud LLMs like Qwen 4b etc. available at https://ollama.com/library/qwen which may need high specifications of hardware for a better response.
If you have any questions or run into any issues, feel free to leave a comment below.
Data Visualization with Alibaba Cloud ECS: From Installation to Data Analysis
Regional Content Hub - October 28, 2024
Regional Content Hub - November 11, 2024
Regional Content Hub - November 4, 2024
Regional Content Hub - November 4, 2024
Regional Content Hub - October 28, 2024
Alibaba Cloud Community - November 22, 2024
Accelerate innovation with generative AI to create new business success
Learn MoreOffline SDKs for visual production, such as image segmentation, video segmentation, and character recognition, based on deep learning technologies developed by Alibaba Cloud.
Learn MoreAccelerate AI-driven business and AI model training and inference with Alibaba Cloud GPU technology
Learn MoreTop-performance foundation models from Alibaba Cloud
Learn MoreMore Posts by JwdShah