Alibaba Cloud AI Containers (AC2) provide ChatGLM3 series large model images for ready-to-use container deployment. The container images contain all dependencies required for running ChatGLM3 large models, including the Python runtime environment, deep learning frameworks, and dependency libraries, which ensures that ChatGLM3 large models can be efficiently and stably deployed and run in different environments. The ChatGLM3 series images do not contain large model weight files. You must download the large model weight files or use the download tool provided by the images to download the files. The ChatGLM3 series images provide external services in the form of web demos. You can also use the ChatGLM3 series images as base images to provide custom service provisions.
Images
ChatGLM3 series large model images include the quick deployment (devel) image and runtime image.
Runtime image: contains only all software environments required to run a model, including system components and Python dependencies.
Quick deployment (devel) image: contains a runtime environment, a startup script, and a web demo script.
Category | Supported large model | CPU/GPU | Image address |
Runtime image | ChatGLM3-6B | GPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/chatglm3:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 |
ChatGLM3-6B | CPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/chatglm3:runtime-pytorch2.2.0.1-alinux3.2304 | |
Quick deployment (devel) image | GPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/chatglm3:6b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 | |
CPU | ac2-registry.cn-hangzhou.cr.aliyuncs.com/ac2/chatglm3:6b-pytorch2.2.0.1-alinux3.2304 |
Image content
chatglm3:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304
gradio: 3.41.0
sentencepiece: 0.1.99
safetensors: 0.4.0
accelerate: 0.26.1
transformers: 4.36.2
PyTorch: 2.2.0.1
CUDA: 12.1.1
Python: 3.10.13
BaseOS: Alinux 3.2304
chatglm3:runtime-pytorch2.2.0.1-alinux3.2304
gradio: 3.41.0
sentencepiece: 0.1.99
safetensors: 0.4.0
accelerate: 0.26.1
transformers: 4.36.2
PyTorch: 2.2.0.1
Python: 3.10.13
BaseOS: Alinux 3.2304
chatglm3:6b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304
Image components are inherited from the chatglm3:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 image.
chatglm3:6b-pytorch2.2.0.1-alinux3.2304
Image components are inherited from the chatglm3:runtime-pytorch2.2.0.1-alinux3.2304 image.
Operational requirements
The chatglm3:runtime-pytorch2.2.0.1-alinux3.2304 and chatglm3:6b-pytorch2.2.0.1-alinux3.2304 images are CPU-based images and do not have requirements on drivers.
The chatglm3:runtime-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 and chatglm3:6b-pytorch2.2.0.1-cuda12.1.1-alinux3.2304 images are GPU-based images that include CUDA 12.1.1. The images require NIVIDA Driver release 530 or later and are compatible with NIVIDA Driver releases 470 and 525.
Important features
The devel image supports the quick deployment capability and a built-in script. For more information about the built-in script, see web_demo_gradio.py.
Updates
2024.07
Released the ChatGLM3 runtime image.
2024.08
Released the ChatGLM3 devel image.