Guide to Using NIM models - Platform For AI - Alibaba Cloud Documentation Center

Alibaba Cloud Platform for AI (PAI) is an authorized NIM partner of NVIDIA in China.

NIM is a set of easy-to-use prebuilt container tools introduced by NVIDIA, aimed at helping enterprise customers securely and reliably deploy high-performance AI models in the cloud, data centers, and workstations. NIM models are performance-optimized models processed by the NIM optimization tool, offering significantly improved inference performance compared to original open source models.

Model Gallery of PAI provides multiple NIM models. To obtain NIM models, select NIM for Model Source in the left-side filter pane of Model Gallery. You can deploy NIM models by using one of the following methods:

Deploy a model in Model Gallery of PAI
Deploy a model on your on-premises machine

Supported NIM models

The following table describes the NIM models that can be deployed in Model Gallery of PAI.

Model name	Model page in Model Gallery	Supported model type for NIM model inference optimization
qwen2.5-7b-instruct-NIM	qwen2.5-7b-instruct-NIM	ecs.gn7e ecs.gn8is
MolMIM	MolMIM	General-purpose GPU
Earth-2 FourCastNet	FourCastNet	General-purpose GPU
NVIDIA Retrieval QA Mistral 7B Embedding v2	NV-EmbedQA-Mistral-7b-v2	ecs.gn7e edition
Eye Contact	eyecontact	General-purpose GPU
NV-CLIP	nvclip	ecs.gn7e edition ecs.gn7i edition
AlphaFold2-Multimer	AlphaFold2-Multimer	General-purpose GPU
Snowflake Arctic Embed Large Embedding	Arctic-embed-l	ecs.gn7e edition ecs.gn7i edition
NVIDIA Retrieval QA Mistral 4B Reranking v3	NV-RerankQA-Mistral-4B-v3	ecs.gn7e edition ecs.gn7i edition
NVIDIA Retrieval QA E5 Embedding v5	NV-EmbedQA-E5-v5	ecs.gn7e edition ecs.gn7i edition
Parakeet CTC Riva 1.1b	Parakeet-CTC-1.1b-asr	General-purpose GPU
FastPitch HifiGAN Riva	FastPitchHifi-GAN	General-purpose GPU
VISTA-3D	Vista-3D	General-purpose GPU
AlphaFold2	AlphaFold2	General-purpose GPU
ProteinMPNN	ProteinMPNN	General-purpose GPU
megatron-1b-nmt	megatron-1b-nmt	General-purpose GPU

Deploy a model in Model Gallery of PAI

Go to Model Gallery.
In the left-side filter pane, select NIM for Model Source to search for NIM models.
Find the model that you want to manage and click the model card. In the upper-right corner of the Overview tab, click Deploy.
Note
Before you deploy an NIM model in PAI, make sure that you are an NVIDIA Al Enterprise user or an NVIDIA Developer Program user.
In the Deploy panel, configure the resources and click Deploy. For information about how to call the deployed model service, see the introduction on the Overview tab of the model details page.

Deploy a model on your on-premises machine

You can download the image and model files of an NIM model and deploy the model on your on-premises machine. Before you deploy an NIM model in PAI, make sure that you are an NVIDIA Al Enterprise user or an NVIDIA Developer Program user.

Configure the environment. For more information, see Getting Started.
On the Overview tab of the model details page, click Resource Download. In the dialog box that appears, select the download statements and click OK to obtain the image URL and model URL.
Run the following command to pull the image: (Replace ${Image URL} with the actual image URL.)
```
docker pull ${Image URL}
```
Download the model file using the Alibaba Cloud tool ossutil.
Run the following commands to start the container: (In this example, the /local/model/ directory is used. Replace ${Model URL} with the actual model URL and replace ${Image URL} with the actual image URL.)
```
docker run --rm \
--runtime=nvidia \
--gpus all \
-u $(id -u) \
-v /local/model/:${Model URL} ${Image URL}
```

Appendix: Activation process for using PAI

For users new to Alibaba Cloud, the following steps outline the initial setup process for using Model Gallery of PAI:

Visit Alibaba Cloud, click Sign Up in the upper-right corner, and then enter your account and password, or register a new user.
After logon and real-name verification, visit Platform for AI (PAI).
If you use PAI for the first time, you must complete real-name verification and authorization. Retain the default settings for all options and click Activate. After a few seconds, PAI is activated. Then, you can access the default workspace for deployment and usage.