This topic describes how to deploy the Grounded-SAM model using Model Gallery and use it for image segmentation and annotation.
Background information
The Grounded SAM model is a powerful zero-shot visual application that can detect, segment, and generate any image from text input. It was created by a professional Chinese team led by IDEA and is based on Meta's SAM model and three other powerful zero-shot models.
The Grounded SAM model consists of two main models: Grounding DINO and Segment Anything Model (SAM). SAM is a zero-shot segmentation model. It can generate masks for any object in an image or video, including objects and images not seen during training. Grounding DINO is an advanced zero-shot detector that can find any object in an image from a text description.
Go to the model details page
Go to the Model Gallery page.
Log on to the PAI console.
In the upper-left corner of the top navigation bar, select a region.
In the left-side navigation pane, click Workspaces and then click the name of the workspace you want to open.
In the left-side navigation pane, choose Quick Start > Model Gallery.
-
On the Model Gallery home page, in the Computer Vision area, click Image Segmentation. Then, in the model list on the right, click the Modelscope_Segment-Anything model card to open the model details page.
Deploy the online service
Model Gallery is integrated with Elastic Algorithm Service (EAS). You can use it to deploy the Grounded-SAM model as an online service and generate a WebUI application. The WebUI application lets you easily test the powerful image segmentation features of the Grounded-SAM model. To deploy the online service from Model Gallery, perform the following steps:
-
On the model details page, click Deploy.
-
Confirm the basic service and resource information.
Model Gallery pre-configures the compute resources and service name for each model deployment based on the model's characteristics. This example uses the default configurations. You can also change the deployment configurations as needed. For more information about how to configure parameters, see Deploy and debug models.
The default Resource Type is public resources. The Instance Type is ecs.gn7i-c8g1.2xlarge (8 vCPU, 30 GiB, NVIDIA A10 × 1), and the number of instances is 1. After you confirm the configurations, click Deploy.
-
Click Deploy. In the Billing reminder dialog box, click OK.
The page automatically redirects to the Service Details page, where you can view the deployment status in the Basic Information section. Because the model is large, the deployment process takes about 10 minutes. When the Status changes to Running, the model service is deployed. You can also view the deployment task on the Inference Service tab of the Elastic Algorithm Service (EAS) page. For more information, see Service deployment.
You can then start the WebUI application to test the inference capabilities of the Grounded SAM model. For more information, see Experience the Grounded SAM model.
Experience the Grounded SAM model
-
On the Service details page for the deployed service, click View Web App in the upper-right corner to start the WebUI application.
-
On the WebUI application page, you can test the model's inference capabilities.
The Grounded SAM model provides multiple task modes. In the Image area on the left side of the page, you can upload the image that you want to segment. At the bottom of the page, select a task_type and enter a corresponding Text Prompt. Then, click Run to start the image segmentation task.
This example describes the following two task modes (task_type). These modes allow you to test the powerful image segmentation features of the Grounded-SAM model.
-
Automatic mode
Set task_type to automatic. You do not need to provide any other information. After you upload an image, click Run to automatically perform image segmentation. This performs non-interactive detection and segmentation.
-
Scribble mode
Set task_type to scribble. You can use your mouse to click and mark the parts of the image that you want to segment. Similar to the automatic mode, you do not need to enter a Text Prompt. Then, click Run to perform image segmentation.
-