All Products
Search
Document Center

Platform For AI:Use ComfyUI to deploy an AI video generation model service

Last Updated:Apr 29, 2025

You can use ComfyUI to create complex AI-powered content generation processes based on nodes. This helps generate short videos and animations on social media platforms. Elastic Algorithm Service (EAS) of Platform for AI (PAI) allows you to quickly deploy an AI video generation service based on ComfyUI and Stable Video Diffusion models. This topic describes how to deploy a service based on a ComfyUI image and common methods for calling a service.

Select a deployment method

The following table describes the available editions for scenario-based deployment: Standard Edition, API Edition, Cluster Edition WebUI, and Serverless.

Suitable scenario

Call method

Billing method

Standard Edition

A single user calls the service on the WebUI or by using API operations when the service is deployed on a single instance.

  • WebUI

  • Online debugging

  • Synchronous API calls

You are charged based on your deployment configurations. For more information, see Billing of EAS.

API Edition

High concurrency is required. The system creates a queue service instance in addition to the asynchronous inference service instances.

Asynchronous API calls

Cluster Edition WebUI

Multiple users call the service on the WebUI at the same time for group work or teaching purposes.

WebUI

Serverless

The demand computing resources fluctuate significantly. The system automatically scales the service based on your service requests.

WebUI

The Serverless edition is free of charge. You are billed based on the duration of service calls.

  • Cluster Edition WebUI: Each user has their own backend environment and working directory. This helps implement efficient GPU sharing and file management. For information about how a Cluster Edition WebUI service works, see Principles of the Cluster Edition WebUI service.

  • Serverless: The Serverless edition is supported only in the China (Shanghai) and China (Hangzhou) regions.

If scenario-based deployment cannot meet your business requirements, you can deploy a custom model by using the Alibaba Cloud images of the Standard Edition, Cluster Edition WebUI, and API Edition based on your business requirements. You can also configure parameters for custom deployment in the console to implement other features.

Deploy a model service in EAS

Method 1: Scenario-based model deployment (recommend)

  1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).

  2. On the Elastic Algorithm Service (EAS) page, click Deploy Service. In the Scenario-based Model Deployment section, click AI Video Generation: ComfyUI-based Deployment.

  3. On the AI Video Generation: ComfyUI-based Deployment page, configure the key parameters described in the following table.

    Parameter

    Description

    Basic Information

    Service Name

    The name of the model service.

    Edition

    The edition of the service. For more information, see Select a deployment method.

    Model Settings

    You must add model settings in the following scenarios:

    Supported types:

    • OSS: Click the image icon to select an existing Object Storage Service (OSS) directory.

    • NAS: Configure a File Storage NAS (NAS) mount target and NAS source path.

    Resource Configuration

    Instance Count

    If you select Standard Edition, we recommend that you set the value to 1.

    Resource Configuration

    We recommend that you use the GU30, A10 or T4 GPU types. By default, the system uses the ml.gu7i.c16m60.1-gu30 (GPU-accelerated) instance type to ensure cost-effectiveness.

    Note

    ComfyUI supports only single-GPU mode, which means tasks can run on a single-GPU instance or multiple single-GPU instances. ComfyUI does not support multi-GPU concurrent operations.

  4. Click Deploy. The deployment requires approximately 5 minutes to complete. If Service Status changes to Running, the service is deployed.

Method 2: Custom deployment

  1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).

  2. Click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.

  3. On the Custom Deployment page, configure the key parameters described in the following table.

    Parameter

    Description

    Basic Information

    Service Name

    The name of the service. In this example, the name comfyui_svd_demo is used.

    Environment Information

    Deployment Method

    Select Image-based Deployment and Enable Web App.

    Image Configuration

    Select comfyui:1.7 from Alibaba Cloud Image. Take note of the following version suffixes:

    • x.x: indicates Standard Edition.

    • x.x.-api: indicates API version.

    • x.x.-cluster: indicates Cluster Edition WebUI.

    Note
    • The image version is updated frequently. We recommend that you select the latest version.

    • For information about the suitable scenarios for each edition, see Select a deployment method.

    Model Settings

    You must add model settings in the following scenarios:

    Supported types:

    • OSS

      • OSS: Click the image icon to select an existing OSS directory. Example: oss://bucket-test/data-oss/.

      • Mount Path: Enter /code/data-oss. The OSS directory is mounted to the /code/data-oss path of the image.

    • General-purpose NAS

      • Select a file system: Select a NAS file system.

      • Mount Target: the mount target of the NAS file system. The EAS service uses the mount target to access the NAS file system.

      • File System Path: the NAS path in which the files are stored. Example: /data-oss.

      • Mount Path: Enter /code/data-oss. The path of the NAS file system is mounted to the /code/data-oss path of the image.

    Command

    • After you configure the image version, the system automatically sets the value to python main.py --listen --port 8000.

    • Port number: 8000.

    After you configure the model settings, you must append the --data-dir mount directory to the command that you enter. The mount directory must be the same as the Mount Path that you specify in the Model Settings section. Example: python main.py --listen --port 8000 --data-dir /code/data-oss.

    Resource Information

    Resource Type

    Select Public Resources.

    Instances

    If you use Standard Edition, we recommend that you set the value to 1.

    Deployment Resources

    You must select a GPU-accelerated instance type. We recommend that you use ml.gu7i.c16m60.1-gu30 to ensure cost-effectiveness. If this instance type is unavailable, you can select the ecs.gn6i-c16g1.4xlarge instance type.

    Note

    ComfyUI supports only single-GPU mode, which means tasks can run on a single-GPU instance or multiple single-GPU instances. ComfyUI does not support multi-GPU concurrent operations.

  4. Click Deploy. The deployment requires approximately 5 minutes to complete. If Service Status changes to Running, the service is deployed.

Call an EAS service

Call an EAS service on the WebUI

You can call EAS services of Standard Edition, Cluster Edition WebUI, or Serverless on the WebUI.

Procedure:

  1. Find the service that you want to call and click View Web App in the Service Type column.

    Note

    The WebUI page takes approximately 1 minute to load.

  2. Perform model inference on the WebUI page.

    Select a model for text-to-image generation and a model for image-to-video generation. In this example, the default settings are used. Enter text prompts in the CLIP Text Encode (Prompt) editor, such as Rocket takes off from the ground, fire, sky, airplane and click Queue Prompt. The system starts to run the workflow and generate the video.85453c9fcadd222fbb087c5acddb6e90.png

  3. Right-click the generated video and select Save Image to save the generated video to your on-premises device.image.png

    Sample video:

Call an EAS service by using API operations

You can use API operations to call services. Standard Edition supports synchronous calls and online debugging. API Edition supports asynchronous calls.

Note

Common definitions of synchronous calls and asynchronous calls in EAS:

  • Synchronous calls directly send requests to the inference instance without using the queue service of EAS.

  • Asynchronous calls use the queue service of EAS to send requests to the input queue. You can subscribe to the queue service to obtain asynchronous inference results.

ComfyUI processes synchronous calls in an asynchronous manner. After you submit a request, the system returns a prompt ID. You need to use the prompt ID to poll the inference results.

1. Generate a request body

You can obtain the API request body of ComfyUI by setting a workflow. You must configure the workflow on the WebUI and save the API format to obtain a JSON file of the workflow.

  • For synchronous calls, the JSON file content must be included in the prompt field of the request body.

  • For asynchronous calls, the request body is the the JSON file content itself.

Procedure for obtaining the API request body

  1. On the WebUI page, click the image icon. In the Settings dialog box, select Enable Dev mode Options.image

  2. Configure the workflow based on your business requirements.

    You can select a model in the Load Checkpoint editor, enter prompts in the CLIP Text Encode (Prompt) editor, or adjust sampler configurations. After you complete the configuration, click Queue Prompt to obtain the AI-generated video.

  3. If the workflow works as expected, click Save (API Format) to download the JSON file of the workflow.image

    Sample request body for synchronous calls

    {
        "prompt": {
            "3": {
                "inputs": {
                    "seed": 367490676387803,
                    "steps": 40,
                    "cfg": 7,
                    "sampler_name": "dpmpp_sde_gpu",
                    "scheduler": "karras",
                    "denoise": 1,
                    "model": [
                        "4",
                        0
                    ],
                    "positive": [
                        "6",
                        0
                    ],
                    "negative": [
                        "7",
                        0
                    ],
                    "latent_image": [
                        "5",
                        0
                    ]
                },
                "class_type": "KSampler",
                "_meta": {
                    "title": "K sampler"
                }
            },
            "4": {
                "inputs": {
                    "ckpt_name": "LandscapeBING_v10.safetensors"
                },
                "class_type": "CheckpointLoaderSimple",
                "_meta": {
                    "title": "Checkpoint loader (simple)"
                }
            },
            "5": {
                "inputs": {
                    "width": 720,
                    "height": 1280,
                    "batch_size": 1
                },
                "class_type": "EmptyLatentImage",
                "_meta": {
                    "title": "Empty Latent"
                }
            },
            "6": {
                "inputs": {
                    "text": "Rocket takes off from the ground, fire, sky, airplane",
                    "clip": [
                        "4",
                        1
                    ]
                },
                "class_type": "CLIPTextEncode",
                "_meta": {
                    "title": "CLIP text encoder"
                }
            },
            "7": {
                "inputs": {
                    "text": "",
                    "clip": [
                        "4",
                        1
                    ]
                },
                "class_type": "CLIPTextEncode",
                "_meta": {
                    "title": "CLIP text encoder"
                }
            },
            "8": {
                "inputs": {
                    "samples": [
                        "3",
                        0
                    ],
                    "vae": [
                        "4",
                        2
                    ]
                },
                "class_type": "VAEDecode",
                "_meta": {
                    "title": "VAE decoding"
                }
            },
            "9": {
                "inputs": {
                    "filename_prefix": "ComfyUI",
                    "images": [
                        "8",
                        0
                    ]
                },
                "class_type": "SaveImage",
                "_meta": {
                    "title": "Save the image"
                }
            },
            "13": {
                "inputs": {
                    "seed": 510424455529432,
                    "steps": 40,
                    "cfg": 2.5,
                    "sampler_name": "euler_ancestral",
                    "scheduler": "karras",
                    "denoise": 1,
                    "model": [
                        "17",
                        0
                    ],
                    "positive": [
                        "16",
                        0
                    ],
                    "negative": [
                        "16",
                        1
                    ],
                    "latent_image": [
                        "16",
                        2
                    ]
                },
                "class_type": "KSampler",
                "_meta": {
                    "title": "K sampler"
                }
            },
            "14": {
                "inputs": {
                    "samples": [
                        "13",
                        0
                    ],
                    "vae": [
                        "18",
                        2
                    ]
                },
                "class_type": "VAEDecode",
                "_meta": {
                    "title": "VAE decoding"
                }
            },
            "15": {
                "inputs": {
                    "filename_prefix": "ComfyUI",
                    "fps": 10,
                    "lossless": false,
                    "quality": 85,
                    "method": "default",
                    "images": [
                        "14",
                        0
                    ]
                },
                "class_type": "SaveAnimatedWEBP",
                "_meta": {
                    "title": "Save WEBP"
                }
            },
            "16": {
                "inputs": {
                    "width": 512,
                    "height": 768,
                    "video_frames": 35,
                    "motion_bucket_id": 140,
                    "fps": 15,
                    "augmentation_level": 0.15,
                    "clip_vision": [
                        "18",
                        1
                    ],
                    "init_image": [
                        "8",
                        0
                    ],
                    "vae": [
                        "18",
                        2
                    ]
                },
                "class_type": "SVD_img2vid_Conditioning",
                "_meta": {
                    "title": "SVD_Image to Video_Condition"
                }
            },
            "17": {
                "inputs": {
                    "min_cfg": 1,
                    "model": [
                        "18",
                        0
                    ]
                },
                "class_type": "VideoLinearCFGGuidance",
                "_meta": {
                    "title": "Linear CFG Bootstrap"
                }
            },
            "18": {
                "inputs": {
                    "ckpt_name": "svd_xt_image_decoder.safetensors"
                },
                "class_type": "ImageOnlyCheckpointLoader",
                "_meta": {
                    "title": "Checkpoint loader (image only)"
                }
            },
            "19": {
                "inputs": {
                    "frame_rate": 10,
                    "loop_count": 0,
                    "filename_prefix": "comfyUI",
                    "format": "video/h264-mp4",
                    "pix_fmt": "yuv420p",
                    "crf": 20,
                    "save_metadata": true,
                    "pingpong": false,
                    "save_output": true,
                    "images": [
                        "14",
                        0
                    ]
                },
                "class_type": "VHS_VideoCombine",
                "_meta": {
                    "title": "Merge to video"
                }
            }
        }
    }

2. Initiate a call

Online debugging

Only Standard Edition services support online debugging.

  1. On the Elastic Algorithm Service (EAS) page, find the service that you want to debug and click Online Debugging in the Actions column.

  2. Send a POST request to obtain the prompt ID.

    1. In the Request Parameter Online Tuning section, enter /prompt in the request URL input box and enter the request body in the Body code editor.image

    2. Click Send Request and view the returned result in the Debugging Information section. The following figure shows an example.image

  3. Send a GET request to obtain the inference result based on the prompt ID.

    1. In the Request Parameter Online Tuning section, select GET from the drop-down list and enter /history/<prompt id> in the input box.image

      Replace <prompt id> with the prompt ID that you obtained in Step 3.

    2. Click Send Request to obtain the inference result.

      You can view the inference result in the output directory of the mounted storage.

Synchronous calls

Only Standard Edition services support synchronous calls.

  1. View the information about API calls.

    1. In the service list, click the name of the Standard Edition service. In the Basic Information section, click View Endpoint Information.

    2. On the Public Endpoint tab of the Invocation Method dialog box, obtain the service endpoint and token.image

  2. Send a POST request to obtain the prompt ID.

    Curl

    • HTTP request method: POST

    • Request URL: <service_url>/prompt

    • Request header:

      Request header

      Value

      Description

      Authorization

      <token>

      The authorization key.

      Content-Type

      application/json

      The format of the request body.

    • Sample code

      curl --location --request POST '<service_url>/prompt' \
      --header 'Authorization: <token>' \
      --header 'Content-Type: application/json' \
      --data-raw '{
          "prompt":
          Omitted
      }'

      The following table describes the key parameters.

      Parameter

      Description

      <service_url>

      Replace the value with the endpoint that you obtained in Step 1. Delete the forward slash ( /) at the end of the endpoint. Example: http://comfyui****.175805416243****.cn-beijing.pai-eas.aliyuncs.com.

      <token>

      Replace the value with the token that you obtained in Step 1. Example: ZGJmNzcwYjczODE1MmVlNWY1NTNiNGYxNDkzODI****NzU2NTFiOA==.

      data-raw

      Set the value to the request body. Example:

      Important

      The first letter of the Boolean value (true and false) in the request body must be lowercase.

      Click to view a sample request body

      {
          "prompt": {
              "3": {
                  "inputs": {
                      "seed": 367490676387803,
                      "steps": 40,
                      "cfg": 7,
                      "sampler_name": "dpmpp_sde_gpu",
                      "scheduler": "karras",
                      "denoise": 1,
                      "model": [
                          "4",
                          0
                      ],
                      "positive": [
                          "6",
                          0
                      ],
                      "negative": [
                          "7",
                          0
                      ],
                      "latent_image": [
                          "5",
                          0
                      ]
                  },
                  "class_type": "KSampler",
                  "_meta": {
                      "title": "K sampler"
                  }
              },
              "4": {
                  "inputs": {
                      "ckpt_name": "LandscapeBING_v10.safetensors"
                  },
                  "class_type": "CheckpointLoaderSimple",
                  "_meta": {
                      "title": "Checkpoint loader (simple)"
                  }
              },
              "5": {
                  "inputs": {
                      "width": 720,
                      "height": 1280,
                      "batch_size": 1
                  },
                  "class_type": "EmptyLatentImage",
                  "_meta": {
                      "title": "Empty Latent"
                  }
              },
              "6": {
                  "inputs": {
                      "text": "Rocket takes off from the ground, fire,sky, airplane",
                      "clip": [
                          "4",
                          1
                      ]
                  },
                  "class_type": "CLIPTextEncode",
                  "_meta": {
                      "title": "CLIP text encoder"
                  }
              },
              "7": {
                  "inputs": {
                      "text": "",
                      "clip": [
                          "4",
                          1
                      ]
                  },
                  "class_type": "CLIPTextEncode",
                  "_meta": {
                      "title": "CLIP text encoder"
                  }
              },
              "8": {
                  "inputs": {
                      "samples": [
                          "3",
                          0
                      ],
                      "vae": [
                          "4",
                          2
                      ]
                  },
                  "class_type": "VAEDecode",
                  "_meta": {
                      "title": "VAE decoding"
                  }
              },
              "9": {
                  "inputs": {
                      "filename_prefix": "ComfyUI",
                      "images": [
                          "8",
                          0
                      ]
                  },
                  "class_type": "SaveImage",
                  "_meta": {
                      "title": "Save the image"
                  }
              },
              "13": {
                  "inputs": {
                      "seed": 510424455529432,
                      "steps": 40,
                      "cfg": 2.5,
                      "sampler_name": "euler_ancestral",
                      "scheduler": "karras",
                      "denoise": 1,
                      "model": [
                          "17",
                          0
                      ],
                      "positive": [
                          "16",
                          0
                      ],
                      "negative": [
                          "16",
                          1
                      ],
                      "latent_image": [
                          "16",
                          2
                      ]
                  },
                  "class_type": "KSampler",
                  "_meta": {
                      "title": "K sampler"
                  }
              },
              "14": {
                  "inputs": {
                      "samples": [
                          "13",
                          0
                      ],
                      "vae": [
                          "18",
                          2
                      ]
                  },
                  "class_type": "VAEDecode",
                  "_meta": {
                      "title": "VAE decoding"
                  }
              },
              "15": {
                  "inputs": {
                      "filename_prefix": "ComfyUI",
                      "fps": 10,
                      "lossless": false,
                      "quality": 85,
                      "method": "default",
                      "images": [
                          "14",
                          0
                      ]
                  },
                  "class_type": "SaveAnimatedWEBP",
                  "_meta": {
                      "title": "Save WEBP"
                  }
              },
              "16": {
                  "inputs": {
                      "width": 512,
                      "height": 768,
                      "video_frames": 35,
                      "motion_bucket_id": 140,
                      "fps": 15,
                      "augmentation_level": 0.15,
                      "clip_vision": [
                          "18",
                          1
                      ],
                      "init_image": [
                          "8",
                          0
                      ],
                      "vae": [
                          "18",
                          2
                      ]
                  },
                  "class_type": "SVD_img2vid_Conditioning",
                  "_meta": {
                      "title": "SVD_Image to Video_Condition"
                  }
              },
              "17": {
                  "inputs": {
                      "min_cfg": 1,
                      "model": [
                          "18",
                          0
                      ]
                  },
                  "class_type": "VideoLinearCFGGuidance",
                  "_meta": {
                      "title": "Linear CFG Bootstrap"
                  }
              },
              "18": {
                  "inputs": {
                      "ckpt_name": "svd_xt_image_decoder.safetensors"
                  },
                  "class_type": "ImageOnlyCheckpointLoader",
                  "_meta": {
                      "title": "Checkpoint loader (image only)"
                  }
              },
              "19": {
                  "inputs": {
                      "frame_rate": 10,
                      "loop_count": 0,
                      "filename_prefix": "comfyUI",
                      "format": "video/h264-mp4",
                      "pix_fmt": "yuv420p",
                      "crf": 20,
                      "save_metadata": true,
                      "pingpong": false,
                      "save_output": true,
                      "images": [
                          "14",
                          0
                      ]
                  },
                  "class_type": "VHS_VideoCombine",
                  "_meta": {
                      "title": "Merge to video"
                  }
              }
          }
      }

    Python

    Sample code:

    import requests
    
    url = "<service_url>/prompt"
    
    payload = {
        "prompt":
        Omitted
    }
    
    session = requests.session()
    session.headers.update({"Authorization":"<token>"})
    
    
    response = session.post(url=f'{url}', json=payload)
    if response.status_code != 200:
        raise Exception(response.content)
    
    data = response.json()
    print(data)

    The following table describes the key parameters.

    Parameter

    Cluster Description

    <service_url>

    Replace the value with the endpoint that you obtained in Step 1. Delete the forward slash (/) at the end of the endpoint. Example: http://comfyui****.175805416243****.cn-beijing.pai-eas.aliyuncs.com.

    <token>

    Replace the value with the token that you obtained in Step 1. ZGJmNzcwYjczODE1MmVlNWY1NTNiNGYxNDkzODI****NzU2NTFiOA==

    payload

    Set the value to the request body. Example:

    Important

    The first letter of the Boolean value (True and False) in the request body must be uppercase.

    Click to view a sample request body

    {
        "prompt": {
            "3": {
                "inputs": {
                    "seed": 367490676387803,
                    "steps": 40,
                    "cfg": 7,
                    "sampler_name": "dpmpp_sde_gpu",
                    "scheduler": "karras",
                    "denoise": 1,
                    "model": [
                        "4",
                        0
                    ],
                    "positive": [
                        "6",
                        0
                    ],
                    "negative": [
                        "7",
                        0
                    ],
                    "latent_image": [
                        "5",
                        0
                    ]
                },
                "class_type": "KSampler",
                "_meta": {
                    "title": "K sampler"
                }
            },
            "4": {
                "inputs": {
                    "ckpt_name": "LandscapeBING_v10.safetensors"
                },
                "class_type": "CheckpointLoaderSimple",
                "_meta": {
                    "title": "Checkpoint loader (simple)"
                }
            },
            "5": {
                "inputs": {
                    "width": 720,
                    "height": 1280,
                    "batch_size": 1
                },
                "class_type": "EmptyLatentImage",
                "_meta": {
                    "title": "Empty Latent"
                }
            },
            "6": {
                "inputs": {
                    "text": "Rocket takes off from the ground, fire,sky, airplane",
                    "clip": [
                        "4",
                        1
                    ]
                },
                "class_type": "CLIPTextEncode",
                "_meta": {
                    "title": "CLIP text encoder"
                }
            },
            "7": {
                "inputs": {
                    "text": "",
                    "clip": [
                        "4",
                        1
                    ]
                },
                "class_type": "CLIPTextEncode",
                "_meta": {
                    "title": "CLIP text encoder"
                }
            },
            "8": {
                "inputs": {
                    "samples": [
                        "3",
                        0
                    ],
                    "vae": [
                        "4",
                        2
                    ]
                },
                "class_type": "VAEDecode",
                "_meta": {
                    "title": "VAE decoding"
                }
            },
            "9": {
                "inputs": {
                    "filename_prefix": "ComfyUI",
                    "images": [
                        "8",
                        0
                    ]
                },
                "class_type": "SaveImage",
                "_meta": {
                    "title": "Save the image"
                }
            },
            "13": {
                "inputs": {
                    "seed": 510424455529432,
                    "steps": 40,
                    "cfg": 2.5,
                    "sampler_name": "euler_ancestral",
                    "scheduler": "karras",
                    "denoise": 1,
                    "model": [
                        "17",
                        0
                    ],
                    "positive": [
                        "16",
                        0
                    ],
                    "negative": [
                        "16",
                        1
                    ],
                    "latent_image": [
                        "16",
                        2
                    ]
                },
                "class_type": "KSampler",
                "_meta": {
                    "title": "K sampler"
                }
            },
            "14": {
                "inputs": {
                    "samples": [
                        "13",
                        0
                    ],
                    "vae": [
                        "18",
                        2
                    ]
                },
                "class_type": "VAEDecode",
                "_meta": {
                    "title": "VAE decoding"
                }
            },
            "15": {
                "inputs": {
                    "filename_prefix": "ComfyUI",
                    "fps": 10,
                    "lossless": False,
                    "quality": 85,
                    "method": "default",
                    "images": [
                        "14",
                        0
                    ]
                },
                "class_type": "SaveAnimatedWEBP",
                "_meta": {
                    "title": "Save WEBP"
                }
            },
            "16": {
                "inputs": {
                    "width": 512,
                    "height": 768,
                    "video_frames": 35,
                    "motion_bucket_id": 140,
                    "fps": 15,
                    "augmentation_level": 0.15,
                    "clip_vision": [
                        "18",
                        1
                    ],
                    "init_image": [
                        "8",
                        0
                    ],
                    "vae": [
                        "18",
                        2
                    ]
                },
                "class_type": "SVD_img2vid_Conditioning",
                "_meta": {
                    "title": "SVD_Image to Video_Condition"
                }
            },
            "17": {
                "inputs": {
                    "min_cfg": 1,
                    "model": [
                        "18",
                        0
                    ]
                },
                "class_type": "VideoLinearCFGGuidance",
                "_meta": {
                    "title": "Linear CFG Bootstrap"
                }
            },
            "18": {
                "inputs": {
                    "ckpt_name": "svd_xt_image_decoder.safetensors"
                },
                "class_type": "ImageOnlyCheckpointLoader",
                "_meta": {
                    "title": "Checkpoint loader (image only)"
                }
            },
            "19": {
                "inputs": {
                    "frame_rate": 10,
                    "loop_count": 0,
                    "filename_prefix": "comfyUI",
                    "format": "video/h264-mp4",
                    "pix_fmt": "yuv420p",
                    "crf": 20,
                    "save_metadata": True,
                    "pingpong": False,
                    "save_output": True,
                    "images": [
                        "14",
                        0
                    ]
                },
                "class_type": "VHS_VideoCombine",
                "_meta": {
                    "title": "Merge to video"
                }
            }
        }
    }

    Sample response:

    {
        "prompt_id": "021ebc5b-e245-4e37-8bd3-00f7b949****",
        "number": 5,
        "node_errors": {}
    }

    You can obtain the prompt ID from the response.

  3. Send a request to obtain the inference result.

    Curl

    • HTTP request method: GET

    • Request URL: <service_url>/history/<prompt_id>

    • Request header:

      Request header

      Value

      Description

      Authorization

      <token>

      The authorization key.

    • Sample code:

      curl --location --request GET '<service_url>/history/<prompt_id>' \
           --header 'Authorization: <token>'

      The following table describes the key parameters.

      Parameter

      Cluster Description

      <service_url>

      Replace the value with the endpoint that you obtained in Step 1. Delete the forward slash ( /) at the end of the endpoint. Example: http://comfyui****.175805416243****.cn-beijing.pai-eas.aliyuncs.com.

      <token>

      Replace the value with the token that you obtained in Step 1. Example: ZGJmNzcwYjczODE1MmVlNWY1NTNiNGYxNDkzODI****NzU2NTFiOA==.

      <prompt_id>

      Replace the value with the prompt ID that you obtained in Step 2.

    Python

    Sample code:

    import requests
    
    # Create the request URL. 
    url = "<service_url>/history/<prompt_id>"
    
    session = requests.session()
    session.headers.update({"Authorization":"<token>"})
    
    response = session.get(url=f'{url}')
    
    if response.status_code != 200:
        raise Exception(response.content)
    
    data = response.json()
    print(data)

    The following table describes the parameters.

    Parameter

    Description

    <service_url>

    Replace the value with the endpoint that you obtained in Step 1. Delete the forward slash (/) at the end of the endpoint. Example: http://comfyui****.175805416243****.cn-beijing.pai-eas.aliyuncs.com.

    <token>

    Replace the value with the token that you obtained in Step 1. Example: ZGJmNzcwYjczODE1MmVlNWY1NTNiNGYxNDkzODI****NzU2NTFiOA==.

    <prompt_id>

    Replace the value with the prompt ID that you obtained in Step 2.

    Sample response:

    Click to view a sample response

    {
        "130bcd6b-5bb5-496c-9c8c-3a1359a0****": {
            "prompt": ...omitted,
            "outputs": {
                "9": {
                    "images": [
                        {
                            "filename": "ComfyUI_1712645398_18dba34d-df87-4735-a577-c63d5506a6a1_.png",
                            "subfolder": "",
                            "type": "output"
                        }
                    ]
                },
                "15": {
                    "images": [
                        {
                            "filename": "ComfyUI_1712645867_.webp",
                            "subfolder": "",
                            "type": "output"
                        }
                    ],
                    "animated": [
                        true
                    ]
                },
                "19": {
                    "gifs": [
                        {
                            "filename": "comfyUI_00002.mp4",
                            "subfolder": "",
                            "type": "output",
                            "format": "video/h264-mp4"
                        }
                    ]
                }
            },
            "status": {
                "status_str": "success",
                "completed": true,
                "messages": ...omitted,
            }
        }
    }
    

    In the preceding sample response, the generated image, WEBP file, and MP4 video are returned in the outputs section. You can find the files by name in the output directory of the mounted storage.

Asynchronous calls

Only API Edition services support asynchronous calls, and the calls are sent by using api_prompt.

  1. View the information about API calls.

    Click Invocation Method in the Service Type column of the service. In the Invocation Method dialog box, view the service endpoint and token on the Asynchronous Invocation tab. image

  2. Send request.

    Sample code:

    import requests,io,base64
    from PIL import Image, PngImagePlugin
    
    url = "<service_url>"
    session = requests.session()
    session.headers.update({"Authorization":"<token>"})
    
    work_flow = {
        '3': 
        ...Omitted
      }
    
    for i in range(5):
      payload = work_flow
      response = session.post(url=f'{url}/api_prompt?task_id=txt2img_{i}', json=payload)
      if response.status_code != 200:
        exit(f"send request error:{response.content}")
      else:
        print(f"send {i} success, index is {response.content}")

    The following table describes the key parameters.

    Parameter

    Description

    <service_url>

    Replace the value with the endpoint that you obtained in Step 1. Delete the forward slash (/) at the end of the endpoint. Example: http://175805416243****.cn-beijing.pai-eas.aliyuncs.com/api/predict/comfyui_api.

    <token>

    Replace the value with the token that you obtained in Step 1. Example: ZTJhM****TBhMmJkYjM3M2U0NjM1NGE3OGNlZGEyZTdjYjlm****Nw==.

    work_flow

    Configure the request body, which is the content of the JSON file of the workflow.

    Important

    The first letter of the Boolean values (True and False) in the file must be uppercase.

    Click to view a sample JSON file

    {
      "3": {
        "inputs": {
          "seed": 1021224598837526,
          "steps": 40,
          "cfg": 7,
          "sampler_name": "dpmpp_sde_gpu",
          "scheduler": "karras",
          "denoise": 1,
          "model": [
            "4",
            0
          ],
          "positive": [
            "6",
            0
          ],
          "negative": [
            "7",
            0
          ],
          "latent_image": [
            "5",
            0
          ]
        },
        "class_type": "KSampler",
        "_meta": {
          "title": "K sampler"
        }
      },
      "4": {
        "inputs": {
          "ckpt_name": "LandscapeBING_v10.safetensors"
        },
        "class_type": "CheckpointLoaderSimple",
        "_meta": {
          "title": "Checkpoint loader (simple)"
        }
      },
      "5": {
        "inputs": {
          "width": 720,
          "height": 1280,
          "batch_size": 1
        },
        "class_type": "EmptyLatentImage",
        "_meta": {
          "title": "Empty Latent"
        }
      },
      "6": {
        "inputs": {
          "text": "Rocket takes off from the ground, fire, sky, airplane",
          "clip": [
            "4",
            1
          ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
          "title": "CLIP text encoder"
        }
      },
      "7": {
        "inputs": {
          "text": "",
          "clip": [
            "4",
            1
          ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
          "title": "CLIP text encoder"
        }
      },
      "8": {
        "inputs": {
          "samples": [
            "3",
            0
          ],
          "vae": [
            "4",
            2
          ]
        },
        "class_type": "VAEDecode",
        "_meta": {
          "title": "VAE decoding"
        }
      },
      "9": {
        "inputs": {
          "filename_prefix": "ComfyUI",
          "images": [
            "8",
            0
          ]
        },
        "class_type": "SaveImage",
        "_meta": {
          "title": "Save the image"
        }
      },
      "13": {
        "inputs": {
          "seed": 1072245043382649,
          "steps": 40,
          "cfg": 2.5,
          "sampler_name": "euler_ancestral",
          "scheduler": "karras",
          "denoise": 1,
          "model": [
            "17",
            0
          ],
          "positive": [
            "16",
            0
          ],
          "negative": [
            "16",
            1
          ],
          "latent_image": [
            "16",
            2
          ]
        },
        "class_type": "KSampler",
        "_meta": {
          "title": "K sampler"
        }
      },
      "14": {
        "inputs": {
          "samples": [
            "13",
            0
          ],
          "vae": [
            "18",
            2
          ]
        },
        "class_type": "VAEDecode",
        "_meta": {
          "title": "VAE decoding"
        }
      },
      "15": {
        "inputs": {
          "filename_prefix": "ComfyUI",
          "fps": 10,
          "lossless": False,
          "quality": 85,
          "method": "default",
          "images": [
            "14",
            0
          ]
        },
        "class_type": "SaveAnimatedWEBP",
        "_meta": {
          "title": "Save WEBP"
        }
      },
      "16": {
        "inputs": {
          "width": 512,
          "height": 768,
          "video_frames": 35,
          "motion_bucket_id": 140,
          "fps": 15,
          "augmentation_level": 0.15,
          "clip_vision": [
            "18",
            1
          ],
          "init_image": [
            "8",
            0
          ],
          "vae": [
            "18",
            2
          ]
        },
        "class_type": "SVD_img2vid_Conditioning",
        "_meta": {
          "title": "SVD_Image to Video_Condition"
        }
      },
      "17": {
        "inputs": {
          "min_cfg": 1,
          "model": [
            "18",
            0
          ]
        },
        "class_type": "VideoLinearCFGGuidance",
        "_meta": {
          "title": "Linear CFG Bootstrap"
        }
      },
      "18": {
        "inputs": {
          "ckpt_name": "svd_xt_image_decoder.safetensors"
        },
        "class_type": "ImageOnlyCheckpointLoader",
        "_meta": {
          "title": "Checkpoint loader (image only)"
        }
      },
      "19": {
        "inputs": {
          "frame_rate": 10,
          "loop_count": 0,
          "filename_prefix": "comfyUI",
          "format": "video/h264-mp4",
          "pix_fmt": "yuv420p",
          "crf": 20,
          "save_metadata": True,
          "pingpong": False,
          "save_output": True,
          "images": [
            "14",
            0
          ]
        },
        "class_type": "VHS_VideoCombine",
        "_meta": {
          "title": "Merge to video"
        }
      }
    }
  3. Subscribe to results.

    1. Run the following command to install the eas_prediction SDK:

      pip install eas_prediction  --user
    2. Run the following code to obtain the response:

      from eas_prediction import QueueClient
      
      sink_queue = QueueClient('<service_domain>', '<service_name>/sink')
      sink_queue.set_token('<token>')
      sink_queue.init()
      
      watcher = sink_queue.watch(0, 5, auto_commit=False)
      for x in watcher.run():
          if 'task_id' in x.tags:
              print('index {} task_id is {}'.format(x.index, x.tags['task_id']))
          print(f'index {x.index} data is {x.data}')
          sink_queue.commit(x.index)
      

      The following table describes the key parameters.

      Parameter

      Description

      <service_domain>

      Replace the value with the service endpoint that you obtained in Step 1. Example: 139699392458****.cn-hangzhou.pai-eas.aliyuncs.com.

      <service_name>

      Replace the value with the name of the EAS service.

      <token>

      Replace the value with the token that you obtained in Step 1.

      Sample response:

      index 42 task_id is txt2img_0
      index 42 data is b'[{"type": "executed", "data": {"node": "9", "output": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "prompt_id": "c3c983b6-f92b-4dd5-b4dc-442db4d1736f"}}, {"type": "executed", "data": {"node": "15", "output": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "prompt_id": "c3c983b6-f92b-4dd5-b4dc-442db4d1736f"}}, {"type": "executed", "data": {"node": "19", "output": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}, "prompt_id": "c3c983b6-f92b-4dd5-b4dc-442db4d1736f"}}, {"9": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "15": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "19": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}}]'
      index 43 task_id is txt2img_1
      index 43 data is b'[{"9": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "15": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "19": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}}]'
      index 44 task_id is txt2img_2
      index 44 data is b'[{"9": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "15": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "19": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}}]'
      index 45 task_id is txt2img_3
      index 45 data is b'[{"9": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "15": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "19": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}}]'
      index 46 task_id is txt2img_4
      index 46 data is b'[{"9": {"images": [{"filename": "ComfyUI_1712647318_8e7f3c93-d2a8-4377-92d5-8eb552adc172_.png", "subfolder": "", "type": "output"}]}, "15": {"images": [{"filename": "ComfyUI_1712647895_.webp", "subfolder": "", "type": "output"}], "animated": [true]}, "19": {"gifs": [{"filename": "comfyUI_00001.mp4", "subfolder": "", "type": "output", "format": "video/h264-mp4"}]}}]'

      You can view the inference result files in the output directory of the mounted storage.

Others

Use a workflow in ComfyUI

The following figure shows how to open your workflow in your on-premises file system.

comfyui使用自己的workflow

Mount a custom model and install the ComfyUI plug-in

To use a custom model or install the ComfyUI plug-in, you must mount an OSS or NAS storage to the service. If you use custom deployment, you must append the --data-dir mount directory to the command that you enter in the Command editor. After you deploy the service, the system automatically creates the directory structure shown in the following figure in the mounted OSS bucket or NAS file system.

image

Take note of the following directories:

  • custom_nodes: This directory is used to store the ComfyUI plug-in.

  • models: This directory is used to store model files.

Procedure:

  1. Upload the model file or the plug-in. If you use OSS, upload an object.

Upload the model file

Upload the model file to the subdirectory of the models directory of the mounted storage. Refer to the instructions for the open source project library of the corresponding node to determine the subdirectory to which to upload the model. Examples:

  • For a Checkpoint loader node, you must upload the model to the models/checkpoints path.

  • For a style loader node, you must upload the model to the models/style_models path.

Upload the plug-in

We recommend that you upload the third-party ComfyUI plug-in to the custom_nodes directory of the mounted storage.

Note

We recommend that you do not directly install the plug-in in the Manager. This is because the network connection may fail when pulling code from the GitHub platform.

  1. Restart the service.

    The model or plug-in that you uploaded in the mounted storage takes effect after the service is restarted. The restart process requires approximately 5 minutes. After you click the restart button in the ComfyUI Manager, the service is restarted and runs as expected.

FAQ

What do I do when the service gets stuck or ComfyUI fails to generate images?

In most cases, this is because the resource specifications do not meet your business requirements. Check whether the service image and resource specifications are configured as expected. We recommend that you use the GU30, A10 or T4 GPU types. For example, you can select ml.gu7i.c16m60.1-gu30 to ensure cost-effectiveness.

What do I do if the model loader displays "undefined"?

Check whether the model directory is correctly configured based on the requirements of the model loader.

If you upload a model file when the service is running, restart the service for the model to take effect.

How to accelerate image generation?

Use xFormers, an open source acceleration tool based on Transformer, to effectively accelerate image and video generation and improve GPU utilization. By default, xFormers acceleration is enabled for ComfyUI image-based deployment. The acceleration effect varies depending on the size of the workflow. Models accelerated by GPUs, especially those running on NVIDIA GPUs, have better acceleration effects.

What is the difference between EAS and Function Compute when you deployed the Serverless edition of ComfyUI?

  • EAS: suitable for stateful services with a long running duration. The Serverless edition of ComfyUI deployed in EAS allows you to deploy models as online inference services or AI-powered web applications with few clicks and provides features such as automatic scaling and blue-green deployment. You can deploy ComfyUI in EAS by using scenario-based model deployment or custom deployment.

  • Function Compute: suitable for services that require high-quality image generation. The Serverless edition of ComfyUI deployed in Function Compute is based on a Serverless architecture and uses the pay-as-you-go billing method. It provides features such as automatic scaling and allows you to use ComfyUI custom models and install plug-ins. You can create applications, select ComfyUI templates, and specify configuration items in the Function Compute 3.0 console.

How to check the available model files and ComfyUI plug-ins?

  • If you use a default ComfyUI workflow, check the available model files on the related node. For example, you can view the available model files in the drop-down list of the Checkpoint loader.

  • Right-click the WebUI page and select Add Node from the short-cut menu. You can view all ComfyUI plug-ins that are installed.

References

  • ComfyUI-based API Edition services use asynchronous queues. For more information about the asynchronous calls, see Deploy an asynchronous inference service.

  • EAS also supports the following scenario-based deployment:

    • You can use EAS to deploy an LLM and then call the model on the WebUI or by using API operations. After you deploy an LLM, you can use the LangChain framework to build a Q&A chatbot that is connected to an enterprise knowledge base. For more information, see Quickly deploy LLMs in EAS.

    • You can deploy a Retrieval-Augmented Generation (RAG)-based LLM chatbot. The chatbot is suitable for Q&A, summarization, and other natural language processing (NLP) tasks that rely on specific knowledge bases. For more information, see RAG-based LLM chatbot.

Appendixes

Principles of the Cluster Edition WebUI service

The following figure shows how a Cluster Edition WebUI service works.

image
  • The Cluster Edition WebUI service is suitable for multi-user scenarios. The service decouples the client and backend inference instances to allow multiple users to reuse backend inference instances at different times. This improves instance utilization and reduces inference costs.

  • The proxy is used to manage client processes and inference instances. Your operations are processed in your processes. Only files in public directories and personal directories can be managed. This isolates the working directories of team members. If you want to use an inference instance to process a request, the proxy identifies an idle instance from the available backend inference instances to process the inference request.