Platform For AI:Call a ComfyUI service deployed on EAS from Dify

Solution overview

The core of this solution is to use Dify as an application orchestrator to call the ComfyUI inference service deployed on PAI-EAS. ComfyUI inference tasks are executed asynchronously. The entire call chain is as follows:

1. Submit a task: Dify sends a POST request to the EAS service. The request body contains the ComfyUI workflow in JSON format. When EAS receives the task, it immediately returns a unique prompt_id.

2. Poll the status: Dify uses the prompt_id from the previous step to poll the task execution status by sending GET requests. EAS returns the task status, such as completed, successful, or failed.

3. Retrieve the result: After you confirm that the task is successfully executed, you can send another GET request to retrieve the final result. This result contains the filename of the generated file on Object Storage Service (OSS). Dify concatenates the filename with the OSS endpoint and displays the complete image or video link.

The following figure shows the complete workflow.

Step 1: (Optional) Deploy Dify

1. You can go to the Dify Community Edition product page on Compute Nest and click Deploy Now.

2. Select configurations as needed. For a basic demo, select the Single ECS Version template, set Instance Type to ecs.u1-c1m2.xlarge, set an instance password, and select a zone.

3. Click Next: Confirm Order, and then click Create Now on the next page.

4. After the service is deployed, click Details. In the Use Now section on the Overview tab, find the Dify access link. You can then register an account and log on to create a Dify application.

Step 2: Deploy Wan ComfyUI

1. Go to ModelGallery. In the search box, enter Wan 2.1-Text-to-Video-1.3B and click the card to open the model's product page.

2. In the upper-right corner, click Deploy. On the configuration page, configure the following parameters and leave the other parameters at their default values.

   • Set Deployment Method to ComfyUI > Standard deployment.

   • For Service Configuration, in the JSON editor, set the storage.oss.path parameter to your OSS path, such as oss://examplebucket/wan. This path is the working directory where ComfyUI stores generated content. For more information about OSS operations, see Quick Start for the OSS console.

3. Click Deploy. The deployment is complete in about 3 minutes when the service status changes to Running.

4. On the Service Details page, click View Invocation Information. Obtain the public endpoint and token for the EAS service. This information is required to call the service in Dify.

   

Step 3: Create a Dify application

In Dify, import the sample DSL file wan-comfyui-dify.yml. This topic uses this file to explain the key node settings for calling the ComfyUI service.

def main(arg: str) -> dict:
    data = json.loads(arg)
    return {"result": data["prompt_id"]}

import time

def main()-> dict:
    time.sleep(12)
    return {
        "result": ""
    }

import json

def main(input_str: str) -> dict:
    data = json.loads(input_str)
    images = []
    image_names = []
    first_key = next(iter(data))
    status = data[first_key]["status"]
    
    if status["completed"] and status["status_str"] == "success":
        images = data[first_key]["outputs"]["28"]["images"]
    
    for img in images:
        image_names.append(img["filename"])
    
    # Convert the list to a string
    image_names_str = ",".join(image_names)
    
    return {
        "result": image_names_str,
    }

Step 4: Test the application

In Dify, click Preview in the upper-right corner. Enter the positive and negative prompts. Enter `1` in the dialog box. The workflow does not use this field, so you can enter any value. After you send the request, wait for the video to be generated.

Recommendations for production applications

• Security: In a production environment, setting an OSS bucket to public-read creates a data security risk. You can restrict access to specific IP addresses or VPCs. For more information, see Bucket Policy.

• Stability and cost: Set a reasonable waiting time for polling. A short interval increases the number of API calls and the server load, which can result in unnecessary costs. A long interval increases the user waiting time. You can adjust the interval based on the average execution time of your tasks.

FAQ

1. Why can't I access the generated image or video link?

   Check the following:

   • Check whether the OSS bucket permission is set to public-read.

   • Check whether the `OSS_URL` environment variable in Dify is correctly configured. You must also verify that the OSS domain name matches the region where the bucket is located.

2. What should I do if the workflow is stuck at the polling step?

   This issue usually occurs because the task on the EAS server failed to complete. You can log on to the PAI console and view the real-time log of the EAS service to check for runtime errors.