All Products
Search
Document Center

Platform For AI:Quickly deploy a Stable Diffusion API service in EAS

Last Updated:Apr 29, 2024

Elastic Algorithm Service (EAS) allows you to deploy a synchronous or asynchronous API service for Stable Diffusion (SD). After you deploy the service, you can send synchronous or asynchronous requests and use the responses to verify the model performance. This topic describes how to use EAS to deploy and call an SD API service in a synchronous or asynchronous manner, including the additional parameters that you can use to invoke the API.

Prerequisites

  • Platform for AI (PAI) is activated by using the pay-as-you-go billing method and a default workspace is created. For more information, see Activate PAI and create a default workspace.

  • A general-purpose Apsara File Storage NAS file system is created to store model files. For more information, see Create a NAS file system.

  • An Object Storage Service (OSS) bucket is created to store generated images. For more information, see Create buckets.

Deploy a synchronous service

You can use one of the following methods to deploy a synchronous API service based on your business requirements. We recommend that you use Method 1 the first time you deploy a service.

Method 1: Use the PAI console

  1. Go to the EAS-Online Model Services page.

    1. Log on to the PAI console.

    2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace to which the model service that you want to manage belongs.

    3. In the left-side navigation pane, choose Model Deployment>Elastic Algorithm Service (EAS) to go to the EAS-Online Model Services page. image.png

  2. On the PAI-EAS Model Online Service page, click Deploy Service. In the dialog box that appears, select Custom Deployment and click OK.

  3. On the Create Service page, configure the parameters. The following table describes the key parameters.

    Parameter

    Description

    Service Name

    The name of the service. This example uses the name sdapi_demo.

    Deployment Method

    Select Deploy Service by Using Image.

    Select Image

    Select PAI Image. On the drop-down lists that appear, select stable-diffusion-webui and 3.2.

    Note

    The image version is updated frequently. We recommend that you select the latest version.

    Model Settings

    Click Specify Model Settings to configure the model settings.

    • We recommend that you configure both OSS and Apsara File Storage NAS. The OSS path is used to store the generated images and the NAS file system is used to store the models.

    • You can also configure only OSS or Apsara File Storage NAS.

      • Apsara File Storage NAS allows you to switch models and generate images in a faster manner.

      • OSS allows you to upload files in an easier manner, but model switching and image generation may take a longer time than NAS.

    To configure both OSS and Apsara File Storage NAS, perform the following steps:

    • Configure OSS

      • Set the OSS path to the path of an OSS bucket that you created.

      • Set the Mount Path parameter to /code/stable-diffusion-webui/data-oss.

    • Configure Apsara File Storage NAS

      • NAS Mount Target: Select an existing NAS file system and mount target.

      • NAS Source Path: Set the value to /.

      • Mount Path: Set the value to /code/stable-diffusion-webui/data-nas.

    Command to Run

    Command: Set the value to ./webui.sh --listen --port=8000 --skip-version-check --no-hashing --no-download-sd-model --skip-install --api --filebrowser --data-dir data-nas.

    Port Number: Set the value to 8000.

    Resource Group Type

    Select Public Resource Group from the drop-down list.

    Resource Configuration Mode

    Select General.

    Resource Configuration

    Select GPU. On the Instance Type list that appears, select an instance type. We recommend that you select ml.gu7i.c16m60.1-gu30 for cost efficiency.

    VPC

    The system automatically selects the virtual private cloud (VPC) where the specified NAS file system resides.

  4. Click Deploy. The deployment requires approximately five minutes to complete.

    When the Model Status changes to Running, the service is deployed.

Method 2: Use JSON configuration

  1. Go to the Create Service page as described in Method 1.

  2. Navigate to the Configuration Editor section and click JSON Deployment in the upper right corner. Paste the following code to the editor:

    {
        "metadata": {
            "instance": 1,
            "name": "sd_v32",
        },
        "containers": [
            {
                "image": "eas-registry-vpc.<region>.cr.aliyuncs.com/pai-eas/stable-diffusion-webui:3.2",
                "script": "./webui.sh --listen --port=8000 --skip-version-check --no-hashing --no-download-sd-model --skip-install --api --filebrowser --data-dir data-nas",
                "port": 8000
            }
        ],
        "cloud": {
            "computing": {
                "instance_type": "ml.gu7i.c16m60.1-gu30",
                "instances": null
            },
            "networking": {
                "vpc_id": "vpc-t4nmd6nebhlwwexk2****",
                "vswitch_id": "vsw-t4nfue2s10q2i0ae3****",
                "security_group_id": "sg-t4n85ksesuiq3wez****"
            }
        },
        "storage": [
            {
                "oss": {
                    "path": "oss://examplebucket/data-oss",
                    "readOnly": false
                },
                "properties": {
                    "resource_type": "model"
                },
                "mount_path": "/code/stable-diffusion-webui/data-oss"
            },
            {
                "nfs": {
                    "path": "/",
                    "server": "726434****-aws0.ap-southeast-1.nas.aliyuncs.com"
                },
                "properties": {
                    "resource_type": "model"
                },
                "mount_path": "/code/stable-diffusion-webui/data-nas"
            }
        ]
    } 

    Modify the following parameters in the preceding code:

    • metadata.name: Enter the name of the service.

    • containers.image: Replace <region> with the region ID of your current region. For information about how to obtain the region ID, see Regions and Zones.

    • cloud.networking: Configure the VPC that you want to use, including the VPC ID (vpc_id), vSwitch ID (vswitch_id), and the security group ID (security_group_id). The VPC must be the same as the VPC of the general-purpose NAS file system.

    • storage.oss.path: Set the value to the path of an existing OSS bucket.

    • storage.nfs.server: Set the value to an existing NAS file system.

  3. Click Deploy. The deployment requires approximately five minutes to complete.

    When the Model Status changes to Running, the service is deployed.

Call a synchronous service

After you deploy a synchronous API service, perform the following steps to send a synchronous request to the service:

  1. Obtain the invocation information.

    On the EAS-Online Model Services page, find the service that you deployed and click Invocation Method in the Service Type column. In the Invocation Method dialog box, click the Public Endpoint tab to view the endpoint and token of the service. image.png

  2. Use one of the following methods to send a synchronous request.

    Use a cURL command

    Sample code:

    curl --location --request POST '<service_url>/sdapi/v1/txt2img' \
    --header 'Authorization: <token>' \
    --header 'Content-Type: application/json' \
    --data-raw '{
      "prompt":"cut dog ",
      "steps":20
    }'

    Modify the following parameters in the preceding code:

    • <service_url>: Set this parameter to the endpoint that you obtained in Step 1.

    • <token>: Set this parameter to the token that you obtained in Step 1.

    If the command runs successfully, an image is returned in the Base64-encoded format.

    Use Python code

    For details about how to send requests to the SD API, see the GitHub project wiki.

    • Example 1: Use the following Python code to send a request to obtain an image file

      import requests
      import io
      import base64
      from PIL import Image, PngImagePlugin
      
      url = "<service_url>"
      
      payload = {
          "prompt": "puppy dog",
          "steps": 20,
          "n_iter": 2
      }
      
      session = requests.session()
      session.headers.update({"Authorization": "<token>"})
      
      
      response = session.post(url=f'{url}/sdapi/v1/txt2img', json=payload)
      if response.status_code != 200:
          raise Exception(response.content)
      
      data = response.json()
      
      # Obtain a Base64-encoded image file. We recommend that you obtain an image URL instead, as shown in Example 2. 
      for idx, im in enumerate(data['images']):
          image = Image.open(io.BytesIO(base64.b64decode(im.split(",", 1)[0])))
      
          png_payload = {
              "image": "data:image/png;base64," + im
          }
          resp = session.post(url=f'{url}/sdapi/v1/png-info', json=png_payload)
      
          pnginfo = PngImagePlugin.PngInfo()
          pnginfo.add_text("parameters", resp.json().get("info"))
          image.save(f'output-{idx}.png', pnginfo=pnginfo)
      

      Modify the following parameters in the preceding code:

      • <service_url>: Set this parameter to the endpoint that you obtained in Step 1.

      • <token>: Set this parameter to the token that you obtained in Step 1.

      If the code runs successfully, the returned image file is saved in the directory of the Python code.

    • Example 2: Use the following Python code to send a request to obtain an image URL

      import requests
      import oss2
      
      url = "<service_url>"
      # The mount_path configuration for OSS that you specified when you deploy the service. 
      mount_path = "/code/stable-diffusion-webui/data-oss"
      # The OSS path that you specified when you deploy the service. 
      oss_url = "oss://examplebucket/data-oss"
      
      # Do not use the AccessKey pair of an Alibaba Cloud account as it grants permissions to all API operations and poses a high risk. We recommend that you use the AccessKey pair of a Resource Access Management (RAM) user for API operations and O&M. To create a RAM user, log on to the RAM console. 
      auth = oss2.Auth('<yourAccessKeyId>', '<yourAccessKeySecret>')
      # Specify the endpoint and examplebucket parameters based on your business requirements. 
      bucket = oss2.Bucket(auth, '<endpoint>', '<examplebucket>')
      
      
      payload = {
          "alwayson_scripts": {
              "sd_model_checkpoint": "deliberate_v2.safetensors",
              "save_dir": "/code/stable-diffusion-webui/data-oss/outputs"
          },
          "steps": 30,
          "prompt": "girls",
          "batch_size": 1,
          "n_iter": 2,
          "width": 576,
          "height": 576,
          "negative_prompt": "ugly, out of frame"
      }
      
      
      session = requests.session()
      session.headers.update({"Authorization": "<token>"})
      
      
      response = session.post(url=f'{url}/sdapi/v1/txt2img', json=payload)
      if response.status_code != 200:
          raise Exception(response.content)
      
      data = response.json()
      
      for idx, path in enumerate(data['parameters']['image_url'].split(',')):
          url = path.replace(mount_path, oss_url)
          print(idx, path, url)
      
          # Download the OSS object to the local file system. 
          bucket.get_object_to_file(url[len("oss://<examplebucket>/"):], f'output-{idx}.png')
      

      Modify the following parameters in the preceding code:

      • <service_url>: Set this parameter to the endpoint that you obtained in Step 1.

      • <token>: Set this parameter to the token that you obtained in Step 1.

      • auth: Replace <yourAccessKeyId> and <yourAccessKeySecret> with the AccessKey ID and AccessKey secret of your Alibaba Cloud account or a RAM user.

      • <endpoint>: Replace <endpoint> with the endpoint that is used to access OSS. For example, you can set this value to http://oss-cn-hangzhou.aliyuncs.com for the China (Hangzhou) region. For more information, see Regions and endpoints.

      • <examplebucket>: Set this parameter to the name of the OSS bucket.

      If the code runs successfully, the following results are returned. You can go to the OSS console and view the generated images in the OSS path that you specified when you deployed the service.

      0 /code/stable-diffusion-webui/data-oss/outputs/txt2img-grids/2023-08-03/grid-b2565642-8ec0-4250-aa9e-f8402e7b87f7.png oss://examplebucket/aohai-singapore/outputs/txt2img-grids/2023-08-03/grid-b2565642-8ec0-4250-aa9e-f8402e7b87f7.png
      1 /code/stable-diffusion-webui/data-oss/outputs/txt2img-images/2023-08-03/dc380f1f-efa0-4681-9b1a-298e2513b929-4084873146.png oss://examplebucket/aohai-singapore/outputs/txt2img-images/2023-08-03/dc380f1f-efa0-4681-9b1a-298e2513b929-4084873146.png
      2 /code/stable-diffusion-webui/data-oss/outputs/txt2img-images/2023-08-03/3df9fd96-7c5a-487a-8e43-24f87c042806-4084873147.png oss://examplebucket/aohai-singapore/outputs/txt2img-images/2023-08-03/3df9fd96-7c5a-487a-8e43-24f87c042806-4084873147.png

    In addition, you can add LoRA and ControNnet configurations in the request data based on your business requirements.

    LoRA configurations

    Add <lora:yaeMikoRealistic_Genshin:1000> in the prompt parameter to use LoRA models. For more information, see LORA and alwayson_scripts example.

    Sample request body:

    {
      "prompt":"girls <lora:yaeMikoRealistic_Genshin:1>",
      "steps":20,
      "save_images":true
    }

    ControlNet configurations

    You can add the controlnet parameter in the request to perform common operations on the generated image, such as keeping the image horizontal or vertical. For more information, see the "Example: txt2img with ControlNet" section of this topic.

EAS provides additional parameters on top of the SD WebUI API. You can configure these parameters to use advanced features and achieve customization. For more information, see the "Additional request parameters provided by EAS" section of this topic.

Deploy an asynchronous service

EAS includes a queue service that can efficiently handle a high volume of requests. You only need to send asynchronous requests and subscribe to their results. Perform the following steps to deploy an asynchronous service:

  1. Go to the Deploy Service page. For more information, see the "Deploy a synchronous service" section of this topic.

  2. Navigate to the Configuration Editor section, click JSON Deployment in the upper right corner, and then paste the following code into the editor. For information about how to configure the queue service, see Asynchronous inference and queue service.

    {
        "metadata": {
            "name": "sd_async",
            "instance": 1,
            "rpc.work_threads": 1,
            "type": "Async"
        },
        "cloud": {
            "computing": {
                "instance_type": "ml.gu7i.c16m60.1-gu30",
                "instances": null
            },
            "networking": {
                "vpc_id": "vpc-bp1t2wukzskw9139n****",
                "vswitch_id": "vsw-bp12utkudylvp4c70****",
                "security_group_id": "sg-bp11nqxfd0iq6v5g****"
            }
        },
        "queue": {
            "cpu": 1,
            "max_delivery": 1,
            "memory": 4000,
            "resource": ""
        },
        "storage": [
            {
                "oss": {
                    "path": "oss://examplebucket/aohai-singapore/",
                    "readOnly": false
                },
                "properties": {
                    "resource_type": "model"
                },
                "mount_path": "/code/stable-diffusion-webui/data-oss"
            },
            {
                "nfs": {
                    "path": "/",
                    "server": "0c9624****-fgh60.cn-hangzhou.nas.aliyuncs.com"
                },
                "properties": {
                    "resource_type": "model"
                },
                "mount_path": "/code/stable-diffusion-webui/data-nas"
            }
        ],
        "containers": [
            {
                "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/stable-diffusion-webui:3.2",
                "script": "./webui.sh --listen --port 8000 --skip-version-check --no-hashing --no-download-sd-model --skip-install --api --nowebui --time-log",
                "port": 8000
            }
        ]
    } 

    The JSON configuration file of an asynchronous service has the following changes compared with the configuration file of a synchronous service. For information about how to configure the parameters, see the "Method 2: Use JSON configuration" section of this topic.

    • Deletion:

      • Deletes the metadata.enable_webservice parameter.

      • Deletes the --filebrowser option that is specified in the containers.script parameter to accelerate the service startup.

    • Addition:

      • Adds the metadata.type parameter and sets the value to Async.

      • Adds the metadata.rpc.work_threads parameter and sets the value to 1, which specifies that an instance can process only one request at the same time.

      • Adds the queue.max_delivery parameter and sets the value to 1, which specifies that retries are not allowed after an error occurs during processing.

      • Adds the --nowebui and --time-log options in the containers.script parameter to accelerate the service startup and log the response time, respectively.

  3. Click Deploy and wait for the deployment to complete.

Call an asynchronous service

After you deploy an asynchronous API service, you can send synchronous or asynchronous requests based on your business requirements. If you send a synchronous request, the client pauses execution and awaits the return of the result. If you send an asynchronous request, the client proceeds without delay and subscribes to the result. Once the result is ready, the server automatically pushes the result to the client. For more information about how to send synchronous and asynchronous requests, see Asynchronous inference and queue service.

Send asynchronous requests and subscribe to the results

  1. Obtain the invocation information.

    On the EAS-Online Model Services page, find the service that you deployed and click Invocation Method in the Service Type column. In the Invocation Method dialog box, click the Asynchronous Invocation tab. On the Request URL (Internet) tab, obtain the endpoint and token of the service. image.png

  2. Send asynchronous requests.

    Note
    • The queue service requires that the size of input queues or output queues cannot exceed 8 KB. Take note of the following items:

      • If the request data contains an image, we recommend that you use a URL to pass the image data. SD WebUI automatically downloads and parses the image data.

      • To ensure that the response does not contain any original image data, we recommend that you use the save_dir parameter to specify the path where the generated image is saved. For more information, see the "Additional request parameters provided EAS" section of this topic.

    • By default, EAS cannot access the Internet. If you set the image_link parameter to an Internet URL, you must complete the required configurations so that EAS can access the image. For more information, see Configure network connectivity and Configure Internet access and a whitelist.

    Use SDK for Python

    Sample code:

    import requests
    
    url = "<service_url>"
    session = requests.session()
    session.headers.update({"Authorization": "<token>"})
    
    prompts = ["cute dog", "cute cat", "cute girl"]
    
    for i in range(5):
        p = prompts[i % len(prompts)]
        payload = {
            "prompt": p,
            "steps": 20,
            "alwayson_scripts": {
                "save_dir": "/code/stable-diffusion-webui/data-oss/outputs/txt2img"
            },
        }
        response = session.post(url=f'{url}/sdapi/v1/txt2img?task_id=txt2img_{i}', json=payload)
        if response.status_code != 200:
            exit(f"send request error:{response.content}")
        else:
            print(f"send {p} success, index is {response.content}")
    
    
    for i in range(5):
        p = prompts[i % len(prompts)]
        payload = {
            "prompt": p,
            "steps": 20,
            "alwayson_scripts": {
                "save_dir": "/code/stable-diffusion-webui/data-oss/outputs/img2img",
                "image_link": "https://eas-cache-cn-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/stable-diffusion-cache/tests/boy.png",
            },
        }
        response = session.post(url=f'{url}/sdapi/v1/img2img?task_id=img2img_{i}', json=payload)
        if response.status_code != 200:
            exit(f"send request error:{response.content}")
        else:
            print(f"send {p} success, index is {response.content}")
    

    Modify the following parameters in the preceding code:

    • <service_url>: Set this parameter to the endpoint that you obtained in Step 1.

    • <token>: Set this parameter to the token that you obtained in Step 1.

    Note
    • You can use SDK for Python to send POST requests to the API endpoints provided by SD WebUI. Select an endpoint based on your business requirements.

    • If you want to pass custom information to the service, specify a custom tag by using a URL parameter. For example, you can append ?task_id=task_abc to the request path to specify a tag named task_id. The tag information is included in the result within the tags parameter.

    If the code runs successfully, the following result is returned. Your actual result may vary. image.png

    Use SDK for Java

    Refer to the following code to send asynchronous requests. Since Maven is commonly used to manage Java projects, you must add EAS SDK for Java as a dependency in the pom.xml file. For more information, see SDK for Java.

    import com.aliyun.openservices.eas.predict.http.HttpConfig;
    import com.aliyun.openservices.eas.predict.http.QueueClient;
    import com.aliyun.openservices.eas.predict.queue_client.QueueUser;
    import org.apache.commons.lang3.tuple.Pair;
    
    import java.util.HashMap;
    
    public class SDWebuiAsyncPutTest {
        public static void main(String[] args) throws Exception {
            // Create a client for the queue service. 
            String queueEndpoint = "http://166233998075****.cn-hangzhou.pai-eas.aliyuncs.com";
            String queueToken = "xxxxx==";
            // The name of the input queue consists of the service name and the request path that you want to use. 
            String inputQueueName = "<service_name>/sdapi/v1/txt2img";
    
            // Create the input queue. After request data is added to the input queue, the inference service automatically reads the request data from the input queue. 
            QueueClient inputQueue =
                new QueueClient(queueEndpoint, inputQueueName, queueToken, new HttpConfig(), new QueueUser());
            // Clear queue data. Proceed with caution. 
            // input_queue.clear();
    
            // Add request data to the input queue. 
            int count = 5;
            for (int i = 0; i < count; ++i) {
                // Create request data. 
                String data = "{\n" +
                    "    \"prompt\": \"cute dog\", \n" +
                    "    \"steps\":20,\n" +
                    "    \"alwayson_scripts\":{\n" +
                    "        \"save_dir\":\"/code/stable-diffusion-webui/data-oss/outputs/txt2img\"\n" +
                    "    }\n" +
                    "  }";
                // Create a custom tag. 
                HashMap<String, String> map = new HashMap<String, String>(1);
                map.put("task_id", "txt2img_" + i);
                Pair<Long, String> entry = inputQueue.put(data.getBytes(), map);
    
                System.out.println(String.format("send success, index is %d, request_id is %s", entry.getKey(), entry.getValue()));
                // The queue service supports multi-priority queues. You can use the put function to set the priority level of the request. The default value is 0. A value of 1 specifies a high priority. 
                //  inputQueue.put(data.getBytes(), 0L, null);
            }
            // Close the client. 
            inputQueue.shutdown();
    
    
            inputQueueName = "<service_name>/sdapi/v1/img2img";
            inputQueue =
                new QueueClient(queueEndpoint, inputQueueName, queueToken, new HttpConfig(), new QueueUser());
            for (int i = 0; i < count; ++i) {
                // Create request data. 
                String data = "{\n" +
                    "    \"prompt\": \"cute dog\", \n" +
                    "    \"steps\":20,\n" +
                    "    \"alwayson_scripts\":{\n" +
                    "        \"save_dir\":\"/code/stable-diffusion-webui/data-oss/outputs/img2img\",\n" +
                    "        \"image_link\":\"https://eas-cache-cn-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/stable-diffusion-cache/tests/boy.png\"\n" +
                    "    }\n" +
                    "  }";
                HashMap<String, String> map = new HashMap<String, String>(1);
                map.put("task_id", "img2img_" + i);
                Pair<Long, String> entry = inputQueue.put(data.getBytes(), map);
    
                System.out.println(String.format("send success, index is %d, requestId is %s", entry.getKey(), entry.getValue()));
            }
    
            // Close the client. 
            inputQueue.shutdown();
        }
    }

    Modify the following parameters in the preceding code:

    • queueEndpoint: Set this parameter to the endpoint that you obtained in Step 1. You can refer to the sample code to configure this parameter.

    • queueToken: Set this parameter to the token that you obtained in Step 1.

    • <service_name>: Set this parameter to the name of the asynchronous service that you deployed.

    Note

    If you want to pass custom information to the service, specify a custom tag in the put function. You can refer to the sample code to configure custom tags. The tag information is included in the result within the tags parameter.

    If the code runs successfully, the following result is returned. Your actual result may vary.

    send success, index is 21, request_id is 05ca7786-c24e-4645-8538-83d235e791fe
    send success, index is 22, request_id is 639b257a-7902-448d-afd5-f2641ab77025
    send success, index is 23, request_id is d6b2e127-eba3-4414-8e6c-c3690e0a487c
    send success, index is 24, request_id is 8becf191-962d-4177-8a11-7e4a450e36a7
    send success, index is 25, request_id is 862b2d8e-5499-4476-b3a5-943d18614fc5
    send success, index is 26, requestId is 9774a4ff-f4c8-40b7-ba43-0b1c1d3241b0
    send success, index is 27, requestId is fa536d7a-7799-43f1-947f-71973bf7b221
    send success, index is 28, requestId is e69bdd32-5c7b-4c8f-ba3e-e69d2054bf65
    send success, index is 29, requestId is c138bd8f-be45-4a47-a330-745fd1569534
    send success, index is 30, requestId is c583d4f8-8558-4c8d-95f7-9c3981494007
    
    Process finished with exit code 0
  3. Subscribe to the results of the asynchronous requests.

    Use SDK for Python

    Sample code:

    import json
    import oss2
    
    from eas_prediction import QueueClient
    
    sink_queue = QueueClient('139699392458****.cn-hangzhou.pai-eas.aliyuncs.com', 'sd_async/sink')
    sink_queue.set_token('<token>')
    sink_queue.init()
    
    mount_path = "/code/stable-diffusion-webui/data-oss"
    oss_url = "oss://<examplebucket>/aohai-singapore"
    # Do not use the AccessKey pair of an Alibaba Cloud account as it grants permissions to all API operations and poses a high risk. We recommend that you use the AccessKey pair of a Resource Access Management (RAM) user for API operations and O&M. To create a RAM user, log on to the RAM console. 
    auth = oss2.Auth('<yourAccessKeyId>', '<yourAccessKeySecret>')
    # Specify the endpoint and examplebucket parameters based on your business requirements. 
    bucket = oss2.Bucket(auth, 'http://oss-cn-hangzhou.aliyuncs.com', '<examplebucket>')
    
    
    watcher = sink_queue.watch(0, 5, auto_commit=False)
    for x in watcher.run():
        if 'task_id' in x.tags:
            print('index {} task_id is {}'.format(x.index, x.tags['task_id']))
        print(f'index {x.index} data is {x.data}')
        sink_queue.commit(x.index)
        try:
            data = json.loads(x.data.decode('utf-8'))
            for idx, path in enumerate(data['parameters']['image_url'].split(',')):
                url = path.replace(mount_path, oss_url)
                # Download the OSS object to the local file system. 
                bucket.get_object_to_file(url[len("oss://<examplebucket>/"):], f'{x.index}-output-{idx}.png')
                print(f'save {url} to {x.index}-output-{idx}.png')
        except Exception as e:
            print(f'index {x.index} process data error {e}')
    

    Modify the following parameters in the preceding code:

    • sink_queue: Update the service endpoint and service name that are specified in this parameter. For more information about how to obtain the service endpoint, see the "Obtain the invocation information" section of this topic. You can refer to the sample code to configure this parameter.

    • <token>: Set this parameter to the token that you obtained in Step 1.

    • oss_url: Set this parameter to the OSS path that you specified when you deployed the service.

    • <examplebucket>: Set this parameter to the name of the OSS bucket.

    • bucket: This example uses the China (Hangzhou) region. Configure the parameter based on your actual situation. For more information, see Regions and endpoints.

    Note
    • You can manually commit the data or set the auto_commit parameter to true to automatically commit the data.

    • If the client of the queue service stops consuming data, we recommend that you close the client to release resources.

    • You can also run a cURL command or call an API operation to subscribe to the results. For more information, see Asynchronous inference and queue service.

    If the code runs successfully, the following result is returned. Your actual results may vary. You can go to the OSS console and view the generated images in the OSS path that you specified when you deployed the service.

    index 1 task_id is txt2img_0
    index 1 data is b'{"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-16/ad2e270b-dfcc-4ec3-8718-f19824857834-919429454.png","seed":"919429454","error_msg":"","total_time":88.55384421348572},"info":""}'
    save oss://<examplebucket>/aohai-singapore/outputs/txt2img/txt2img-images/2023-08-16/ad2e270b-dfcc-4ec3-8718-f19824857834-919429454.png to 1-output-0.png
    index 2 task_id is txt2img_1
    index 2 data is b'{"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-16/374ceae2-deb5-4448-b213-cfdb11a53892-3273662200.png","seed":"3273662200","error_msg":"","total_time":2.76196551322937},"info":""}'
    save oss://<examplebucket>/aohai-singapore/outputs/txt2img/txt2img-images/2023-08-16/374ceae2-deb5-4448-b213-cfdb11a53892-3273662200.png to 2-output-0.png
    index 3 task_id is txt2img_2
    index 3 data is b'{"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-16/a37a0384-2ed4-42f9-9166-fe0fd80c624c-210444956.png","seed":"210444956","error_msg":"","total_time":2.7492804527282715},"info":""}'
    save oss://<examplebucket>/aohai-singapore/outputs/txt2img/txt2img-images/2023-08-16/a37a0384-2ed4-42f9-9166-fe0fd80c624c-210444956.png to 3-output-0.png
    index 4 task_id is txt2img_3
    index 4 data is b'{"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-16/482dcc64-4848-48f8-b9fd-7f0d3992f1ae-198191449.png","seed":"198191449","error_msg":"","total_time":2.8161191940307617},"info":""}'
    save oss://<examplebucket>/aohai-singapore/outputs/txt2img/txt2img-images/2023-08-16/482dcc64-4848-48f8-b9fd-7f0d3992f1ae-198191449.png to 4-output-0.png

    Use SDK for Java

    Sample code:

    import com.aliyun.openservices.eas.predict.http.HttpConfig;
    import com.aliyun.openservices.eas.predict.http.QueueClient;
    import com.aliyun.openservices.eas.predict.queue_client.DataFrame;
    import com.aliyun.openservices.eas.predict.queue_client.QueueUser;
    import com.aliyun.openservices.eas.predict.queue_client.WebSocketWatcher;
    
    public class SDWebuiAsyncWatchTest {
        public static void main(String[] args) throws Exception {
            // Create a client for the queue service. 
            String queueEndpoint = "http://166233998075****.cn-hangzhou.pai-eas.aliyuncs.com";
            String queueToken = "xxxxx==";
            // The name of the output queue consists of the service name and "/sink". 
            String sinkQueueName = "<service_name>/sink";
    
            // The output queue. The inference service processes the input data and writes the results to the output queue. 
            QueueClient sinkQueue =
                new QueueClient(queueEndpoint, sinkQueueName, queueToken, new HttpConfig(), new QueueUser());
    
            // Clear queue data. Proceed with caution. 
            //  sinkQueue.clear();
    
            // Subscribe to the queue and obtain the results. 
            WebSocketWatcher watcher = sinkQueue.watch(0L, 5L, false, false, null);
            try {
                while (true) {
                    DataFrame df = watcher.getDataFrame();
                    if (df.getTags().containsKey("task_id")) {
                        System.out.println(String.format("task_id = %s", df.getTags().get("task_id")));
                    }
                    System.out.println(String.format("index = %d, data = %s, requestId = %s", df.getIndex(), new String(df.getData()), df.getTags().get("requestId")));
                    sinkQueue.commit(df.getIndex());
                }
            } catch (Exception e) {
                System.out.println("watch error:" + e.getMessage());
                e.printStackTrace();
                watcher.close();
            }
    
            // Close the client. 
            sinkQueue.shutdown();
        }
    }

    Modify the following parameters in the preceding code:

    • queueEndpoint: Set this parameter to the endpoint that you obtained in Step 1. You can refer to the sample code to configure this parameter.

    • queueToken: Set this parameter to the token that you obtained in Step 1.

    • <service_name>: Set this parameter to the name of the asynchronous service that you deployed.

    Note
    • You can manually commit the data or set the auto_commit parameter to true to automatically commit the data.

    • If the client of the queue service stops consuming data, we recommend that you close the client to release resources.

    • You can also run a cURL command or call an API operation to subscribe to the results. For more information, see Asynchronous inference and queue service.

    If the code runs successfully, the following result is returned. Your actual results may vary. You can go to the OSS console and view the generated image in the OSS path that you specified when you deployed the service.

    2023-08-04 16:17:31,497 INFO [com.aliyun.openservices.eas.predict.queue_client.WebSocketWatcher] - WebSocketClient Successfully Connects to Server: 1396993924585947.cn-hangzhou.pai-eas.aliyuncs.com/116.62.4.210:80
    task_id = txt2img_0
    index = 21, data = {"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-04/54363a9d-24a5-41b5-b038-2257d43b8e79-412510031.png","seed":"412510031","error_msg":"","total_time":2.5351321697235107},"info":""}, requestId = 05ca7786-c24e-4645-8538-83d235e791fe
    task_id = txt2img_1
    index = 22, data = {"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-04/0c646dda-4a53-43f4-97fd-1f507599f6ae-2287341785.png","seed":"2287341785","error_msg":"","total_time":2.6269655227661133},"info":""}, requestId = 639b257a-7902-448d-afd5-f2641ab77025
    task_id = txt2img_2
    index = 23, data = {"images":[],"parameters":{"id_task":null,"status":0,"image_url":"/code/stable-diffusion-webui/data-oss/outputs/txt2img/txt2img-images/2023-08-04/4d542f25-b9cc-4548-9db2-5addd0366d32-1158414078.png","seed":"1158414078","error_msg":"","total_time":2.6604185104370117},"info":""}, requestId = d6b2e127-eba3-4414-8e6c-c3690e0a487c
    task_id = txt2img_3

Send a synchronous request

After you deploy an asynchronous service, you can still send synchronous requests to the service. You can use a cURL command or a Python client to send a synchronous request to verify the model performance and service availability. Perform the following steps:

  1. Obtain the invocation information.

    On the EAS-Online Model Services page, find the service that you deployed and click Invocation Method in the Service Type column. In the Invocation Method dialog box, click the Synchronous Invocation tab. On the Public Endpoint tab, you can view the endpoint and token of the service. image.png

  2. Use a cURL command to send a request. Sample code:

    curl -v --location '<service_url>/sdapi/v1/sd-models' \
    --header 'Authorization: <service_token>' 

    Modify the following parameters in the preceding code:

    • <service_url>: Set this parameter to the endpoint that you obtained in Step 1.

    • <token>: Set this parameter to the token that you obtained in Step 1.

    If the command runs successfully, the list of supported models is returned.

EAS provides additional parameters on top of the SD WebUI API. You can configure these parameters to use advanced features and achieve customization. For more information, see the "Additional request parameters provided by EAS" section of this topic.

Additional request parameters provided by EAS

EAS provides additional parameters on top of the SD WebUI API. You can configure these parameters to use advanced features and achieve customization.

  • Configure the SD model, the Variational Autoencoder (VAE) model, and the path to save the generated images.

  • Use URL parameters to send requests, for which status codes are provided.

  • Access the generated images through URLs, including the images that are processed by ControlNet.

Example: txt2img

Sample request body:

{
      "alwayson_scripts": {
          "sd_model_checkpoint": "deliberate_v2.safetensors",  
          "save_dir": "/code/stable-diffusion-webui/data-oss/outputs",
          "sd_vae": "Automatic"
      },
      "steps": 20,
      "prompt": "girls",          
      "batch_size": 1,                                            
      "n_iter": 2,                                                 
      "width": 576, 
      "height": 576,
      "negative_prompt": "ugly, out of frame"
  }

Parameters:

  • sd_model_checkpoint: the SD model that you want to use. The SD model can automatically switch to a foundation model.

  • sd_vae: the VAE model that you want to use.

  • save_dir: the path to save the generated images.

Sample synchronous request:

# Send a synchronous request to verify the model performance. 

curl --location --request POST '<service_url>/sdapi/v1/txt2img' \
--header 'Authorization: <token>' \
--header 'Content-Type: application/json' \
--data-raw '{
      "alwayson_scripts": {
          "sd_model_checkpoint": "deliberate_v2.safetensors",
          "save_dir": "/code/stable-diffusion-webui/data-oss/outputs",
          "sd_vae": "Automatic"
      },
      "prompt": "girls",          
      "batch_size": 1,                                            
      "n_iter": 2,                                                 
      "width": 576, 
      "height": 576,
      "negative_prompt": "ugly, out of frame"
  }'

Sample response to a synchronous request:

{
  "images": [],
  "parameters": {
    "id_task": "14837",
    "status": 0,
    "image_url": "/code/stable-diffusion-webui/data-oss/outputs/txt2img-grids/2023-07-24/grid-29a67c1c-099a-4d00-8ff3-1ebe6e64931a.png,/code/stable-diffusion-webui/data-oss/outputs/txt2img-images/2023-07-24/74626268-6c81-45ff-90b7-faba579dc309-1146644551.png,/code/stable-diffusion-webui/data-oss/outputs/txt2img-images/2023-07-24/6a233060-e197-4169-86ab-1c18adf04e3f-1146644552.png",
    "seed": "1146644551,1146644552",
    "error_msg": "",
    "total_time": 32.22393465042114
  },
  "info": ""
}

Sample asynchronous request:

# Send the request data to the input queue. 
curl --location --request POST '<service_url>/sdapi/v1/txt2img' \
--header 'Authorization: <token>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "alwayson_scripts": {
        "sd_model_checkpoint": "deliberate_v2.safetensors",
        "id_task": "14837",
        "uid": "123",
        "save_dir": "tmp/outputs"
    },
    "prompt": "girls",
    "batch_size": 1,
    "n_iter": 2,
    "width": 576,
    "height": 576,
    "negative_prompt": "ugly, out of frame"
}'

Example: img2img

Sample request body:

{
    "alwayson_scripts": {
        "image_link":"https://eas-cache-cn-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/stable-diffusion-cache/tests/boy.png",
        "sd_model_checkpoint": "deliberate_v2.safetensors",
        "sd_vae": "Automatic",
        "save_dir": "/code/stable-diffusion-webui/data-oss/outputs"
    },
    "prompt": "girl",
    "batch_size": 1,                                            
    "n_iter": 2,                                                 
    "width": 576, 
    "height": 576,
    "negative_prompt": "ugly, out of frame",
    "steps": 20, # Sampling steps
    "seed": 111,   
    "subseed": 111, # Variation seed
    "subseed_strength": 0, # Variation strength
    "seed_resize_from_h": 0, # Resize seed from height
    "seed_resize_from_w": 0, # Resize seed from width
    "seed_enable_extras": false, # Extra
    "sampler_name": "DDIM", # Sampling method
    "cfg_scale": 7.5, # CFG Scale
    "restore_faces": true, # Restore faces
    "tiling": false, # Tiling
    "init_images": [], # image base64 str, default None
    "mask_blur": 4, # Mask blur
    "resize_mode": 1, # 0 just resize, 1 crop and resize, 2 resize and fill, 3 just resize
    "denoising_strength": 0.75, # Denoising strength
    "inpainting_mask_invert": 0, #int, index of ['Inpaint masked', 'Inpaint not masked'], Mask mode
    "inpainting_fill": 0, #index of ['fill', 'original', 'latent noise', 'latent nothing'], Masked content
    "inpaint_full_res": 0, # index of ["Whole picture", "Only masked"], Inpaint area
    "inpaint_full_res_padding": 32, #minimum=0, maximum=256, step=4, value=32, Only masked padding, pixels
    #"image_cfg_scale": 1, # resized by scale
    #"script_name": "Outpainting mk2", # The name of the script. Skip this field if you do not use a script.
    #"script_args": ["Outpainting", 128, 8, ["left", "right", "up", "down"], 1, 0.05] # The parameters of the script in the following order: fixed fields, pixels, mask_blur, direction, noise_q, and color_variation.
}

Sample response:

{
    "images":[],
    "parameters":{
        "id_task":"14837",
        "status":0,
        "image_url":"/data/api_test/img2img-grids/2023-06-05/grid-0000.png,/data/api_test/img2img-images/2023-06-05/00000-1003.png,/data/api_test/img2img-images/2023-06-05/00001-1004.png",
        "seed":"1003,1004",
        "error_msg":""
    },
    "info":""
}

Example: txt2img with ControlNet

Sample request body:

{
    "alwayson_scripts": {
        "sd_model_checkpoint": "deliberate_v2.safetensors", # The name of the model. 
        "save_dir": "/code/stable-diffusion-webui/data-oss/outputs",
        "controlnet":{
            "args":[
                {
                    "image_link": "https://pai-aigc-dataset.oss-cn-hangzhou.aliyuncs.com/pixabay_images/00008b87bf3ff6742b8cf81c358b9dbc.jpg",
                    "enabled": true, 
                    "module": "canny", 
                    "model": "control_v11p_sd15_canny", 
                    "weight": 1, 
                    "resize_mode": "Crop and Resize", 
                    "low_vram": false, 
                    "processor_res": 512, 
                    "threshold_a": 100, 
                    "threshold_b": 200, 
                    "guidance_start": 0, 
                    "guidance_end": 1, 
                    "pixel_perfect": true, 
                    "control_mode": "Balanced", 
                    "input_mode": "simple", 
                    "batch_images": "", 
                    "output_dir": "", 
                    "loopback": false
                }
            ]
        }
    },
    # Key parameters.
    "prompt": "girls",          
    "batch_size": 1,                                            
    "n_iter": 2,                                                 
    "width": 576, 
    "height": 576,
    "negative_prompt": "ugly, out of frame"
}

Sample response:

{
    "images":[],
    "parameters":{
        "id_task":"14837",
        "status":0,
        "image_url":"/data/api_test/txt2img-grids/2023-06-05/grid-0007.png,/data/api_test/txt2img-images/2023-06-05/00014-1003.png,/data/api_test/txt2img-images/2023-06-05/00015-1004.png",
        "seed":"1003,1004",
        "error_msg":"",
        "image_mask_url":"/data/api_test/controlnet_mask/2023-06-05/00000.png,/data/api_test/controlnet_mask/2023-06-05/00001.png"
    },
    "info":""
}

References