This topic describes how to use a data cache to deploy the Stable Diffusion application and the ControlNet extension to implement light and shadow images of texts. Before you deploy the Stable Diffusion application, you can store the Stable Diffusion and ControlNet model data to data caches. When you create the elastic container instance that corresponds to the Stable Diffusion application, you can mount the model data to the instance. This way, the system does not need to pull the model data from the elastic container instance and can deploy the application faster.
Background information
Stable Diffusion is a model that can generate and modify images based on text descriptions. Stable Diffusion supports extensions to enrich drawing effects. ControlNet is a plugin that is used to control the generation of AI images. After you install the ControlNet extension on Stable Diffusion, you can use ControlNet to control the image effects and make AI images closer to your needs.
Alibaba Cloud does not guarantee the legality, security, or accuracy of third-party models. Alibaba Cloud is not liable for any damages caused thereby.
You must abide by the user agreements, usage specifications, and relevant laws and regulations of the third-party models. You agree that your use of the third-party models is at your sole risk.
Procedure
Create data caches for the models
Console mode
Visit Hugging Face and obtain the IDs of the models.
In this example, the following models are used. After you find the models in Hugging Face, you can copy the model IDs at the top of the model details pages.
Stable Diffusion model: hanafuusen2001/BeautyProMix
ControlNet model: ioclab/ioc-controlnet
Log on to the Elastic Container Instance console.
In the top navigation bar, select a region.
ImportantIf your business is deployed in the Chinese mainland, you may not be able to cache model data from Hugging Face because of network issues. In this case, create the data cache in the Hong Kong (China) region and then copy it to the required region.
In the navigation pane on the left, click Data Cache.
Create data caches for the models.
Click Create Data Cache to create data caches for the hanafuusen2001/BeautyProMix and ioclab/ioc-controlnet models separately. The following table describes the sample parameter values that are used in this example. You must use the specified values for the Cache Data Source parameter. These specified values are fixed configurations for pulling the models. You can specify other parameters as needed. For more information, see Create a data cache.
Parameter
hanafuusen2001/BeautyProMix
ioclab/ioc-controlnet
Cache Bucket
test
test
Cache Directory
/model/beautypromix/
/model/ioc/
Cache Name
beautypromix
ioc
Cache Size
20 GiB
20 GiB
Cache Data Source
Type: URL
Parameters
repoSource: HuggingFace/Model
repoId: hanafuusen2001/BeautyProMix
Type: URL
Parameters
repoSource: HuggingFace/Model
repoId: ioclab/ioc-controlnet
EIP
Auto Create
Maximum bandwidth: 5 Mbps
Auto Create
Maximum bandwidth: 5 Mbps
View the status of the data caches.
On the Data Cache page, refresh the page to view the status of the data caches. If the status becomes Available, the data caches are ready for use.
Copy the data caches.
If a data cache is in a different region from your application, you can find the target data cache on the Data Cache page and click Copy to copy it to the required region.
OpenAPI
Visit Hugging Face and obtain the IDs of the models.
In this example, the following models are used. After you find the models in Hugging Face, you can copy the model IDs at the top of the model details pages.
Stable Diffusion model: hanafuusen2001/BeautyProMix
ControlNet model: ioclab/ioc-controlnet
Call the CreateDataCache API operation to create data caches.
ImportantIf your business is deployed in the Chinese mainland, you may not be able to cache model data from Hugging Face because of network issues. In this case, create the data cache in the Hong Kong (China) region and then copy it to the required region.
If you use an SDK to create a DataCache, you do not need to prefix the length of the parameter name to each parameter in DataSource.Options. For example, write
repoSourceinstead of#10#repoSource. WriterepoIdinstead of#6#repoId.
Create a data cache for the hanafuusen2001/BeautyProMix model.
The following code provides sample parameter values. The system pulls the model data from Hugging Face and saves it to the
/model/beautypromix/directory of the bucket named test. The data cache is named beautypromix and is retained for one day. When the system pulls the data from the model to create the data cache, it automatically creates an elastic IP address (EIP) and associates the EIP with the Virtual Private Cloud (VPC) that the data cache belongs to. This allows the system to pull data over the internet. After the data cache is created, the EIP is automatically released.{ "RegionId": "cn-hongkong", "Path": "/model/beautypromix/", "Name": "beautypromix", "DataSource": { "Type": "URL", "Options": { "#10#repoSource": "HuggingFace/Model", "#6#repoId": "hanafuusen2001/BeautyProMix" } }, "RetentionDays": 1, "EipCreateParam": { "Bandwidth": 5 }, "Bucket": "test" }Create a data cache for the ioclab/ioc-controlnet model.
The following code provides sample parameter values. The system pulls the model data from Hugging Face and saves it to the
/model/ioc/directory of the bucket named test. The data cache is named ioc and is retained for one day. When the system pulls the data from the model to create the data cache, it automatically creates an EIP and associates the EIP with the VPC that the data cache belongs to. This allows the system to pull data over the internet. After the data cache is created, the EIP is automatically released.{ "RegionId": "cn-hongkong", "Path": "/model/ioc/", "Name": "ioc", "DataSource": { "Type": "URL", "Options": { "#10#repoSource": "HuggingFace/Model", "#6#repoId": "ioclab/ioc-controlnet" } }, "RetentionDays": 1, "EipCreateParam": { "Bandwidth": 5 }, "Bucket": "test" }
Query the status of the data caches.
Use the returned DataCache ID to call the DescribeDataCaches API operation and query the information of the DataCache. If the status of the DataCache that is indicated by DataCaches.Status is Available, the DataCache is ready for use.
Copy the data caches.
If the data cache is in a different region from where you want to deploy the application, you can call the CopyDataCache API operation to copy the data cache to the required region.
Deploy the Stable Diffusion application and the extension of the application
Console mode
On the Container Group page in the Elastic Container Instance console, click Create Container Group.
Configure the parameters that are used to create the container group (elastic container instance) and then click Confirm Configuration.
The following table describes the sample parameter values used to create the Elastic Container Instance. The instance is created based on a GPU-accelerated ECS instance type and is mounted with the data caches for the hanafuusen2001/BeautyProMix and ioclab/ioc-controlnet models. The container in the Elastic Container Instance uses an image provided by the Elastic Container Instance service that contains the Stable Diffusion application. After the container starts, it runs the
python3 launch.py --listen --skip-torch-cuda-test --port 8888 --no-halfcommand to start the WebUI.ImportantIf the VPC to which the elastic container instance belongs is associated with an Internet NAT gateway, you may not associate an EIP with the elastic container instance when you create the instance. After the instance is created, you can configure DNAT entries to allow external access to the instance.
Section
Parameter
Example
Container Group Configurations
Specify Instance Type
ecs.gn6i-c16g1.4xlarge
Name
sd-webui
Container Configurations
Container Name
sd
Image
Image: registry.cn-hangzhou.aliyuncs.com/eci_open/stable-diffusion
Image Tag: 1.0.1
Executable Command
/bin/sh
Parameter
-c
python3 launch.py --listen --skip-torch-cuda-test --port 8888 --no-half
Data Cache
Data Cache Bucket
test
Click Add to mount the data cache of the Hanafuusen2001/BeautyProMix model.
Data Cache Directory: /model/beautypromix/
Destination Container: sd
Container Mount Directory: /stable-diffusion-webui/models/Stable-diffusion/
Click Add to mount the data cache of the ioclab/ioc-controlnet model.
Data Cache Directory: /model/ioc/
Destination Container: sd
Container Mount Directory: /stable-diffusion-webui/extensions/sd-webui-controlnet/models/
Enable Burst
Enable
EIP
EIP
Auto Create
Maximum Bandwidth: 5 Mbps
Check the configurations of the instance, read and select the Terms of Service, and then click Confirm Order.
Return to the Container Group page, check whether the application is deployed, and view the EIP of the instance.
On the Container Group page, you can view the instance status. Click an instance ID to open the instance details page and view the container status. The application is successfully deployed when both the instance and container statuses are Running. You can find the EIP address of the instance in the IP Address column.

OpenAPI
Use the data caches to create an elastic container instance. Deploy the Stable Diffusion application and the extension of the application on the instance.
The following example shows the parameters used to create an ECI instance by calling the CreateContainerGroup API operation. The instance uses a GPU specification and is mounted with the data caches for hanafuusen2001/BeautyProMix and ioclab/ioc-controlnet. The container in the instance uses an image that is provided by ECI and contains the Stable Diffusion application. After the container starts, it runs the
python3 launch.py --listen --skip-torch-cuda-test --port 8888 --no-halfcommand to start the WebUI.NoteIn the following example, the system automatically creates an EIP and associates the EIP with the elastic container instance. If the VPC to which the elastic container instance belongs is associated with an Internet NAT gateway, you may not associate an EIP with the elastic container instance when you create the instance. After the instance is created, you can configure DNAT entries to allow external access to the instance.
{ "RegionId": "cn-beijing", "SecurityGroupId": "sg-2ze7l1o0ql1cbk******", "VSwitchId": "vsw-2ze23nqzig8inpr******", "ContainerGroupName": "sd-web", "InstanceType": "ecs.gn6i-c16g1.4xlarge", "DataCacheBucket": "test", "Container": [ { "Arg": [ "-c", "python3 launch.py --listen --skip-torch-cuda-test --port 8888 --no-half" ], "Command": [ "/bin/sh" ], "Gpu": 1, "Name": "sd", "Image": "registry.cn-hangzhou.aliyuncs.com/eci_open/stable-diffusion:1.0.1", "VolumeMount": [ { "Name": "model1", "MountPath": "/stable-diffusion-webui/models/Stable-diffusion/" }, { "MountPath": "/stable-diffusion-webui/extensions/sd-webui-controlnet/models/", "Name": "model2" } ] } ], "Volume": [ { "Type": "HostPathVolume", "HostPathVolume.Path": "/model/beautypromix/", "Name": "model1" }, { "Type": "HostPathVolume", "HostPathVolume.Path": "/model/ioc/", "Name": "model2" } ], "DataCacheProvisionedIops": 35000, "DataCacheBurstingEnabled": true, "AutoCreateEip": true }Check whether the application is deployed.
Use the returned instance ID to call the DescribeContainerGroupStatus API operation and query the status of the instance and the container. If the status of the instance that is indicated by Status and the status of the container that is indicated by ContainerStatuses.State are Running, the instance is created and the container is running.
Query the EIP of the elastic container instance.
Use the returned instance ID to call the DescribeContainerGroups API operation and query the instance details. You can obtain the EIP of the instance from the InternetIP parameter.
Test the model and the extension
Add an inbound rule to the security group to which the elastic container instance belongs and allow port 8888.
Use a browser to visit the web page of the Stable Diffusion application.
In this example, the Stable Diffusion application is bound to a dedicated EIP, which lets you access it directly by entering the EIP address of the instance and the port exposed by the container, for example,
182.92.XX.XX:8888.Check whether the ControlNet extension is installed.
On the Extensions tab, the extension is automatically installed and can be used.

Test the extension.
Click the img2img tab.
Enter a text description about the resulting image at the top of the page as needed, and upload reference images for AI drawing.
In the lower part of the page, unfold ControlNet configurations. Upload an image whose light and shadow effects you want to control. Configure parameters for the image.
NoteTo implement the light and shadow effects of the texts, upload an image that has white lines on a black background.
The light and shadow matching effects of different images and different texts are quite different. Fine-tune the parameters based on actual image results.
Description of the parameters:
Model: the model of brightness.
Control Weight: the weight of the ControlNet impact.
Starting Control Step: the starting time when ControlNet is involved. An earlier starting time indicates less freedom that you have in AI drawing, and more striking light and shadow effects of texts.
Ending Control Step: the ending time when ControlNet is involved. An earlier ending time indicates better fusing effects of images and texts and less striking light and shadow effects of texts.

Click Generate to display the result graph.
Example:
