Platform for AI (PAI) provides AI Portrait SDK for Python that allows you to call the algorithm service operations of the AI Portrait service to train models and create portraits. You can use the SDK to configure a custom LoRA model and create a portrait based on a template image. This topic describes the preparations must complete before you can use AI Portrait SDK for Python to call the related operations. This topic also provides examples on how to call the operations.
Prerequisites
A Python environment is prepared. Python 3.4 or later is supported.
For model training and portrait creation, 5 to 20 training images and 1 template image are prepared. The following image formats are supported:
.jpg,.jpeg, and.png. Make sure that the size of each image is greater than 512 x 512 pixels.Single-person portrait: The template image must contain the face of a person. The faces in multiple training images belong to the same person.
Multi-person portrait: The template image must contain multiple faces, and the number of faces must be the same as the value of the model_id parameter specified for model training.
Preparations
Run the following command to install the Python SDK:
wget https://ai-service-data.oss-cn-beijing.aliyuncs.com/python-sdk/ai_service_python_sdk-1.1.3-py3-none-any.whl pip install ai_service_python_sdk-1.1.3-py3-none-any.whlInitialize the client.
Run the following command to initialize the environment:
from ai_service_python_sdk.client.api_client import ApiClient client = ApiClient('<HOST>', '<YOUR-APPID>', '<YOUR-TOKEN>')Modify the following parameters based on your business requirements.
Parameter
Description
<HOST>
The server address. Example:
http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com.<YOUR-APPID>
The application ID. After you activate AI Portrait, you can view the application ID on the AI Portrait page.
<YOUR-TOKEN>
The token. After you activate AI Portrait, you can view the token on the AI Portrait page.
Sample code
AI Portrait is a resource-intensive service, which involve the model training and portrait creation processes. Model training usually takes several minutes, whereas portrait creation takes only several dozens seconds. The following figure shows the process of creating AI portraits by calling API operations.
The following sections provide sample requests and sample responses for the related API operations and the sample code for end-to-end processes.
Check Request (aigc_images_check)
Sample request:
from ai_service_python_sdk.client.api_client import ApiClient from ai_service_python_sdk.client.api.ai_service_aigc_images_api import AIGCImagesApi host = 'http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com' appId = 'YOUR-APPID' token = 'YOUR-TOKEN' client = ApiClient(host, appId, token) api = AIGCImagesApi(client) # noqa: E501 # The training images in the URL format. images = [ 'https://xxx/0.jpg', 'https://xxx/1.jpg' ] response = api.aigc_images_check(images, <model_name>, <configure>) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The details of the request status. message = response.message # The content returned by the model service. data = response.data # Display the return value. print(response)The following table describes the key parameters.
Parameter
Description
appId
The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page.
token
The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page.
images
The URLs of the images that you want to check. Separate multiple URLs with commas (,).
response
The response parameter contains the following fields:
<model_name>: the name of the model. By default, an empty string is used. Example:
''.<configure>: the returned configuration items. Default value:
None. No additional configuration items are specified.
Sample response:
{ "request_id":"07988c97-caa4-4512-a2b2-e9173c1a03c6", "code":"OK", "message":"success", "data":{ "check_results":[ { "code":1, "frontal":true, "message":"success", "url":"https://xxx/0.jpg" },{ "code":1, "frontal":false, "message":"success", "url":"https://xxx/1.jpg" } ], "cost_time":0.47095775604248047, "images":["https://xxx/0.jpg","https://xxx/1.jpg"], "request_id":"07988c97-caa4-4512-a2b2-e9173c1a03c6"}}The following table describes the response parameters.
Parameter
Description
request_id
The request ID, which is of the STRING type.
code
The status code of the request, which is of the STRING type. Valid values:
OK: The request was successful.
error: The request failed.
message
The details of the request status. If the request is successful, success is returned. In other scenarios, a different message is returned.
data
The details of the returned data, which is of the JSON Object type. Parameters:
check_results: The detection result for each input image. Each image corresponds to a dictionary that contains the following keys: url, message, and frontal.
url: the URL of the image.
message: the detection details of the image. For information about possible values, see Possible values of the message field in check_results.
frontal: indicates whether the face in the image is front-facing.
cost_time: the time required by the server to call the API.
images: the URLs of the images that are checked. The value is of the LIST type.
request_id: the ID of the request. The value is of the STRING type.
Model training initiation (aigc_images_train)
Sample request:
from ai_service_python_sdk.client.api_client import ApiClient from ai_service_python_sdk.client.api.ai_service_aigc_images_api import AIGCImagesApi host = 'http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com' appId = 'YOUR-APPID' token = 'YOUR-TOKEN' client = ApiClient(host, appId, token) api = AIGCImagesApi(client) # noqa: E501 # The training images in the URL format. images = [ 'https://xxx/0.jpg', 'https://xxx/1.jpg', 'https://xxx/2.jpg', 'https://xxx/3.jpg', 'https://xxx/4.jpg', 'https://xxx/5.jpg' ] response = api.aigc_images_train(images, <model_name>, <configure>) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The details of the request status. message = response.message # The content returned by the model service. data = response.data # job id job_id = response.data['job_id'] # model_id model_id = response.data['model_id'] # Display the return value. print(response)The following table describes the key parameters.
Parameter
Description
appId
The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page.
token
The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page.
images
The URLs of the training images. Separate multiple URLs with commas (,).
response
The response parameter contains the following fields:
<model_name>: the name of the model. By default, an empty string is used. Example:
''.<configure>: the returned configuration items. Default value:
None. No additional configuration items are specified.
Sample response:
{'code': 'OK', 'data': {'job_id': 11***, 'model_id': 'fa4ba43a-df55-45a4-9c31-bb0dc1e5****'}, 'message': 'success', 'request_id': 'de314ef5-114d-4db1-b54a-332d5300780b'}The following table describes the parameters.
Parameter
Description
request_id
The request ID, which is of the STRING type.
code
The status code of the request, which is of the STRING type. Valid values:
OK: The request was successful.
error: The request failed.
message
The details of the request status. If the request is successful, success is returned. In other scenarios, a different message is returned.
data
The details of the returned data, which is of the JSON Object type. Parameters:
job_id: the ID of the task. The value is of the INT type.
model_id: the ID of the model that is used in the training. The ID is a string that is 36 characters in length.
Save the job_id and model_id parameters in the preceding response to your on-premises device. You need to use the job_id to query training results and model_id to send a request for portrait creation.
Training result query (get_async_job)
Sample request:
from ai_service_python_sdk.client.api_client import ApiClient from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi host = 'http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com' appId = 'YOUR-APPID' token = 'YOUR-TOKEN' client = ApiClient(host, appId, token) ai_service_job_api = AiServiceJobApi(client) # Query the result. result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>) # Display the return value. print(result)The following table describes the key parameters.
Parameter
Description
appId
The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page.
token
The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page.
result
Replace the <YOUR-JOB-ID> parameter with the job_id returned by the aigc_images_train operation.
Sample response:
The following code block shows a sample response when the model training is incomplete:
{'code': 'OK', 'data': {'job': {'Result': '', 'app_id': '2******6', 'create_time': '2023-08-22T16:43:35.36+08:00', 'id': 11***, 'message': 'model requesting', 'state': 1, 'type': 'Image'}}, 'message': 'success', 'request_id': '8639143a-e147-4107-8e25-fcdeae24b0c5'}The following code block shows a sample response when the model training is complete:
{'code': 'OK', 'data': {'job': {'Result': '{"cost_time":1589.0886301994324, "states": [{"code":1,"frontal":false,"message":"success","url":"xxx.jpg"}],"model_id":"fa4ba43a-df55-45a4-9c31-bb0dc1e5****"}', 'app_id': '2******6', 'create_time': '2023-08-22T15:54:41.046+08:00', 'id': 11***, 'message': 'success', 'state': 2, 'type': 'Image'}}, 'message': 'success', 'request_id': '93b77f4b-56cb-4120-b45b-f5c88941bff5'}
The following table describes the response parameters.
Parameter
Description
request_id
The request ID, which is of the STRING type.
code
The request status code, which is of the STRING type. Valid values:
OK: The request was successful.
error: The request failed.
message
The details of the request status. If the request is successful, success is returned. In other scenarios, a different message is returned.
data
The details of the returned data, which is of the JSON Object type. For information about internal fields, view the following sections in this topic:
Description of related error codes
Request error codes
HTTP status code
code
message
Description
400
PARAMETER_ERROR
not found appid
The application ID is invalid.
EXCEEDED_QUOTA_ERROR
exceeded quota
The quota for service calls of the account is used up.
401
PARAMETER_ERROR
sign error
The token is invalid.
404
PARAMETER_ERROR
model not found
The requested model service is not deployed.
Response error codes
HTTP status code
code
message
Description
462
error
Invalid input data
An error occurred while parsing the input data.
Image not provided
The image for training is not provided.
Make dir in oss Error
Failed to create the Object Storage Service folder. Check whether OSS is mounted.
Image process error
An error occurred while preprocessing an image.
469
error
Training - Not get best template image
The training crashed and the template image is not generated.
Training - Not get lora weight
The training crashed and the LoRA weights are not generated.
Portrait creation
Sample requests:
Solo portrait creation (aigc_images_create)
import base64 import cv2 import numpy as np from ai_service_python_sdk.client.api.ai_service_aigc_images_api import \ AIGCImagesApi # noqa: E501 from ai_service_python_sdk.client.api_client import ApiClient def decode_image_from_base64jpeg(base64_image): image_bytes = base64.b64decode(base64_image) np_arr = np.frombuffer(image_bytes, np.uint8) image = cv2.imdecode(np_arr, cv2.IMREAD_COLOR) return image host = "http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com" appId = 'YOUR-APPID' token = 'YOUR-TOKEN' client = ApiClient(host, appId, token) api = AIGCImagesApi(client) # noqa: E501 response = api.aigc_images_create( '<Your-Model-ID>', '<url>', '<model_name>', <configure> ) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The details of the request status. message = response.message # The content returned by the model service. data = response.data print(response) image = data['image'] image = decode_image_from_base64jpeg(image) cv2.imwrite("1.jpg", image) print(data['cost_time'])The following table describes the parameters.
Parameter
Description
appId
The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page.
token
The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page.
response
The response parameter contains the following fields:
<Your-Model-ID>: Replace the value with the model ID returned by the aigc_images_train operation.
<url>: Replace the value with the URL of the template image that contains a single face. Example:
https://xxx/single_template.jpg.<model_name>: the name of the model. By default, an empty string is used. Example:
''.<configure>: the returned configuration items. Default value:
None. For information about the internal fields, see Parameters of the configure field.
Multi-person portrait creation (aigc_images_create_by_multi_model_ids)
import base64 import cv2 import numpy as np from ai_service_python_sdk.client.api.ai_service_aigc_images_api import \ AIGCImagesApi # noqa: E501 from ai_service_python_sdk.client.api_client import ApiClient def decode_image_from_base64jpeg(base64_image): image_bytes = base64.b64decode(base64_image) np_arr = np.frombuffer(image_bytes, np.uint8) image = cv2.imdecode(np_arr, cv2.IMREAD_COLOR) return image host = "http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com" appId = 'YOUR-APPID' token = 'YOUR-TOKEN' client = ApiClient(host, appId, token) api = AIGCImagesApi(client) # noqa: E501 response = api.aigc_images_create_by_multi_model_ids( ['<Your-Model-ID1>', '<Your-Model-ID2>'], '<url>', '<model_name>', <configure> ) # The ID of the request. request_id = response.request_id # The status of the request. code = response.code # The details of the request status. message = response.message # The content returned by the model service. data = response.data print(response) image = data['image'] image = decode_image_from_base64jpeg(image) cv2.imwrite("1.jpg", image) print(data['cost_time'])The following table describes the parameters.
Parameter
Description
appId
The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page.
token
The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page.
response
The response parameter contains the following fields:
<Your-Model-ID1> and <Your-Model-ID2>: Replace the values with the model IDs returned by the aigc_images_train operation.
<url>: Replace the value with the URL of the template image that contains multiple faces. The number of faces in the image must be the same as the value of the model_id parameter specified for model training. Example:
http://xxx/multi_template.jpg.<model_name>: the name of the model. By default, an empty string is used. Example:
''.<configure>: the returned configuration items. Default value:
None. For information about the internal fields, see Parameters of the configure field.
Sample response:
{ 'code': 'OK', 'data': { 'cost_time': 21.705406427383423, 'image': '/9j/4AAQSkZJRgABAQAAAQABAAD/.............2wBDAAgGBgcGBQgHBwcJCQgK', 'model_id': 'fa4ba43a-df55-45a4-9c31-bb0dc1e5****' }, 'message': 'success', 'request_id': 'df5454ca-07ec-4a15-be50-7beaba42f36b' }The following table describes the parameters.
Parameter
Description
request_id
The request ID, which is of the STRING type.
code
The status code of the request. Valid values:
OK: The request was successful.
error: The request failed.
message
The details of the request status. If the request is successful, success is returned. In other scenarios, a different message is returned.
data
The details of the returned data, which is of the JSON Object type. Parameters:
model_id: the model ID of the user.
image: the template image used to generate the AI portrait, which is Base64-encoded.
cost_time: the time required for portrait creation. The value is of the FLOAT type.
Error codes
Request error codes
HTTP status code
code
message
Description
400
PARAMETER_ERROR
not found appid
The application ID is invalid.
EXCEEDED_QUOTA_ERROR
exceeded quota
The quota for service calls of the account is used up.
401
PARAMETER_ERROR
sign error
The token is invalid.
404
PARAMETER_ERROR
model not found
The requested model service is not deployed.
Response error codes
HTTP status code
code
message
Description
462
error
Invalid input data. Please check the input dict.
An error occurred while parsing the input data.
mage not provided. Please check the template_image.
The template image that is used for portrait creation is not provided.
Prompts get error. Please check the model_id.
The format of the model ID is invalid.
Roop image decord error. Pleace check the user's lora is trained or not.
The Roop image does not exist. Check whether the model is trained.
Template image decord error. Please Give a new template.
An error occurred while decoding the template image. Provide a new template image.
There is not face in template. Please Give a new template.
No face exists in the template image. Provide a new template image.
Template image process error. Please Give a new template.
An error occurred while preprocessing the template image. Provide a new template image.
469
error
First Face Fusion Error, Can't get face in template image.
An error occurred during the first portrait merge.
First Stable Diffusion Process error. Check the webui status.
An error occurred during the first image generation by using Stable Diffusion.
Second Face Fusion Error, Can't get face in template image.
An error occurred during the second portrait merge.
Second Stable Diffusion Process error. Check the webui status.
An error occurred during the second image generation by using Stable Diffusion.
Please confirm if the number of faces in the template corresponds to the user ID.
The number of user IDs that you provided does not match the number of faces.
Third Stable Diffusion Process error. Check the webui status.
An error occurred while preprocessing the background. Provide a new template image.
500
error
Face id image decord error. Pleace check the user's lora is trained or not.
An error occurred while decoding the uploaded images. Check whether the model is trained.
Sample code of an end-to-end portrait creation process
The following code block shows a sample end-to-end portrait creation process. After the code is run, an AI portrait is created in the current directory.
from ai_service_python_sdk.client.api_client import ApiClient
from ai_service_python_sdk.client.api.ai_service_aigc_images_api import AIGCImagesApi
from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi
import logging
import sys
import time
import base64
import cv2
import numpy as np
host = "http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com"
appId = 'YOUR-APPID'
token = 'YOUR-TOKEN'
def decode_image_from_base64jpeg(base64_image):
image_bytes = base64.b64decode(base64_image)
np_arr = np.frombuffer(image_bytes, np.uint8)
image = cv2.imdecode(np_arr, cv2.IMREAD_COLOR)
return image
def Check(client, images):
# Check the request.
response = api.aigc_images_check(images, '', None)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The details of the request status.
message = response.message
# The content returned by the model service.
data = response.data
# Display the return value.
if code != "OK":
logging.error(f"aigc_images_check failed,request id is {request_id}")
logging.error(message)
sys.exit(-1)
else:
for check_result in data['check_results']:
if check_result['code'] != 1:
logging.error(f"check {check_result['url']} failed, message is {check_result['message']}")
sys.exit(-1)
logging.info("check images done")
def Train(client, images):
# The training request.
response = api.aigc_images_train(images, '', None)
request_id = response.request_id
code = response.code
message = response.message
data = response.data
# Display the return value.
if code != "OK":
logging.error(f"aigc_images_train failed, request id is {request_id}")
logging.error(message)
sys.exit(-1)
else:
# job id
job_id = response.data['job_id']
# model_id
model_id = response.data['model_id']
logging.info(f"train job_id is {job_id}")
logging.info(f"train moded_id is {model_id}")
if isinstance(job_id, int):
while True:
ai_service_job_api = AiServiceJobApi(client)
response = ai_service_job_api.get_async_job(job_id)
request_id = response.request_id
code = response.code
message = response.message
data = response.data
if code != "OK":
logging.error(f"get_async_job failed, request id is {request_id}")
logging.error(message)
sys.exit(-1)
else:
state = data["job"]["state"]
if state == 2:
logging.info(f"model {model_id} trained successfully")
break
elif state != 3:
logging.info(f"training model {model_id}")
time.sleep(10)
else:
logging.error(f"model {model_id} trained failed")
logging.error(f"message: {message}")
break
if state != 2:
sys.exit(-1)
return model_id
def Create(model_id, template_image):
# Create an image.
response = api.aigc_images_create(model_id, template_image,'', None)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The details of the request status.
message = response.message
# The content returned by the model service.
data = response.data
if code != "OK":
logging.error(f"aigc_images_create failed, model_id is {model_id}")
sys.exit(-1)
else:
image = data['image']
image = decode_image_from_base64jpeg(image)
cv2.imwrite("single_out.jpg", image)
logging.info(f"single create {data['cost_time']}")
def CreateMulti(model_ids, template_image):
response = api.aigc_images_create_by_multi_model_ids(
model_ids,
template_image,
'', None
)
# The ID of the request.
request_id = response.request_id
# The status of the request.
code = response.code
# The details of the request status.
message = response.message
# The content returned by the model service.
data = response.data
if code != "OK":
logging.error(f"aigc_images_create_by_multi_model_ids failed, model_id is {model_id}")
sys.exit(-1)
else:
image = data['image']
image = decode_image_from_base64jpeg(image)
cv2.imwrite("multi_out.jpg", image)
logging.info(f"multi create cost time {data['cost_time']}")
if __name__ == "__main__":
client = ApiClient(host, appId, token)
api = AIGCImagesApi(client) # noqa: E501
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
# The training images in the URL format.
images = [
'https://xxx/0.jpg',
'https://xxx/1.jpg',
'https://xxx/2.jpeg',
'https://xxx/3.jpeg',
]
#check
Check(client, images)
model_id = Train(client, images)
template_image = 'https://xxx/single_template.jpg'
Create(model_id, template_image)
#read the image single_out.jpg
model_ids = [model_id, model_id]
multi_template_image = 'https://xxx/multi_template.jpg'
CreateMulti(model_ids, multi_template_image)
#read the image multi_out.jpgThe following table describes the parameters.
Parameter | Description |
appId | The application ID. After you activate the AI Portrait service, you can view the application ID on the AI Portrait page. |
token | The token. After you activate the AI Portrait service, you can view the token on the AI Portrait page. |
images | The URLs of the training images. Separate multiple URLs with commas (,). |
template_image | The URL of the template image that contains a single face. The template image is used to generate a single-person portrait. |
multi_template_image | The URL of the template image that contains multiple faces. The number of faces must be the same as the number of model IDs that you specify. The template image is used to generate a multi-person portrait. |