LivePortrait video generation API reference - Alibaba Cloud Model Studio

The LivePortrait model quickly generates lightweight, dynamic portrait videos from portrait images detected by the LivePortrait-detect model and human voice recordings. This document describes how to use the API for video generation of this model.

Important

This document applies only to the China (Beijing) region. To use the model, you must use an API key for the China (Beijing) region.

Model overview

Model	Description
liveportrait	Liveportrait is a portrait video generation model. It can quickly generate lightweight, dynamic portrait videos based on portrait images and human voice recordings.

HTTP API

Feature description

This API is used to generate dynamic portrait videos.

Prerequisites

You have activated Alibaba Cloud Model Studio and obtained an API key. For more information, see Create an API key.
The input image has been processed by LivePortrait image detection.

Input limitations

Image format: The image must be in the JPEG, JPG, PNG, BMP, or WebP format.
Image resolution: The image file must be smaller than 10 MB. The aspect ratio must be less than or equal to 2. The maximum side length cannot exceed 4,096 pixels.
Audio format: The audio file must be in the WAV or MP3 format.
Audio limits: The audio file must be smaller than 15 MB. The duration must be longer than 1 second and shorter than 3 minutes.
Audio content: The audio must contain a clear and loud human voice. Environmental noise, background music, or other sound interference must be removed.
Only HTTP URLs are supported for uploading images and audio files. Local file paths are not supported.

Task submission

POST https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis/

Note

Because the algorithm invocation is time-consuming, the task is submitted asynchronously.
After you submit the task, the system returns a task ID. You can then use the 'Query task status and retrieve results' API to retrieve the task status and results.

Request parameters

Parameter	Type	Method	Required	Description	Example
Content-Type	String	Header	Yes	The request type. Set the value to application/json.	application/json
Authorization	String	Header	Yes	The API key. Example: Bearer d1**2a.	Bearer d1**2a
X-DashScope-Async	String	Header	Yes	Set the value to enable to submit the task asynchronously.	enable
model	String	Body	Yes	The model to be called. Set the value to liveportrait.	liveportrait
input.image_url	String	Body	Yes	The URL of the image that you upload. The image must be processed by the LivePortrait image detection API first. The image file must be smaller than 10 MB. The aspect ratio must be less than or equal to 2. The maximum side length cannot exceed 4096 pixels. Supported formats: JPEG, JPG, PNG, BMP, and WebP. Note File uploads support only HTTP links, not local file paths.	"image_url": "http://a/a.jpg"
input.audio_url	String	Body	Yes	The URL of the audio file that you upload. The audio file must be smaller than 15 MB. The duration must be longer than 1 second and shorter than 3 minutes. Supported formats: WAV and MP3. Note File uploads support only HTTP links, not local file paths.	http://aaa/bbb.wav
parameters.template_id	String	Body	No	Controls the posture and range of head movements based on a template. The following templates are supported: normal, calm, and active. The default value is normal.	"normal"
parameters.eye_move_freq	Float	Body	No	The number of blinks per second. The value can be set from 0 to 1. A larger value indicates a higher blinking frequency. The default value is 0.5.	0.5
parameters.video_fps	Integer	Body	No	The frame rate of the output video. The value can be set from 15 to 30. The default value is 24.	24
parameters.mouth_move_strength	Float	Body	No	The magnitude of mouth movements. The value can be set from 0 to 1.5. A larger value indicates a larger mouth shape. If this parameter is set to 0, the mouth does not move. The default value is 1.	1
parameters.paste_back	Boolean	Body	No	Specifies whether to paste the generated face back to the original image. The value can be true or false. If this parameter is set to false, only the generated face is output, and the body is ignored. The default value is true.	true
parameters.head_move_strength	Float	Body	No	The magnitude of head movements. The value can be set from 0 to 1. A larger value indicates a larger range of head movements. The default value is 0.7.	0.7

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The ID of the submitted asynchronous task. The actual task result must be obtained using the asynchronous task query API.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The status of the task after the asynchronous task is submitted.	"PENDING"
request_id	String	The request ID.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Action templates

template_id	Effect description
normal	The default action template. The range of head movements is moderate. This template is suitable for various scenarios.
calm	The character appears calm, and the range of head movements is small. This template is recommended for scenarios such as broadcasting.
active	The character appears lively, and the range of head movements is large. This template is recommended for scenarios such as singing.

Sample request

curl --location 'https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis/' \
--header 'X-DashScope-Async: enable' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--header 'Content-Type: application/json' \
--data '{
    "model": "liveportrait",
    "input": {
        "image_url": "http://xxx/1.jpg",
        "audio_url": "http://xxx/1.wav"
    },
      "parameters": {
         "template_id": "normal",
         "eye_move_freq": 0.5,
         "video_fps":30,
         "mouth_move_strength":1,
         "paste_back": true,
         "head_move_strength":0.7
    }
  }'

Sample response

{
    "output": {
	"task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "PENDING"
    }
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
}

Query task status and retrieve results

GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}

Request parameters

Parameter	Type	Method	Required	Description	Example
Authorization	String	Header	Yes	The API key. Example: Bearer d1**2a.	Bearer d1**2a
task_id	String	Url Path	Yes	The ID of the task to be queried.	a8532587-fa8c-4ef8-82be-0c46b17950d1

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The ID of the queried task.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The status of the queried task.	Task status: PENDING RUNNING SUCCEEDED FAILED UNKNOWN
output.results.video_url	String	If the task is successful, this parameter contains the result object generated by the model. Each object contains the URL of the generated result as required. The video_url is valid for 24 hours after the task is completed.	https://xxx/1.mp4
usage.video_duration	Float	The duration of the video generated for this request, in seconds.	10.23
usage.video_ratio	String	The frame type of the video generated for this request. The value is standard.	"video_ratio": "standard"
request_id	String	The request ID.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl -X GET \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
https://dashscope.aliyuncs.com/api/v1/tasks/<YOUR_TASK_ID>

Sample response (successful)

{
    "request_id":"7574ee8f-38a3-4b1e-9280-11c33ab46e51",
    "output":{
        "task_id":"a8532587-fa8c-4ef8-82be-0c46b17950d1",
	"task_status":"SUCCEEDED",
        "results":{
            "video_url":"https://xxx/1.mp4"
        }
    },
    "usage":{
        "video_duration": 10.23,
        "video_ratio": "standard"
    }
}

Sample response (failed)

{
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
    "output": {
        "task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "FAILED",
    	"code": "xxx", 
    	"message": "xxxxxx", 
    }  
}

Status codes

For information about the common status codes, see Error messages.

This model also has the following specific error codes:

HTTP return code*	code	message	Description
400	InvalidParameter.UnsupportedFileFormat	Input files format not supported.	The audio or image format is not supported. Supported audio formats: MP3, WAV, and AAC. Supported image formats: JPG, JPEG, PNG, BMP, and WebP.
400	InvalidParameter.InputDownloadFailed	Failed to download input files.	Failed to download the input file.
400	InvalidImage.ImageSize	The size of image is beyond limit.	The image size exceeds the limit. The aspect ratio of the image must be less than or equal to 2, and the longest side cannot exceed 4,096 pixels.
400	InvalidFile.AudioLengthError	Audio length must be between 1s and 180s.	The audio length is invalid. It must be within the range of [1, 180] seconds.
400	InvalidImage.NoHumanFace	No human face detected.	No human face is detected (only for the asynchronous query API for generation tasks).
400	InvalidParamter.OutOfDefinition	The type or value of {parameter} is out of definition.	The parameter type or value is invalid.
500	InternalError.Algo	An internal error has occured during algorithm execution.	An error occurred during algorithm execution.
500	InternalError.Upload	Failed to upload result.	Failed to upload the generated result.