AnimateAnyone video generation API - Alibaba Cloud Model Studio

The AnimateAnyone model generates character animation videos based on action templates created by AnimateAnyone-template and character images detected by AnimateAnyone-detect. This topic describes the how to call the API for the video generation capability provided by this model.

Important

This document applies only to the China (Beijing) region. To use the model, you must use an API key from the China (Beijing) region.

Model overview

Model introduction

Model	Description
animate-anyone-gen2	animate-anyone-gen2 is a character animation model that generates videos based on character images and action templates.

Performance showcase

Character image	Action template	Output (image background)	Output (video background)

Note

The preceding examples were generated by the Tongyi App, which integrates AnimateAnyone.
Action templates need to be created through the AnimateAnyone action template generation API. Please ensure that the source video for the action template complies with relevant laws and regulations, and you have obtained a license to use the video content (including audio).

Use HTTP

Feature description

Used to generate character action videos.

Prerequisites

You have activated Alibaba Cloud Model Studio and obtained an API key. For more information, see Obtain API key.
The input action template has been generated by the AnimateAnyone action template generation API.
The input image has been detected by the AnimateAnyone image detection API.

Input limitations

Image format: JPEG, PNG, BMP.
Image requirements: Image file size < 5MB, maximum aspect ratio of 2:1, maximum edge length ≤ 4096 pixels.
Uploaded image files support HTTP links but do not support local paths. You can also use the temporary storage space provided by Model Studio to upload local files and create links.

Relationship between input images and video generation types:

Input image

Generate with image background

(use_ref_img_bg set to true)

Generate with video background

(use_ref_img_bg set to false)

Full-body portrait

Supported

Half-body portrait

Supported

Not recommended

Note

When generating with video background, the character in the image needs to be matched to the corresponding position of the character in the video. For areas not appearing in half-body portrait images (such as legs), the model will randomly generate the missing parts, which introduces significant uncertainty. Therefore, video generation under this condition is not recommended.

Task submission

POST https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis/

Note

Because this algorithm takes a long time, an asynchronous invocation method is used to submit tasks.
After the task is submitted, the system will return the corresponding task ID, which can be used to obtain the task status and corresponding results through the "Task status query and result retrieval interface".

Request parameters

Parameter	Type	Method	Required	Description	Example
Content-Type	String	Header	Yes	Request type: application/json.	application/json
Authorization	String	Header	Yes	API key, for example: Bearer d1**2a.	Bearer d1**2a
X-DashScope-Async	String	Header	Yes	Set to "enable" to indicate that the task is submitted asynchronously.	enable
model	String	Body	Yes	Specifies the model to be called.	animate-anyone-gen2
input.image_url	String	Body	Yes	URL of the user-uploaded image. This image should first be processed through the AnimateAnyone image detection API and appropriately cropped according to the desired output frame. Image file size < 5MB, maximum aspect ratio of 2:1, maximum edge length ≤ 4096 Supported formats: JPEG, PNG, BMP. Note File uploads support only HTTP links, not local file paths.	http://aaa/bbb.jpg
input.template_id	String	Body	Yes	Action template ID, used to specify the action template to use. You should input the template_id generated by the AnimateAnyone action template generation API. Note The action template ID will undergo verification. Make sure that the template_id you use was created by your current Alibaba Cloud account. Usage suggestion: Increasing the resolution and frame rate of the template video can effectively improve the quality of the generated video.	AACT.xxx.xxx-xxx.xxx
parameters.use_ref_img_bg	Boolean	Body	No	Background control for the generated video. When set to true, the video will use the background of the input image. When set to false, the video will use the background of the original video. This is the default value.	false
parameters.video_ratio	String	Body	No	When choosing to generate video with image background, the available aspect ratios are "9:16" or "3:4", with the default being "9:16". Note When choosing to generate with video background (use_ref_img_bg set to false), this parameter has no effect. The new video will be generated according to the aspect ratio of the template video. Note You should ensure that the aspect ratio of the input image matches the selected aspect ratio to avoid distortion in the generated video.	"9:16"

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The task ID of the submitted asynchronous task. The task result needs to be obtained through the task query interface.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The task status after submitting the asynchronous task.	"PENDING"
request_id	String	The request ID.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl --location 'https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis/' \
--header 'X-DashScope-Async: enable' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model": "animate-anyone-gen2",
    "input": {
        "image_url": "http://xxx/1.jpg",
        "template_id": "AACT.xxx.xxx-xxx.xxx"
    },
      "parameters": {
         "use_ref_img_bg": false,
         "video_ratio": "9:16"
    }
  }'

Sample response

{
    "output": {
	"task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "PENDING"
    }
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
}

Task status query and result retrieval

GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}

Request parameters

Parameter	Type	Method	Required	Description	Example
Authorization	String	Header	Yes	API key, for example: Bearer d1**2a.	Bearer d1**2a
task_id	String	Url Path	Yes	The task_id of the task to be queried.	a8532587-fa8c-4ef8-82be-0c46b17950d1

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The task_id of the queried task	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The status of the queried task	Task status: PENDING PRE-PROCESSING RUNNING POST-PROCESSING SUCCEEDED FAILED UNKNOWN - Task does not exist or status unknown
output.video_url	String	The video result output by the platform. The video_url is valid for 24 hours after the task is completed	https://xxx/1.mp4"
usage.video_duration	Float	The duration of the generated video, in seconds	"video_duration": 10.23
usage.video_ratio	String	The aspect ratio type of the generated video for this request, with a value of "standard"	"video_ratio": "standard"
request_id	String	The request ID	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl -X GET \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
https://dashscope.aliyuncs.com/api/v1/tasks/<YOUR_TASK_ID>

Sample response

{
    "request_id":"7574ee8f-38a3-4b1e-9280-11c33ab46e51",
    "output":{
        "task_id":"a8532587-fa8c-4ef8-82be-0c46b17950d1",
	"task_status":"SUCCEEDED",
        "video_url":"https://xxx/1.mp4"
    },
    "usage":{
        "video_duration": 10.23,
        "video_ratio": "standard"
    }
}

Error sample response

{
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
  	"output": {
        "task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "FAILED",
    	"code": "xxx", 
    	"message": "xxxxxx", 
    }  
}

Status codes

For general status codes, see: Error messages.

This model also has the following specific error codes:

HTTP status code*	code	message	Description
400	InvalidURL.ConnectionRefused	Connection to ${url} refused, please provide available URL	Download refused, please provide an available URL
400	InvalidURL.Timeout	Download ${url} timeout, please check network connection.	The download timed out. The timeout period is 10s
400	InvalidFile.Size	Invalid file size, The image file size must be less than 5 MB.	Image file exceeds 5 MB
400	InvalidFile.Resolution	Please ensure the longest side is less than 4096 pixels and the shortest side is greater than 224 pixels.	Uploaded image size does not meet requirements
400	InvalidFile.AspectRatio	Invalid file ratio. The file aspect ratio (height/width) must be between 0.5 and 2.0	Image file must have a maximum aspect ratio of 2:1
400	InvalidFile.Template.Content	Invalid template content.	Action template has no permission or template content does not meet requirements.
400	InvalidParameter	parameters.video_ratio must be 9:16 or 3:4	video_ratio input parameter can only be "9:16" or "3:4"