AnimateAnyone action template generation API - Alibaba Cloud Model Studio

The AnimateAnyone action template generation model extracts character movements from a video of character motion and generates a character action template. This template can be used by the AnimateAnyone video generation model. This document describes how to call the API for the action template generation feature of this model.

Important

This document applies only to the China (Beijing) region. To use the model, you must use an API key for the China (Beijing) region.

Model overview

Model	Description
animate-anyone-template-gen2	animate-anyone-template-gen2 is a model for generating character action templates. It can extract character movements from a video of character motion and create a template.

Model input requirements

Correct examples:

Examples of videos that meet the requirements for creating action templates

Note

The person in the uploaded video must be fully visible, with no body parts obscured, and the face must be clear.
The character must appear from the first frame of the video. The action must be continuous and captured in a single take. We recommend that you split videos with scene changes into multiple segments.
Recommendations: The character should face the camera in the first frame. Avoid actions such as excessive bending, squatting, or crouching.

Incorrect examples:

Crouched or obscured body

Multiple people in the frame

Blurry character

Character too small

Unclear face

Character too large

Character not fully visible

身体遮挡@3x

多人物@3x

f3e2df6643d44db7a7d65d4571609bff_3 (1)

人物过小@3x

人物过大@3x

Note

To ensure the quality of the generated template, make sure that the uploaded video does not contain any of the issues shown in the incorrect examples.
If any of the issues shown in the incorrect examples appear in the first frame of the video, the submitted task may fail and be aborted.

Important

Make sure that the video content complies with relevant laws and regulations.
The generated action template contains the same audio as the uploaded video file. If you do not want to use this audio, or if you have not obtained the license to use the audio (such as music), remove the audio from the video file before you upload it.

Use HTTP

Feature description

This API generates a character action template. This template can be used as an input for the AnimateAnyone video generation API to generate a character action video.

Prerequisites

You have activated the service and obtained an API key. For more information, see Get an API key.

Input limits

Video format: MP4, AVI, or MOV.
The video file size must be less than 200 MB.
The length of each side of the video must be between 200 and 2,048 pixels. The video frame rate must be 24 frames per second (fps) or higher. The video must be encoded in H.264 or H.265.
The video duration must be 2 seconds to 60 seconds.
The aspect ratio of the video must be between 1:3 and 3:1.
Only HTTP URLs are supported; local file paths are not allowed.

Task submission

POST https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/aa-template-generation/

Note

Because this algorithm takes a long time to run, the task is submitted using an asynchronous invocation.
After you submit the task, the system returns a task ID. You can then use the API for querying the task status and result to retrieve the task status and the result.

Request parameters

Parameter	Type	Method	Required	Description	Example
Content-Type	String	Header	Yes	Request type: application/json	application/json
Authorization	String	Header	Yes	API key. Example: Bearer d1**2a	Bearer d1**2a
X-DashScope-Async	String	Header	Yes	Set to `enable` to indicate that the task is submitted asynchronously.	enable
model	String	Body	Yes	Specifies the model to be called. In this case, set to `animate-anyone-template-gen2`.	animate-anyone-template-gen2
input.video_url	String	Body	No	The URL of the video that you upload. This video is used to generate a motion template. The video file size must be less than 200 MB. The width and height of the video must each be between 200 and 2,048 pixels. The video frame rate must be 24 fps or higher. The video must be encoded in H.264 or H.265. The video duration must be 2 seconds to 60 seconds. The aspect ratio of the video must be between 1:3 and 3:1. Supported video formats: MP4, AVI, and MOV. Note File uploads support only HTTP links, not local file paths.	http://aaa/bbb.mp4

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The ID of the submitted asynchronous task. The actual task result must be obtained through the task query interface.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	Task status after submission	"PENDING"
request_id	String	Request ID	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl --location 'https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/aa-template-generation/' \
--header 'X-DashScope-Async: enable' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--header 'Content-Type: application/json' \
--data '{
    "model": "animate-anyone-template-gen2",
    "input": {
        "video_url": "http://xxx/1.mp4"
    },
    "parameters": {
    }
  }'

Sample response

{
    "output": {
	"task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "PENDING"
    }
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
}

Task status query and result retrieval

GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}

Request parameters

Parameter	Type	Method	Required	Description	Example
Authorization	String	Header	Yes	API key. Example: Bearer d1**2a.	Bearer d1**2a
task_id	String	Url Path	Yes	The ID of the task whose task you want to query.	a8532587-fa8c-4ef8-82be-0c46b17950d1

Response parameters

Parameter	Type	Description	Example
output.task_id	String	The ID of the task that is queried.	a8532587-fa8c-4ef8-82be-0c46b17950d1
output.task_status	String	The status of the queried task.	Task status: PENDING RUNNING SUCCEED FAILED UNKNOWN: The task does not exist or its status is unknown.
output.template_id	String	The action template ID output by the platform. This ID can be used as a request parameter for the 'Animate-Anyone video generation API'. The action template ID will undergo verification. Make sure that the template_id you use was created by your current Alibaba Cloud account.	AACT.xxx.xxx-xxx.xxx
usage.video_duration	Float	The duration of the template generated for this request. Unit: seconds.	"video_duration": 10.23
usage.video_ratio	String	The aspect ratio type of the video template generated for this request. The value is `standard`.	"video_ratio": "standard"
request_id	String	The request ID.	7574ee8f-38a3-4b1e-9280-11c33ab46e51

Sample request

curl -X GET \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
https://dashscope.aliyuncs.com/api/v1/tasks/<YOUR_TASK_ID>

Sample response

{
    "request_id":"7574ee8f-38a3-4b1e-9280-11c33ab46e51",
    "output":{
        "task_id":"a8532587-fa8c-4ef8-82be-0c46b17950d1",
		"task_status":"SUCCEEDED",
        "template_id":"AACT.xxx.xxx-xxx.xxx"
    },
    "usage":{
        "video_duration": 10.23,
        "video_ratio": "standard"
    }
}

Error sample response:

{
    "request_id": "7574ee8f-38a3-4b1e-9280-11c33ab46e51"
  	"output": {
        "task_id": "a8532587-fa8c-4ef8-82be-0c46b17950d1", 
    	"task_status": "FAILED",
    	"code": "xxx", 
    	"message": "xxxxxx", 
    }  
}

Status codes

For information about the common status codes, see Error messages.

This model has the following specific error codes:

HTTP status code*	code	message	Description
400	InvalidParameter	Field required: xxx	A request parameter is missing or the format is incorrect.
400	InvalidURL.ConnectionRefused	Connection to ${url} refused, please provide available URL	Download refused, please provide an available URL
400	InvalidURL.Timeout	Download ${url} timeout, please check network connection.	The download timed out. The timeout period is 60s.
400	InvalidFile.Size	Invalid file size. The video file size must be less than 200MB, and the audio file size must be less than 15MB.	The video file size must be less than 200 MB, and the audio file size must be less than 15 MB.
400	InvalidFile.Resolution	Invalid video resolution. The height or width of video must be 200 ~ 2048.	The width and height of the video must each be between 200 and 2,048 pixels.
400	InvalidFile.Duration	Invalid file duration. The file duration must be 2s ~ 60s.	The duration of the video or audio file must be between 2 and 60 seconds.
400	InvalidFile.AspectRatio	Invalid file ratio. The file aspect ratio (height/width) must be between 3:1 and 1:3.	The aspect ratio of the video file must be between 3:1 and 1:3.
400	InvalidFile.Openerror	Invalid file, cannot open file as video.	The video file cannot be opened.
400	InvalidFile.Content	No human body	A person must be present in the first frame of the video.
400	InvalidFile.Content	The person is too small in the first frame of input video. Please choose another clip.	Person too small
400	InvalidFile.Content	The person is not clear in the first frame of input video. Please choose another clip.	Unclear person
400	InvalidFile.FullBody	Human not fullbody	Person not fully visible