Multimedia analysis SDK for Python - Platform For AI - Alibaba Cloud Documentation Center

The multimedia analysis SDK for Python lets you call Platform for AI (PAI) algorithm-powered services — image quality scoring, face attribute detection, video classification, and more — from your Python application.

Prerequisites

Before you begin, make sure you have:

Python 3 or later installed
An activated multimedia analysis service with AppId and Token (available on the Multimedia Analysis page in the PAI console)

Step 1: Install the SDK

pip install ai-service-python-sdk==1.1.5

Step 2: Initialize the client

from ai_service_python_sdk.client.api_client import ApiClient

client = ApiClient('<HOST>', '<YOUR-APPID>', '<YOUR-TOKEN>')

Replace the placeholders with your actual values:

Placeholder	Description
`<HOST>`	Server address: `http://ai-service.ce8cc13b6421545749e7b4605f3d02607.cn-hangzhou.alicontainer.com`
`<YOUR-APPID>`	AppId shown on the Multimedia Analysis page after activation
`<YOUR-TOKEN>`	Token shown on the Multimedia Analysis page after activation

Sync vs. async calls

Image services return results synchronously — call the method and read response.data directly. Video services and multimodal services are asynchronous because they take longer to process: they return a job ID immediately, and you poll for the result using AiServiceJobApi.

Polling pattern for async services:

Submit the job and save the returned job_id or batch_id.
Call get_async_job(job_id) or get_batch_job_with_id(batch_id).
Check state in the response. If state is 1 (running), wait and retry.
Stop polling when state is 2 (completed) or 3 (failed).

Call type	Method pattern	Return value
Sync	call method → read `response.data`	Data directly
Async (single)	call method → get `job_id` → call `get_async_job(job_id)` until done	`job_id` (INT)
Async (batch)	call method → get `batch_id` → call `get_batch_job_with_id(batch_id)` until done	`batch_id` (INT)

Examples

All examples reuse the client initialized in Step 2.

Image quality evaluation

Scores the quality of an image on a scale from 0 to 100.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = ''   # Leave blank to use the general-purpose model
configure = {}

response = ai_service_api.image_qa(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. Keep the file size under 5 MB and the resolution at 240×240 or higher.
`model_name`	STRING	Yes	Leave blank to use the general-purpose model. To use a custom model, pass its name. Contact your account manager to get a custom model.
`configure`	Dict	Yes	Response settings. Pass an empty dictionary for this service.

Sample response

{
    "code": "OK",
    "data": {
        "image_height": 214,
        "image_width": 398,
        "iqa_result": 66.88
    },
    "message": "success",
    "request_id": "1d4572a1-2800-4213-9e9c-33f9fa7e****"
}

Response parameters

Parameter	Type	Description
`request_id`	STRING	The request ID.
`code`	STRING	The status code.
`message`	STRING	Status details. See Error codes.
`data.image_height`	INT	Image height in pixels.
`data.image_width`	INT	Image width in pixels.
`data.iqa_result`	FLOAT	Quality score, accurate to two decimal places.

Face attribute analysis

Returns face-related attributes such as face shape, hair color, hairstyle, and bounding box coordinates.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = ''
configure = {}

response = ai_service_api.face_attr_image(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. The face area must have a resolution higher than 100×100.
`model_name`	STRING	Yes	Leave blank to use the general-purpose model. To use a custom model, pass its name.
`configure`	Dict	Yes	Response settings. Pass an empty dictionary for this service.

Sample response

{'code': 'OK',
 'data': {'attr': [{'FaceShape': {'Triangle': 0.0013,
                                  'Round': 0.0147,
                                  'Heart': 0.0405,
                                  'Square': 0.0001,
                                  'Oval': 0.9266,
                                  'Diamond': 0.0027,
                                  'Oblong': 0.0141},
                    'FemaleHairStyle': {'BangsType': {'Center parted bangs': 0.5932,
                                                      'Braided bangs': 0.0058,
                                                      'Side-swept bangs': 0.0575,
                                                      'No bangs': 0.3078,
                                                      'Wispy bangs': 0.0143,
                                                      'Blunt bangs': 0.0213},
                                        'FemaleCurlyHairType': {'Cloud curls': 0.1289,
                                                                'Loose waves': 0.2526,
                                                                'Soft waves': 0.084,
                                                                'Bouncy curls': 0.2027,
                                                                'Airy curls': 0.1434,
                                                                'Tight curls': 0.1539,
                                                                'Egg rolls': 0.0345},
                                        'FemaleHairType': {'Curly': 0.1941,
                                                           'Updo': 0.0112,
                                                           'Straight': 0.7947},
                                        'FemaleLongShortHair': {'Medium': 0.2807,
                                                                'Short': 0.0346,
                                                                'Long': 0.6846},
                                        'Tied hair': 0.3054,
                                        'Braided hair': 0.3453},
                    'HairColor': {'Other': 0.0001,
                                  'Coffee brown': 0,
                                  'Silver gray': 0,
                                  'Chestnut': 0.0025,
                                  'Brown': 0.0622,
                                  'Gradient': 0.0004,
                                  'Wine red': 0.0001,
                                  'Golden': 0,
                                  'Yellow': 0,
                                  'Black': 0.9347},
                    'MaleHairStyle': {'Side part': 0.9939,
                                      'Buzz cut': 0,
                                      'Crew cut': 0.0033,
                                      'Flat top': 0,
                                      'Induction cut': 0.001,
                                      'Textured crop': 0.0008,
                                      'Tousled': 0,
                                      'Slicked back': 0.0001,
                                      'Mohawk': 0.0006},
                    'bbox': [91, 0, 430, 452],
                    'face_score': 3.2431,
                    'index': 0}]},
 'message': 'success',
 'request_id': '9b2f2c0e-d2c0-45f8-b922-058d00edc9ad'}

Response parameters — data

Parameter	Type	Description
`attr`	LIST	List of detected faces. An empty list means no faces were found. Each element contains the attributes below.
`FaceShape`	DICT	Face shape probabilities.
`FemaleHairStyle`	DICT	Female hairstyle attributes.
`BangsType`	DICT	Female bangs type.
`FemaleCurlyHairType`	DICT	Female curl type. Read when `FemaleHairType` most likely indicates a curly style.
`FemaleHairType`	DICT	Female hair type.
`FemaleLongShortHair`	DICT	Female hair length.
`Tied hair`	FLOAT	Whether the hair is tied. Values ≥ 0.5 indicate tied hair.
`Braided hair`	FLOAT	Whether the hair is braided. Values ≥ 0.5 indicate braided hair.
`HairColor`	DICT	Hair color probabilities.
`MaleHairStyle`	DICT	Male hairstyle probabilities.
`bbox`	LIST	Bounding box of the face in `[xmin, ymin, xmax, ymax]` format.
`index`	INT	Face index in the image, starting from 0.

Age analysis

Important

This is a custom model. Contact your account manager and provide your AppId to enable it.

Identifies the age range of the main face in an image.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = 'fairface'   # Required: custom model name
configure = {}

response = ai_service_api.face_attr_image(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. The face area must have a resolution higher than 100×100.
`model_name`	STRING	Yes	Set to `fairface`.
`configure`	Dict	Yes	Response settings. Pass an empty dictionary for this service.

Sample response

{
    "request_id": "10000",
    "code": "OK",
    "message": "success",
    "data": {
        "age_range": "3-9",
        "age_range_prob": 0.9525
    }
}

Response parameters — data

Parameter	Type	Description
`age_range`	STRING	Predicted age range. Possible values: `0-2`, `3-9`, `10-19`, `20-29`, `30-39`, `40-49`, `50-59`, `60-69`, `70+`.
`age_range_prob`	FLOAT	Probability of the predicted age range.

Multiple image labels

Returns the top K labels for an image sorted by probability, and optionally returns 512-dimensional image embeddings.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = ''
configure = {'tag_top_k': 5, 'output_embedding': False}

response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. Keep the file size under 5 MB and the resolution at 240×240 or higher.
`model_name`	STRING	Yes	Leave blank to use the general-purpose model. Contact PAI technical support to get a custom model.
`configure`	Dict	Yes	Response settings. Default: `None`. If set to `None`, all configure parameters use their defaults. See the configure parameters table below.

Configure parameters

Parameter	Type	Required	Description
`tag_top_k`	INT	No	Number of top labels to return. Valid values: 1–10. Default: 10.
`output_embedding`	BOOL	No	Whether to return 512-dimensional image embeddings. `True` (default): return embeddings. `False`: skip embeddings.

Sample response

{'code': 'OK',
 'data': {'image_height': 510,
          'image_width': 516,
          'tag_result': [{'score': 0.505, 'tag': 'Smile'},
                         {'score': 0.493, 'tag': 'Woman'},
                         {'score': 0.475, 'tag': 'Beauty'},
                         {'score': 0.438, 'tag': 'Grin'},
                         {'score': 0.432, 'tag': 'Honey brown'}]},
 'message': 'success',
 'request_id': 'c9e5aa33-4d8d-49ce-8991-bcbb5f12e43b'}

Response parameters — data

Parameter	Type	Description
`tag_result`	LIST	Labels sorted by score in descending order. Length depends on `tag_top_k`.
`score`	FLOAT	Label probability, accurate to three decimal places.
`tag`	STRING	Label name.
`embedding_result`	STRING	512-dimensional features separated by commas. Returned only when `output_embedding` is `True`.
`image_height`	INT	Image height in pixels.
`image_width`	INT	Image width in pixels.

Labeling AI paintings

Important

This is a custom model. Provide your AppId to your account manager to enable it.

Generates a synthesized caption and label scores for images produced by Stable Diffusion models. Use this to improve training data for image generation.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = 'tagger'
configure = {'threshold': 0.5, 'tagger_type': "SmilingWolf/wd-v1-4-convnextv2-tagger-v2"}

response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. Keep the file size under 5 MB and the resolution at 240×240 or higher.
`model_name`	STRING	Yes	Set to `tagger`.
`configure`	Dict	Yes	Response settings. Default: `None`. See configure parameters below.

Configure parameters

Parameter	Type	Required	Description	Example
`threshold`	FLOAT	No	Confidence threshold for label inclusion.	`0.5`
`tagger_type`	STR	No	Labeling model to use. Valid values: `SmilingWolf/wd-v1-4-convnext-tagger-v2`, `SmilingWolf/wd-v1-4-convnextv2-tagger-v2`, `SmilingWolf/wd-v1-4-vit-tagger-v2`, `SmilingWolf/wd-v1-4-swinv2-tagger-v2`, `RAM`, `BLIP`, `GIT`.	`SmilingWolf/wd-v1-4-convnextv2-tagger-v2`

Sample response

{
    "data": {
        "caption": "sensitive, 1girl, solo, long hair, looking at viewer, smile, black hair, brown eyes, scarf, lips, realistic",
        "probs": [1.1920928955078125e-07, "............", 3.5762786865234375e-07, 4.76837158203125e-07]
    },
    "request_id": "12***",
    "code": "OK",
    "message": "success"
}

Response parameters — data

Parameter	Type	Description
`caption`	STR	Labeling results separated by commas.
`probs`	List	Scores from the specified labeling model.

Portrait figure modification

Important

This is a custom model. Provide your AppId to your account manager to enable it.

Modifies the figure in a portrait image. Set degree to a positive value to slim the figure, or to a negative value to enlarge it.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = 'body_reshaping'
# Positive degree values slim the figure; negative values enlarge it.
# To enlarge, use a value between -2.5 and -1.5.
configure = {'degree': -1.5}

response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the portrait image. Keep the file size under 5 MB and the resolution at 240×240 or higher.
`model_name`	STRING	Yes	Set to `body_reshaping`.
`configure`	Dict	Yes	Response settings. Default: `None`. See configure parameters below.

Configure parameters

Parameter	Type	Required	Description
`degree`	FLOAT	No	Degree of figure modification. Values greater than `0` slim the figure. Default: `-1.5`.

Sample response

{
    "code": "OK",
    "data": "imByev+I3vrryTGN****",
    "message": "success",
    "request_id": "1d4572a1-2800-4213-9e9c-33********"
}

Response parameters

Parameter	Type	Description
`request_id`	STRING	The request ID.
`code`	STRING	The status code.
`message`	STRING	Status details.
`data`	STRING	Base64-encoded output image.

Watermark removal

Important

This is a custom model. Provide your AppId to your account manager to enable it.

Removes watermarks from images, targeting common watermarks from videos and social media platforms.

from ai_service_python_sdk.client.api.ai_service_image_api import AiServiceImageApi

image_url = 'https://your_image_url.jpg'
ai_service_api = AiServiceImageApi(client)
model_name = 'watermark_remover'
configure = {}

response = ai_service_api.multi_label_image_v2(image_url, model_name, configure)
print(response)

Request parameters

Parameter	Type	Required	Description
`image_url`	STRING	Yes	URL of the image. Keep the file size under 5 MB and the resolution at 240×240 or higher.
`model_name`	STRING	Yes	Set to `watermark_remover`.
`configure`	Dict	Yes	Pass an empty dictionary.

Sample response

{
    "code": "OK",
    "data": "imByev+I3vr****",
    "message": "success",
    "request_id": "1d4572a****"
}

Response parameters

Parameter	Type	Description
`request_id`	STRING	The request ID.
`code`	STRING	The status code.
`message`	STRING	Status details.
`data`	STRING	Base64-encoded output image.

Post categorization and labeling

This is a custom model. Contact your account manager and provide your AppId to enable it.

Classifies and labels multimodal posts (text + images, or text + video) and returns 32-dimensional embedding features. This service is asynchronous — submit the job, then poll for results using the returned job ID.

Step 1: Submit the job

from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi

ai_service_api = AiServiceVideoApi(client)
model_name = 'multi_modal_tag'
video_url = 'https://YOUR_VIDEO.mp4'   # Pass an empty string if no video
images = [
    'https://YOUR_IMAGE_0.jpg',
    'https://YOUR_IMAGE_1.jpg',
    'https://YOUR_IMAGE_2.jpg'
]   # Pass an empty array if no images
text = 'Text content'   # Post title and introduction
configure = {
    'output_classification': True,
    'classification_top_k': 1,
    'output_tag': True,
    'tag_top_k': 5,
    'output_embedding': True
}

response = ai_service_api.multimodal_classify(video_url, images, text, model_name, configure)
job_id = response.data['job_id']
print(response)

Request parameters

Parameter	Type	Required	Description
`video_url`	STRING	Yes	Video URL. Pass an empty string if no video.
`images`	List[STRING]	Yes	Image URLs. Pass an empty array if no images. Separate multiple URLs with commas.
`text`	STRING	Yes	Post content, consisting of the title and introduction.
`model_name`	STRING	Yes	Set to `multi_modal_tag`.
`configure`	Dict	Yes	Additional model configuration. See configure parameters below.

Configure parameters

Parameter	Type	Required	Description
`output_classification`	BOOL	No	Whether to return classification results. Default: `True`.
`classification_top_k`	INT	No	Number of top classification results. Valid values: 1–10. Default: 1.
`output_tag`	BOOL	No	Whether to return labeling results. Default: `True`.
`tag_top_k`	INT	No	Number of top labels. Valid values: 1–10. Default: 5.
`output_embedding`	BOOL	No	Whether to return 32-dimensional multimodal embeddings. Default: `True`.

Sample response (job submission)

{'code': 'OK',
 'data': {'job_id': 1134***},
 'message': 'success',
 'request_id': '49b0c9c2-06be-4f30-9b9f-f04baecf4577'}

Step 2: Poll for results

from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi

ai_service_job_api = AiServiceJobApi(client)
result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>)
print(result)

Replace <YOUR-JOB-ID> with the job_id value (INT type) from the submission response. Poll until state is 2 (completed) or 3 (failed).

Sample response — job running

{'code': 'OK',
 'data': {'job': {'Result': '',
                  'app_id': 'YOUR_APPID',
                  'create_time': '2023-10-18T14:32:42.619+08:00',
                  'id': 1134206,
                  'message': 'video downloading',
                  'state': 1}},
 'message': 'success',
 'request_id': '49b0c9c2-06be-4f30-9b9f-f04baecf4577'}

Sample response — job completed

{
'code': 'OK',
'data': {'job':
            {'Result':
                        '{"classification_result":[{"class":"Travel","score":0.67}],
                        "duration":15.16,
                        "embedding_result":"0.915,0.882,0.943,0.978,1.027,1.181,1.066,1.029,0.866,0.716,0.628,1.203,0.689,0.533,0.734,1.038,0.98,0.613,0.96,0.88,0.586,0.702,1.515,0.697,0.987,0.699,1.179,4.274,0.757,0.89,0.805,0.901",
                        "tag_result":[{"score":0.56,"tag":"Subtitles"},{"score":0.456,"tag":"Aerial shot"},{"score":0.453,"tag":"Tibet"},{"score":0.437,"tag":"Vehicle"},{"score":0.429,"tag":"Machinery"}],
                        "video_height":918,
                        "video_width":552}',
                        'app_id': '202211140827340****',
                        'create_time': '2023-10-18T14:32:42.619+08:00',
                        'id': 113****,
                        'message': 'success',
                        'model_id': 0,
                        'request_id': '',
                        'state': 2,
                        'type': 'Video'
            }
        },
'message': 'success',
'request_id': '139a40ad-fdf6-45de-b760-161c52ff3f3c'
}

Response parameters — data.job

Parameter	Type	Description
`id`	INT	Job ID.
`app_id`	STRING	AppId.
`state`	INT	Job status. Valid values: `0` = initializing, `1` = running, `2` = completed, `3` = failed.
`message`	STRING	Execution status. Valid values: `success`, `video downloading`.
`create_time`	STRING	Job creation time.
`Result`	STRING	Model output. See Result parameters below.

Result parameters

Parameter	Type	Description
`classification_result`	List	Classification results sorted by score. Length depends on `classification_top_k`. Each item is a DICT. Returned only when `output_classification` is `True`.
`tag_result`	List	Labels sorted by score. Length depends on `tag_top_k`. Each item is a DICT. Returned only when `output_tag` is `True`.
`score`	FLOAT	Probability, accurate to three decimal places.
`class`	STRING	Class name.
`tag`	STRING	Label name.
`embedding_result`	STRING	32-dimensional multimodal features, separated by commas.
`video_height`	INT	Video height in pixels. Empty string if `video_url` is empty.
`video_width`	INT	Video width in pixels. Empty string if `video_url` is empty.
`duration`	FLOAT	Video duration in seconds. Empty string if `video_url` is empty.

Video quality evaluation

Scores the quality of a video on a scale from 0 to 100. This service is asynchronous.

Evaluate a single video

Step 1: Submit the job

from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi

video_url = 'https://your_video_url.mp4'
ai_service_api = AiServiceVideoApi(client)
model_name = ''   # Leave blank to use the general-purpose model
video_title = ''  # Leave blank; not required by this service
configure = {}

response = ai_service_api.video_qa(video_url, model_name, video_title, configure)
job_id = response.data['job_id']
print(response)

Request parameters

Parameter	Type	Required	Description
`video_url`	STRING	Yes	URL of the video. Keep the file size under 100 MB and the resolution at 320×240 or higher.
`video_title`	STRING	Yes	Leave blank; this service does not require a title.
`model_name`	STRING	Yes	Leave blank to use the general-purpose model. To use a custom model, pass its name.
`configure`	Dict	Yes	Response settings. Pass an empty dictionary for this service.

Sample response (job submission)

{'code': 'OK',
 'data': {'job_id': 10**},
 'message': 'success',
 'request_id': '228235a6-db4f-466a-b20b-a4453f81****'}

Step 2: Poll for results

from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi

ai_service_job_api = AiServiceJobApi(client)
result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>)
print(result)

Replace <YOUR-JOB-ID> with the job_id value (INT type). Poll until state is 2 (completed) or 3 (failed).

Sample response — job running

{'code': 'OK',
 'data': {'job': {'Result': '',
                  'app_id': '202209070402410****',
                  'create_time': '2022-09-07T14:03:33.8+08:00',
                  'id': 1012,
                  'message': 'video downloading',
                  'state': 1}},
 'message': 'success',
 'request_id': '5259ec57-e55c-4772-b01e-3deafd74****'}

Sample response — job completed

{'code': 'OK',
 'data': {'job': {'Result': '{"avg_fps":30,"duration":14,"height":1920,"single_img":0.74072265625,"video_score":20.57750291278764,"width":1080}',
                  'app_id': '202209290847310****',
                  'create_time': '2022-10-12T10:51:30.015+08:00',
                  'id': 1012,
                  'message': 'success',
                  'state': 2}},
 'message': 'success',
 'request_id': '0b4f658f-cd96-4606-8d81-d2fe0a6d****'}

Response parameters — data.job

Parameter	Type	Description
`id`	INT	Job ID.
`app_id`	STRING	AppId.
`state`	INT	Job status. Valid values: `0` = initializing, `1` = running, `2` = completed, `3` = failed.
`message`	STRING	Execution status. Valid values: `success`, `video downloading`.
`create_time`	STRING	Job creation time.
`Result`	STRING	Model output. See Result parameters below.

Result parameters

Parameter	Type	Description
`video_score`	FLOAT	Quality score (0–100).
`avg_fps`	INT	Average frame rate.
`duration`	INT	Video duration in seconds.
`single_img`	FLOAT	Frame-to-frame similarity (0–1). Lower values indicate the video consists mostly of static images.
`height`	INT	Video height in pixels.
`width`	INT	Video width in pixels.

Evaluate multiple videos in a batch

Step 1: Submit the batch job

from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi
from ai_service_python_sdk.client.models.batch_video_request_videos import BatchVideoRequestVideos

ai_service_api = AiServiceVideoApi(client)
videos = [
    BatchVideoRequestVideos('https://your_video_1_url.mp4', ''),
    BatchVideoRequestVideos('https://your_video_2_url.mp4', '')
]
model_name = ''
configure = {}

response = ai_service_api.batch_video_qa(videos, model_name, configure)
batch_id = response.data['batch_id']
print(response)

Sample response (batch submission)

{'code': 'OK',
 'data': {'batch_id': 14},
 'message': 'success',
 'request_id': '6d5ef1b5-c01f-4e63-9eec-150ddde9****'}

Step 2: Poll for results

from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi

ai_service_job_api = AiServiceJobApi(client)
result = ai_service_job_api.get_batch_job_with_id(<YOUR-BATCH-ID>)
print(result)

Replace <YOUR-BATCH-ID> with the batch_id value (INT type).

Sample response

{'code': 'OK',
 'data': {'jobs': [{'Result': '{"avg_fps":30,"duration":14.633333333333333,"height":1280,"single_img":0.486328125,"video_score":55.3773279862801,"width":720}',
                    'app_id': '202209290847310****',
                    'batch_id': 14,
                    'create_time': '2022-10-26T19:44:47.501+08:00',
                    'id': 44**,
                    'message': 'success',
                    'object': 'https://your_video_1_url.mp4',
                    'state': 2},
                   {'Result': '{"avg_fps":30,"duration":18.2,"height":568,"single_img":0.5791015625,"video_score":67.21071975649835,"width":320}',
                    'app_id': '202209290847310****',
                    'batch_id': 14,
                    'create_time': '2022-10-26T19:44:47.501+08:00',
                    'id': 4484,
                    'message': 'success',
                    'object': 'https://your_video_2_url.mp4',
                    'state': 2}]},
 'message': 'success',
 'request_id': 'e99697ff-7d42-4c0a-a0a0-c6a0c0ba****'}

The data['jobs'] field is an array. Each element has the following parameters:

Parameter	Type	Description
`id`	INT	Entry ID within the batch.
`batch_id`	INT	Batch job ID.
`app_id`	STRING	AppId.
`state`	INT	Job status. Valid values: `0` = initializing, `1` = running, `2` = completed, `3` = failed.
`message`	STRING	Execution status.
`create_time`	STRING	Job creation time.
`object`	STRING	Video URL.
`Result`	STRING	Model output. The Result parameters are identical to those of the single video evaluation.

Video classification by label

Classifies short videos and returns their class, top K labels with probabilities, and 64-dimensional video embeddings. This service is asynchronous.

Classify a single video

Step 1: Submit the job

from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi

video_url = 'https://your_video_url.mp4'
ai_service_api = AiServiceVideoApi(client)
model_name = ''
video_title = 'your_video_title'   # Pass an empty string if no title
configure = {
    'output_classification': True,
    'classification_top_k': 1,
    'output_tag': True,
    'tag_top_k': 5,
    'output_embedding': True
}

response = ai_service_api.classify_label_video(video_url, video_title, model_name, configure)
job_id = response.data['job_id']
print(response)

Request parameters

Parameter	Type	Required	Description
`video_url`	STRING	Yes	URL of the video. Keep the file size under 100 MB and the resolution at 320×240 or higher.
`video_title`	STRING	Yes	Title of the video. Pass an empty string if no title.
`model_name`	STRING	Yes	Leave blank to use the general-purpose model. Contact PAI technical support to get a custom model.
`configure`	Dict	Yes	Response settings. Default: `None`. If set to `None`, all configure parameters use their defaults.

Configure parameters

Parameter	Type	Required	Description
`output_classification`	BOOL	No	Whether to return classification results. Default: `True`.
`classification_top_k`	INT	No	Number of top classes. Valid values: 1–10. Default: 1.
`output_tag`	BOOL	No	Whether to return labels. Default: `True`.
`tag_top_k`	INT	No	Number of top labels. Valid values: 1–10. Default: 5.
`output_embedding`	BOOL	No	Whether to return 64-dimensional video embeddings. Default: `True`.

Sample response (job submission)

{'code': 'OK',
 'data': {'job_id': 5},
 'message': 'success',
 'request_id': '4f6caecb-e3d6-4997-89fe-26ac7536****'}

Step 2: Poll for results

from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi

ai_service_job_api = AiServiceJobApi(client)
result = ai_service_job_api.get_async_job(<YOUR-JOB-ID>)
print(result)

Replace <YOUR-JOB-ID> with the job_id value (INT type). Poll until state is 2 (completed) or 3 (failed).

Sample response — job running

{'code': 'OK',
 'data': {'job': {'Result': '',
                  'app_id': '202209070402410****',
                  'create_time': '2022-09-07T14:03:33.8+08:00',
                  'id': 5,
                  'message': 'video downloading',
                  'state': 1}},
 'message': 'success',
 'request_id': '5259ec57-e55c-4772-b01e-3deafd74****'}

Sample response — job completed

{'code': 'OK',
 'data': {'job': {'Result':
      '{"classification_result":[
          {"class":"Agriculture","score":0.997}
          ],
      "duration":14.633333333333333,
      "embedding_result":"-0.162,0.483,0.881,0.686,0.319,-1.569,0.266,0.227,-1.352,1.157,-0.784,-1.016,-0.987,-1.214,-0.493,-1.182,0.85,1.473,-1.27,-3.391,3.394,-0.471,-0.434,0.256,2.146,0.787,0.348,1.67,0.229,-0.878,0.299,0.392,-2.161,-0.756,-3.438,-0.158,0.713,-5.812,-1.248,0.253,-1.111,-0.165,2.644,-2.368,-1.636,-0.533,0.468,1.2,-1.431,-1.151,-1.276,-1.954,1.167,-0.938,-0.111,-2.636,-1.248,4.662,1.892,5.485,-1.137,1.589,-1.935,1.018",
      "tag_result":[
          {"score":0.728,"tag":"Potted plant"},
          {"score":0.576,"tag":"Plant"},
          {"score":0.539,"tag":"Planting technique"},
          {"score":0.466,"tag":"Agricultural technology"},
          {"score":0.458,"tag":"Planting"}
          ],
      "video_height":1280,
      "video_width":720}\n',
    'app_id': '202209070402410****',
    'create_time': '2022-09-07T14:03:33.8+08:00',
    'id': 5,
    'message': 'success',
    'state': 2}},
 'message': 'success',
 'request_id': '9283066a-f414-40fd-8fd3-0f77c1ef****'}

Response parameters — data.job

Parameter	Type	Description
`id`	INT	Job ID.
`app_id`	STRING	AppId.
`state`	INT	Job status. Valid values: `0` = initializing, `1` = running, `2` = completed, `3` = failed.
`message`	STRING	Execution status. Valid values: `success`, `video downloading`.
`create_time`	STRING	Job creation time.
`Result`	STRING	Model output. See Result parameters below.

Result parameters

Parameter	Type	Description
`classification_result`	LIST	Classes sorted by score. Length depends on `classification_top_k`. Returned only when `output_classification` is `True`.
`tag_result`	LIST	Labels sorted by score. Length depends on `tag_top_k`. Returned only when `output_tag` is `True`.
`score`	FLOAT	Probability, accurate to three decimal places.
`class`	STRING	Class name.
`tag`	STRING	Label name.
`embedding_result`	STRING	64-dimensional video features, separated by commas. Returned only when `output_embedding` is `True`.
`video_height`	INT	Video height in pixels.
`video_width`	INT	Video width in pixels.
`duration`	FLOAT	Video duration in seconds.

Classify multiple videos in a batch

Step 1: Submit the batch job

from ai_service_python_sdk.client.api.ai_service_video_api import AiServiceVideoApi
from ai_service_python_sdk.client.models.batch_video_request_videos import BatchVideoRequestVideos

ai_service_api = AiServiceVideoApi(client)
configure = {
    'output_classification': True,
    'classification_top_k': 1,
    'output_tag': True,
    'tag_top_k': 5,
    'output_embedding': True
}
videos = [
    BatchVideoRequestVideos('https://your_video_1_url.mp4', 'your_video_title_1'),
    BatchVideoRequestVideos('https://your_video_2_url.mp4', 'your_video_title_2')
]
model_name = ''

response = ai_service_api.batch_classify_label_video(videos, model_name, configure)
batch_id = response.data['batch_id']
print(response)

Sample response (batch submission)

{'code': 'OK',
 'data': {'batch_id': 1},
 'message': 'success',
 'request_id': 'c59eb912-9df4-4afe-8129-23f7b6dc****'}

Step 2: Poll for results

from ai_service_python_sdk.client.api.ai_service_job_api import AiServiceJobApi

ai_service_job_api = AiServiceJobApi(client)
result = ai_service_job_api.get_batch_job_with_id(<YOUR-BATCH-ID>)
print(result)

Replace <YOUR-BATCH-ID> with the batch_id value (INT type).

Sample response

{'code': 'OK',
 'data': {'jobs': [{'Result': '{"classification_result":[{"class":"Lifestyle","score":0.998}],"duration":14.633333333333333,"embedding_result":"0.998,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.002","tag_result":[{"score":0.917,"tag":"Potted plant"},{"score":0.777,"tag":"Plant"},{"score":0.693,"tag":"Gardening"},{"score":0.58,"tag":"Lucky tree"},{"score":0.509,"tag":"Subtitles"}],"video_height":1280,"video_width":720}',
                    'app_id': '202209290847310****',
                    'batch_id': 13,
                    'create_time': '2022-10-26T18:13:20.928+08:00',
                    'id': 4090,
                    'message': 'success',
                    'object': 'https://your_video_1_url.mp4',
                    'state': 2},
                   {'Result': '{"classification_result":[{"class":"Sports","score":0.998}],"duration":18.2,"embedding_result":"0.0,0.0,0.0,0.998,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.001,0.0,0.0,0.0,0.0,0.001,0.0,0.0,0.0,0.0","tag_result":[{"score":0.981,"tag":"National Games"},{"score":0.901,"tag":"Table tennis"},{"score":0.798,"tag":"Sports"},{"score":0.583,"tag":"Subtitles"},{"score":0.498,"tag":"Table tennis match"}],"video_height":568,"video_width":320}',
                    'app_id': '202209290847310****',
                    'batch_id': 13,
                    'create_time': '2022-10-26T18:13:20.928+08:00',
                    'id': 4091,
                    'message': 'success',
                    'object': 'https://your_video_2_url.mp4',
                    'state': 2}]},
 'message': 'success',
 'request_id': '73e4b48e-42b8-4347-9cb4-7a49c1f8****'}

The data['jobs'] field is an array. Each element has the following parameters:

Parameter	Type	Description
`id`	INT	Entry ID within the batch.
`batch_id`	INT	Batch job ID.
`app_id`	STRING	AppId.
`state`	INT	Job status. Valid values: `0` = initializing, `1` = running, `2` = completed, `3` = failed.
`message`	STRING	Execution status.
`create_time`	STRING	Job creation time.
`object`	STRING	Video URL.
`Result`	STRING	Model output. The Result parameters are identical to those of the single video classification.

Error codes

Image analysis services

Request errors

HTTP status code	code	message	Cause	Action
400	PARAMETER_ERROR	`image is empty`	No image was included in the request.	Set `image_url` to a valid image URL.
400	PARAMETER_ERROR	`note found appid`	The AppId is invalid.	Verify your AppId on the Multimedia Analysis page.
400	PARAMETER_ERROR	`Unmarshal type error: expected=string, got=number, field=image, offset=42, internal=json: cannot unmarshal number into Go struct field ImageRequest.image of type string`	The `image` parameter has an invalid data type.	Pass the image as a string (URL).
401	PARAMETER_ERROR	`sign error`	The Token is invalid.	Verify your Token on the Multimedia Analysis page.
404	PARAMETER_ERROR	`model not found`	The requested model service is not deployed.	Check that `model_name` is correct and that the model has been configured for your AppId.
422	SERVER_ERROR	(see returned message)	The model service encountered an error.	Check the `message` field for details.

Video analysis services

Request errors

HTTP status code	code	message	Cause	Action
400	PARAMETER_ERROR	`not found appid`	The AppId is invalid.	Verify your AppId on the Multimedia Analysis page.
401	PARAMETER_ERROR	`sign error`	The Token is invalid.	Verify your Token on the Multimedia Analysis page.
404	PARAMETER_ERROR	`model not found`	The requested model service is not deployed.	Check the `model_name` value and confirm the model is configured for your AppId.

Result query errors

state	message	Cause	Action
3	`failed to download video, err=fetch resource failed\tcode = 404`	The system could not download the video from the specified URL.	Verify that the URL is publicly accessible.
3	(see returned message)	The model service encountered an error.	Check the `message` field for details.