Video label detection

Use cases

Video classification: Categorize videos into topics such as news, entertainment, gaming, technology, food, sports, travel, animation, dance, music, film and television, and automobile.
Video retrieval: Index labels to build a searchable video library. For example, find all videos containing outdoor scenes or specific objects.
Personalized recommendation: Match video content labels with user preference labels to deliver targeted recommendations.

How it works

Upload a video to an OSS bucket.
Call CreateVideoLabelClassificationTask to create an asynchronous label detection task.
After the task completes, call GetVideoLabelClassificationResult to retrieve the detected labels.

Important

Task results are retained for seven days after the task starts. After this period, results are no longer available through the API.

Prerequisites

Make sure that you have:

An AccessKey pair. Create an AccessKey pair
An OSS bucket with the video uploaded. Upload objects
IMM activated. Activate IMM
An IMM project created in the target region. Create a project

Note

You can also manage projects programmatically. Call CreateProject to create a project, or ListProjects to query existing projects in a region.

Track task status

Besides calling GetVideoLabelClassificationResult, you can track task progress with these methods:

Method	Description
API polling	Call GetTask or ListTasks to query task details.
Simple Message Queue (SMQ)	Subscribe to task notifications in the same region as your IMM project. Asynchronous message examples and Receive and delete the message.
ApsaraMQ for RocketMQ 4.0	Create a RocketMQ instance, topic, and group in the same region to receive task notifications. Asynchronous message examples and Send and subscribe to normal messages.
EventBridge	Receive task completion events through EventBridge. IMM events.

Response structure

A successful task returns a Labels array with up to three hierarchical levels. Each label includes a confidence score and a centricity score.

Label fields

Field	Type	Description
`LabelName`	String	The detected label name, such as "Natural landscape" or "Car".
`LabelConfidence`	Float	Probability that the label is correct. Range: 0 to 1. Higher values indicate greater confidence.
`CentricScore`	Float	How prominent the labeled content is in the video. Range: 0 to 1. Higher values mean greater prominence.
`LabelLevel`	Integer	The position in the label hierarchy. `1` = top-level category, `2` = mid-level subcategory, `3` = leaf-level detail.
`ParentLabelName`	String	The name of the parent label. Empty for top-level labels (level 1).
`Language`	String	The language of the label name, such as `zh-Hans`.

Label hierarchy

Labels follow a three-level hierarchy. Each child label references its parent through ParentLabelName:

Level 1 (category)      Level 2 (subcategory)     Level 3 (detail)
---------------------   ----------------------    -----------------
Tourism & geography  ->  Natural landscape     ->  Moon, Sky
Others               ->  Color                 ->  Blue, Green, Black, White
                     ->  Astronomical object
Daily necessities    ->  Text
                     ->  Letter
Virtual scene        ->  Web page
                     ->  Website
Artwork              ->  Illustration
Other scenes         ->  Mobile phone screenshot

Sample response

{
    "ProjectName": "test-project",
    "RequestId": "D65E8038-C584-0809-9BF0-****",
    "StartTime": "2022-08-22T05:01:17.572Z",
    "EndTime": "2022-08-22T05:01:20.49Z",
    "TaskType": "VideoLabelClassification",
    "TaskId": "VideoLabelClassification-1b77de73-ff9f-4c39-b254-****",
    "Status": "Succeeded",
    "Labels": [
        {
            "Language": "zh-Hans",
            "LabelName": "Color",
            "LabelConfidence": 0.999,
            "CentricScore": 0.77,
            "LabelLevel": 2,
            "ParentLabelName": "Others"
        },
        {
            "Language": "zh-Hans",
            "LabelName": "Others",
            "LabelConfidence": 0.999,
            "CentricScore": 0.77,
            "LabelLevel": 1,
            "ParentLabelName": ""
        },
        {
            "Language": "zh-Hans",
            "LabelName": "Blue",
            "LabelConfidence": 1,
            "CentricScore": 0.716,
            "LabelLevel": 3,
            "ParentLabelName": "Color"
        },
        {
            "Language": "zh-Hans",
            "LabelName": "Natural landscape",
            "LabelConfidence": 0.897,
            "CentricScore": 0.801,
            "LabelLevel": 2,
            "ParentLabelName": "Tourism & geography"
        },
        {
            "Language": "zh-Hans",
            "LabelName": "Moon",
            "LabelConfidence": 0.859,
            "CentricScore": 0.756,
            "LabelLevel": 3,
            "ParentLabelName": "Natural landscape"
        }
    ]
}

Note

This sample is abbreviated. A full response typically contains more labels and may include UserData if set in the request, or Code and Message fields on errors.

Supported video formats

Supported formats (22 total): AVI, MPEG, MPG, DAT, DIVX, XVID, RM, RMVB, MOV, QT, ASF, WMV, VOB, 3GP, MP4, FLV, AVS, MKV, TS, OGM, NSV, and SWF.

FAQ

Can I specify which labels to detect?

No. The feature uses a predefined taxonomy. You cannot specify, include, or exclude individual labels.

What categories do labels fall into?

Labels fall into three broad categories:

Scenes: Natural landscapes (forests, beaches, snow-capped mountains), living spaces (homes, restaurants), and disaster scenes.
Events: Talent shows, office activities, performances, and production processes.
Objects: Tableware, electronics (phones, computers), furniture, and vehicles.