Intelligent video processing models - AI Computing Asset Management| Alibaba Cloud Documentation Center

Background information

The following table describes the intelligent video processing models that are provided by Model Hub of PAI.


Model	Description
General video classification model	Uses the UCF101 dataset for model training based on the ResNet3D framework.
Video highlight generation model	Generates a five-second clip of video highlights from a video.
Super-resolution model	Uses super-resolution technologies to improve video resolution.

To go to Model Hub, perform the following steps:

Log on to the PAI console.
In the left-side navigation pane, choose AI Computing Asset Management > Model Management.
On the Model Management page, click the Model Hub tab.

Overview
The general video classification model uses the UCF101 dataset for model training based on the ResNet3D framework. For more information, see Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
Input format
The input data must be in the JSON format. It contains the url and play_duration fields. The value of the url field is the URL of the input video. The value of the play_duration field is the first several microseconds of the input video to be processed for classification. If the play_duration field is not specified, the total length of the input video is processed. The following code provides an example of the input data:
```
{
  "input": {
    "url": "URL of the input video",
    "play_duration": "Length of the input video clip to be processed"
  }
}
```

Output format

The output data is key-value pairs (KVPs) in the JSON format. The following table describes the fields in the output data.


Field	Description	Dimension	Type
class	The name of the category.	[]	STRING

Overview
This model generates a five-second clip of video highlights from a video.
Input format
The input data must be in the JSON format. It contains the url and play_duration fields. The value of the url field is the URL of the input video. The value of the play_duration field is the first several microseconds of the input video to be processed to generate the clip. If the play_duration field is not specified, the total length of the input video is processed. The following code provides an example of the input data:
```
{
  "input": {
    "url": "URL of the input video",
    "play_duration": "Length of the input video clip to be processed"
  }
}
```

Output format

The output data is KVPs in the JSON format. The following table describes the fields in the output data.


Field	Description	Dimension	Type
oss_path	The Object Storage Service (OSS) path of the generated clip.	[]	STRING
result	The result field.	[]	STRING
success	Indicates whether the clip is generated. Valid values: true: The clip is generated. false: The clip failed to be generated.	[]	BOOL

Example

The following code provides an example of the input data of the model:

{"input" : {"url": "http://pai-vision-data-sh.oss-cn-shanghai-internal.aliyuncs.com/tmp/50005632219.mp4"}}

PAI displays information that is similar to the following output:

{
   "oss_path": "oss://experience-ai/video_5s/50005632219.mp4",
   "result": "",
   "success": true
}

Overview
The super-resolution model uses super-resolution technologies to improve video resolution.

Input format

The input data is must be in the JSON format. The following table describes the fields in the input data.


Field	Required	Description	Type
url	Yes	The URL of the input video.	STRING
play_duration	No	The first several microseconds of the input video from which a clip with higher resolution to be generated. If the play_duration field is not specified, the entire input video is processed.	STRING

The following code provides an example of the input data:

{
  "input": {
    "url": "URL of the input video",
    "play_duration": "Length of the input video clip to be processed"
  }
}

Output format


Field	Description	Dimension	Type
oss_path	The OSS path of the generated clip.	[]	STRING
success	Indicates whether a clip with higher resolution is obtained after super resolution. Valid values: true: A clip with higher resolution is obtained after super resolution. false: A clip with higher resolution failed to be obtained after super resolution.	[]	BOOL
result	The result field.	[]	STRING

Example

The following code provides an example of the input data of the model:

{
    "input" : {
        "url": "http://pai-vision-data-sh.oss-cn-shanghai-internal.aliyuncs.com/tmp/5000563****.mp4"
    }
}

PAI displays information that is similar to the following output:

{
    "oss_path": "oss://experience-ai/video_sr/5000563****.mp4", 
    "result": "", 
    "success": true
}