Highlight Extraction API Reference - ApsaraVideo VOD

This topic describes the data structures of the request parameters of SubmitHighlightExtractionJob and the response parameters of GetSmartHandleJob.

Important

When you call relevant operations, the region in the OSS URLs of media assets must be the same as the region in the OpenAPI endpoint.
Supported regions: China (Shanghai), China (Beijing), China (Hangzhou), China (Shenzhen), US (Silicon Valley), Singapore. The action recognition capability (corresponding to Strategy.EnableActionRecog and Strategy.CustomActions) is only available in the China (Shanghai) region.
Videos must contain either subtitles or human voices. Materials lacking both are unsupported.

Relevant APIs

To submit a highlight extraction job, see SubmitHighlightExtractionJob. The data structures of key parameters (InputConfig and OutputConfig) are described in the following sections.
To obtain details about a highlight extraction job, see GetSmartHandleJob. The result parameters (AiResult) are described in the following sections.

InputConfig

Configure InputConfig to specify video materials and highlight extraction strategy.

Parameter

Type

Description

Required

MediaArray

List<Media>

The video materials. You can specify a list of media IDs or OSS URLs. The total video duration can be up to two hours, and the maximum number of videos is 5.
For supported formats, see Video formats.

Yes

Strategy

JSON

The highlight extraction strategy.

Count: The number of highlight clips to extract from each material. Valid values: [1,10]. Default value: 5.
ClipDuration: Expected duration of each highlight clip in seconds. Valid values: [3,60]. Default value: 15. The actual duration of each highlight may vary slightly from this value.
EnableActionRecog: Specifies whether to enable action recognition. Default value: false.
CustomActions: Custom action tags, which are mapped according to the input tag names. For example: ["Fight","Cry"]. The maximum number of tags is 50, with each no longer than 5 characters.

Media

Parameter

Type

Description

Required

MediaId

String

The ID of the media asset.

You must specify either MediaId or MediaURL.

If both are specified, MediaId takes precedence.

MediaURL

String

The OSS URL to the media file.

Example

{
  "MediaArray": [
    {
      "MediaId": "1cb94770da*******75e6e6c5486302"
    }
  ],
  "Strategy": {
    "Count": 5,
    "ClipDuration": 15,
    "EnableActionRecog": true,
    "CustomActions":  ["Fight","Cry"]
  }
}

OutputConfig

Configure OutputConfig to specify output settings, such as the storage location and naming of output videos.

Parameter	Type	Description	Required	Example
NeedExport	Boolean	Specifies whether to directly export clips. Valid values: true: returns the extracted highlight clips. false: only the time ranges of highlight clips are returned. Default value: false.	No	false
OutputMediaTarget	String	Required only if `NeedExport` is set to `true`. oss-object: saves the outputs as objects in OSS buckets.	No	oss-object
Endpoint	String	The S3 protocol-compatible endpoint. Specify an OSS endpoint in which the region is the same as where the service is called.	No	https://oss-cn-shanghai.aliyuncs.com
Bucket	String	Required only if `NeedExport` is set to `true`. Specify your OSS bucket, which is S3 protocol-compatible.	No	your bucket
ObjectKey	String	Required only if `NeedExport` is set to `true`. The naming for the OSS objects. Supported placeholder: {index}: must be included in the object path.	No	dir/to/testOutput_{index}.mp4
ExportAsNewMedia	Boolean	Optional when `NeedExport` is set to `true`. Specifies whether to output as new media assets. Only supported when `OutputMediaTarget` is set to `oss-object`. Default value: false.	No	false
Width	Integer	Optional when `NeedExport` is set to `true`. The output video width in pixels. If not specified, it will be the same as the source video	No	1280
Height	Integer	Optional when `NeedExport` is set to `true`. The output video height in pixels. If not specified, it will be the same as the source video	No	720
Video	JSONObject	Optional when `NeedExport` is set to `true`. The stream configuration of the output video, such as CRF and codec.	No	{ "Bitrate": 3000 }

Example

 {
    "NeedExport": true,
    "OutputMediaTarget": "oss-object",
    "Endpoint": "https://oss-cn-shanghai.aliyuncs.com"
    "Bucket": "your-bucket",
    "ObjectKey": "dir/to/testOutput_{index}.mp4",
    "ExportAsNewMedia": false,
    "Width": 1280,
    "Height": 720,
    "Video": {
      "Bitrate": 3000
    }
  }

GetSmartHandleJob

This section shows the response parameters when you call GetSmartHandleJob to query the result of a highlight extraction job.

AiResult

{
  "HighlightResults": [
    {
      "Media": "MediaId1", //If URL was specified in InputConfig, then URL will be returned here.
      "TimeRanges": [
        {
          "In": 20,
          "Out": 30,
          "Tags": ["Fight","Cry"], // Detected action tags.
          "OutputURL": "http://your bucket.oss-cn-shanghai.aliyuncs.com/output_0.mp4", // Only returned when needExport is set to true.
          "MediaId": "MediaId11", // Only returned when ExportAsNewMedia is set to true.
        }
      ]
    },
    {
      "Media": "MediaId2", //If URL was specified in InputConfig, then URL will be returned here.
      "TimeRanges": [
        {
          "In": 2,
          "Out": 10,
          "Tags": ["Run","Shout"],
          "OutputURL": "http://your bucket.oss-cn-******.aliyuncs.com/output_1.mp4" // Only returned when needExport is set to true.
        },
        {
          "In": 40,
          "Out": 50,
          "OutputURL": "http://your bucket.oss-cn-******.aliyuncs.com/output_2.mp4" // Only returned when needExport is set to true.
        }
      ]
    }
  ]
}