The intelligent production feature is implemented based on Alibaba Cloud media AI technologies. The feature allows you to generate and process media content in various formats. It provides capabilities such as intelligent thumbnail, intelligent landscape-to-portrait, image matting, figure cutout, intelligent logo blurring, intelligent caption removal, caption extraction, chorus identification, and music rhythm identification. These capabilities improve the efficiency and quality of media content generation and processing.
Overview
Scenarios
Intelligent production of media and entertainment videos: The intelligent production feature allows you to efficiently generate and process video content such as logos, captions, and objects.
Video delivery to multiple terminals and platforms: The intelligent production feature allows you to convert the orientation of a video from landscape to portrait or vice versa. This allows videos to be played on different terminals and platforms and improves content distribution efficiency.
Capabilities
For more information about the supported capabilities, see the Intelligent production section of the "Functions and features" topic.
To try the feature online, visit Intelligent Production Experience Center.
Limits
For more information about the supported regions, see Regions and endpoints.
For more information about the maximum number of jobs that an ApsaraVideo Media Processing (MPS) queue can concurrently process, see the Limits section of the "Overview" topic.
Prerequisites
The following operations are complete before you use the intelligent production feature:
Create an MPS queue. For more information, see Create an MPS queue. If no MPS queue for intelligent thumbnail creation exists in the MPS queue list, create an MPS queue for intelligent thumbnail creation first.
Upload the video to be processed to OSS.
If you want to create a job, directly upload the video to OSS. For more information, see Upload a video.
If you want to create a workflow and automatically trigger the workflow, you must add media buckets and create a workflow before you upload the video. After you specify the input and output media buckets and upload a media file to the input media bucket, MPS receives a message about the upload and triggers the associated workflow. After the workflow is complete, the processed media file is uploaded to the output media bucket. For more information, see Add media buckets.
Optional. Enable the notification feature. If you want MPS to send notifications on jobs and workflows to MNS topics or queues, configure an MNS queue or topic as required. For more information, see Enable the notification feature for MPS jobs or workflows.
Create an intelligent production job in the MPS console
You can create only intelligent thumbnail jobs or workflows in the MPS console. You can create other intelligent production jobs by calling API operations.
Create an intelligent thumbnail job
Log on to the MPS console.
In the top navigation bar, select a region from the drop-down list.
In the left-side navigation pane, choose Tasks > Job Management.
On the Tasks page, click the Intelligent Thumbnail tab. On the Intelligent Thumbnail tab, click Create Task.
Configure the parameters that are described in the following table.
Parameter
Description
Input File URL
Click Select next to the Input File URL field. In the Select File dialog box, select a bucket from the Bucket drop-down list. Then, folders in the bucket are automatically displayed in the Path section. Select a file from a folder as the input file.
Intelligent Thumbnail Pipeline
Select an MPS queue for intelligent thumbnail creation from the drop-down list.
NoteIf no MPS queue is displayed in the Intelligent Thumbnail Pipeline drop-down list, create an MPS queue for intelligent thumbnail creation. For more information, see Create an MPS queue.
Output Path
Click Select next to the Output Path field. In the Select Output Path dialog box, select a bucket from the Bucket drop-down list. Then, folders in the bucket are automatically displayed in the Path section. Select a folder as the output path.
NoteIf the output path is the same as the input path and the output file name is different from the input file name, the input file is not overwritten by the output file. If the input file has the same name as the output file, you can configure OSS to prevent the input file from being overwritten. For more information, see Prevent objects from being overwritten by objects that have the same name. In this case, if the transcoded output file shares the same name as the input file, the output file fails to be written to the OSS bucket. We recommend that you do not specify a name that is the same as the name of the input file for the output file.
Click Submit. The intelligent thumbnail job is created.
After the intelligent thumbnail job is created, you can view the state and file path of the job on the Intelligent Thumbnail tab. You can also click the name of the job to view the job details, such as the input file and job configurations.
Create an intelligent thumbnail workflow
Log on to the MPS console.
In the top navigation bar, select a region from the drop-down list.
In the left-side navigation pane, choose .
On the Workflow Orchestration page, click Create Workflow.
Configure the parameters that are described in the following table.
Parameter
Description
Workflow Name
Enter a name for the workflow.
Configuration Plan
Select Custom from the drop-down list.
Configure the Input node.
Click the Pen icon on the right side of the Input node. The Input dialog box appears.
In the Input dialog box, configure the parameters that are described in the following table.
Parameter
Description
Input Bucket
You do not need to configure this parameter. The name of the input bucket automatically appears after you configure the Input Path parameter.
Input Path
Click Select next to the Input Path field. In the Select Input Path dialog box, select a bucket from the Bucket drop-down list. Then, folders in the bucket are automatically displayed in the Path section. Select a folder as the input path.
Encoding Pipeline
Select an MPS queue for transcoding from the drop-down list.
NoteThis MPS queue is used for transcoding and snapshot capture. An MPS queue for intelligent thumbnail creation is required when you configure the Intelligent Thumbnail node.
If no MPS queue is displayed in the drop-down list, create an MPS queue for transcoding. For more information, see Create an MPS queue.
Notifications
Optional. If you turn on Notifications, configure the Message Type parameter and select a Message Service (MNS) queue or topic from the Queue Name or Topic Name drop-down list.
NoteIf no MNS queue or topic is displayed in the drop-down list, enable the notification feature based on an MNS queue or topic. For more information, see Enable the notification feature for a media workflow.
Click OK. The Input node of the workflow is configured.
Add an Intelligent Thumbnail node.
Click the Plus icon on the right side of the Input node and select Intelligent Thumbnail.
Click the Pen icon on the right side of the Intelligent Thumbnail node.
In the Intelligent Thumbnail dialog box, configure the parameters that are described in the following table.
Parameter
Description
Name
Enter a name for the Intelligent Thumbnail node.
Intelligent Thumbnail Pipeline
Select an MPS queue for intelligent thumbnail creation from the drop-down list.
NoteIf no MPS queue name is displayed in the Intelligent Thumbnail Pipeline drop-down list, create an MPS queue for intelligent thumbnail creation. For more information, see Create an MPS queue.
Output Bucket
You do not need to configure this parameter. The name of the output bucket automatically appears after you configure the Output Path parameter.
Output Path
Click Select next to the Output Path field. In the Select Output Path dialog box, select a folder as the output path.
Set as Thumbnail
Optional. Specify whether to use the captured image as a thumbnail.
Click OK. The Intelligent Thumbnail node is configured.
Configure the Publish node.
Click the Pen icon on the right side of the Publish node.
In the Publish dialog box, configure the parameter that is described in the following table.
Parameter
Description
Publish Type
In this example, the Publish Type parameter is set to Auto. Valid values:
Manual: The output video is not published after the workflow is run. You must manually publish the output video. For more information, see Manage media files.
Auto: The output video is automatically published after the workflow is run.
NoteIf this parameter is set to Auto, the access permissions on all resources related to the output video, including playback files and captured video images, are the same as those on the OSS bucket in which the output video resides.
If this parameter is set to Manual, all these resources can be accessed only by the current user.
Click OK. The Publish node is configured.
Click Save.
If a file is uploaded to the path that is specified by the Input Path parameter after the workflow is created, the workflow is triggered.
After the workflow is run, you can query the details of the Intelligent Thumbnail node. To do so, perform the following operations: Click Details in the Actions column of the workflow to go to the details page of the workflow. On the details page of the workflow, find the Intelligent Thumbnail node and copy the task name. In the left-side navigation pane of the MPS console, choose Tasks > Job Management. On the Tasks page, click the Intelligent Thumbnail tab. On the Intelligent Thumbnail tab, select Search by ID.
Create an intelligent production job by calling an API operation
Create an intelligent production job.
For media files such as videos, images, and audio files that have been uploaded to OSS, you can call the SubmitIProductionJob operation to create an intelligent production job. You can specify the intelligent production capability that you want to use by configuring the FunctionName parameter.
Receive a callback notification.
If you have activated MNS and configured a queue or topic to receive notifications, MPS sends a notification to the queue or topic after the intelligent production job is complete. For more information, see Receive notifications.
Query the results of the intelligent production job.
You can call the QueryIProductionJob operation to query the state and results of the intelligent video production job by job ID.