How to complete image/text-to-video - Intelligent Media Services

This topic describes how to configure Timeline parameters for the SubmitMediaProducingJob API operation to produce videos based on images and texts. For more information, see Timeline configurations.

Usage notes

Intelligent production supports editing and compositing, effect rendering, and templates for live streams, VOD files, and material files from Object Storage Service (OSS). For more information, see Intelligent production overview.
You can produce a video from one or more videos, audio files, images, and subtitle materials by configuring Timeline parameters and calling the SubmitMediaProducingJob operation.
A timeline is created when you add materials and configure effects to create a video. A timeline consists of tracks, materials, and effects. For more information, see Timeline configurations.
For more information about how to use the IMS SDK to edit audio and video files, see Preparations.

Automatic alignment between images and voice-over scripts in webpage-to-video and document-to-video scenarios

Output video

Timeline example

{
  "VideoTracks": [{
    "VideoTrackClips": [{
      "ReferenceClipId": "speech1",
      "MediaURL": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/a1.png",
      "Type": "Image",
      "Effects": [{
        "Type": "Text",
        "Content": "Confucius said: "Is it not a pleasure to learn something and constantly put it into practice? Is it not a delight to have friends from afar? Is it not a gentleman to remain unsoured even though one's merits are unrecognized by others?"",
        "Alignment": "CenterCenter",
        "FontSize": 55,
        "AdaptMode": "AutoWrap",
        "TextWidth": 0.7,
        "Font": "FZKai-Z03S",
        "FontColor": "#ffffe0",
        "Outline": 2,
        "OutlineColour": "#000000",
        "FontFace": {
          "Bold": true,
          "Italic": false,
          "Underline": false
        }
      }]
    },{
      "ReferenceClipId": "speech2",
      "MediaURL": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/a2.png",
      "Type": "Image",
      "Effects": [{
        "Type": "Text",
        "Content": "Confucius said: "Isn't it a pleasure to review and practice what you have learned on time? Isn't it happy to have friends from far away? Isn't it a cultured gentleman's merit to remain unsoured even though one is unrecognized by others?"",
        "Alignment": "CenterCenter",
        "FontSize": 55,
        "AdaptMode": "AutoWrap",
        "TextWidth": 0.7,
        "Font": "FZKai-Z03S",
        "FontColor": "#ffffe0",
        "Outline": 2,
        "OutlineColour": "#000000",
        "FontFace": {
          "Bold": true,
          "Italic": false,
          "Underline": false
        }
      }]
    },{
      "ReferenceClipId": "speech3",
      "MediaURL": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/a3.png",
      "Type": "Image",
      "Effects": [{
        "Type": "Text",
        "Content": "Confucius said: "If a man keeps cherishing his old knowledge, so as continually to be acquiring new, he may be a teacher of others."",
        "Alignment": "CenterCenter",
        "FontSize": 55,
        "AdaptMode": "AutoWrap",
        "TextWidth": 0.7,
        "Font": "FZKai-Z03S",
        "FontColor": "#ffffe0",
        "Outline": 2,
        "OutlineColour": "#000000",
        "FontFace": {
          "Bold": true,
          "Italic": false,
          "Underline": false
        }
      }]
    },{
      "ReferenceClipId": "speech4",
      "MediaURL": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/a4.png",
      "Type": "Image",
      "Effects": [{
        "Type": "Text",
        "Content": "Confucius said: "If you gain something new when you review your old knowledge, you can become a teacher."",
        "Alignment": "CenterCenter",
        "FontSize": 55,
        "AdaptMode": "AutoWrap",
        "TextWidth": 0.7,
        "Font": "FZKai-Z03S",
        "FontColor": "#ffffe0",
        "Outline": 2,
        "OutlineColour": "#000000",
        "FontFace": {
          "Bold": true,
          "Italic": false,
          "Underline": false
        }
      }]
    }]
  }],
  "AudioTracks": [{
    "MainTrack": true,
    "AudioTrackClips": [{
      "ClipId": "speech1",
      "Type": "AI_TTS",
      "Voice": "zhichu",
      "Content": "The Analects, Chapter 12. Confucius said: "Is it not a pleasure to learn something and constantly put it into practice? Is it not a delight to have friends from afar? Is it not a gentleman to remain unsoured even though one's merits are unrecognized by others? "",
      "SpeechRate": -200
    },{
      "ClipId": "speech2",
      "Type": "AI_TTS",
      "Voice": "zhichu",
      "Content": "Translation. Confucius said: "Isn't it a pleasure to review and practice what you have learned on time? Isn't it happy to have friends from far away? Isn't it a cultured gentleman's merit to remain unsoured even though one is unrecognized by others?"",
      "SpeechRate": -200
    },{
      "ClipId": "speech3",
      "Type": "AI_TTS",
      "Voice": "zhichu",
      "Content": "Confucius said: "If a man keeps cherishing his old knowledge, so as continually to be acquiring new, he may be a teacher of others."",
      "SpeechRate": -200
    },{
      "ClipId": "speech4",
      "Type": "AI_TTS",
      "Voice": "zhichu",
      "Content": "Translation. Confucius said: "If you gain something new when you review your old knowledge, you can become a teacher."",
      "SpeechRate": -200
    }]
  },{
    "AudioTrackClips": [{
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
      "LoopMode": true
    }]
  }],
  "SubtitleTracks":[
    {
      "SubtitleTrackClips": [
        {
          "Type": "Text",
          "Y": 150,
          "Content": "The Analects, Chapter 12",
          "Font": "HappyZcool-2016",
          "Alignment": "TopCenter",
          "EffectColorStyle": "CS0004-000005",
          "AdaptMode": "AutoWrap",
          "FontSize": 70,
          "TextWidth": 900,
          "FontFace": {
            "Bold": false,
            "Italic": false,
            "Underline": false
          }
        }
      ]
    }
  ]
}

Images and audio content for producing music albums

Output video

Timeline example

{
  "VideoTracks": [{
    "VideoTrackClips": [{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/01.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/02.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/03.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/04.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/05.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/06.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    },{
      "Type": "Image",
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/image/07.jpg",
      "Duration": 3,
      "Effects": [{
        "Type": "Background",
        "SubType": "Blur",
        "Radius": 0.1
      }, {
        "Type": "Transition",
        "SubType": "random",
        "Duration": 0.5
      }]
    }]
  }],
  "AudioTracks": [{
    "AudioTrackClips": [{
      "MediaUrl": "http://ice-document-materials.oss-cn-shanghai.aliyuncs.com/test_media/music/m1.wav",
      "LoopMode": true
    }]
  }]
}

Intelligent Media Services:Image/text-to-video

Usage notes

Automatic alignment between images and voice-over scripts in webpage-to-video and document-to-video scenarios

Images and audio content for producing music albums

References