Script-to-Video FAQ

• Long input videos are split into shorter segments first. During composition, the system follows the group order, randomly picks one segment from each group, and concatenates them into the final video. The duration of each segment after splitting is controlled by the SingleShotDuration.

• If global voice-over scripts are provided, the system distributes them without repeating when there are enough scripts. For example, if you provide 3 scripts to generate 5 videos, all 3 scripts are used at least once, and some are reused to fill the remaining videos.

• If a global voice-over is provided and the assembled video is shorter than the voice-over duration, the system randomly selects additional segments from non-start/end groups until the video duration matches the voice-over.

• The final video duration follows this priority order:

  • If a global voice-over script is provided, the final duration equals the voice-over duration.

  • If no global voice-over script is provided:

    • If FixedDuration is set, the final duration equals the value of FixedDuration.

    • If MaxDuration is set, the final duration equals SingleShotDuration multiplied by the number of groups (the length of SpeechTextArray), capped at the value of MaxDuration.

• In this mode, voice-over scripts are configured within each MediaGroup using the SpeechTextArray field. Long input videos are split into segments first, as in Global Scripts mode. Do not set InputConfig.SpeechText or a global SpeechTextArray, as doing so causes an error.

• If MediaGroup.SpeechTextArray is empty for a group, that group has no voice-over. Its duration defaults to 5 seconds and is controlled by MediaGroup.Duration.

• To control the audio volume for assets within a specific MediaGroup, set MediaGroup.Volume. This parameter takes priority over EditingConfig.MediaConfig.Volume.

• Each group can have multiple voice-over scripts. If all groups with voice-overs have the same number of scripts, they are combined in order — all groups use their Nth script together. If the number of scripts differs across groups, the system randomly selects a segment from each group to form the complete voice-over.

Use the shot analysis feature of Smart Tagging to automatically split your media into meaningful shots. This prevents the system from cutting across natural scene boundaries and produces smoother transitions. The following steps apply to a single video asset; repeat them for each additional asset.

1. For the video asset, call SubmitSmarttagJob. Set TemplateId to the fixed value "S00000103-000003".

2. Retrieve the shot analysis results by calling QuerySmarttagJobI. In the response, find the data entry where type=ClipSplit. The value is a JSON array where StartTime and EndTime mark the boundaries of each shot in seconds:

   [
     {
       "EndTime": 5.4,
       "ClipType": "opening",
       "StartTime": 0.0
     },
     {
       "EndTime": 9.16,
       "ClipType": "opening",
       "StartTime": 5.4
     },
     {
       "EndTime": 12.88,
       "ClipType": "opening",
       "StartTime": 9.16
     },
     {
       "EndTime": 16.0,
       "ClipType": "opening",
       "StartTime": 12.88
     }
   ]
   

3. Pass the shot boundaries from step 2 to the EditingConfig.MediaConfig.MediaMetaDataArray.TimeRangeList parameter in your Script-to-Video job. Each In/Out pair corresponds to one shot from the ClipSplit result:

   {
     "MediaConfig": {
       "MediaMetaDataArray": [
         {
           "Media": "https://******.****.****/public-template/video/movie_apsara_4.mp4",
           "GroupName": "opening",
           "TimeRangeList": [
             {
               "In": 0,
               "Out": 5.4
             },
             {
               "In": 5.4,
               "Out": 9.16
             },
             {
               "In": 9.16,
               "Out": 12.88
             },
             {
               "In": 12.88,
               "Out": 16.0
             }
           ]
         }
       ]
     }
   }

4. Set MediaGroup.SplitMode to NoSplit for the corresponding MediaGroup. This prevents the system from re-splitting the pre-segmented clips, preserving shot integrity and producing smoother transitions.

Use these two parameters depending on the asset type:

• For images: use ImageDuration to set how long each image appears in the output video.

• For videos: use the SingleShotDuration parameter to control the duration of segments after a video asset is split.

When a media group contains both image and video assets and an image is selected, its display duration is determined by the following priority order:

• If ImageDuration is set, that value is used.

• If a voice-over is provided, the image duration equals the total voice-over duration divided by the number of items in the MediaGroupArray.

• If FixedDuration is set but no voice-over is provided, the image duration equals FixedDuration divided by the number of items in the MediaGroupArray.

• Set the group's MediaGroup.SplitMode to NoSplit and set MediaGroup.Duration to the exact length of the source clip. For example, to play a 20-second clip in full, set MediaGroup.SplitMode to NoSplit and MediaGroup.Duration to 20.

• If the selected asset is shorter than the specified MediaGroup.Duration, the system adjusts playback speed to fit. For example, if MediaGroup.Duration is 10 seconds but a 20-second clip is selected, the clip plays at 2x speed.

• Use MediaGroup.Volume to control the volume of assets within each media group. Set the value to 1 to play a clip with its original audio, or to 0 to mute it and let the voice-over take over.

• For example, with three groups — MediaGroup1, MediaGroup2, and MediaGroup3 — where MediaGroup1 and MediaGroup3 have voice-overs and MediaGroup2 plays with its original audio, configure MediaGroupArray as follows:

[{
"GroupName": "MediaGroup1",
"MediaArray": ["https://ice-*****-test.oss-cn-*****.aliyuncs.com/0-test-batch-editing-materials/160134%2B9859695-2032aa5c-2803-47cd-bf65-8a40d66598db.png", "https://ice-auto-test.oss-cn-shanghai.aliyuncs.com/0-test-batch-editing-materials/cloud.mp4"],
"SpeechTextArray": ["Grand opening! A Freshippo store opens today at the nearby mall.", "A Freshippo store opens today at the nearby mall", "<speak>The battle is <phoneme alphabet=\"py\" ph=\"zheng4 hao3\">fierce</phoneme>. The table tennis legend Ma Long, is charging towards the pinnacle of glory. In the quarter-finals against Togami Shunsuke, Ma Long shows no fear, giving his all in every rally. His precise ball placement and calm judgment give him the upper hand in this match. In the end, Ma Long defeats his opponent and advances to the semi-finals.</speak>"]
},
{
"GroupName": "MediaGroup2",
"MediaArray": ["https://ice-*****-test.oss-cn-*****.aliyuncs.com/0-test-batch-editing-materials/normal%20video.mp4", "https://ice-auto-test.oss-cn-shanghai.aliyuncs.com/0-test-batch-editing-materials/3.jpeg"],
"Duration": 5,
"SplitMode": "NoSplit",
"Volume": 1
},
{
"GroupName": "MediaGroup3",
"MediaArray": ["https://ice-*****-test.oss-cn-*****.aliyuncs.com/0-test-batch-editing-materials/young_sunset_walk.mp4"],
"SpeechTextArray": ["Come and take a look", "Come and take a look now"]
}]