Timeline Data Structures in the Video Editing SDK Explained - Intelligent Media Services

This topic describes whether timeline data is supported in the video editing SDK for web and the preview component. This can help you understand how to configure different types of video tracks and clips to edit various materials in combination, including videos, audio, images, subtitles, and special effects.

Backend support for timeline data

For more information, see the "VideoTrack" section of the Timeline configurations topic.

Frontend support for timeline data

This section describes the frontend support for timeline data. The following content is applicable to the basic video editing SDK for web and the preview component whose versions are later than V5.2.2.

Track

Parameter	Type	Supported	Description
VideoTracks	VideoTrack[]	Yes	The video tracks. The video tracks are overlaid based on the sequence of MediaId array elements. For example, the track that corresponds to the first MediaId array element is placed below the track that corresponds to the second MediaId array element.
AudioTracks	AudioTrack[]	Yes	The audio tracks.
ImageTracks (The image track capability is merged into the video track capability and no longer iterated or maintained.)	ImageTrack[]	Yes	The image tracks. Image tracks must be placed above video tracks. If an image used as a material is mixed with a video in image and video production scenarios, use video tracks. The image tracks are overlaid based on the sequence of MediaId array elements. For example, the track that corresponds to the first MediaId array element is placed below the track that corresponds to the second MediaId array element.
SubtitleTracks	SubtitleTrack[]	No (We recommend that you use Type="Subtitle" in VideoTracks.)	The subtitle tracks.
EffectTracks	EffectTrack[]	No (We recommend that you use Type="Effects" in VideoTracks.)	The effect tracks.

VideoTrackClip

Parameter	Type	Supported	Required	Description
MediaId	String	Yes	No	The ID of the Intelligent Media Services (IMS) or ApsaraVideo VOD media asset to which the material in the video track corresponds. Note You must specify the MediaId or MediaURL parameter.
MediaURL	String	Yes (This parameter is equal to mediaOrigin, which is the third parameter in the getMediaInfo or getDynamicSrc method.)	No	The Object Storage Service (OSS) URL of the material in the video track. The OSS URL is in the following format: https://your-bucket.oss-cn-shanghai.aliyuncs.com/your-object.mp4 Note You must specify the MediaId or MediaURL parameter. You can specify only a public OSS URL for this parameter. HTTP URLs, accelerated OSS URLs, and CDN URLs are not supported. For more information, see Edit images and videos.
Type	String	Yes	No	The type of the material. Default value: Video. Valid values: Video Image
X	Float	Yes	No	The horizontal distance between the upper-left corner of the material and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Y	Float	Yes	No	The vertical distance between the upper-left corner of the material and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Width	Float	Yes	No	If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Height	Float	Yes	No	If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
AdaptMode	String	Yes	No	The mode in which the material is automatically resized. This parameter takes effect only if you specify the width and height of the view. The material is scaled based on the width and height of the view. Valid values: Contain: resizes the material based on the original aspect ratio of the material. Cover: resizes the material based on the original aspect ratio to fill in the view and crop the overflow. If the aspect ratio of the object does not match the view, the object is cropped based on the width and height of the view. Fill (default): resizes the material based on the original aspect ratio and stretches the material to fill in the view. The object completely fills in the view. If the aspect ratio of the object does not match the view, the object is stretched based on the width and height of the view.
In	Float	Yes	No	The start time of the material in the source file. The source file must be an audio or video file. Unit: seconds. The value is accurate to four decimal places. If you do not specify In, 0 is used.
Out	Float	Yes	No	The end time of the material in the source file. The source file must be an audio or video file. Unit: seconds. The value is accurate to four decimal places. If you do not specify Out, the material ends when the source file ends.
MaxOut	Float	Yes	No	The maximum end time of the material. If you specify this parameter, the end time of the material is the smaller value between the duration of the source file and the value of MaxOut. This parameter takes effect only if the source file is an audio or video file. Unit: seconds. The value is accurate to four decimal places. If you specify Out, MaxOut becomes invalid.
Duration	Float	Yes	No	The duration of the material. In most cases, this parameter is used for images. Unit: seconds. The value is accurate to four decimal places.
DyncFrames	Int	Yes	No	The number of frames of the animated sticker. This parameter takes effect only if the source file is an animated image. For more information, see the Add a watermark, sticker, or GIF sticker section of the "Multi-video mixing timelines" topic.
TimelineIn	Float	Yes	Yes	The start time of the material in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineIn, the material starts to be displayed when the display of the previous material in the timeline ends.
TimelineOut	Float	Yes	Yes	The end time of the material in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineOut, the material stops being displayed when the display of the next material in the timeline starts.
Speed	Float	Yes	No	The playback speed of the material. Valid values: 0.1 to 100. For example, if you set this parameter to 2, the material is played at twice the original speed in the output video. In this case, the duration of the material is reduced to half. For more information, see Edit images and videos.
Opacity	Float	Yes	No	The opacity of the video. Valid values: 0 to 1. A value of 0 specifies that the video is completely transparent. A value of 1 specifies that the video is completely opaque.
MaskVideoUrl	String	Yes	No	The URL of the mask file. In most cases, the mask file is a video that contains alpha channels. You can use alpha channels to add transparency information to the original video. Note You can set this parameter only to a public OSS URL.
ClipId	String	Yes	No	The track alignment value. If the same value of the ReferenceClipId parameter is set for materials in other audio or video tracks, the start time and end time of the materials in other tracks in the timeline are aligned with the start time and end time of the current material. For more information, see Automatically align materials and the material duration.
ReferenceClipId	String	Yes	No	The track alignment value. If the same value of the ClipId parameter is set for materials in other audio or video tracks, the start time and end time of the current material in the timeline are aligned with the start time and end time of the materials in other tracks. For more information, see Automatically align materials and the material duration.
Effects	Effect[]	Yes	No	The effects that are used on the material.

AudioTrackClip

Parameter	Type	Supported	Required	Description
MediaId	String	Yes	No	The ID of the IMS or ApsaraVideo VOD media asset to which the material in the audio track corresponds. Note You must specify the MediaId or MediaURL parameter.
MediaURL	String	Yes (This parameter is equal to mediaOrigin, which is the third parameter in the getMediaInfo or getDynamicSrc method.)	No	The OSS URL of the material in the audio track. The OSS URL is in the following format: https://your-bucket.oss-cn-shanghai.aliyuncs.com/your-object.mp4 Note You must specify the MediaId or MediaURL parameter. You can specify only a public OSS URL for this parameter. HTTP URLs, accelerated OSS URLs, and CDN URLs are not supported. For more information, see Edit images and videos.
In	Float	Yes	No	The start time of the material in the source file. Unit: seconds. The value is accurate to four decimal places. If you do not specify In, 0 is used.
Out	Float	Yes	No	The end time of the material in the source file. Unit: seconds. The value is accurate to four decimal places. If you do not specify Out, the material ends when the source file ends.
TimelineIn	Float	Yes	Yes	The start time of the material in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineIn, the material starts to be displayed when the display of the previous material in the timeline ends.
TimelineOut	Float	Yes	Yes	The end time of the material in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineOut, the material stops being displayed when the display of the next material in the timeline starts.
Speed	Float	Yes	No	The playback speed of the material. Valid values: 0.1 to 100. For example, if you set this parameter to 2, the material is played at twice the original speed in the output audio. In this case, the duration of the material is reduced to half. For more information, see Edit images and videos.
Effects	Effect[]	Yes	No	The effects that are used on the material.
LoopMode	Boolean	Yes	No	Specifies whether to enable loop playback for the material in the timeline. True: enables loop playback. False (default): disables loop playback. For more information, see Edit images and videos.
ClipId	String	Yes	No	The track alignment value. If the same value of the ReferenceClipId parameter is set for materials in other audio or video tracks, the start time and end time of the materials in other tracks in the timeline are aligned with the start time and end time of the current material. For more information, see Automatically align materials and the material duration.
ReferenceClipId	String	Yes	No	The track alignment value. If the same value of the ClipId parameter is set for materials in other audio or video tracks, the start time and end time of the current material in the timeline are aligned with the start time and end time of the materials in other tracks. For more information, see Automatically align materials and the material duration.

ImageTrackClip

Parameter	Type	Supported	Required	Description
MediaId	String	Yes	No	The ID of the IMS or ApsaraVideo VOD media asset to which the material in the image track corresponds. Note You must specify the MediaId or MediaURL parameter.
MediaURL	String	Yes (This parameter is equal to mediaOrigin, which is the third parameter in the getMediaInfo or getDynamicSrc method.)	No	The OSS URL of the material in the image track. The OSS URL is in the following format: https://your-bucket.oss-cn-shanghai.aliyuncs.com/your-object.mp4. Note You must specify the MediaId or MediaURL parameter. You can specify only a public OSS URL for this parameter. HTTP URLs, accelerated OSS URLs, and CDN URLs are not supported. For more information, see Edit images and videos.
X	Float	Yes	No	The horizontal distance between the upper-left corner of the material and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Y	Float	Yes	No	The vertical distance between the upper-left corner of the material and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Width	Float	Yes	No	The width of the material in the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Height	Float	Yes	No	The height of the material in the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
TimelineIn	Float	Yes	Yes	The start time of the image in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineIn, 0 is used.
TimelineOut	Float	Yes	Yes	The end time of the image in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineOut, the maximum duration of the video track is used.
DyncFrames	Int	Yes	No	The number of frames of the animated sticker. This parameter is required if the source file is an animated image. For more information, see the Add a watermark, sticker, or GIF sticker section of the "Multi-video mixing timelines" topic.
Effects	Effect[]	Yes	No	The effects that are used on the material.

SubtitleTrackClip

Parameter	Type	Supported	Required	Description
Type	String	Partially supported (Only Text is supported.)	Yes	The type of the subtitle material. Valid values: Subtitle: an external subtitle file Text: banner text
SubType	String	Yes	No	The subtype of the subtitle material. Valid values: srt: an external subtitle file in the SubRip Subtitle (SRT) format ass: an external subtitle file in the Advanced SubStation Alpha (ASS) format Ignore this parameter if you set the Type parameter to Text.
FileURL	String	No	No	The OSS URL of the subtitle file. This parameter is required if you set the Type parameter to Subtitle. The OSS URL is in the following format: https://your-bucket.oss-cn-shanghai.aliyuncs.com/your-object.srt Note You can specify only a public OSS URL for this parameter. HTTP URLs, accelerated OSS URLs, and CDN URLs are not supported. For more information, see Subtitles and subtitle templates.
X	Float	Yes	No	The horizontal distance between the upper-left corner of the banner text and the upper-left corner of the output video. This parameter takes effect if you set the Type parameter to Text. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Y	Float	Yes	No	The vertical distance between the upper-left corner of the banner text and the upper-left corner of the output video. This parameter takes effect if you set the Type parameter to Text. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
TimelineIn	Float	Yes	Yes	The start time of the banner text in the timeline. This parameter takes effect if you set the Type parameter to Text. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineIn, the text starts to be displayed when the display of the previous text in the timeline ends.
TimelineOut	Float	Yes	Yes	The end time of the banner text in the timeline. This parameter takes effect if you set the Type parameter to Text. Unit: seconds. The value is accurate to four decimal places. If you do not specify TimelineOut, the text stops being displayed when the display of the next text in the timeline starts.
Content	String	Yes	Yes	The banner text. This parameter is required if you set the Type parameter to Text.
Font	String	Yes	No	The font of the banner text. This parameter takes effect if you set the Type parameter to Text. Valid values: `"Alibaba PuHuiTi", // Alibaba PuhuiTi "STFangsong", // Fangsong "KaiTi", // KaiTi "SimSun", // SimSun "Source Han Sans CN Normal", // Source Han Sans "Source Han Serif", // Source Han Serif "WenQuanYi Zen Hei Mono", // WenQuanYi Zen Hei Mono "WenQuanYi Zen Hei Sharp", // WenQuanYi Zen Hei Sharp "Wenquanyi Micron Black", // WenQuanYi Micro Hei "zcool-gdh", // zcool-gdh "HappyZcool-2016", // HappyZcool "zcoolwenyiti" // zcoolwenyiti`
FontSize	Int	Yes	Yes	The font size of the banner text. This parameter takes effect if you set the Type parameter to Text.
SizeRequestType	String	Yes	No	The calculation method of the size of the rendered font of the banner text. This parameter takes effect if you set the Type parameter to Text. Default value: Nominal. Nominal: The subtitle rendering height (in pixels) is equal to the font size. RealDim: For some fonts, the subtitle rendering height (in pixels) may be less than the font size.
FontColor	String	Yes	No	The color of the banner text. This parameter takes effect if you set the Type parameter to Text. The value is in the format of a number sign (#) and a hexadecimal value. Example: #ffffff.
FontColorOpacity	String	Yes	No	The opacity of the banner text. This parameter takes effect if you set the Type parameter to Text. Valid values: 0 to 1. Default value: 1. A value of 1 specifies that the text is completely opaque. A value of 0 specifies that the text is completely transparent.
FontFace	FontFace	Yes	No	The appearance of the banner text. This parameter takes effect if you set the Type parameter to Text.
Spacing	Int	Yes	No	The word spacing of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: pixel. Default value: 0.
LineSpacing	Int	Yes	No	The line spacing of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: pixel. Default value: 0.
Angle	Float	Yes	No	The rotation angle of the banner text in the counterclockwise direction. This parameter takes effect if you set the Type parameter to Text. Unit: degree. Default value: 0.
BorderStyle	Int	Yes	No	The border style of the banner text. Valid values: 1 and 3. A value of 1 specifies that the text has a border and a shadow. A value of 3 specifies that the text has a background. Default value: 1.
Outline	Int	Yes	No	The outline width of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: pixel. Default value: 0.
OutlineColour	String	Yes	No	The outline color of the banner text. This parameter takes effect if you set the Type parameter to Text. The value is in the format of a number sign (#) and a hexadecimal value. Example: #ffffff.
Shadow	Int	Yes	No	The shadow size of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: pixel. Default value: 0.
BackColour	String	Yes	No	The shadow color of the banner text. This parameter takes effect if you set the Type parameter to Text. The value is in the format of a number sign (#) and a hexadecimal value. Example: #ffffff.
Alignment	String	Yes	No	The text alignment. This parameter takes effect if you set the Type parameter to Text. Default value: TopLeft. Valid values: TopLeft: the upper-left corner of the video. TopCenter: the upper part of the vertical axis. TopRight: the upper-right corner of the video. CenterLeft: the left part of the horizontal axis. CenterCenter: the center of the video. CenterRight: the right part of the horizontal axis. BottomLeft: the lower-left corner of the video. BottomCenter: the lower part of the vertical axis. BottomRight: the lower-right corner of the video. If you need to specify the accurate position of the subtitle in a specific alignment mode, we recommend that you set one of the following configurations: If you specify left alignment, specify the coordinates (x,y) of the upper-left corner of the subtitle relative to the upper-left corner of the video. If you specify center alignment, specify the coordinates (x,y) of the upper contact point of the subtitle with the axis relative to the upper-left corner of the video. If you specify right alignment, specify the coordinates (x,y) of the upper-right corner of the subtitle relative to the upper-left corner of the video.
AdaptMode	String	Partially supported (AutoScale is not supported.)	No	Specifies whether to enable automatic line breaking or automatic scaling when the banner text exceeds the video width or the specified text box width. Valid values: AutoWrap: enables automatic line breaking. AutoScale: enables automatic scaling. AutoWrapAtSpaces: automatically wraps text by using the white space only when the text is displayed in English.
TextWidth	Float	Yes	No	The width of the text box. This parameter takes effect if you specify the AdaptMode parameter. This parameter is used when you enable automatic line breaking or automatic scaling. If you do not specify this parameter, the video width is used for automatic line breaking or automatic scaling. When the value is greater than 0 and less than or equal to 1, the value indicates the percentage of the width of the text box to the width of the output video. When the value is greater than 1, the value indicates the number of pixels.
FontUrl	String	Yes	No	The OSS URL of the font file. This parameter takes effect if you set the Type parameter to Text. The following formats of font files are supported: ttf, otf, and woff. Example: https://your-bucket.oss-cn-shanghai.aliyuncs.com/example-font.ttf.
EffectColorStyle	String	Yes	No	The word art effect of the banner text. This parameter takes effect if you set the Type parameter to Text. For more information about word art effects, see Subtitles and subtitle templates.
SubtitleEffects	SubtitleEffect[]	Yes	No	The multilayer effects of the banner text. This parameter takes effect if you set the Type parameter to Text. You can configure multilayer outlines, multilayer shadows, and Gaussian blurs. Gaussian blurs are supported only if subtitle text is set to have a shadow. Note If you specify SubtitleEffects, the Outline and Shadow parameters in SubtitleTrackClip become invalid. If you configure multilayer outlines or multilayer shadows, the subtitle effects are overlaid based on the sequence of SubtitleEffects array elements. The layer that corresponds to the first subtitle effect in the array is placed at the bottom, and the layer that corresponds to the last subtitle effect in the array is placed at the top.
AaiMotionInEffect	String	Yes	No	The entrance effect of the banner text. This parameter takes effect if you set the Type parameter to Text. For more information about entrance effects of subtitles, see Subtitle effect examples.
AaiMotionIn	Float	Yes	No	The duration of the entrance effect of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: seconds. The value is accurate to four decimal places. If you do not specify the AaiMotionIn parameter, the default duration of 0.5 seconds is used. If the text is displayed for less than 0.5 seconds, the duration of the entrance effect is the total duration of the text minus the duration of the exit effect.
AaiMotionOutEffect	String	Yes	No	The exit effect of the banner text. This parameter takes effect if you set the Type parameter to Text. For more information about exit effects of subtitles, see Subtitle effect examples.
AaiMotionOut	Float	Yes	No	The duration of the exit effect of the banner text. This parameter takes effect if you set the Type parameter to Text. Unit: seconds. The value is accurate to four decimal places. If you do not specify the AaiMotionOut parameter, the default duration of 0.5 seconds is used. If the text is displayed for less than 0.5 seconds, the duration of the exit effect is the total duration of the text.
AaiMotionLoopEffect	String	Yes	No	The loop effect of the banner text. This parameter takes effect if you set the Type parameter to Text. The loop effect cannot take effect together with the entrance effect or the exit effect. For more information about loop effects of text, see Subtitle effect examples.
Ratio	Float	Yes	No	The speed of the loop effect of the banner text. This parameter takes effect if you set the Type parameter to Text. The value is accurate to four decimal places. If you do not specify this parameter, 1 is used. A value greater than 1 indicates a higher speed, and a value less than 1 indicates a lower speed.

EffectTrackItem

Parameter	Type	Supported	Required	Description
Type	String	Yes	Yes	The type of the effect. Valid values: VFX and Filter.
SubType	String	Yes	Yes	The subtype of the effect. For more information about the supported effects, see the following topics: Special effect examples and Filter examples.
TimelineIn	Float	Yes	Yes	The start time of the effect in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify the TimelineIn parameter, 0 is used.
TimelineOut	Float	Yes	Yes	The end time of the effect in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify the TimelineOut parameter, the effect ends when the video ends.
Duration	Float	Yes	No	The duration of the effect in the timeline. Unit: seconds. The value is accurate to four decimal places. If you do not specify the Duration parameter, the duration of the video is used. Note You cannot specify the Duration and TimelineOut parameters at the same time.
X	Float	Yes	No	This parameter takes effect only if you set the SubType parameter to mosaic_rect/blur. The horizontal distance between the upper-left corner of the effect area and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Y	Float	Yes	No	This parameter takes effect only if you set the SubType parameter to mosaic_rect/blur. The vertical distance between the upper-left corner of the effect area and the upper-left corner of the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Width	Float	Yes	No	This parameter takes effect only if you set the SubType parameter to mosaic_rect/blur. The width of the effect area in the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.
Height	Float	Yes	No	This parameter takes effect only if you set the SubType parameter to mosaic_rect/blur. The height of the effect area in the output video. Note You can set the value to a percentage or a number of pixels. If the value ranges from 0 to 0.9999, the value indicates a percentage. If the value is an integer greater than or equal to 2, the value indicates a number of pixels.

VideoClips effect

Parameter

Type

Supported

Required

Description

Type

String

Partially supported (The following types are supported:

Text, Crop, Scale, Transition, Filter, VFX, Volume, and AFade.)

Yes

The type of the effect. Valid values:

Text: adds banner text to a material in the video track.
DeWatermark: blurs the watermark on a material in the video track.
Crop: crops a material in the video track.
Pad: adds paddings to a material in the video track.
Scale: scales a material in the video track.
Transition: adds a transition effect to a material in the video track.
VFX: adds a visual effect to a material in the video track.
Volume: adjusts the volume of a material in the audio track.
AFade: configures audio fade-in and fade-out for a material in the audio track.
AI_ASR: generates subtitles from the audio of a material in the audio or video track.

SubType

String

Yes

The subtype of the effect.

If you set the Type parameter to Transition, this parameter specifies the subtype of the transition effect. For more information about this parameter, see Transition effect examples.
If you set the Type parameter to VFX, this parameter specifies the subtype of the visual effect. For more information about this parameter, see Special effect examples.
If you set the Type parameter to Filter, this parameter specifies the subtype of the filter effect. For more information about this parameter, see Filter examples.