All Products
Search
Document Center

Intelligent Media Services:Video translation parameters

Last Updated:Aug 24, 2026

Use SubmitVideoTranslationJob to submit video translation tasks and GetSmartHandleJob to retrieve the results. This topic covers the request and response parameters for both operations.

Availability

Translation type

Supported regions

Subtitle translation

China (Shanghai), China (Beijing), China (Shenzhen), China (Hangzhou), Singapore, US (Silicon Valley)

Speech translation

China (Shanghai), China (Beijing), China (Shenzhen), China (Hangzhou), Singapore, US (Silicon Valley)

Lip-sync translation

China (Shanghai), Singapore

SubmitVideoTranslationJob

Important

To distinguish between translation types, configure the following boolean parameters:

  • Subtitle translation: Set NeedSpeechTranslate and NeedFaceTranslate to false. Configure subtitle settings in SubtitleTranslate.

  • Speech translation: Set NeedSpeechTranslate to true and NeedFaceTranslate to false. Configure speech settings in SpeechTranslate.

  • Lip-sync translation: Set NeedFaceTranslate to true and NeedSpeechTranslate to false. Configure lip-sync settings in FaceTranslate.

Parameter

Type

Required

Description

InputConfig

String

Yes

Input configurations as a JSON string.

OutputConfig

String

Yes

Output configurations as a JSON string.

EditingConfig

String

Yes

Translation configurations as a JSON string.

Title

String

No

Task name.

Description

String

No

Task description.

UserData

String

No

Custom information as a JSON string. Maximum length: 512 characters. Supports callback URL configuration.

InputConfig

Parameter

Type

Required

Description

Type

String

Yes

Input type. Valid values: Video, Audio, Subtitle.

Video

String

Conditional

Video file ID or URL. Required when Type is Video. Accepts a media asset ID, an OSS URL in the current Alibaba Cloud account, or a publicly accessible URL.

Audio

String

Conditional

Audio file ID or URL. Required when Type is Audio. Accepts a media asset ID, an OSS URL in the current Alibaba Cloud account, or a publicly accessible URL. Supported formats: MP3, WAV.

Subtitle

String

Conditional

Subtitle file ID or URL. Required when Type is Subtitle. Accepts a media asset ID, an OSS URL in the current Alibaba Cloud account, or a publicly accessible URL. Supported format: SRT.

Input type constraints

Translation type

Supported input types

Subtitle translation (NeedSpeechTranslate=false, NeedFaceTranslate=false)

Video, Subtitle

Speech translation (NeedSpeechTranslate=true)

Video, Audio

Lip-sync translation (NeedFaceTranslate=true)

Video only

OutputConfig

Parameter

Type

Required

Description

OutputTarget

String

No

Output destination. Default: OSS. Valid values: OSS (save to OSS), VOD (save to ApsaraVideo VOD).

MediaURL

String

Yes

OSS URL in the current Alibaba Cloud account. The path must include the file extension. Supported extensions: mp4 (when InputConfig.Type=Video), wav (when InputConfig.Type=Audio), srt (when InputConfig.Type=Subtitle).

StorageLocation

String

Conditional

Required when OutputTarget is VOD. Specify the storage location without protocol prefixes such as http:// or https://. Example: outin-***<b>c7d2a3811eb83da00163e0</b>***.oss-cn-shanghai.aliyuncs.com.

FileName

String

Conditional

Required when OutputTarget is VOD. File name with extension. Supported extensions: mp4, wav, srt.

Width

Integer

No

Output video width in pixels. Defaults to the source video width.

Height

Integer

No

Output video height in pixels. Defaults to the source video height.

Video

JSONObject

No

Output video configurations such as CRF settings and codec. Example: {"Crf": 27}.

EditingConfig

Parameter

Type

Required

Description

SourceLanguage

String

Yes

Source language code (for example, zh).

TargetLanguage

String

Yes

Target language code. Separate multiple languages with commas. Example: en,ja,id.

DetextArea

String

No

Subtitle erasure configuration. If unspecified, source subtitles are retained. Valid values: Auto (automatically detect the subtitle area), or a two-layer array [[x, y, width, height], ...] specifying one or more erasure areas. In the array, x is the ratio of the horizontal distance from the upper-left corner of the area to the upper-left corner of the video, y is the vertical distance ratio, width is the width ratio, and height is the height ratio. All values range from 0 to 1.

SupportEditing

Boolean

No

Enable post-editing to correct translations. Default: false.

BilingualSubtitle

Boolean

No

Generate bilingual subtitles in the output video. Supported by all three translation types. Default: false.

SubtitleTranslate

String

No

Subtitle translation configurations as a JSON string. Required for subtitle translation.

NeedSpeechTranslate

Boolean

No

Enable speech translation. Default: false.

SpeechTranslate

String

No

Speech translation configurations as a JSON string. Speech translation runs when NeedSpeechTranslate is true or when this parameter is specified.

NeedFaceTranslate

Boolean

No

Enable lip-sync translation. Default: false.

FaceTranslate

String

No

Lip-sync translation configurations as a JSON string. Lip-sync translation runs when NeedFaceTranslate is true or when this parameter is specified.

TextSource

String

No

Subtitle source method.

CustomParameter

String

No

Custom parameters to control video translation effects.

HotwordLibraryIdList

String

No

Comma-separated IDs of hotword libraries. Example: "**<b>b4549d46c88681030f6e</b><b>,</b><b>549d46c88b4681030f6e</b>**".

FECanvas

JSON

No

Frontend preview canvas dimensions for adaptive subtitle font scaling. Example: {"Width": 1080, "Height": 1920}. The rendered font size is calculated as: FontSize x Math.min(outputWidth/feWidth, outputHeight/feHeight). For example, with FECanvas={"Width": 1080, "Height": 1920}, FontSize=100, and output resolution 720x1280, the rendered FontSize = 100 x Math.min(720/1080, 1280/1920) = 67.

To produce an output video without subtitles, set SubtitleTrackClip.FontSize to 0.

SourceLanguage valid values

Subtitle translation (TextSource=OCR or OCR_ASR)

Subtitle translation (TextSource=ASR)

External SRT source

Speech translation

Lip-sync translation

zh (Chinese), en (English)

zh (Chinese), en (English), fr (French), tr (Turkish)

zh, en, ja, ko, yue, de, fr, es, ar, it, az, be, bg, bs, bn, cs, cy, da, et, fa, hi, hbs, hu, id, is, lt, lv, mi, mn, mr, ms, mt, ne, nl, no, pl, pt, ro, ru, sk, sl, sq

Same as External SRT source

Same as External SRT source

TargetLanguage valid values

Subtitle translation:

zh (Chinese), zh-tw (Traditional Chinese), en (English), ja (Japanese), id (Indonesian), es (Spanish), pt (Portuguese), ar (Arabic), fr (French), tr (Turkish), yue (Cantonese), de (German), ko (Korean), ru (Russian), th (Thai), vi (Vietnamese), ms (Malay)

Chinese dialects: sichuan (Sichuan), dongbei (Northeastern), henan (Henan), shanghai (Shanghai), tianjin (Tianjin), beijing (Beijing), chongqing (Chongqing), hunan (Hunan), taiwan (Hakka Chinese), shanxi (Shanxi), shaanxi (Shaanxi)

Speech translation:

zh (Chinese), zh-tw (Traditional Chinese), en (English), ja (Japanese), ko (Korean), yue (Cantonese), de (German), fr (French), es (Spanish), ar (Arabic), tr (Turkish), ru (Russian), pt (Portuguese), vi (Vietnamese), ms (Malay), th (Thai), id (Indonesian)

Chinese dialects: sichuan (Sichuan), tianjin (Tianjin)

Lip-sync translation:

en (English) only

TextSource valid values

Value

Description

Default for

ASR

Recognize subtitles through Automatic Speech Recognition (ASR).

-

OCR

Recognize subtitles through Optical Character Recognition (OCR). Set the recognition area through OcrArea.

Subtitle translation

OCR_ASR

Prioritize OCR. If OCR fails, fall back to ASR. Set the recognition area through OcrArea.

Speech and lip-sync translation

ALL

Prioritize ASR-transcribed text and timeline, then refine with OCR to correct inaccuracies. Available for subtitle translation only.

-

SubtitleFile

Use an external SRT file specified in InputConfig.Subtitle.

-

SubtitleTranslate

Parameter

Type

Required

Description

OcrArea

String

No

OCR recognition area. If unspecified, the subtitle area is automatically detected. Valid values: Auto, or a single-layer array [x, y, width, height] where x is the ratio of the horizontal distance from the upper-left corner of the area to the upper-left corner of the video, y is the vertical distance ratio, width is the width ratio, and height is the height ratio. All values range from 0 to 1. Unlike DetextArea, only a single area can be specified.

SubtitleConfig

String

No

Output subtitle styling as a JSON string. Parameters follow the SubtitleTrackClip specification in Timeline. Set FontSize to 0 to hide subtitles.

SpeechTranslate

Parameter

Type

Required

Description

OcrArea

String

No

OCR recognition area. Same format as SubtitleTranslate.OcrArea.

CustomSrtType

String

Conditional

Required if InputConfig.Subtitle is specified. Specifies the input subtitle type. Valid values: SourceSrt (source language), TargetSrt (target language), BilingualSrtSrcFirst (bilingual, target below source), BilingualSrtTgtFirst (bilingual, source below target).

SubtitleTimeForce

Boolean

No

Translate only audio within the subtitle time range. Default: false. Valid only when TextSource is OCR or OCR_ASR.

SubtitleConfig

String

No

Output subtitle styling. Same format as SubtitleTranslate.SubtitleConfig.

OriginalJobId

String

No

Initial task ID for post-editing resubmission. Use the SpeechTranslationJobId value returned from the initial task (when SupportEditing is true). The initial translation result is available from SpeechTranslatedSubtitleURL. Modify the SRT file and pass the URL to InputConfig.Subtitle.

SkipSong

Integer

No

Skip translation for sung musical sections. Valid values: 0 (default, translate all detected speech including vocals), 1 (identify and exclude sung vocals). The music detection algorithm is not 100% accurate. Set to 1 only when the source video contains singing.

FaceTranslate

Parameter

Type

Required

Description

OcrArea

String

No

OCR recognition area. Same format as SubtitleTranslate.OcrArea.

CustomSrtType

String

Conditional

Required if InputConfig.Subtitle is specified. Same valid values as SpeechTranslate.CustomSrtType.

SubtitleTimeForce

Boolean

No

Same behavior as SpeechTranslate.SubtitleTimeForce.

SubtitleConfig

String

No

Output subtitle styling. Same format as SubtitleTranslate.SubtitleConfig.

SpeechDurationThres

Float

No

Minimum segment duration in seconds for lip-sync processing. Segments shorter than this threshold skip lip-sync. Default: 1. Cannot exceed the total duration of the source video.

FacialClarity

Float

No

Face clarity level. Valid range: 0 to 1. Default: 1 (highest clarity). Use a smaller value for low-resolution source videos.

SRT format examples

For Chinese-to-English translation:

SourceSrt (source language only):

1
00:00:01,000 --> 00:00:05,000
你好,世界!

TargetSrt (target language only):

1
00:00:01,000 --> 00:00:05,000
Hello,World!

BilingualSrtSrcFirst (target below source):

1
00:00:01,000 --> 00:00:05,000
你好,世界!
Hello,World!

BilingualSrtTgtFirst (source below target):

1
00:00:01,000 --> 00:00:05,000
Hello,World!
你好,世界!

CustomParameter

Value

Description

--add -dl

Use when the input audio contains multiple languages. For example, in a Korean-to-French translation task, English segments in the input are translated into French. For Chinese-to-English tasks with mixed Chinese and English input, do not use this parameter because the English input is used directly in the output.

Sample code

Subtitle translation

Subtitle file input
Important

SRT file input is charged based on text translation pricing.

{
  "InputConfig": {
    "Type": "Subtitle",
    "Subtitle": "https://******.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a0720f71eeb755f6f7d6496302/snapshots/sprite/test.srt"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en"
  },
  "Title": "1735798516693.srt",
  "OutputConfig": {
    "MediaURL": "https://****.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a0720f71eeb755f6f7d6496302/snapshots/sprite/new.srt"
  }
}
Video file input
{
  "InputConfig": {
    "Type": "Video",
    "Video": "1628ae20c36******f6f7c77a6302"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "DetextArea": [[0, 0.64, 1, 0.13]],
    "BilingualSubtitle": false,
    "SubtitleTranslate": {
      "OcrArea": [0, 0.64, 1, 0.15],
      "SubtitleConfig": {
        "Type": "Text",
        "FontSize": 95,
        "FontColorOpacity": 1,
        "Color": "#ffffff",
        "X": 0.5,
        "Y": 0.686,
        "Angle": 0,
        "Spacing": 0,
        "TextWidth": 0.9,
        "Font": "Alibaba PuHuiTi",
        "FontColor": "#ffffff",
        "FontFace": {
          "Bold": false,
          "Italic": false,
          "Underline": false
        },
        "SizeRequestType": "RealDim",
        "SubtitleEffects": [],
        "LineSpacing": 0,
        "BorderStyle": 1,
        "Outline": 0,
        "Alignment": "Center"
      }
    },
    "SupportEditing": true,
    "NeedSpeechTranslate": false
  },
  "Title": "have a test",
  "OutputConfig": {
    "MediaURL": "https://*****.oss-cn-shanghai.aliyuncs.com/ice-generated/test.mp4"
  }
}
Video and subtitle file input
{
  "InputConfig": {
    "Type": "Video",
    "Video": "4e92fa60c995*****6f7c77a6302",
    "Subtitle": "https://*****.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a072****5f6f7d6496302/snapshots/sprite/test.srt"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "DetextArea": [[0, 0.64, 1, 0.14]],
    "BilingualSubtitle": false,
    "SubtitleTranslate": {
      "OcrArea": "Auto",
      "SubtitleConfig": {
        "Type": "Text",
        "FontSize": 95,
        "FontColorOpacity": 1,
        "Color": "#ffffff",
        "X": 0.5,
        "Y": 0.686,
        "Angle": 0,
        "Spacing": 0,
        "TextWidth": 0.9,
        "Font": "Alibaba PuHuiTi",
        "FontColor": "#ffffff",
        "FontFace": {
          "Bold": false,
          "Italic": false,
          "Underline": false
        },
        "SizeRequestType": "RealDim",
        "SubtitleEffects": [],
        "LineSpacing": 0,
        "BorderStyle": 1,
        "Outline": 0,
        "Alignment": "Center"
      }
    },
    "SupportEditing": true,
    "NeedSpeechTranslate": false
  },
  "Title": "1735898570421.mp4",
  "OutputConfig": {
    "MediaURL": "https://****.oss-cn-shanghai.aliyuncs.com/ice-generated/******/snapshots/sprite/1735898570421.mp4"
  }
}

Speech translation

Video file input
{
  "InputConfig": {
    "Type": "Video",
    "Video": "1628ae20c36******8f6f7c77a6302"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "DetextArea": [[0, 0.64, 1, 0.15]],
    "SupportEditing": true,
    "BilingualSubtitle": false,
    "NeedSpeechTranslate": true,
    "SpeechTranslate": {
      "SubtitleTimeForce": false,
      "SubtitleConfig": {
        "Type": "Text",
        "FontSize": 95,
        "FontColorOpacity": 1,
        "Color": "#ffffff",
        "X": 0.5,
        "Y": 0.686,
        "Angle": 0,
        "Spacing": 0,
        "TextWidth": 0.9,
        "Font": "Alibaba PuHuiTi",
        "FontColor": "#ffffff",
        "FontFace": {
          "Bold": false,
          "Italic": false,
          "Underline": false
        },
        "SizeRequestType": "RealDim",
        "SubtitleEffects": [],
        "LineSpacing": 0,
        "BorderStyle": 1,
        "Outline": 0,
        "Alignment": "Center"
      }
    }
  },
  "Title": "have a test",
  "OutputConfig": {
    "MediaURL": "https://******.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a0720f71eeb755f6f7d6496302/snapshots/sprite/1735798757385.mp4"
  }
}
Video and subtitle file input
{
  "InputConfig": {
    "Type": "Video",
    "Video": "738d94a0ce87******af6f7c6696302",
    "Subtitle": "https://********.oss-cn-test.aliyuncs.com/test.srt"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "DetextArea": "Auto",
    "SupportEditing": true,
    "BilingualSubtitle": false,
    "NeedSpeechTranslate": true,
    "SpeechTranslate": {
      "SubtitleTimeForce": false,
      "SubtitleConfig": {
        "Type": "Text",
        "FontSize": 95,
        "FontColorOpacity": 1,
        "Color": "#ffffff",
        "X": 0.5,
        "Y": 0.686,
        "Angle": 0,
        "Spacing": 0,
        "TextWidth": 0.9,
        "Font": "Alibaba PuHuiTi",
        "FontColor": "#ffffff",
        "FontFace": {
          "Bold": false,
          "Italic": false,
          "Underline": false
        },
        "SizeRequestType": "RealDim",
        "SubtitleEffects": [],
        "LineSpacing": 0,
        "BorderStyle": 1,
        "Outline": 0,
        "Alignment": "Center"
      },
      "OcrArea": "Auto",
      "CustomSrtType": "SourceSrt"
    }
  },
  "Title": "1736485935837.mp4",
  "OutputConfig": {
    "MediaURL": "https://*****.oss-cn-***.aliyuncs.com/test.mp4"
  }
}
Audio file input
{
  "InputConfig": {
    "Type": "Audio",
    "Audio": "2f552010c8d******e7f7f4586303"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "NeedSpeechTranslate": true
  },
  "Title": "have a test",
  "OutputConfig": {
    "MediaURL": "https://******.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a0720f***f6f7d6496302/snapshots/sprite/test.wav"
  }
}

Lip-sync translation

{
  "InputConfig": {
    "Type": "Video",
    "Video": "1628ae20c36******8f6f7c77a6302"
  },
  "EditingConfig": {
    "SourceLanguage": "zh",
    "TargetLanguage": "en",
    "DetextArea": [[0, 0.64, 1, 0.15]],
    "SupportEditing": true,
    "BilingualSubtitle": false,
    "NeedFaceTranslate": true,
    "FaceTranslate": {
      "SubtitleConfig": {
        "Type": "Text",
        "FontSize": 95,
        "FontColorOpacity": 1,
        "Color": "#ffffff",
        "X": 0.5,
        "Y": 0.686,
        "Angle": 0,
        "Spacing": 0,
        "TextWidth": 0.9,
        "Font": "Alibaba PuHuiTi",
        "FontColor": "#ffffff",
        "FontFace": {
          "Bold": false,
          "Italic": false,
          "Underline": false
        },
        "SizeRequestType": "RealDim",
        "SubtitleEffects": [],
        "LineSpacing": 0,
        "BorderStyle": 1,
        "Outline": 0,
        "Alignment": "Center"
      },
      "SpeechDurationThres": 1,
      "FacialClarity": 1,
      "SubtitleTimeForce": false
    }
  },
  "Title": "have a test",
  "OutputConfig": {
    "MediaURL": "https://******.oss-cn-shanghai.aliyuncs.com/ice-generated/4e1021a0720f71eeb755f6f7d6496302/snapshots/sprite/1735798757385.mp4"
  }
}

GetSmartHandleJob

AiResult response parameters

Single target language

Category

Parameter

Type

Description

Common

EditingProjectId

String

Post-editing project ID.

MediaURL

String

Output media asset URL.

MediaId

String

Output media asset ID.

DetextVideoURL

String

URL of the subtitle-erased video.

DetextVideoMediaId

String

ID of the subtitle-erased video.

Subtitle translation

OriginalSubtitleMediaId

String

Source SRT file ID.

OriginalSubtitleURL

String

Source SRT file URL.

TranslatedSubtitleMediaId

String

Translated SRT file ID.

TranslatedSubtitleURL

String

Translated SRT file URL.

TranslatedText

String

Translated text.

TranslatedTextArray

String

Array of translated text for multiple subtitle inputs.

Speech translation

SpeechTranslatedSubtitleMediaId

String

SRT file ID for subtitle display in the output video.

SpeechTranslatedSubtitleURL

String

SRT file URL for subtitle display in the output video.

SpeechTranslatedSubtitleURLSigned

String

Signed SRT file URL for subtitle display in the output video.

SpeechTranslatedSubtitleMediaIdForFix

String

SRT file ID for translation correction.

SpeechTranslatedSubtitleURLForFix

String

SRT file URL for translation correction.

SpeechBilingualSubtitleMediaId

String

Bilingual SRT file ID.

SpeechBilingualSubtitleURL

String

Bilingual SRT file URL.

SpeechTranslationJobId

String

Speech translation task ID for post-editing.

TranslatedAudioMediaId

String

Translated audio file ID.

TranslatedAudioMediaURL

String

Translated audio file URL.

Lip-sync translation

FaceTranslationMediaId

String

Lip-synced video file ID.

Sample response:

{
  "SpeechTranslatedSubtitleURL": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test1.srt",
  "MediaId": "****9cb015bb71f0b96ff7f6d449****",
  "TranslatedAudioMediaId": "****df80e79d71efa44ae7f6c449****",
  "SpeechTranslatedSubtitleURLForFix": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test2.srt",
  "EditingProjectId": "****febec2bf4377b923cf5b1742****",
  "DetextVideoURL": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test3.mp4",
  "FaceTranslationMediaId": "****138015bb71f0b96ff7f6d449****",
  "SpeechTranslationJobId": "****0c827a664b4f88b33d8d46e3****"
}

Multiple target languages

Parameter

Type

Description

DetextVideoURL

String

URL of the subtitle-removed video.

DetextVideoMediaId

String

ID of the subtitle-removed video.

VideoTranslationAiResultMap

JSON

Translation outputs grouped by target language. Each key is a language code (for example, en), and the value contains the translation result parameters for that language.

VideoTranslationAiResultMap

Category

Parameter

Type

Description

Common

MediaURL

String

Output media asset URL for the target language.

MediaId

String

Output media asset ID for the target language.

Subtitle translation

OriginalSubtitleMediaId

String

Source SRT file ID.

OriginalSubtitleURL

String

Source SRT file URL.

TranslatedSubtitleMediaId

String

Translated SRT file ID.

TranslatedSubtitleURL

String

Translated SRT file URL.

TranslatedText

String

Translated text.

TranslatedTextArray

String

Array of translated text for multiple subtitle entries.

Speech translation

SpeechTranslatedSubtitleMediaId

String

SRT file ID for subtitle display in the output video.

SpeechTranslatedSubtitleURL

String

SRT file URL for subtitle display in the output video.

SpeechTranslatedSubtitleURLSigned

String

Signed SRT file URL for subtitle display in the output video.

SpeechTranslatedSubtitleMediaIdForFix

String

SRT file ID for translation correction.

SpeechTranslatedSubtitleURLForFix

String

Signed SRT file URL for translation correction.

SpeechBilingualSubtitleMediaId

String

Bilingual output ID.

SpeechBilingualSubtitleURL

String

Bilingual output URL.

SpeechTranslationJobId

String

Speech translation task ID for post-editing.

TranslatedAudioMediaId

String

Translated audio file ID.

TranslatedAudioMediaURL

String

Translated audio file URL.

Lip-sync translation

FaceTranslationMediaId

String

Lip-synced video file ID.

Sample response
{
  "VideoTranslationAiResultMap": {
    "en": {
      "SpeechTranslatedSubtitleURL": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test1.srt",
      "MediaId": "****4d00087171f0bf81f6f7d44b****",
      "TranslatedAudioMediaId": "****df80e79d71efa44ae7f6c449****",
      "SpeechTranslatedSubtitleURLForFix": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test2.srt",
      "EditingProjectId": "****983759c045c387146888713d****",
      "SpeechTranslationJobId": "****33cc65164f13aace55a0f0d3****"
    },
    "es": {
      "SpeechTranslatedSubtitleURL": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test3.srt",
      "MediaId": "****cec0087171f09033f7f6d449****",
      "TranslatedAudioMediaId": "****a7f0085671f0bf81f6f7d44b****",
      "SpeechTranslatedSubtitleURLForFix": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test4.srt",
      "EditingProjectId": "****d16f4a874c4898d254d4b68e****",
      "SpeechTranslationJobId": "****f705193c404e9bec19859a11****"
    }
  },
  "DetextVideoURL": "http://[your-bucket].oss-[your-region-id].aliyuncs.com/test5.mp4"
}

Scenario examples

Speech translation with post-editing

Correct the generated SRT file and resubmit to refine the translation output.

Important

Lip-sync translation supports corrections to voice output only. Post-editing of lip movements is not available.

Procedure

  1. Submit a speech translation task with SupportEditing set to true.

       {
         "InputConfig": {
           "Type": "Video",
           "Video": "*****a0052ff71efbfd4e7e6c66*****"
         },
         "OutputConfig": {
           "MediaURL": "https://your-bucket.oss-cn-shanghai.aliyuncs.com/video.mp4"
         },
         "EditingConfig": {
           "SourceLanguage": "zh",
           "TargetLanguage": "en",
           "SupportEditing": true,
           "NeedSpeechTranslate": true
         }
       }
  2. Retrieve the task result. The response includes:

    Field

    Description

    MediaId

    Output media asset ID.

    TranslatedAudioMediaId

    Translated audio file ID.

    SpeechTranslatedSubtitleURL

    Translated SRT file URL for subtitle display in the output video.

    SpeechTranslatedSubtitleURLSigned

    Signed URL of the translated SRT file.

    SpeechTranslatedSubtitleURLForFix

    Signed URL of the SRT file for translation correction.

    SpeechTranslationJobId

    Speech translation task ID. Use this value for OriginalJobId in the resubmission.

       {
         "MediaId": "*****d306b6d71efbf98f6f7f55*****",
         "TranslatedAudioMediaId": "*****d306b6d71efbf98f6f7f5*****",
         "SpeechTranslatedSubtitleURL": "http://your-bucket.oss-cn-shanghai.aliyuncs.com/video_subtitle_asr_en.srt",
         "SpeechTranslatedSubtitleURLSigned": "http://your-bucket.oss-cn-shanghai.aliyuncs.com/***.srt",
         "SpeechTranslatedSubtitleURLForFix": "http://your-bucket.oss-cn-shanghai.aliyuncs.com/***.srt",
         "SpeechTranslationJobId": "*****74f329d4c03b63e7f7dac8*****"
       }
  3. Download the SRT file from SpeechTranslatedSubtitleURLForFix, modify the content, and resubmit the task. The system regenerates the output based on the modified subtitles.

    Important

    The InputConfig.Subtitle parameter must contain the manually corrected file, which is the modified version of the file from SpeechTranslatedSubtitleURLForFix.

       {
         "InputConfig": {
           "Type": "Video",
           "Video": "*****a0052ff71efbfd4e7e6c66*****",
           "Subtitle": "https://your-bucket.oss-cn-shanghai.aliyuncs.com/new_subtitle.srt"
         },
         "OutputConfig": {
           "MediaURL": "https://your-bucket.oss-cn-shanghai.aliyuncs.com/video.mp4"
         },
         "EditingConfig": {
           "SourceLanguage": "zh",
           "TargetLanguage": "en",
           "SupportEditing": true,
           "NeedSpeechTranslate": true,
           "SpeechTranslate": {
             "OriginalJobId": "*****b5d5d604916bb898b3066*****",
             "SubtitleConfig": {
               "Type": "Text",
               "FontSize": 95,
               "FontColorOpacity": 1,
               "Color": "#ffffff",
               "X": 0.5,
               "Y": 0.686,
               "Angle": 0,
               "Spacing": 0,
               "TextWidth": 0.9,
               "Font": "Alibaba PuHuiTi",
               "FontColor": "#ffffff",
               "FontFace": {
                 "Bold": false,
                 "Italic": false,
                 "Underline": false
               },
               "SizeRequestType": "RealDim",
               "SubtitleEffects": [],
               "LineSpacing": 0,
               "BorderStyle": 1,
               "Outline": 0,
               "Alignment": "Center"
             }
           }
         }
       }

Post-editing marks

Important

A subtitle line cannot have both the content and the post-editing mark empty (including spaces and tab characters). To display an empty line without changing the audio, add <--onlymodifytext> to the line.

Mark

Effect

Description

<--copy>

Use original audio

Replace the audio within the timestamp range with the original audio from the source video.

<--fsttran>

Use initial translation

Replace the audio within the timestamp range with the initial translation result.

<--mute>

Mute speaker

Mute the speaker's voice within the timestamp range.

<--mute_all>

Mute all audio

Mute all audio within the timestamp range.

<--onlymodifytext>

Modify text only

Change the displayed subtitle text without changing the audio.

SRT file example:

1
00:00:01,000 --> 00:00:03,000
大家好,
<--copy>Hello everyone,

2
00:00:03,500 --> 00:00:06,000
欢迎来到今天的讲座。
<--fsttran>welcome to today's lecture.

3
00:00:07,000 --> 00:00:10,000
我们将讨论未来的技术趋势。
<--mute>We'll discuss the future technology trends.

4
00:00:11,000 --> 00:00:14,000
请大家准备好提问。
<--mute_all>Please get ready to ask questions.

5
00:00:15,000 --> 00:00:18,000
现在让我们开始吧。
<--onlymodifytext>Now, let's get started.

Custom hotword libraries

Use EditingConfig.HotwordLibraryIdList to define hotword mappings and improve transcription and translation accuracy.

How it works

image

A custom hotword library provides:

  • Define term translations: Specify how particular terms should be translated. For example, map "Intelligent Media Services" to "IMS", or map "YYDS" (Chinese internet slang for "the eternal god") to "The GOAT".

  • Improve recognition accuracy: Enhance recognition of industry-specific terms, including geographic locations, product names, homophones, and uncommon words.

API reference:

Code example

Maven dependencies:

<dependency>
  <groupId>org.projectlombok</groupId>
  <artifactId>lombok</artifactId>
  <version>1.18.8</version>
</dependency>
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.78</version>
</dependency>
<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-core</artifactId>
    <version>4.6.3</version>
</dependency>

Create and use a hotword library:

import com.alibaba.fastjson.JSONObject;
import com.alibaba.fastjson.annotation.JSONField;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import org.apache.commons.lang3.StringUtils;

import java.util.ArrayList;
import java.util.List;

import static com.aliyuncs.http.ProtocolType.HTTPS;

public class VideoTranslationHotwordTest {

    // Obtain AccessKey credentials from environment variables.
    static String ak = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
    static String sk = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");

    // Specify the endpoint and region.
    static String endpoint = "ice.cn-hangzhou.aliyuncs.com";
    static String region = "cn-hangzhou";

    @Data
    @Builder
    @AllArgsConstructor
    public static class Hotword {
        @JSONField(name = "Text")
        private String text;
        @JSONField(name = "Weight")
        private Integer weight;
        @JSONField(name = "Language")
        private String language;
        @JSONField(name = "TranspositionResultList")
        private List<TranspositionResult> transpositionResultList;
    }

    @Data
    @Builder
    @AllArgsConstructor
    public static class TranspositionResult {
        @JSONField(name = "TranslatedText")
        private String translatedText;
        @JSONField(name = "TargetLanguage")
        private String targetLanguage;
    }

    public static String createHotwordLibrary(String name, String description, List<Hotword> hotwords) throws ClientException {
        CommonRequest request = new CommonRequest();
        request.setSysMethod(MethodType.POST);
        request.setSysDomain(endpoint);
        request.setSysVersion("2020-11-09");
        request.setHttpContentType(FormatType.JSON);
        request.setSysProtocol(ProtocolType.HTTPS);

        request.setSysAction("CreateHotwordLibrary");
        request.putBodyParameter("Name", name);
        request.putBodyParameter("Description", description);
        request.putBodyParameter("Hotwords", JSONObject.parseArray(JSONObject.toJSONString(hotwords)));
        request.putBodyParameter("UsageScenario", "VideoTranslation");

        DefaultProfile profile = DefaultProfile.getProfile(region, ak, sk);
        IAcsClient ascClient = new DefaultAcsClient(profile);

        CommonResponse commonResponse = ascClient.getCommonResponse(request);
        JSONObject data = JSONObject.parseObject(commonResponse.getData());
        String hotwordLibraryId = data.getString("HotwordLibraryId");
        System.out.println("HotwordLibraryId:" + hotwordLibraryId);
        return hotwordLibraryId;
    }

    public static String getHotwordLibrary(String hotwordLibraryId) throws ClientException {
        CommonRequest request = new CommonRequest();
        request.setSysMethod(MethodType.POST);
        request.setSysDomain(endpoint);
        request.setSysVersion("2020-11-09");
        request.setHttpContentType(FormatType.JSON);
        request.setSysProtocol(ProtocolType.HTTPS);

        request.setSysAction("GetHotwordLibrary");
        request.putBodyParameter("HotwordLibraryId", hotwordLibraryId);
        DefaultProfile profile = DefaultProfile.getProfile(region, ak, sk);
        IAcsClient ascClient = new DefaultAcsClient(profile);
        CommonResponse commonResponse = ascClient.getCommonResponse(request);
        System.out.println(JSONObject.toJSONString(commonResponse.getData()));
        return commonResponse.getData();
    }

    public static void listHotwordLibraries() throws ClientException {
        CommonRequest request = new CommonRequest();
        request.setSysMethod(MethodType.POST);
        request.setSysDomain(endpoint);
        request.setSysVersion("2020-11-09");
        request.setHttpContentType(FormatType.JSON);
        request.setSysProtocol(ProtocolType.HTTPS);

        request.setSysAction("ListHotwordLibraries");
        request.putBodyParameter("UsageScenario", "VideoTranslation");
        request.putBodyParameter("MaxResults", 10);
        DefaultProfile profile = DefaultProfile.getProfile(region, ak, sk);
        IAcsClient ascClient = new DefaultAcsClient(profile);
        CommonResponse commonResponse = ascClient.getCommonResponse(request);
        System.out.println(JSONObject.toJSONString(commonResponse.getData()));
    }

    public static void testCreateHotword() throws InterruptedException, ClientException {
        String hotwordList = "宋察=Song Cha;朴颂=Piao Song;万妮=Wanni;妮妮=Nini;万母=Wanni's mother;万父=Wanni's father;胡恩=Hu En;朱琳=Zhu Lin;素莎=Susha;希娜=Sheena;胡母=Mother Hu;万卡=Wanka;医院院长=Hospital director;死老头=Damn old man;缅甸=Myanmar;济岛=Ji Island;韩城=Hancheng;丘比特=Cupid;璀璨之心=The heart of brilliance;避孕药=contraceptive pill;黄豆酱汤=Soybean paste soup;接待处=Reception;贱人=bitch;虎帮=The Tiger Gang;鸿帮=The Hong Gang;混蛋=Bastart";
        String sourceLanguage = "zh";
        String targetLanguage = "en";

        List<Hotword> hotwords = new ArrayList<>();
        String[] hotwordTranslationArray = hotwordList.split(";");

        for (String hotwordTranslation : hotwordTranslationArray) {
            String originalWord = hotwordTranslation.split("=")[0];
            String translationWord = hotwordTranslation.split("=")[1];

            // Add translation results in multiple target languages.
            List<TranspositionResult> transpositionList = new ArrayList<>();
            transpositionList.add(TranspositionResult.builder().translatedText(translationWord).targetLanguage(targetLanguage).build());
            hotwords.add(Hotword.builder().language(sourceLanguage).weight(4).text(originalWord).transpositionResultList(transpositionList).build());
        }

        // Create a hotword library.
        String hotwordLibraryId = createHotwordLibrary("Name", "Description", hotwords);

        // Retrieve details about a specific hotword library.
        getHotwordLibrary(hotwordLibraryId);

        // List all hotword libraries.
        listHotwordLibraries();
    }

    public static void testVideoTranslation() throws ClientException {

        // Set HotwordLibraryIdList in EditingConfig to the ID of the hotword library.
        String subtitleTranslationConfigStr = "{\n" +
                "\t\"InputConfig\": {\n" +
                "        \"Type\": \"Video\",\n" +
                "        \"Video\": \"https://your-bucket.oss-cn-shanghai.aliyuncs.com/translation/your.mp4\"\n" +
                "    },\n" +
                "    \"OutputConfig\": {\n" +
                "        \"MediaURL\": \"https://your-bucket.oss-cn-shanghai.aliyuncs.com/speech_translate/your_outout.mp4\",\n" +
                "    },\n" +
                "    \"EditingConfig\": {\n" +
                "        \"SourceLanguage\": \"zh\",\n" +
                "        \"TargetLanguage\": \"en\",\n" +
                "        \"SupportEditing\": \"true\",\n" +
                "        \"TextSource\": \"OCR\",\n" +
                "        \"HotwordLibraryIdList\": \"*****00ef45f54a87a61be313a221*****\",\n" +
                "        \"SubtitleTranslate\": {\n" +
                "            \"SubtitleConfig\": {\n" +
                "                \"FontSize\": 40,\n" +
                "                \"Alignment\": \"TopCenter\",\n" +
                "                \"Y\": 0.8,\n" +
                "                \"FontColor\": \"#ffffff\",\n" +
                "                \"TextWidth\": 0.9,\n" +
                "                \"Shadow\": 2,\n" +
                "                \"BackColour\": \"#000000\",\n" +
                "                \"AdaptMode\": \"AutoWrapAtSpacesStrict\"\n" +
                "            }\n" +
                "        }\n" +
                "    }\n" +
                "}";

        // Set HotwordLibraryIdList in EditingConfig to the ID of the hotword library.
        String speechTranslationConfigStr = "{\n" +
                "\t\"InputConfig\": {\n" +
                "        \"Type\": \"Video\",\n" +
                "        \"Video\": \"https://your-bucket.oss-cn-shanghai.aliyuncs.com/translation/your.mp4\"\n" +
                "    },\n" +
                "    \"OutputConfig\": {\n" +
                "        \"MediaURL\": \"https://your-bucket.oss-cn-shanghai.aliyuncs.com/speech_translate/your_outout.mp4\",\n" +
                "    },\n" +
                "    \"EditingConfig\": {\n" +
                "        \"SourceLanguage\": \"zh\",\n" +
                "        \"TargetLanguage\": \"en\",\n" +
                "        \"SupportEditing\": \"true\",\n" +
                "        \"TextSource\": \"ASR\",\n" +
                "        \"HotwordLibraryIdList\": \"*****00ef45f54a87a61be313a221*****\",\n" +
                "        \"SpeechTranslate\": {\n" +
                "            \"SubtitleConfig\": {\n" +
                "                \"FontSize\": 40,\n" +
                "                \"Alignment\": \"TopCenter\",\n" +
                "                \"Y\": 0.8,\n" +
                "                \"FontColor\": \"#ffffff\",\n" +
                "                \"TextWidth\": 0.9,\n" +
                "                \"Shadow\": 2,\n" +
                "                \"BackColour\": \"#000000\",\n" +
                "                \"AdaptMode\": \"AutoWrapAtSpacesStrict\"\n" +
                "            }\n" +
                "        }\n" +
                "    }\n" +
                "}";
        JSONObject subtitleTranslationConfig = JSONObject.parseObject(subtitleTranslationConfigStr);
        JSONObject speechTranslationConfig = JSONObject.parseObject(speechTranslationConfigStr);

        doSubmitVideoTranslationJob(subtitleTranslationConfig.getString("InputConfig"), subtitleTranslationConfig.getString("OutputConfig"), subtitleTranslationConfig.getString("EditingConfig"));
        doSubmitVideoTranslationJob(speechTranslationConfig.getString("InputConfig"), speechTranslationConfig.getString("OutputConfig"), speechTranslationConfig.getString("EditingConfig"));
    }


    private static void doSubmitVideoTranslationJob(String inputConfig, String outputConfig, String editingConfig) throws ClientException {

        try {
            CommonRequest request = new CommonRequest();
            request.setSysMethod(MethodType.POST);
            request.setSysDomain(endpoint);
            request.setSysProtocol(HTTPS);
            request.setSysVersion("2020-11-09");
            request.setHttpContentType(FormatType.JSON);
            request.setSysAction("SubmitVideoTranslationJob");
            request.putBodyParameter("InputConfig", inputConfig);
            request.putBodyParameter("EditingConfig", editingConfig);

            if (StringUtils.isNotBlank(outputConfig)) {
                outputConfig = outputConfig.replaceAll("placeholder", "test" + System.currentTimeMillis());
                request.putBodyParameter("OutputConfig", outputConfig);
            }

            DefaultProfile profile = DefaultProfile.getProfile(region, ak, sk);
            IAcsClient ascClient = new DefaultAcsClient(profile);
            CommonResponse commonResponse = ascClient.getCommonResponse(request);

            JSONObject data = JSONObject.parseObject(commonResponse.getData());

            System.out.printf("doSubmitVideoTranslationJobBody:" + data.toJSONString());
        } catch (ClientException clientException) {
            throw clientException;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

References

Billing

For pricing details, see Billing of video translation.