Intelligent Media Services:Video translation

Method 1: Use the console

1. Go to the Video Translation page in the Intelligent Media Services console.

2. In the upper-left corner, select a region based on your requirements.

3. Click Create Translation Task to go to the Create Translation Task page.

4. Configure the following parameters:

   • Translation Method: Select Subtitle Translation, Speech Translation, or Lip-sync Translation.

   • Select Source File: Upload the video file that you want to translate. MP4, WebM, and MOV formats are supported.

   • Subtitle Source: Configure options such as whether to erase the original subtitles and the subtitle source. Supported sources are OCR, ASR, and Specified SRT File.

     • OCR: If your video contains hardcoded subtitles but you do not have a subtitle file, use OCR to extract subtitle text from the video frames. To improve efficiency and accuracy, you can also specify an OCR Range.

     • ASR: If your video does not have subtitles, use ASR to generate subtitles by transcribing speech from the audio track.

     • OCR/ASR: This is a hybrid method that combines OCR and ASR. It prioritizes OCR for subtitle extraction and falls back to ASR if OCR fails.

     • The system does not support a fully automatic subtitle source detection mode. If you are not sure whether the source video contains audio, or you want the task to adapt automatically, set the TextSource parameter to OCR_ASR. With this setting, the system first attempts OCR to extract subtitle text from the video frames, and automatically falls back to ASR to transcribe speech if OCR fails.

     • If a task fails because the video contains only background music (BGM) and no valid human speech, set the NeedSpeechTranslate parameter to false to disable speech translation. If you still need on-screen text subtitles, use this setting together with the OCR or OCR_ASR subtitle source mode.

     • Specified SRT File: If you have an existing subtitle file (such as in .srt format), you can upload it to be used as the source.

   • Target Language: You can select multiple target languages at a time. After you submit the translation task, the system generates a video file for each selected language.

   • Storage Directory and File Name: Specify the storage location and name for the translated files.

5. After you confirm that all parameters are correct, click Submit Translation Task to create the task.

6. You can monitor the task status, parameters, and results in the task list. When the task status changes to Processed, click View Details to view the task details.

   The details page has three sections: Basic Settings, Advanced Settings, and Output. The Basic Settings section displays information such as the source file type, translation method, target languages (for example, Chinese > English, Japanese), creation time, and task status. The Advanced Settings section displays configurations such as whether to erase source subtitles, the target subtitle type, the subtitle source, and whether post-editing is enabled. The Output section lists the output path for each target language and provides Edit and Download options.

Method 2: Use Cloud Editing

Before you begin

If you are unfamiliar with Cloud Editing, we recommend that you first review the Cloud Editing operation guides.

Procedure

1. Go to the Cloud Editing page.

2. In the upper-left corner, select a region based on your requirements.

3. On the Video Editing Project tab, click Create Editing Project. Follow the on-screen instructions to create the project, and then click Edit in the project list to open the Cloud Editing workspace.

4. In the upper-left Materials panel, click Import. In the Add Material dialog box on the right, select the files you want to translate and add them to the library. Then, add a file to a track in the timeline below by clicking the icon or by dragging the file directly. The Cloud Editing workspace provides several function menus on the left, such as Materials, Stock Media, Digital Avatars, Smart Dubbing, Subtitles, and Stickers. You can add video, audio, or image files by clicking the + Import button in the upper-right corner.

5. On the timeline, select the audio or video track that you want to translate. In the properties panel on the right, click AI translation to open the video translation panel. Follow the on-screen instructions to configure settings such as translation type, subtitle extraction, target languages, and subtitle erasure, and then click Submit. This example uses speech translation.

6. Wait a few minutes. The translation results will appear on the timeline. After the task is complete, three tracks appear in the timeline area: a subtitle track with the target-language subtitles, a video track with the original subtitles removed, and an audio track with the translated audio and its waveform.

7. When the task is complete, click Export > Export Video in the upper-right corner. The system displays a dialog box for composing the video. Configure the parameters as prompted and click OK to compose and export the translated video. You can manage the translated clips in batches in the Clip Settings panel on the right.

Method 3: Create a video translation via OpenAPI

1. Create a translation task

   Set the API parameters based on your business requirements and call the SubmitVideoTranslationJob operation. Before you submit the request, make sure you have read Parameters and examples for video translation to correctly configure the parameters.

   The system does not support a fully automatic subtitle source detection mode. If you are not sure whether the source video contains audio, or you want the task to adapt automatically, set TextSource to OCR_ASR. The system first attempts OCR to extract subtitle text from the video frames and automatically falls back to ASR to transcribe speech if OCR fails. If a task fails because the video contains only background music (BGM) and no valid human speech, set NeedSpeechTranslate to false to disable speech translation. If you still need on-screen text subtitles, use this setting together with the OCR or OCR_ASR subtitle source mode.

2. Query the result of a single translation task:

   Call the GetSmartHandleJob operation to get the status and result of a specific video translation task. This API allows you to retrieve detailed information about a specified task, including its processing progress, completion time, and final output URL.

3. Query the list of translation tasks

   If you need to view all ongoing or completed video translation tasks, you can call the ListSmartJobs operation to list these tasks.

4. Delete a translation task

   Call the DeleteSmartJob operation to delete completed tasks and release system resources.

Method 1: Use OpenAPI

You can correct the results of speech translation by calling the relevant API operation. For more information, see Speech translation - Manual correction.

Method 2: Use Cloud Editing (Web SDK)

Before you begin

If you are unfamiliar with Cloud Editing, we recommend that you first review the Cloud Editing operation guides.

Procedure

1. Go to the Video Translation page, and select the task that you want to correct.

2. In the Actions column, click Edit to open the corresponding Cloud Editing project. For subsequent operations, refer to the video tutorial below:

Align subtitle timing with the audio waveform

For example, if you want to split the translated subtitle "Great where are you" into two segments, "Great" and "where are you", adjust the start and end times to align the subtitle start and end points with the troughs of the audio waveform. This optimizes the post-editing result for speech translation.

Control subtitle word count during post-editing

When post-editing translated subtitles, ensure the word count of the edited subtitle does not exceed 1.5 times the word count of the original translation. Otherwise, the speech in the post-edited audio may be too fast.

For example, a line from the initial translation is: Let's talk about this later. We need to go home now

Incorrect example: Let's discuss this matter in more detail at a later time. Right now, we should focus on heading back home as it's important to ensure we get there safely and in good time. We can revisit this conversation when we are both more relaxed and have ample opportunity to explore all the aspects thoroughly.

Correct example: Let's pick this up another time. We should be going home now.

How long does it take to process a video translation task?

For a video shorter than one minute, processing usually takes several minutes to more than ten minutes. In general, the processing time is about three times the duration of the source video, excluding any time spent waiting in the queue. The actual processing time is affected by factors such as the video bitrate, frame rate, current queue load, and the complexity of the translation algorithm. After you submit a task, we recommend that you call the [GetSmartHandleJob](t2161217.xdita#) operation for querying the result of a single translation task to get the final result instead of waiting on the page.

Why is the output video's aspect ratio abnormal or distorted? How do I keep the original aspect ratio?

• Default behavior: In a standard request, do not include the Width and Height fields in OutputConfig (when you serialize the request, skip fields whose value is 0 or null). By default, the system outputs the video by using the aspect ratio of the source video.

• Cause of the issue: If you explicitly set Width or Height in OutputConfig to 0, or if the source video uses a non-standard format that relies on the sample aspect ratio (SAR) for stretching (for example, a PAL widescreen DVD source), the aspect ratio calculation may be incorrect.

• Solution: For a special-format video, such as one with black bars or a non-standard SAR, that is not resolved by relying on the default source aspect ratio, first use media analysis to obtain the actual display aspect ratio, and then explicitly set matching Width and Height values in OutputConfig to force the correct aspect ratio.

How do I troubleshoot a failed video translation task or a video that cannot be downloaded?

• Download fails or the page responds slowly: Press F12 to open the browser developer tools and check the Network and Console tabs for errors. Try switching to a different browser (such as Safari) to rule out browser-side blocking, and check whether the API response includes a RequestId.

• Content moderation failure: If a task fails because it was blocked for containing prohibited content, check and revise the content to ensure it is compliant, and then resubmit the task.

• The error src property must be a valid json object is returned: This error usually occurs because the request is missing output configuration. For example, the Timeline parameter contains only input tracks without any output configuration. Refer to the official examples to complete the OutputConfig parameter.

• Calling video translation from a VOD workflow fails: VOD workflows do not support calling video translation directly, because you cannot configure the translation language in a VOD workflow. Use OpenAPI to call video translation instead.