Integrate External Subtitles into Android & iOS via Player SDK - ApsaraVideo VOD

ApsaraVideo Player SDK lets you add, parse, and render external subtitle streams in WebVTT, SRT, and ASS formats. The software development kit (SDK) also supports parsing and rendering subtitle streams embedded in M3U8 files. This topic describes how to implement this feature on Android and iOS platforms.

Important

For all code and implementation details in this topic, refer to the API-Example project and adapt it based on best practices.

For a specific implementation, use the solutions in this topic and refer to the source code of the ExternalSubtitle module in the corresponding API-Example-Android and API-Example-iOS projects.

Limits

Only external subtitle files in WebVTT, SRT, and ASS formats are supported.

Rendering example

To package subtitle streams for rendering, see Best practices for multi-subtitle transcoding and packaging.

Key implementation on Android

Render external subtitles

Import the dependency libraries.

import com.aliyun.subtitle.SubTitleBase;
import com.cicada.player.utils.webVtt.VttSubtitleView;
import com.aliyun.player.IPlayer;

Set the display view.

vttSubtitleView = new VttSubtitleView(getContext());
vttSubtitleView.setId(R.id.cicada_player_vtt_subtitle);


// Set the display position of the subtitles. Add them to the center of the layout.
FrameLayout.LayoutParams params = new FrameLayout.LayoutParams(
        FrameLayout.LayoutParams.WRAP_CONTENT,
        FrameLayout.LayoutParams.WRAP_CONTENT
);
params.gravity = Gravity.CENTER; // Add to the center of the layout.

// Add the VTT subtitle view to the root layout view.
mRootFrameLayout.addView(mVttSubtitleView, params);

Set the callbacks.

// In the following example, mVideoListPlayer is a listPlayer. The process is the same if you use aliPlayer.
// 1. Set OnSubtitleDisplayListener.
mVideoListPlayer.setOnSubtitleDisplayListener(new IPlayer.OnSubtitleDisplayListener() {
    @Override
    public void onSubtitleExtAdded(int trackIndex, String url) {
      // Because we are only testing the addition of a single VTT file, select the subtitle track directly after the VTT file is added.
        mVideoListPlayer.selectExtSubtitle(trackIndex, true);
    }

    @Override
    public void onSubtitleShow(int trackIndex, long id, String data) {
        if (vttSubtitleView != null) {
            vttSubtitleView.show(id, data);
        }
    }

    @Override
    public void onSubtitleHide(int trackIndex, long id) {
        if (vttSubtitleView != null) {
            vttSubtitleView.dismiss(id);
        }
    }

    @Override
    public void onSubtitleHeader(int trackIndex, String header) {
        if (vttSubtitleView != null) {
            vttSubtitleView.setVttHeader(header);
        }
    }
});

// 2. Set VideoSizeChangedListener. This is a required step. The data must be passed back to vttSubtitleView.
mVideoListPlayer.setOnVideoSizeChangedListener(new IPlayer.OnVideoSizeChangedListener() {
    @Override
    public void onVideoSizeChanged(int width, int height) {
        int viewWidth = getWidth();
        int viewHeight = getHeight();
        IPlayer.ScaleMode mode = mVideoListPlayer.getScaleMode();
        SubTitleBase.VideoDimensions videoDimensions = SubTitleBase.getVideoDimensionsWhenRenderChanged(width, height, viewWidth, viewHeight, mode);
        vttSubtitleView.setVideoRenderSize(videoDimensions.videoDisplayWidth, videoDimensions.videoDisplayHeight);
    }
});

After the player executes the prepare/moveTo/moveToNext/moveToPrev operation, clear the display view.

source = getSource(someParams); // pseudocode

mVideoListPlayer.moveTo(mCurrentUUID + "");
if (vttSubtitleView != null) {
    vttSubtitleView.clearAll();
}

if (!source.getExtSubtitleUrl().isEmpty()) {
    mCurrentExtSubtitle = source.getExtSubtitleUrl(); // Get the external subtitle URL.
}

After the player's onPrepared callback is invoked, add the subtitle file.

// Currently, you can call addExtSubtitle only after the player's onPrepared callback is invoked.
mVideoListPlayer.addExtSubtitle(mCurrentExtSubtitle);

Render M3U8-packaged subtitles

Important

Do not render M3U8-packaged subtitles and external subtitles together.

Import the dependency libraries.

import com.aliyun.subtitle.SubTitleBase;
import com.cicada.player.utils.webVtt.VttSubtitleView;
import com.aliyun.player.IPlayer;

Set the display view.

vttSubtitleView = new VttSubtitleView(getContext());
vttSubtitleView.setId(R.id.cicada_player_vtt_subtitle);


// Set the display position of the subtitles. Add them to the center of the layout.
FrameLayout.LayoutParams params = new FrameLayout.LayoutParams(
        FrameLayout.LayoutParams.WRAP_CONTENT,
        FrameLayout.LayoutParams.WRAP_CONTENT
);
params.gravity = Gravity.CENTER; // Add to the center of the layout.

// Add the VTT subtitle view to the root layout view.
mRootFrameLayout.addView(mVttSubtitleView, params);

Set the callbacks.

// In the following example, mVideoListPlayer is a listPlayer. The process is the same if you use aliPlayer.
// 1. Set OnSubtitleDisplayListener.
mVideoListPlayer.setOnSubtitleDisplayListener(new IPlayer.OnSubtitleDisplayListener() {
    @Override
    public void onSubtitleExtAdded(int trackIndex, String url) {
      // Because we are only testing the addition of a single VTT file, select the subtitle track directly after the VTT file is added.
        mVideoListPlayer.selectExtSubtitle(trackIndex, true);
    }

    @Override
    public void onSubtitleShow(int trackIndex, long id, String data) {
        if (vttSubtitleView != null) {
            vttSubtitleView.show(id, data);
        }
    }

    @Override
    public void onSubtitleHide(int trackIndex, long id) {
        if (vttSubtitleView != null) {
            vttSubtitleView.dismiss(id);
        }
    }

    @Override
    public void onSubtitleHeader(int trackIndex, String header) {
        if (vttSubtitleView != null) {
            vttSubtitleView.setVttHeader(header);
        }
    }
});

// 2. Set VideoSizeChangedListener. This is a required step. The data must be passed back to vttSubtitleView.
mVideoListPlayer.setOnVideoSizeChangedListener(new IPlayer.OnVideoSizeChangedListener() {
    @Override
    public void onVideoSizeChanged(int width, int height) {
        int viewWidth = getWidth();
        int viewHeight = getHeight();
        IPlayer.ScaleMode mode = mVideoListPlayer.getScaleMode();
        SubTitleBase.VideoDimensions videoDimensions = SubTitleBase.getVideoDimensionsWhenRenderChanged(width, height, viewWidth, viewHeight, mode);
        vttSubtitleView.setVideoRenderSize(videoDimensions.videoDisplayWidth, videoDimensions.videoDisplayHeight);
    }
});

After the player executes the prepare/moveTo/moveToNext/moveToPrev operation, clear the display view.

source = getSource(someParams); // pseudocode

mVideoListPlayer.moveTo(mCurrentUUID + "");
if (vttSubtitleView != null) {
    vttSubtitleView.clearAll();
}

if (!source.getExtSubtitleUrl().isEmpty()) {
    mCurrentExtSubtitle = source.getExtSubtitleUrl(); // Get the external subtitle URL.
}

Set the onTrackReady callback to obtain the substream information.

mAliPlayer.setOnTrackReadyListener(new IPlayer.OnTrackReadyListener() {
    @Override
    public void onTrackReady(MediaInfo mediaInfo) {
        List<TrackInfo>  trackInfos = mediaInfo.getTrackInfos();
        for (TrackInfo trackInfo : trackInfos) {
            if (trackInfo.getType() == TrackInfo.Type.TYPE_SUBTITLE) {
            // TODO 
            }
        }
    }
});

After the player's onPrepared callback is invoked, call selectTrack to switch the subtitle track.
```
// index is the TrackIndex of the target subtitle stream.
mAliPlayer.selectTrack(index);
```

Key implementation on iOS

Render external subtitles

Set the callback.

/**
 @brief An external subtitle is added.
 @param player The player pointer.
 @param trackIndex The index of the subtitle track to be displayed.
 @param URL The subtitle URL.
 */
- (void)onSubtitleExtAdded:(AliPlayer*)player trackIndex:(int)trackIndex URL:(NSString *)URL {
    NSLog(@"onSubtitleExtAdded: %@, trackIndex: %d", URL, trackIndex);
    [self.listPlayer selectExtSubtitle:trackIndex enable:YES];
}

After the player's onPrepared callback is invoked, add the subtitle file.

if (self.currentModel.extSubtitleUrl != Nil) {
    [self.listPlayer addExtSubtitle:self.currentModel.extSubtitleUrl];
}

Render M3U8-packaged subtitles

Important

Do not render M3U8-packaged subtitles and external subtitles together.

Set the callback.

/**
 @brief An external subtitle is added.
 @param player The player pointer.
 @param trackIndex The index of the subtitle track to be displayed.
 @param URL The subtitle URL.
 */
- (void)onSubtitleExtAdded:(AliPlayer*)player trackIndex:(int)trackIndex URL:(NSString *)URL {
    NSLog(@"onSubtitleExtAdded: %@, trackIndex: %d", URL, trackIndex);
    [self.listPlayer selectExtSubtitle:trackIndex enable:YES];
}

Set the onTrackReady callback to obtain the substream information.

- (void)onTrackReady:(AliPlayer*)player info:(NSArray<AVPTrackInfo*>*)info {
  //
}

After the player's prepare done callback is invoked, call selectTrack to switch the subtitle track.
```
// index is the TrackIndex of the target subtitle stream.
[self.player selectTrack:index];
```

Implement custom rendering based on WebVTT streams

ApsaraVideo Player supports parsing and rendering standard WebVTT streams on both Android and iOS. The player reads CSS styles to create unified subtitle styles on both platforms. It also supports multiple text styles in a single file, which is useful for distinguishing character dialogue, emphasizing key content, and creating special visual effects.

The following code shows an example of a WebVTT file:

REGION: Defines the display area.
STYLE: Defines the font style. The first item is the default style.

WEBVTT

REGION
id:bottom
width:70.000000%
lines:3
viewportanchor:15.000000%,95.000000%
regionanchor:0.000000%,100.000000%
scroll:up

STYLE
::cue {
  color: white;
  font-size: 70%;
  font-family: Noto Sans;
  font-weight: bold;
  background-color: rgba(0, 0, 0, 0.2);
}
::cue(.font1) {
  color: rgba(255, 255, 255, 1);
  font-weight: bold;
  font-family: Noto Sans;
  outline-width: 3px;
  outline-color: rgba(0, 0, 0, 1);
}
::cue(.font2) {
  color: rgba(252, 255, 101, 1);
  font-style: bold;
  font-family: Noto Sans;
  outline-width: 3px;
  outline-color: rgba(0, 0, 0, 1);
}
::cue(.font3) {
  color: rgba(255, 191  23, 1);
  font-style: bold;
  font-family: Noto Sans;
  outline-width: 3px;
  outline-color: rgba(183, 28, 28, 1);
}

00:00:00.200 --> 00:00:01.800 region:bottom align:center
Defendant Bo Hanshi

00:00:01.800 --> 00:00:03.200 region:bottom align:center
Do you have anything else to say

00:00:04.100 --> 00:00:04.800 region:bottom align:center
I

00:00:11.200 --> 00:00:12.200 region:bottom align:center
<v.font1>I have nothing to say</v.font1>

00:00:14.500 --> 00:00:16.200 region:bottom align:center
Defendant Bo Hanshi

...

To specify a style for a text segment, use the following setting:

<v.font1>I have nothing to say</v.font1>