This topic describes how to integrate RTS SDK with third-party FFmpeg players. ijkplayer is used as an example in this topic.

Prerequisites

You have read the compilation instructions for ijkplayer on GitHub. For more information, visit ijkplayer GitHub.

Background information

  • If you use a third-party player, you must integrate RTS SDK as a plug-in into the player to support the Advanced Real-Time Communication (ARTC) protocol. To minimize development workloads, RTS allows you to integrate RTS SDK as the demuxer plug-in into a player that is based on FFmpeg. This plug-in is encapsulated by rtsdec.c. For information about RTS SDK for mobile clients, see Download SDK.
  • RTS SDK for Android provides dynamic libraries only for ARMv7-A and ARM64.
  • The following procedure is based on ijkplayer tag k:0.8.8.

Integrate RTS SDK for Android with third-party players

  1. Integrate RTS SDK as a plug-in into ijkplayer.

    Two integration methods are provided. The following table describes their advantages and disadvantages.

    Integration method Advantage Disadvantage
    Extend FFmpeg by adding a demuxer plug-in This method simplifies development. You do not need to develop extra logic code for ARTC-based URLs. You must re-compile FFmpeg.
    Extend ijkplayer by adding AVInputFormat If you select this method, you do not need to compile FFmpeg. You must add logic code to ff_player.c.
    • Extend FFmpeg
      1. Modify pull_fork in the init-android.sh script in the ijkplayer directory to delete architectures except for ARMv7-A and ARM64. Then, run the modified script. 013
      2. Modify FF_ACT_ARCHS_64 in the compile-ffmpeg.sh script in the ijkplayer/Android/contrib directory to delete architectures except for ARMv7-A and ARM64. 014
      3. Copy the rtsdec.c file in RTS SDK to the ijkplayer/android/contrib/ffmpeg-arm64/libavformat and ijkplayer/android/contrib/ffmpeg-armv7a/libavformat directories.
      4. Modify the Makefile files to compile rtsdec.c.

        Modify the Makefile files in the ijkplayer/android/contrib/ffmpeg-arm64/libavformat/ and ijkplayer/android/contrib/ffmpeg-arm64/libavformat/ directories, as shown in the following figure.

        914
      5. Modify the allformats.c files to support the ARTC protocol.

        Modify the allformats.c files in the ijkplayer/android/contrib/ffmpeg-arm64/libavformat/ and ijkplayer/android/contrib/ffmpeg-arm64/libavformat/ directories, as shown in the following figure.

        016
            extern AVInputFormat ff_rtc_demuxer;
            av_register_input_format(&ff_rtc_demuxer);
      6. Modify the compilation script of FFmpeg to support pulse-code modulation (PCM) decoding. In this case, RTS SDK is used to export PCM data.

        Add the following code to the module-lite.sh script in the /config directory:

        # aliyun rts
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16be_planar"
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le"
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le_planar"
      7. Compile FFmpeg.

        Add the ANDROID_NDK environment variable. In this example, ANDROID_NDK is set to android-ndk-r12b. Run the ./compile-ffmpeg.sh all command in the ijkplayer/android/contrib/ directory.

      8. Check whether an output file of FFmpeg compilation exists in the ijkplayer/android/contrib/build directory.
      9. Modify ACT_ABI_64 in the ijkplayer/Android/compile-ijk.sh script to delete architectures except for ARMv7-A and ARM64. 017
      10. Add the dynamic libraries of RTS.

        Add the dynamic libraries of RTS to the ijkplayer/android/contrib/build/ffmpeg-arm64/output and ijkplayer/android/contrib/build/ffmpeg-armv7a/output directories based on the architecture. Then, copy the rts_api.h and rts_messages.h header files to the ijkplayer/android/contrib/build/ffmpeg-arm64/output/include and ijkplayer/android/contrib/build/ffmpeg-armv7a/output/include directories.

      11. Import the dynamic libraries of RTS.
        Modify the Android.mk files in the ijkplayer/android/ijkplayer/ijkplayer-armv7a/src/main/jni/ffmpeg/ and ijkplayer/android/ijkplayer/ijkplayer-arm64/src/main/jni/ffmpeg/ directories. 018
        include $(CLEAR_VARS)
        LOCAL_MODULE := rtssdk
        LOCAL_SRC_FILES := $(MY_APP_FFMPEG_OUTPUT_PATH)/libRtsSDK.so
        include $(PREBUILT_SHARED_LIBRARY)
      12. Add the dynamic libraries of RTS to ijkplayer as dependencies.
        Modify the Android.mk files in the ijkplayer/ijkmedia/ijkplayer/ directory. 019
      13. Add the logic code of RTS to ff_fflay.c.

        Configure a function pointer to AVInputFormat if streaming URLs start with artc://. Modify the ff_ffplay.c file in the ijkplayer/ijkmedia/ijkplayer/ directory.

        021
            extern AVInputFormat ff_rtc_demuxer;
            extern int artc_reload(AVFormatContext *ctx);
            extern void av_set_rts_demuxer_funcs(const struct rts_glue_funcs *funcs);
            extern void artc_set_rts_param(char* key, char* value);
            extern long long artc_get_state(AVFormatContext *ctx, int key);
        
            int version = 2;
            const struct rts_glue_funcs* rts_funcs = get_rts_funcs(version);
            // set to ffmpeg plugin
            av_set_rts_demuxer_funcs(rts_funcs);
            artc_set_rts_param((char*)"AutoReconnect", (char*)"false");
      14. Compile ijkplayer.

        Run the ./compile-ijk.sh all command in the ijkplayer/android directory.

    • Extend ijkplayer
      1. Modify the compilation script of FFmpeg to support PCM decoding. Then, compile FFmpeg. In this case, you do not need to add RTS SDK dependencies.

        Add the following code to the module-lite.sh script in the /config directory:

        # aliyun rts
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16be_planar"
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le"
        export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le_planar"
      2. Add RTS dependencies to ijkplayer.

        Add the library file of RTS SDK to the project and modify the Android.mk files. For more information, see Extend FFmpeg.

      3. Add the logic code of RTS to ff_fflay.c. If the streaming URLs start with artc://, call the AVInputFormat function. 053
        if(strncmp(is->filename, "artc://", 7) == 0) {
                extern AVInputFormat ff_rtc_demuxer;
                extern int artc_reload(AVFormatContext *ctx);
                extern void av_set_rts_demuxer_funcs(const struct rts_glue_funcs *funcs);
                extern void artc_set_rts_param(char* key, char* value);
                extern long long artc_get_state(AVFormatContext *ctx, int key);
        
                int version = 2;
                const struct rts_glue_funcs* rts_funcs = get_rts_funcs(version);
                // set to ffmpeg plugin
                av_set_rts_demuxer_funcs(rts_funcs);
                artc_set_rts_param((char*)"AutoReconnect", (char*)"false");
                is->iformat = &ff_rtc_demuxer;
            }
            else {
                if(ffp->iformat_name)
                    is->iformat = av_find_input_format(ffp->iformat_name);
            }
      4. Copy rtsdec.c to the ijkplayer/ijkmedia/ijkplayer directory. Modify the android.mk file in the directory to add the rtsdec.c file.
        LOCAL_SRC_FILES += rtsdec.c
  2. Integrate ijkplayer in a project.
    1. Import the RtsNetSDK.jar file to the project.
    2. Add System.loadLibrary("RtsSDK"); to the required activity.
      static {
          System.loadLibrary("RtsSDK");
      }
      You must import the following modules: ijkplayer-arm64, ijkplayer-armv7a, and ijkplayer-java. 060
  3. Call an API operation of ijkplayer.
    1. Create ijkplayer.
      mIjkPlayer = new IjkMediaPlayer();
    2. Configure callbacks.
      mIjkPlayer.setOnPreparedListener(new IMediaPlayer.OnPreparedListener() {
          @Override
          public void onPrepared(IMediaPlayer iMediaPlayer) {
              
          }
      });
      
      mIjkPlayer.setOnInfoListener(new IMediaPlayer.OnInfoListener() {
          @Override
          public boolean onInfo(IMediaPlayer iMediaPlayer, int arg1, int arg2) {
              
      });
      
      mIjkPlayer.setOnVideoSizeChangedListener(new IMediaPlayer.OnVideoSizeChangedListener() {
          @Override
          public void onVideoSizeChanged(IMediaPlayer iMediaPlayer, int width, int height, int sarNum, int sarDen) {
             
          }
      });
      
      mIjkPlayer.setOnErrorListener(new IMediaPlayer.OnErrorListener() {
          @Override
          public boolean onError(IMediaPlayer iMediaPlayer, int framework_err, int impl_err) {
              
              return false;
          }
      });
    3. Configure the UI view.
      mIjkPlayer.setSurface(surface);
    4. Configure the playback source.
      mIjkPlayer.setDataSource("artc://xxx");
    5. Configure playback control.
      public void prepare() {
          if(mIjkPlayer != null){
              mIjkPlayer.prepareAsync();
          }
      }
      
      @Override
      public void start() {
          if(mIjkPlayer != null){
              mIjkPlayer.start();
          }
      }
      
      public void pause() {
          if(mIjkPlayer != null){
              mIjkPlayer.pause();
          }
      }
      
      public void stop() {
          if(mIjkPlayer != null){
              mIjkPlayer.stop();
          }
      }
      public void release() {
          if(mIjkPlayer != null){
              mIjkPlayer.release();
          }
      }

Integrate RTS SDK for iOS with third-party players

  1. Integrate RTS SDK as a plug-in into ijkplayer.

    Two integration methods are provided. The following table describes their advantages and disadvantages.

    Integration method Advantage Disadvantage
    Extend FFmpeg by adding a demuxer plug-in This method simplifies development. You do not need to develop extra logic code for ARTC-based URLs. You must re-compile FFmpeg.
    Extend ijkplayer by adding AVInputFormat If you select this method, you do not need to compile FFmpeg. You must add logic code to ff_player.c.
    • Extend FFmpeg
      1. Run the init-ios.sh command in the ijkplayer directory.
      2. Go to the ijkplayer/ios directory. Copy rtsdec.c to the ffmpeg-$arch/libavformat directory. Then, modify libavformat/Makefile and allformat.c. For more information, see Extend FFmpeg.
      3. Compile ijkplayer.

        Run the ./compile-ffmpeg.sh all command in the ijkplayer/ios directory.

      4. Check whether the output file of FFmpeg compilation exists in the ijkplayer/ios/build/universal/ directory.
      5. Add RtsSDK.framework to the ijkplayer//ios/IJKMediaDemo/IJKMediaDemo directory.
      6. Open ios/IJKMediaDemo/IJKMediaDemo.xcodeproj in Xcode.
      7. Add RtsSDK.framework as a dependency to IJKMediaPlayer.xcodeproj. 264
      8. Add the logic code of RTS to ff_fflay.c. For more information, see Extend FFmpeg.
    • Extend ijkplayer
      1. In this case, you do not need to use RTS SDK to extend FFmpeg.
      2. Add RTS dependencies to ijkplayer. For more information, see Extend FFmpeg.
      3. Import rtsdec.c to your project. 270
      4. Modify the code in ff_ffplay.c. For more information, see Extend FFmpeg.
  2. Integrate ijkplayer and RTS SDK in a project
    Import the frameworks of ijkplayer and the frameworks of RTS SDK to your project. 273
  3. Call an API operation of ijkplayer.
    1. Create ijkplayer.
      _ijkPlayer = [[IJKFFMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:_url] withOptions:options];
      _ijkPlayer.view.autoresizingMask = UIViewAutoresizingFlexibleWidth|UIViewAutoresizingFlexibleHeight;
      _ijkPlayer.view.frame = self.basePlayerView.playerView.bounds;
      _ijkPlayer.scalingMode = IJKMPMovieScalingModeAspectFit; // Configure the scaling mode.
      _ijkPlayer.shouldAutoplay = YES; // Enable autoplay.
    2. Configure playback control.
      - (void)prepareToPlay;
      - (void)play;
      - (void)pause;
      - (void)stop;

Integrate RTS SDK for Windows with third-party players

  1. Download and decompress the RTS SDK for Windows.
  2. Copy the rtsdec.c and rtsenc.c files to the libavformat directory of the FFmpeg player.
  3. Modify the Makefile file.

    Add the descriptions of rtsdec.o and rtsenc.o to the Makefile file, as shown in the following figure:

    001
  4. Modify the allformats.c file.
    002
  5. Compile and run the code.

    ./ffplay -i "artc:// Video URL"

Integrate RTS SDK for Mac with third-party players that are based on FFmpeg

  1. Download and decompress the RTS SDK for Mac.
  2. Copy the rtsdec.c and rtsenc.c files to the libavformat directory of the FFmpeg player.
  3. Modify the Makefile file.

    Add the descriptions of rtsdec.o and rtsenc.o to the Makefile file, as shown in the following figure:

    001
  4. Modify the allformats.c file.
    002
  5. Compile and run the code.

    ./ffplay -i "artc:// Video URL"