:iOS

Procedure

1. Download and decompress the source code package of ijkplayer. To obtain the download link, visit the ijkplayer page.

2. Download and decompress the Native RTS SDK package. For more information about the download link, see SDK download.

3. Integrate Native RTS SDK as a plug-in into ijkplayer. The following table describes the two integration methods.

   Integration method	Description	Advantage	Disadvantage
   Extend FFmpeg.	Extend FFmpeg by adding a demuxer plug-in.	This method simplifies development. You do not need to develop extra logic code for Alibaba Real-Time Communication (ARTC)-based URLs.	You must re-compile FFmpeg.
   Extend ijkplayer.	Extend ijkplayer by adding an AVInputFormat struct.	You do not need to compile FFmpeg.	You must add logic code to the ff_ffplay.c file.

   Extend FFmpeg

   1. In the root directory of ijkplayer, run the ./init-ios.sh command for initialization.

      Take note that Xcode 14 no longer supports architectures such as ARMv7, i386, and x86_64. You must delete the code of these architectures from the init-ios.sh file.

      The following compilation script provides an example when Xcode 14 is used:

      # FF_ALL_ARCHS_IOS8_SDK="armv7 arm64 i386 x86_64"
      FF_ALL_ARCHS_IOS11_SDK="arm64"
      FF_ALL_ARCHS=$FF_ALL_ARCHS_IOS11_SDK

   2. Open the iOS directory of ijkplayer. If you compiled a command for FFmpeg and the ffmpeg-$arch folder from the build directory appears in the iOS directory, run the ./compile-ffmpeg.sh clean command to clean the iOS directory.

   3. Copy the rtsdec.c file in Native RTS SDK to the ijkplayer/ios/ffmpeg-$arch/libavformat directory.

   4. Modify the Makefile files and compile the rtsdec.c file.

   5. Modify the allformats.c file.

      Modify the ijkplayer/ios/ffmpeg-$arch/libavformat/allformats.c file. By default, the Alibaba Real-Time Communication (ARTC) protocol is supported.

      

   6. Modify the FFmpeg compilation script ijkplayer/config/module-lite.sh to support pulse-code modulation (PCM) decoding. This operation is required because Native RTS SDK exports PCM data.

      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16be_planar"
      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le"
      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le_planar"

      Optional: If you want to support streams over the HTTPS protocol, add the following code to the file to enable OpenSSL support.

      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-openssl"

   7. Compile FFmpeg.

      Run the ./compile-ffmpeg.sh all command in the ijkplayer/ios directory. At the same time, delete the code of unsupported architectures from the script, similar to Step a. After you compile FFmpeg, make sure that an output file of FFmpeg compilation exists in the ijkplayer/ios/build/universal directory.

      Optional: If you want to support streams over the HTTPS protocol, run the ./compile-openssl.sh all command first. At the same time, delete the code of unsupported architectures from the script, similar to Step a. After you compile FFmpeg, the libssl.a file and libcrypto.a file are generated in the ijkplayer/ios/build/universal/lib directory. Then, compile ./compile-ffmpeg.sh all.

   8. Copy the RtsSDK.framework file in Native RTS SDK to the ijkplayer/ios/IJKMediaDemo/IJKMediaDemo directory.

   9. Open ios/IJKMediaDemo/IJKMediaDemo.xcodeproj in Xcode.

   10. Add RtsSDK.framework as a dependency.

   11. Add the dynamic frameworks of Native RTS SDK.

       Import the header files rts_api.h and rts_messages.h of Native RTS SDK to the ijkplayer/ios/build/universal/lib directory.

       

   12. Add the logic code of RTS to the ff_ffplay.c file.

       Import the header file rts_api.h.

       #include "rts_api.h"

       Modify the ijkplayer/ijkmedia/ijkplayer/ff_ffplay.c file by configuring the AVInputFormat function pointer for ARTC.

       extern AVInputFormat ff_rtc_demuxer;
       extern int artc_reload(AVFormatContext *ctx);
       extern void av_set_rts_demuxer_funcs(const struct rts_glue_funcs *funcs);
       extern void artc_set_rts_param(char* key, char* value);
       extern long long artc_get_state(AVFormatContext *ctx, int key);
       
       int version = 2;
       const struct rts_glue_funcs* rts_funcs = get_rts_funcs(version);
       // set to ffmpeg plugin
       av_set_rts_demuxer_funcs(rts_funcs);
       artc_set_rts_param((char*)"AutoReconnect", (char*)"false");

   Extend ijkplayer

   1. In the root directory of ijkplayer, run the ./init-ios.sh command for initialization.

      Take note that Xcode 14 no longer supports architectures such as ARMv7, i386, and x86_64. You must delete the code of these architectures from the init-ios.sh file.

      The following compilation script provides an example when Xcode 14 is used:

      # FF_ALL_ARCHS_IOS8_SDK="armv7 arm64 i386 x86_64"
      FF_ALL_ARCHS_IOS11_SDK="arm64"
      FF_ALL_ARCHS=$FF_ALL_ARCHS_IOS11_SDK

   2. Open the iOS directory of ijkplayer. If you compiled a command for FFmpeg and the ffmpeg-$arch folder from the build directory appears in the iOS directory, run the ./compile-ffmpeg.sh clean command to clean the iOS directory.

   3. Execute the FFmpeg compilation script ijkplayer/config/module-lite.sh to support PCM decoding. This operation is required because Native RTS SDK exports PCM data.

      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16be_planar"
      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le"
      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-decoder=pcm_s16le_planar"

      Optional: If you want to support streams over the HTTPS protocol, add the following code to the file to enable OpenSSL support.

      export COMMON_FF_CFG_FLAGS="$COMMON_FF_CFG_FLAGS --enable-openssl"

   4. Compile FFmpeg.

      Run the ./compile-ffmpeg.sh all command in the ijkplayer/ios directory. At the same time, delete the code of unsupported architectures from the script, similar to Step a. After you compile FFmpeg, make sure that an output file of FFmpeg compilation exists in the ijkplayer/ios/build/universal directory.

      Optional: If you want to support streams over the HTTPS protocol, run the ./compile-openssl.sh all command first. At the same time, delete the code of unsupported architectures from the script, similar to Step a. After you compile FFmpeg, the libssl.a file and libcrypto.a file are generated in the ijkplayer/ios/build/universal/lib directory. Then, compile ./compile-ffmpeg.sh all.

   5. Copy the RtsSDK.framework file in Native RTS SDK to the ijkplayer/ios/IJKMediaDemo/IJKMediaDemo directory.

   6. Import the rtsdec.c file in Native RTS SDK into your project.

      Import the header files rts_api.h and rts_messages.h of Native RTS SDK to the ijkplayer/ios/build/universal/lib directory.

      

   7. Add the logic code of RTS to the ff_ffplay.c file.

      Import the header file rts_api.h.

      #include "rts_api.h"

      Modify the ijkplayer/ijkmedia/ijkplayer/ff_ffplay.c file by configuring the AVInputFormat function pointer for ARTC.

      if(strncmp(is->filename, "artc://", 7) == 0) {
          extern AVInputFormat ff_rtc_demuxer;
        extern int artc_reload(AVFormatContext *ctx);
        extern void av_set_rts_demuxer_funcs(const struct rts_glue_funcs *funcs);
        extern void artc_set_rts_param(char* key, char* value);
        extern long long artc_get_state(AVFormatContext *ctx, int key);
      
        int version = 2;
        const struct rts_glue_funcs* rts_funcs = get_rts_funcs(version);
        // set to ffmpeg plugin
        av_set_rts_demuxer_funcs(rts_funcs);
        artc_set_rts_param((char*)"AutoReconnect", (char*)"false");
        is->iformat = &ff_rtc_demuxer;
      }
      else {
        if(ffp->iformat_name)
          is->iformat = av_find_input_format(ffp->iformat_name);
      }

4. Compile the IJKMediaPlayer project.

   After you extend FFmpeg or extend ijkplayer, use the Archive compiler to run the ijkplayer/ios/IJKMediaPlayer/IJKMediaPlayer.xcodeproj project. The IJKMediaFramework.framework file is generated in the Products directory.

   Optional: If you want to support streams over the HTTPS protocol, before you execute the compilation, choose Build Phases > Link Binary With Libraries for the project in Xcode and import the libssl.a and libcrypto.a files in the ijkplayer/ios/build/universal/lib directory that are generated in the preceding steps.

   

5. Import the RtsSDK.framework file in Native RTS SDK and the IJKMediaFramework.framework file in ijkplayer into the project.

   

6. Call the methods provided by ijkplayer to use the RTS feature.

   • Create ijkplayer

     // Generate a URL over the ARTC protocol.
     _url = @"artc://xxxx";
     
     // Create a custom playerView.
     [self.view addSubview:self.playerView];
     ...
     
     IJKFFOptions *options = [IJKFFOptions optionsByDefault];
     // Configure hardware decoding.
     [options setPlayerOptionIntValue:1 forKey:@"videotoolbox"];
     // Configure software decoding.
     //[options setPlayerOptionIntValue:0 forKey:@"videotoolbox"];
     
     _ijkPlayer = [[IJKFFMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:_url] withOptions:options];
     _ijkPlayer.view.autoresizingMask = UIViewAutoresizingFlexibleWidth|UIViewAutoresizingFlexibleHeight;
     _ijkPlayer.view.frame = self.playerView.bounds;
     _ijkPlayer.scalingMode = IJKMPMovieScalingModeAspectFit; // Specify the scaling mode.
     _ijkPlayer.shouldAutoplay = YES; // Enable autoplay.
     [self.playerView addSubview:_ijkPlayer.view];

   • Set playback control

     • Start playback.

       You must call the method in the main thread. We recommend that you stop playback and release the player before you restart playback each time.

       dispatch_async(dispatch_get_main_queue(), ^{
           [_ijkPlayer prepareToPlay];
           [_ijkPlayer play];
       });

     • Stop playback and release the player.

       // Stop playback.
       [_ijkPlayer stop];
       
       // Release the player.
       [_ijkPlayer shutdown];
       _ijkPlayer = nil;

Listen to RTS events

• Listen to the message callback of Native RTS SDK in ijkplayer

  Open the ijkplayer/ios/IJKMediaPlayer/IJKMediaPlayer.xcodeproj project and add adapted methods to relevant files.

  1. Modify the ff_ffplay.c file by inserting the following code block after the static int audio_open(FFPlayer *opaque, int64_t wanted_channel_layout, int wanted_nb_channels, int wanted_sample_rate, struct AudioParams *audio_hw_params){} method.

     extern int artcDemuxerMessage(struct AVFormatContext *s, int type, void *data, size_t data_size);
     //aliyun rts:receive artc message
     int onArtcDemuxerMessage(struct AVFormatContext *s, int type, void *data, size_t data_size)
     {
         return artcDemuxerMessage(s, type, data, data_size);
     }
     
     int artcDemuxerMessage(struct AVFormatContext *s, int type, void *data, size_t data_size)
     {
         //aliyun rts:send message to app
         FFPlayer *ffp = (FFPlayer *)s->opaque;
         const char *data_msg = (const char *)data;
         ffp_notify_msg4(ffp,FFP_MSG_ARTC_DIRECTCOMPONENTMSG,type,0,data_msg,data_size);
         return 0;
     }

  2. Modify the ff_ffplay.c file by inserting the following RTS code block into the static int is_realtime(AVFormatContext *s){} method.

     static int is_realtime(AVFormatContext *s)
     {
         if(   !strcmp(s->iformat->name, "rtp")
            || !strcmp(s->iformat->name, "rtsp")
            || !strcmp(s->iformat->name, "sdp")
            // *** Beginning of the RTS code block ***
            || !strcmp(s->iformat->name, "artc")
            // *** End of the RTS code block ***
         )
             return 1;
     
         if(s->pb && (   !strncmp(s->filename, "rtp:", 4)
                      || !strncmp(s->filename, "udp:", 4)
                     )
         )
             return 1;
         return 0;
      }                           

  3. Modify the ff_ffplay.c file by inserting the following RTS code block into the static int read_thread(void *arg){} method.

     static int read_thread(void *arg)
     {
       ......
       ic = avformat_alloc_context();
       if (!ic) {
         av_log(NULL, AV_LOG_FATAL, "Could not allocate context.\n");
         ret = AVERROR(ENOMEM);
         goto fail;
       }
     
       // *** Beginning of the RTS code block ***
       ic->opaque = ffp;
       ic->control_message_cb = onArtcDemuxerMessage;
       // *** End of the RTS code block ***
     
       ......
       if (ffp->skip_calc_frame_rate) {
          av_dict_set_int(&ic->metadata, "skip-calc-frame-rate", ffp->skip_calc_frame_rate, 0);
          av_dict_set_int(&ffp->format_opts, "skip-calc-frame-rate", ffp->skip_calc_frame_rate, 0);
       }
     
       // *** Beginning of the RTS code block ***
       if(strncmp(is->filename, "artc://", 7) == 0) {
          extern AVInputFormat ff_rtc_demuxer;
          is->iformat = &ff_rtc_demuxer;
       } else {
          if(ffp->iformat_name)
            is->iformat = av_find_input_format(ffp->iformat_name);
        }
       // *** End of the RTS code block ***
       ......
       pkt->flags = 0;
       // *** Beginning of the RTS code block ***
       if(strncmp(is->filename, "artc://", 7) == 0) {
           bool videoExist = is->video_stream >= 0;
         bool audioExist = is->audio_stream >= 0;
         // av_log(NULL, AV_LOG_INFO, "videoDuration %lld audioDuration %lld rate %f videoframeQue %d audioFrameque %d\n",
         // is->videoq.duration, is->audioq.duration, ffp->pf_playback_rate,
         // frame_queue_nb_remaining(&is->pictq), frame_queue_nb_remaining(&is->sampq));
         if(!videoExist) {
            if(is->audioq.duration > 300 ) { // accelerate
                if(ffp->pf_playback_rate <= 1.0) {
                    ffp->pf_playback_rate = 1.3;
                    ffp->pf_playback_rate_changed = 1;
                    av_log(NULL, AV_LOG_INFO, "aliyun rts set rate to %f\n", ffp->pf_playback_rate);
                }
            }
            else if(is->audioq.duration < 200) { // restore speed
                if(ffp->pf_playback_rate > 1.0) {
                    ffp->pf_playback_rate = 1.0;
                    ffp->pf_playback_rate_changed = 1;
                    av_log(NULL, AV_LOG_INFO, "aliyun rts restore rate 1.0\n");
                 }
            }
         }
         else if((!videoExist || (videoExist && is->videoq.duration > 300)) && (!audioExist || (audioExist && is->audioq.duration > 300))) {
            if(ffp->pf_playback_rate <= 1) {
                ffp->pf_playback_rate = 1.3;
                ffp->pf_playback_rate_changed = 1;
                av_log(NULL, AV_LOG_INFO, "aliyun rts set rate 1.1\n");
            }
         } else if((videoExist && is->videoq.duration <= 100) ||  (audioExist && is->audioq.duration <= 100)){
            if(ffp->pf_playback_rate > 1) {
                ffp->pf_playback_rate = 1;
                ffp->pf_playback_rate_changed = 1;
                av_log(NULL, AV_LOG_INFO, "aliyun rts set rate 1\n");
             }
         }
       }
       // *** End of the RTS code block ***
       ......
     }

  4. Modify the ff_ffmsg.h file by adding the method declaration for RTS messages.

     #define FFP_MSG_ARTC_DIRECTCOMPONENTMSG     3000

  5. Send RTS messages to the upper layer.

     • Modify the IJKMediaPlayback.h file by adding the declaration of the listener that listens to external calls of RTS messages.

       IJK_EXTERN NSString *const IJKMPMoviePlayerRtsMsgNotification;

     • Modify the IJKMediaPlayback.m file by adding the definition of the listener that listens to external calls of RTS messages.

       NSString *const IJKMPMoviePlayerRtsMsgNotification = @"IJKMPMoviePlayerRtsMsgNotification";

     • Modify the IJKFFMoviePlayerController.m file to allow you to call the postEvent: method to process RTS messages returned by the listener.

       - (void)postEvent: (IJKFFMoviePlayerMessage *)msg
       {
           ......
         case FFP_MSG_ARTC_DIRECTCOMPONENTMSG:{
            NSString *rtsMsg = [[NSString alloc] initWithUTF8String:avmsg->obj];
            int type = avmsg->arg1;
            if (!rtsMsg) {
               rtsMsg = @"";
            }
            NSDictionary *dic = @{@"type":@(type),@"msg":rtsMsg};
            [[NSNotificationCenter defaultCenter] postNotificationName:IJKMPMoviePlayerRtsMsgNotification
                           object:dic];
            break;
           }
          default:
            // NSLog(@"unknown FFP_MSG_xxx(%d)\n", avmsg->what);
            break;
       }

     • Use the Archive compiler to run the IJKMediaPlayer.xcodeproj project to obtain the latest IJKMediaFramework.framework file.

• Add a method used to perform retries for Native RTS SDK in ijkplayer

  1. Modify the ff_ffplay.h file by adding the definition of the rts_reload_flag variable.

     int       rts_reload_flag;

  2. Modify the ff_ffplay.c file by inserting the following code block after the static int audio_open(FFPlayer *opaque, int64_t wanted_channel_layout, int wanted_nb_channels, int wanted_sample_rate, struct AudioParams *audio_hw_params){} method.

     extern int artc_reload(AVFormatContext *ctx);
     
     void ffp_rts_reload(FFPlayer *ffp){
        if(rts_reload_flag == 0)
        {
           rts_reload_flag = 1;
        }
     }

  3. Modify the ff_ffplay.c file by inserting the following RTS code block into the static int read_thread(void *arg){} method.

     static int read_thread(void *arg)
     {
          ......
     #ifdef FFP_MERGE
        if (is->paused != is->last_paused) {
           is->last_paused = is->paused;
           if (is->paused)
              is->read_pause_return = av_read_pause(ic);
            else
               av_read_play(ic);
        }
     #endif
        // *** Beginning of the RTS code block ***
        if(rts_reload_flag){
             rts_reload_flag = 0;
             av_log(ffp, AV_LOG_ERROR, "param  == ffp_rts_reload\n");
             VideoState *is = ffp->is;
             AVFormatContext *ic = is->ic;
             artc_reload(ic);
         }
        // *** End of the RTS code block ***
        ......
     }

  4. Modify the ijkplayer.h file by adding the definition of the ijkmp_rts_reload method.

     void            ijkmp_rts_reload(IjkMediaPlayer *mp);

  5. Modify the ijkplayer.c file to implement the ijkmp_rts_reload method.

     void ijkmp_rts_reload(IjkMediaPlayer *mp)
     {
         ffp_rts_reload(mp->ffplayer);
     }

  6. Add the rtsReload method to the upper layer.

     • Modify the IJKFFMoviePlayerController.h file by adding the declaration of external calls of the rtsReload method.

       - (void)rtsReload;

     • Modify the IJKFFMoviePlayerController.m file to implement external calls of the rtsReload method.

       - (void)rtsReload {
           ijkmp_rts_reload(_mediaPlayer);
       }

     • Use the Archive compiler to run the IJKMediaPlayer.xcodeproj project to obtain the latest IJKMediaFramework.framework file.

• Call the method for listening to the RTS message callback to live stream over a degraded protocol

  • Listen to the RTS message callback.

    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:@selector(reviceMsg:)
                                                 name:IJKMPMoviePlayerRtsMsgNotification
                                               object:nil];

  • Implement the degradation logic in the method that is used to listen to the RTS message callback.

    You can change the prefix of a source URL from artc:// to rtmp:// or http://xxxx.flv, and then update the UrlSource in ApsaraVideo Player. This way, you can use a degraded protocol to play live streams.

    // Use a degraded protocol, such as http://xxx.flv or rtmp://xxx, for live streaming.
    - (void)convertArtcToRtmpPlay {
        // Obtain the streaming URL and intercept the prefix of the URL.
        NSArray *urlSeparated = [self.url componentsSeparatedByString:@"://"];
        NSString *urlPrefix = urlSeparated.firstObject;
        // Check whether the prefix of the URL is artc. If the prefix is artc, you can use a degraded protocol for standard streaming.
        if ([urlPrefix isEqualToString:@"artc"]) {
            // http://xxx.flv (recommended)
            _url = [[@"http://" stringByAppendingString:urlSeparated.lastObject] stringByAppendingString:@".flv"];
            // rtmp://xxx
            // _url = [@"rtmp://" stringByAppendingString:urlSeparated.lastObject];
    
                // Stop playback and destroy the player.
          [_ijkPlayer stop];
          [_ijkPlayer shutdown];
          _ijkPlayer = nil;
    
          // Re-set the playback source.
          _ijkPlayer = [[IJKFFMoviePlayerController alloc] initWithContentURL:[NSURL URLWithString:_url] withOptions:options];
          ......
                // Start playback.
          dispatch_async(dispatch_get_main_queue(), ^{
                    [_ijkPlayer prepareToPlay];
                    [_ijkPlayer play];
                });
        }
    }

    You must import the RtsSDK API in advance.

    #import <RtsSDK/rts_messages.h>

    Then process the player event callback.

    -(void)reviceMsg:(NSNotification*)notification{
        NSDictionary *dic = [notification object];
      NSNumber *type = dic[@"type"];
      switch (type.intValue) {
        case E_DNS_FAIL:
        case E_AUTH_FAIL:
        case E_CONN_TIMEOUT:
        case E_SUB_TIMEOUT:
        case E_SUB_NO_STREAM:
        {
          // Perform degradation.
          [self convertArtcToRtmpPlay];
        }
            break;
        case E_STREAM_BROKEN:
        {
           static BOOL retryStartPlay = YES;
           // The first time the RTS media times out, retry playback once. If the RTS media still times out, use a degraded protocol for playback.
           if (retryStartPlay) {
               dispatch_async(dispatch_get_main_queue(), ^{
                 [_ijkPlayer rtsReload];
                 });
               retryStartPlay = NO;
            } else {
               // Perform degradation.
               [self convertArtcToRtmpPlay];
            }
         }
          break;
        case E_RECV_STOP_SIGNAL:
                {
              // Stop playback and destroy the player.
              [_ijkPlayer stop];
            [_ijkPlayer shutdown];
            _ijkPlayer = nil;
          }
          break;
        default:
          break;
      }
    }