ApsaraVideo Live:Feature usage

Register the SDK

To request and configure a license, see License Integration Guide.

Call the license registration method at an early stage, before you use the Push SDK.

[AlivcLiveBase registerSDK];

• The AlivcLiveBase class lets you set log levels, set local log paths, and get the SDK version.

• In the onLicenceCheck method of the AlivcLiveBase#setObserver interface, asynchronously check whether the license is configured.

Configure stream ingest parameters

In the ViewController where you need the pusher, import the header file #import <AlivcLivePusher/AlivcLivePusher.h>.

Basic stream ingest parameters have default values. We recommend that you use the default values. You can perform a simple initialization without extra configuration.

AlivcLivePushConfig *config = [[AlivcLivePushConfig alloc] init];// Initialize the stream ingest configuration class. You can also use initWithResolution for initialization.
config.resolution = AlivcLivePushResolution540P;// The default resolution is 540P. The maximum resolution is 720P.
config.fps = AlivcLivePushFPS20; // We recommend that you use 20 fps.
config.enableAutoBitrate = true; // Enable bitrate control. The default value is true.
config.videoEncodeGop = AlivcLivePushVideoEncodeGOP_2;// The default value is 2. A larger GOP size results in higher latency. We recommend that you set this parameter to 1 or 2.
config.connectRetryInterval = 2000; // Unit: milliseconds. The reconnection duration is 2 s. The reconnection interval must be at least 1 second. We recommend that you use the default value.
config.previewMirror = false; // The default value is false. In normal cases, select false.
config.orientation = AlivcLivePushOrientationPortrait; // The default orientation is portrait. You can set the device to landscape with the home button on the left or right.

Ingest a camera stream

1. Initialize.

   After you configure stream ingest parameters, use the initWithConfig method of the Push SDK to perform initialization. Sample code:

   self.livePusher = [[AlivcLivePusher alloc] initWithConfig:config];

   Note

   AlivcLivePusher does not support multiple instances. Therefore, each init call must have a corresponding destroy call.

2. Register stream ingest callbacks.

   The following stream ingest callbacks are supported:

   • Info: the callbacks that are used for notifications and status detection.

   • Error: the callbacks that are returned when errors occur.

   • Network: the callbacks that are related to network.

   Register the delegate to receive the corresponding callbacks. Sample code:

   [self.livePusher setInfoDelegate:self];
   [self.livePusher setErrorDelegate:self];
   [self.livePusher setNetworkDelegate:self];

3. Start preview.

   After the livePusher object is initialized, you can start the preview. When you preview, you must pass in the display view for the camera preview. The view inherits from UIView. Sample code:

   [self.livePusher startPreview:self.view];

4. Start stream ingest.

   You can start stream ingest only after the preview is successful. Therefore, listen for the onPreviewStarted callback of AlivcLivePusherInfoDelegate and add the following code in the callback.

   [self.livePusher startPushWithURL:@"Test ingest URL (rtmp://......)"];

   Note

   • RTMP and RTS (artc://) ingest URLs are supported. To generate ingest URLs, see Generate streaming URLs.

   • ApsaraVideo Live does not support ingesting multiple streams to the same ingest URL at the same time. The second stream ingest request is rejected.

/* Switch between the front and rear cameras. */
[self.livePusher switchCamera];
/* Enable or disable the flash. You cannot enable the flash for the front camera. */
[self.livePusher setFlash:false]; 
/* Adjust the focal length to zoom in or out of the captured frame. Pass a positive number to increase the focal length. Pass a negative number to decrease the focal length. */
CGFloat max = [_livePusher getMaxZoom];
[self.livePusher setZoom:MIN(1.0, max)]; 
/* Manually focus. To manually focus, you must pass two parameters: 1. point: the point to focus on (the coordinates of the point to focus on). 2. autoFocus: specifies whether to enable autofocus. This parameter takes effect only for the focus operation of this API call. Subsequent autofocus operations use the value set by the autofocus API. */
[self.livePusher focusCameraAtAdjustedPoint:CGPointMake(50, 50) autoFocus:true];
/* Set whether to enable autofocus. */
[self.livePusher setAutoFocus:false];
/* Configure mirroring. There are two mirroring interfaces: PushMirror for ingest stream mirroring and PreviewMirror for preview mirroring. The PushMirror setting takes effect only on the playback screen. The PreviewMirror setting takes effect only on the preview screen. The two settings do not affect each other. */
[self.livePusher setPushMirror:false];
[self.livePusher setPreviewMirror:false];

/* You can set pauseImage and then call the pause method to switch from camera stream ingest to static image stream ingest. Audio stream ingest continues. */
[self.livePusher pause];
/* Switch from static image stream ingest to camera stream ingest. Audio stream ingest continues. */
[self.livePusher resume];
/* In the streaming state, you can call this method to stop stream ingest. After the operation is complete, stream ingest stops. */
[self.livePusher stopPush];
/* You can stop the preview only in the previewing state. If you are ingesting a stream, stopping the preview is invalid. After the preview stops, the preview screen freezes on the last frame. */
[self.livePusher stopPreview];
/* In the streaming state or when any Error-related callback is received, you can call this method to restart stream ingest. In an error state, you can only call this method (or reconnectPushAsync for reconnection) or call destroy to destroy the stream ingest instance. After the operation is complete, stream ingest restarts. All internal resources of ALivcLivePusher, including preview and stream ingest, are restarted. */
[self.livePusher restartPush];
/* In the streaming state or when an AlivcLivePusherNetworkDelegate-related Error callback is received, you can call this interface. In an error state, you can only call this interface (or restartPush to restart stream ingest) or call destroy to destroy the stream ingest instance. After the operation is complete, the stream ingest reconnects. The RTMP connection for stream ingest is re-established. */
[self.livePusher reconnectPushAsync];
/* After the stream ingest instance is destroyed, stream ingest and preview stop, and the preview screen is removed. All resources related to AlivcLivePusher are destroyed. */
[self.livePusher destory];
self.livePusher = nil;
/* Get the stream ingest status. */
AlivcLivePushStatus status = [self.livePusher getLiveStatus];

Screen recording (screen sharing) stream ingest

ReplayKit is a feature introduced in iOS 9 that supports screen recording. In iOS 10, a feature was added to ReplayKit that lets you call third-party app extensions to live stream screen content. On iOS 10 and later, you can use the Push SDK with an Extension screen recording process to implement screen sharing for live streaming.

1. Create an App Group.

   Log on to Apple Developer and perform the following operations:

   1. On the Certificates, IDs & Profiles page, register an App Group. For more information, see Register an App Group.

   2. Return to the Identifier page, select App IDs, and then click your App ID to enable the App Group feature. The App IDs for the main app process and the Extension screen recording process must be configured in the same way. For more information, see Enable App Group.

   3. After the operation is complete, download the corresponding Provisioning Profile again and configure it in Xcode.

   After you perform the operations correctly, the Extension screen recording process can communicate with the main app process.

   Note

   After you create an App Group, save the App Group Identifier value. This value is required for subsequent steps.

2. Create an Extension screen recording process.

   The iOS Push SDK demo provides the AlivcLiveBroadcast and AlivcLiveBroadcastSetupUI app extensions that support screen sharing for live streaming. To create an Extension screen recording process in the app, perform the following steps:

   1. In your existing project, choose New > Target… and select Broadcast Upload Extension, as shown in the following figure:

   2. Modify the Product Name, select Include UI Extension, and click Finish to create the live streaming extension and UI, as shown in the following figure:

   3. Configure the Info.plist file for the live streaming extension. In the newly created target, Xcode creates a header file and a source file named SampleHandler by default, as shown in the following figure:

      Drag the AlivcLibReplayKitExt.framework to the project to make the Extension Target depend on it.Replace the code in SampleHandler.m with the following code. Replace KAPP Group in the code with the App Group Identifier that you created in the first step. Sample code:

      
      #import "SampleHandler.h"
      #import <AlivcLibReplayKitExt/AlivcLibReplayKitExt.h>
      
      @implementation SampleHandler
      
      - (void)broadcastStartedWithSetupInfo:(NSDictionary<NSString *,NSObject *>
      *)setupInfo {
      
       //User has requested to start the broadcast. Setup info from the UI extension can
      be supplied but optional.
       [[AlivcReplayKitExt sharedInstance] setAppGroup:kAPPGROUP];
      }
      
      - (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType {
       if (sampleBufferType != RPSampleBufferTypeAudioMic) {
       // Audio is captured and sent by the main app process.
       [[AlivcReplayKitExt sharedInstance] sendSampleBuffer:sampleBuffer withType:sampleBufferType];
       }
      }
      
      - (void)broadcastFinished {
      
       [[AlivcReplayKitExt sharedInstance] finishBroadcast];
      }
      @end
      
                                          

   In your project, create a Broadcast Upload Extension target. In this Extension Target, integrate the AlivcLibReplayKitExt.framework that is customized for the screen recording extension module.

3. Integrate the Live SDK into the main app process for screen sharing stream ingest.

   In the main app process for screen sharing stream ingest, create AlivcLivePushConfig and AlivcLivePusher objects. Set ExternMainStream to True and AudioFromExternal to False. This configuration indicates that audio is still captured by the SDK. Call StartScreenCapture to start receiving screen data from the Extension App. Then, start and stop stream ingest. For more information, see the following steps:

   1. In the main app process for screen sharing stream ingest, add dependencies on AlivcLivePusher.framework, AlivcLibRtmp.framework, RtsSDK.framework, and AlivcLibReplayKitExt.framework.

   2. Initialize the Push SDK and configure it to use an external video source.

      Set ExternMainStream to True and ExternVideoFormat to AlivcLivePushVideoFormatYUV420P. To use the internal SDK for audio capture, set AudioFromExternal to False. Configure other stream ingest parameters. For more information, see the following sample code:

       self.pushConfig.externMainStream = true;
       self.pushConfig.externVideoFormat = AlivcLivePushVideoFormatYUV420P;
       self.pushConfig.audioSampleRate = 44100;
       self.pushConfig.audioChannel = 2;
       self.pushConfig.audioFromExternal = false;
       self.pushConfig.videoEncoderMode = AlivcLivePushVideoEncoderModeSoft;
       self.pushConfig.qualityMode = AlivcLivePushQualityModeCustom;
       self.pushConfig.targetVideoBitrate = 2500;
       self.pushConfig.minVideoBitrate = 2000;
       self.pushConfig.initialVideoBitrate = 2000;
       self.livePusher = [[AlivcLivePusher alloc] initWithConfig:self.pushConfig];
      
                                      

   3. Use AlivcLivePusher to perform live streaming-related functions. Call the following functions:

      • Start receiving screen data.

        Replace kAPPGroup in the code with the App Group Identifier that you created. Sample code:

        [self.livePusher startScreenCapture:kAPPGROUP];

      • Start stream ingest.

        Sample code:

        [self.livePusher startPushWithURL:self.pushUrl]

      • Stop stream ingest.

        Sample code:

        [self.livePusher stopPush];
        [self.livePusher destory];
        self.livePusher = nil;

Preview display modes

The Push SDK supports three preview modes. The preview display mode does not affect stream ingest.

• ALIVC_LIVE_PUSHER_PREVIEW_SCALE_FILL: The preview fills the window. If the aspect ratio of the video is different from the aspect ratio of the window, the preview is distorted.

• ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FIT: The aspect ratio of the video is preserved during preview. If the aspect ratio of the video is different from the aspect ratio of the window, black bars appear in the preview. This is the default mode.

• ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FILL: The video is cropped to fit the window aspect ratio during preview. If the aspect ratio of the video is different from the aspect ratio of the window, the preview is cropped.

Sample code:

mAlivcLivePushConfig.setPreviewDisplayMode(AlivcPreviewDisplayMode.ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FIT);

Image stream ingest

For a better user experience, the SDK provides settings for background image stream ingest and image stream ingest when the bitrate is too low. When the SDK is moved to the background, video stream ingest is paused by default, and only audio is ingested. At this time, you can set an image for image stream ingest. For example, you can display an image to remind users that the streamer is away for a moment and will be back soon. Sample code:

config.pauseImg = [UIImage imageNamed:@"image.png"];// Set the image for background stream ingest.

In addition, when the network is poor, you can set a static image to be ingested based on your needs. After you set the image, the SDK detects when the current bitrate is low and ingests this image to avoid video stream stuttering. Sample code:

config.networkPoorImg = [UIImage imageNamed:@"image.png"];// Set the image to be ingested when the network is poor.

Push external audio and video streams

The Push SDK supports ingesting external audio and video sources, such as an audio or video file.

1. Configure external audio and video input in the stream ingest configuration.

Sample code:

config.externMainStream = true;// Enable external stream input.
config.externVideoFormat = AlivcLivePushVideoFormatYUVNV21;// Set the color format of video data. In this example, the format is set to YUVNV21. You can set the format to another value as needed.
config.externAudioFormat = AlivcLivePushAudioFormatS16;// Set the bit depth format of audio data. In this example, the format is set to S16. You can set the format to another value as needed.

2. Insert external video data.

Sample code:

/* Only continuous buffer data in the external video YUV and RGB formats can be sent using the sendVideoData interface. You can send the video data buffer, length, width, height, timestamp, and rotation angle. */
[self.livePusher sendVideoData:yuvData width:720 height:1280 size:dataSize pts:nowTime rotation:0];
/* If the external video data is in the CMSampleBufferRef format, you can use the sendVideoSampleBuffer interface. */
[self.livePusher sendVideoSampleBuffer:sampleBuffer]
/* You can also convert the CMSampleBufferRef format to a continuous buffer and then pass it to the sendVideoData interface. The following is reference code for the conversion. */
// Get the sample buffer length.
- (int) getVideoSampleBufferSize:(CMSampleBufferRef)sampleBuffer {
if(!sampleBuffer) {
 return 0;
}
int size = 0;
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
if(CVPixelBufferIsPlanar(pixelBuffer)) {
 int count = (int)CVPixelBufferGetPlaneCount(pixelBuffer);
 for(int i=0; i<count; i++) {
 int height = (int)CVPixelBufferGetHeightOfPlane(pixelBuffer,i);
 int stride = (int)CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer,i);
 size += stride*height;
 }
}else {
 int height = (int)CVPixelBufferGetHeight(pixelBuffer);
 int stride = (int)CVPixelBufferGetBytesPerRow(pixelBuffer);
 size += stride*height;
}
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
return size;
}
// Convert the sample buffer to a continuous buffer.
- (int) convertVideoSampleBuffer:(CMSampleBufferRef)sampleBuffer toNativeBuffer:(void*)nativeBuffer
{
if(!sampleBuffer || !nativeBuffer) {
 return -1;
}
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
int size = 0;
if(CVPixelBufferIsPlanar(pixelBuffer)) {
 int count = (int)CVPixelBufferGetPlaneCount(pixelBuffer);
 for(int i=0; i<count; i++) {
 int height = (int)CVPixelBufferGetHeightOfPlane(pixelBuffer,i);
 int stride = (int)CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer,i);
 void *buffer = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, i);
 int8_t *dstPos = (int8_t*)nativeBuffer + size;
 memcpy(dstPos, buffer, stride*height);
 size += stride*height;
 }
}else {
 int height = (int)CVPixelBufferGetHeight(pixelBuffer);
 int stride = (int)CVPixelBufferGetBytesPerRow(pixelBuffer);
 void *buffer = CVPixelBufferGetBaseAddress(pixelBuffer);
 size += stride*height;
 memcpy(nativeBuffer, buffer, size);
}
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
return 0;
}

3. Insert audio data.

Sample code:

/* Only continuous buffer data in the external PCM format is supported. Use sendPCMData to send the audio data buffer, length, and timestamp. */
[self.livePusher sendPCMData:pcmData size:size pts:nowTime];

Configure watermarks

The Push SDK provides a feature to add watermarks. You can add multiple watermarks. The watermark images must be in the PNG format. Sample code:

NSString *watermarkBundlePath = [[NSBundle mainBundle] pathForResource:
[NSString stringWithFormat:@"watermark"] ofType:@"png"];// Set the path of the watermark image.
[config addWatermarkWithPath: watermarkBundlePath
 watermarkCoordX:0.1
 watermarkCoordY:0.1
 watermarkWidth:0.3];// Add a watermark.

Set the video quality

Video quality supports three modes: resolution priority, fluency priority, and custom.

config.qualityMode = AlivcLivePushQualityModeResolutionFirst；// Resolution priority mode

config.qualityMode = AlivcLivePushQualityModeFluencyFirst；// Fluency priority mode

config.qualityMode = AlivcLivePushQualityModeCustom// Set to custom mode.
config.targetVideoBitrate = 1400; // The target bitrate is 1,400 kbit/s.
config.minVideoBitrate = 600; // The minimum bitrate is 600 kbit/s.
config.initialVideoBitrate = 1000; // The initial bitrate is 1,000 kbit/s.

When you set a custom bitrate, refer to the recommended settings from Alibaba Cloud to configure the corresponding bitrate. For more information about the recommended settings, see the following tables:

Adaptive resolution

When dynamic resolution adjustment is enabled, the ingest stream resolution automatically decreases in poor network conditions. This improves video smoothness and clarity. Sample code:

config.enableAutoResolution = YES; // Enable adaptive resolution. The default value is NO.

Background music

The Push SDK supports background music playback, audio mixing, denoising, in-ear monitoring, and muting. Call the related API operations only after the preview starts. The following code provides an example:

/* Start playing background music. */
[self.livePusher startBGMWithMusicPathAsync:musicPath];
/* Stop playing background music. If you want to switch songs while background music is playing, simply call the startBGMWithMusicPathAsync operation. You do not need to stop the current playback. */
[self.livePusher stopBGMAsync];
/* Pause the background music. Call this operation only after the background music starts playing. */
[self.livePusher pauseBGM];
/* Resume the background music. Call this operation only when the background music is paused. */
[self.livePusher resumeBGM];
/* Enable loop playback. */
[self.livePusher setBGMLoop:true];
/* Set the denoising switch. When enabled, non-vocal sounds in the captured audio are filtered. This may cause slight suppression of human voices. Let users decide whether to enable this feature. This feature is disabled by default. */
[self.livePusher setAudioDenoise:true];
/* Set the in-ear monitoring switch. This feature is mainly used in karaoke scenarios. When enabled, the streamer can hear their own voice in the headphones. When disabled, the streamer cannot hear their own voice. This feature does not work if headphones are not connected. */
[self.livePusher setBGMEarsBack:true];
/* Configure audio mixing to adjust the volume of the background music and the captured human voice. */
[self.livePusher setBGMVolume:50];// Set the volume of the background music.
[self.livePusher setCaptureVolume:50];// Set the volume of the captured human voice.
/* Mute the audio. When muted, both the background music and the captured human voice are silenced. To mute the music or the voice separately, use the audio mixing operations to adjust their volumes. */
[self.livePusher setMute:isMute?true:false];

Capture snapshots

The Push SDK lets you capture snapshots of local video streams. The following is sample code:

/* Set the snapshot callback. */
[self.livePushersetSnapshotDelegate:self];
/* Call the snapshot API. */
[self.livePushersnapshot:1interval:1];

Configure the live quiz feature

The live quiz feature works by inserting supplemental enhancement information (SEI) into a live stream. The player then parses the SEI. The Push SDK provides a method to insert SEI messages. Call this method only during stream ingest. The following code is an example:

/*
msg: The SEI message body to insert into the stream. JSON format is recommended. ApsaraVideo Player SDK can receive and parse this message for display.
repeatCount: The number of frames in which to send the message. To prevent the SEI message from being dropped, set the number of repetitions. For example, a value of 100 inserts the message into the next 100 frames. The player deduplicates identical SEI messages.
delayTime: The delay in milliseconds before sending the message.
KeyFrameOnly: Specifies whether to send the message only in keyframes.
*/
[self.livePusher sendMessage:@"Question information" repeatCount:100 delayTime:0 KeyFrameOnly:false];

iPhone X adaptation

Normally, setting the preview view to full screen works correctly. However, the iPhone X has a special aspect ratio. This causes the preview image to stretch when displayed in full screen. Do not use a full-screen view for previews on an iPhone X.

Change the view size during stream ingest

Traverse the UIView assigned in the startPreview or startPreviewAsync call. Change the frame for all subviews in the preview view. For example:

[self.livePusher startPreviewAsync:self.previewView];
for (UIView *subView in [self.previewView subviews]) {
 // ...
}

Play external audio

To play external audio on the stream ingest page, use AVAudioPlayer because the software development kit (SDK) is temporarily incompatible with AudioServicesPlaySystemSound. After playback, update the AVAudioSession settings. The following is sample code:

- (void)setupAudioPlayer {
 NSString *filePath = [[NSBundle
mainBundle] pathForResource:@"sound" ofType:@"wav"];
 NSURL *fileUrl = [NSURL URLWithString:filePath];
 self.player = [[AVAudioPlayer alloc] initWithContentsOfURL:fileUrl error:nil];
 self.player.volume = 1.0;
 [self.player prepareToPlay];
}
 - (void)playAudio {
 self.player.volume = 1.0;
 [self.player play];
 // Configure AVAudioSession.
 AVAudioSession *session = [AVAudioSession sharedInstance];
 [session setMode:AVAudioSessionModeVideoChat error:nil];
 [session overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker error:nil];
 [session setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker|AVAudioSessionCategoryOptionAllowBluetooth
| AVAudioSessionCategoryOptionMixWithOthers error:nil];
 [session setActive:YES error:nil];
}

Background mode and phone calls

The SDK handles background processing automatically. By default, when the app moves to the background, the SDK continues to ingest the audio stream and pauses the video on the last frame. To ensure the app can collect audio in the background, go to your app's Capabilities, enable Background Mode, and select Audio, AirPlay and Picture in Picture.

To stop audio stream ingest when the app moves to the background, destroy the stream ingest engine. When the app returns to the foreground, re-create the engine to resume the stream.

Callbacks

The Push SDK includes the following callbacks:

Common methods and interfaces

/* In custom mode, you can adjust the minimum and target bitrates in real time. */
[self.livePusher setTargetVideoBitrate:800];
[self.livePusher setMinVideoBitrate:200]
/* Get the stream ingest status. */
BOOL isPushing = [self.livePusher isPushing]; 
/* Get the ingest URL. */
NSString *pushURLString = [self.livePusher getPushURL];
/* Get stream ingest performance and debugging information. For details about performance metrics, see the API documentation or the comments in the code. */
AlivcLivePushStatsInfo *info = [self.livePusher getLivePushStatusInfo];
/* Get the SDK version number. */
NSString *sdkVersion = [self.livePusher getSDKVersion];
/* Set the log level to filter debugging information as needed. */
[self.livePusher setLogLevel:(AlivcLivePushLogLevelDebug)];

Debugging tools

The SDK provides the DebugView UI debugging tool, a movable global floating window that is always displayed at the top of the view. It includes debug features such as viewing stream ingest logs, real-time detection of stream ingest performance metrics, and line charts of key performance metrics.

Sample code:

[AlivcLivePusher showDebugView];// Open DebugView.