ApsaraVideo Live:Developer guide to co-streaming

Overview of how to implement co-streaming

Usage notes

• The following section describes how to use the interactive edition of Push SDK for co-streaming. For more information about how to integrate the interactive edition of Push SDK, see Integrate Push SDK for Android or Integrate Push SDK for iOS.

• In co-streaming mode, the streamer and co-streamer can use only the AlivcLivePlayer object to play the stream of each other. The AlivcLivePlayer object supports only streams whose URL starts with artc://live.aliyun.com/play/.

Step 1: Enable the co-streaming feature

For information about how to enable the co-streaming feature, see Get started with co-streaming.

Step 2: Generate ingest and streaming URLs for co-streaming

You can generate the ingest and streaming URLs of the streamer and co-streamer and the CDN streaming URL of viewers by using the ApsaraVideo Live console, or construct these URLs by using the concatenation rules.

Method 1: Generate the URLs by using the console

You can use the URL generator in the ApsaraVideo Live console to generate the ingest and streaming URLs of the streamer and co-streamer and the CDN streaming URL of viewers. For more information, see Co-streaming URL generator.

Method 2: Construct the URLs

Tokens in ingest and streaming URLs of the streamer and co-streamer that are generated by using the console are temporary, and are usually used for testing. If you need ingest and streaming URLs that are used for business purposes, construct the URLs by using the concatenation rules for security reasons. Tokens in URLs that you construct by using the concatenation rules are calculated based on the SdkAppID, AppKey, room ID, UserID, and timestamp by using the SHA-256 encryption algorithm. These tokens provide higher security. For more information, see Ingest and streaming URLs of streamers and co-streamers in co-streaming scenarios and CDN streaming URLs of viewers.

Step 3: Create objects related to stream ingest

This step is required on both the streamer side and the co-streamer side.

1. Create an AlivcLivePushConfig object

   Create an AlivcLivePushConfig object that is used to configure stream ingest. Set livePushMode to AlivcLivePushInteractiveMode, and configure parameters such as the resolution, frame rate, and bitrate.

   Important

   If you want to implement co-streaming between a native client on Android or iOS and the web, the HTML5 compatibility mode must be enabled on the native client. Otherwise, users on the web are seeing a black screen sent from users on the native client. To enable the HTML5 compatibility mode, call AlivcLivePushConfig#setH5CompatibleMode on the Native client. For more information, see the API reference of the native SDK.

   Sample code for Android

   // Initialize the class for stream ingest configurations.
   mAlivcLivePushConfig = new AlivcLivePushConfig();
   // Specify the stream ingest mode. By default, the regular stream ingest mode is used.
   mAlivcLivePushConfig.setLivePushMode(AlivcLiveMode.AlivcLiveInteractiveMode);
   // Specify the resolution. The default resolution is 540p.
   mAlivcLivePushConfig.setResolution(AlivcResolutionEnum.RESOLUTION_540P);
   // Specify the frame rate. The default frame rate is 20 frames per second (FPS).
   mAlivcLivePushConfig.setFps(AlivcFpsEnum.FPS_25);
   // Specify the group of pictures (GOP) size. Unit: seconds. The default GOP size is 2 seconds.
   mAlivcLivePushConfig.setVideoEncodeGop(AlivcVideoEncodeGopEnum.GOP_TWO);
   // Specify whether to enable adaptive bitrate streaming. The default value is true.
   mAlivcLivePushConfig.setEnableBitrateControl(true);
   // Specify the screen orientation. The default screen orientation is portrait. You can configure the settings that allow you to press the Home key to change the orientation to landscape left or landscape right.
   mAlivcLivePushConfig.setPreviewOrientation(AlivcPreviewOrientationEnum.ORIENTATION_PORTRAIT);
   // Specify the audio encoding format. The default format is AAC-LC.
   mAlivcLivePushConfig.setAudioProfile(AlivcAudioAACProfileEnum.AAC_LC);
   // Specify the video encoding mode. By default, hardware encoding is used.
   mAlivcLivePushConfig.setVideoEncodeMode(AlivcEncodeModeEnum.Encode_MODE_HARD);
   // Specify the audio encoding mode. By default, software encoding is used.
   mAlivcLivePushConfig.setAudioEncodeMode(AlivcEncodeModeEnum.Encode_MODE_SOFT);
   // Specify whether to use the front camera or the rear camera. By default, the front camera is used.
   mAlivcLivePushConfig.setCameraType(AlivcLivePushCameraTypeEnum.CAMERA_TYPE_FRONT);
   // Specify the image that is ingested when your app is switched to the background or video stream ingest is paused.
   mAlivcLivePushConfig.setPausePushImage("TODO: Image Path");
   // Specify the image that is ingested in poor network conditions.
   mAlivcLivePushConfig.setNetworkPoorPushImage("TODO: Image Path");

   Sample code for iOS

   AlivcLivePushConfig *rtcPushConfig = [[AlivcLivePushConfig alloc] init];
   rtcPushConfig.livePushMode = AlivcLivePushInteractiveMode;
   rtcPushConfig.resolution = AlivcLivePushResolution540P;
   rtcPushConfig.fps = AlivcLivePushFPS20;
   rtcPushConfig.enableAutoBitrate = true;
   rtcPushConfig.orientation = AlivcLivePushOrientationPortrait;
   rtcPushConfig.enableAutoResolution = YES;

2. Create an AlivcLivePusher object

   Practice 1: The co-streamer can choose either audio co-streaming or video co-streaming, but cannot switch between them

   If the stream is an audio-only stream, add the following code to specify that the stream is an audio-only stream when you create the AlivcLivePushConfig object.

   If the stream is not an audio-only stream, do not add the following code. For an audio-only stream, you do not need to call the method that is used to preview the stream before you start stream ingest. Instead, you can directly call the method that is used to start stream ingest.

   Sample code for Android

   mAlivcLivePushConfig.setAudioOnly(true);

   Sample code for iOS

   rtcPushConfig.audioOnly = YES;

   Practice 2: The co-streamer can switch between audio co-streaming and video co-streaming at any time

   For more information about how the co-streamer can switch between audio co-streaming and video co-streaming, see Solution to use when a co-streamer switches between audio co-streaming and video co-streaming.

3. Start stream ingest in co-streaming mode

   Use an ingest URL for co-streaming to ingest a stream. For more information about how to obtain an ingest URL, see Step 2: Generate ingest and streaming URLs for co-streaming.

   Sample code for Android

   alivcLivePusher.startPreview(context, frameLayout, isAnchor); // The preview settings for the streamer whose stream is displayed in the bigger window. Set isAnchor to true.
   alivcLivePusher.startPushAysnc("artc://live.aliyun.com/push/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX"); // The ingest URL of the streamer or co-streamer.

   Sample code for iOS

   [rtcPusher startPushWithURL:@"artc://live.aliyun.com/push/6375?timestamp=1661596947&token=XXX&userId=7135&sdkAppId=XXX"];  // The ingest URL of the streamer or co-streamer.

Step 4: Create an AliPlayer object for CDN playback

This step is required on the viewer side.

To use an AliPlayer object for CDN playback, obtain the CDN streaming URL of viewers first. For more information, see Step 2: Generate ingest and streaming URLs for co-streaming.

If the stream is an audio-only stream, you need to configure the playback start buffer to speed up the playback of the first frame. In this case, we also recommend that you use a streaming URL in the HTTP-FLV format.

mAliPlayer = AliPlayerFactory.createAliPlayer(mContext);

PlayerConfig playerConfig = mAliPlayer.getConfig();
// The playback start buffer. A larger value indicates more stable playback start. However, a too large value may affect the time it takes to start playing the stream. Specify this parameter as appropriate.
playerConfig.mStartBufferDuration = 1000;
// The cache required for stuttering to recover. Specify a larger value in poor network conditions. We recommend that you set the value to 500 for an audio-only stream or 3000 for a video stream.
playerConfig.mHighBufferDuration = 500;
mAliPlayer.setConfig(playerConfig);

mAliPlayer.setAutoPlay(true);

self.cdnPlayer = [[AliPlayer alloc] init];
self.cdnPlayer.delegate = self;
self.cdnPlayer.autoPlay = YES;
// Configure the playback start buffer to speed up the playback of the first frame if the stream is an audio-only stream.
 AVPConfig *config = [self.cdnPlayer getConfig];
 config.enableStrictFlvHeader = YES; // Specify this parameter if the stream is an audio-only stream or a video stream in the FLV format.
 config.startBufferDuration = 1000; // The playback start buffer. A larger value indicates more stable playback start. However, a too large value may affect the time it takes to start playing the stream. Specify this parameter as appropriate.
 config.highBufferDuration = 500;// The cache required for stuttering to recover. Specify a larger value in poor network conditions. We recommend that you set the value to 500 for an audio-only stream or 3000 for a video stream.
[self.cdnPlayer setConfig:config];

Step 5: Start co-streaming

1. Streamer A ingests a stream based on RTC

   Streamer A uses the AlivcLivePusher object to start stream ingest.

   Sample code for Android

   alivcLivePusher.startPreview(context, frameLayout, isAnchor);
   alivcLivePusher.startPushAysnc("artc://live.aliyun.com/push/123?timestamp=1661596947&token=XXX&userId=userA&sdkAppId=XXX"); // The ingest URL of the streamer.

   Sample code for iOS

   [rtcPusher startPushWithURL:@"artc://live.aliyun.com/push/123?timestamp=1661596947&token=XXX&userId=userA&sdkAppId=XXX"];  // The ingest URL of the streamer.

2. Viewers use a CDN streaming URL to play the stream of Streamer A

   All viewers use the AliPlayer object to play the stream of Streamer A.

   Sample code for Android

   AliPlayer aliPlayer = AliPlayerFactory.createAliPlayer(context);
   aliPlayer.setAutoPlay(true);
   
   aliPlayer.setOnErrorListener(errorInfo -> {
       aliPlayer.prepare();
   });
   
   UrlSource urlSource = new UrlSource();
   urlSource.setUri("http://test.alivecdn.com/live/streamId.flv?auth_key=XXX");  // The CDN streaming URL of viewers.
   aliPlayer.setDataSource(urlSource);
   aliPlayer.prepare();

   Sample code for iOS

   AliPlayer *cdnPlayer = [[AliPlayer alloc] init];
   cdnPlayer.delegate = self;
   cdnPlayer.autoPlay = YES;
   AVPUrlSource *source = [[AVPUrlSource alloc] urlWithString:@"http://test.alivecdn.com/live/streamId.flv?auth_key=XXX"];  // The CDN streaming URL of viewers.
   [self.cdnPlayer setUrlSource:source];
   [self.cdnPlayer prepare];

3. Viewer D initiates co-streaming

   • Viewer D creates an AlivcLivePusher object and use an ingest URL for co-streaming to ingest a stream. In this case, Viewer D becomes a co-streamer.

     Sample code for Android

     // Initialize the class for stream ingest configurations.
     AlivcLivePushConfig alivcLivePushConfig = new AlivcLivePushConfig();
     alivcLivePushConfig.setLivePushMode(AlivcLiveMode.AlivcLiveInteractiveMode);
     // Set the resolution to 540p. The supported maximum resolution is 720p.
     alivcLivePushConfig.setResolution(AlivcResolutionEnum.RESOLUTION_540P);
     // We recommend that you set the frame rate to 20 FPS.
     alivcLivePushConfig.setFps(AlivcFpsEnum.FPS_20);
     // Specify whether to enable adaptive bitrate streaming. The default value is true.
     alivcLivePushConfig.setEnableBitrateControl(true);
     // The default screen orientation is portrait. You can configure the settings that allow you to press the Home key to change the orientation to landscape left or landscape right.
     alivcLivePushConfig.setPreviewOrientation(AlivcPreviewOrientationEnum.ORIENTATION_PORTRAIT);
     // Specify the audio encoding format.
     alivcLivePushConfig.setAudioProfile(AlivcAudioAACProfileEnum.AAC_LC);
     
     AlivcLivePusher alivcLivePusher = new AlivcLivePusher();
     alivcLivePusher.init(context, alivcLivePushConfig);
     alivcLivePusher.setLivePushErrorListener(new AlivcLivePushErrorListener() {});
     alivcLivePusher.setLivePushInfoListener(new AlivcLivePushInfoListener() {});
     alivcLivePusher.setLivePushNetworkListener(new AlivcLivePushNetworkListener() {});
     
     alivcLivePusher.startPreview(context, frameLayout, isAnchor); // The preview settings for the co-streamer. Set isAnchor to false.
     alivcLivePusher.startPushAysnc("artc://live.aliyun.com/push/123?timestamp=1661596947&token=XXX&userId=userD&sdkAppId=XXX");  // The ingest URL of the co-streamer.

     Sample code for iOS

     AlivcLivePusher *rtcPusher = [[AlivcLivePusher alloc] initWithConfig:rtcPushConfig];
     [rtcPusher startPushWithURL:@"artc://live.aliyun.com/push/123?timestamp=1661596947&token=XXX&userId=userD&sdkAppId=XXX"]; // The ingest URL of the co-streamer.

   • Streamer A obtains the streaming URL for co-streaming of Co-streamer D, and creates an AlivcLivePlayer object to pull the stream of Co-streamer D in real time.

     Sample code for Android

     AlivcLivePlayConfig config = new AlivcLivePlayConfig();
     config.isFullScreen = isAnchor;
     
     AlivcLivePlayer alivcLivePlayer = new AlivcLivePlayerImpl(context, AlivcLiveMode.AlivcLiveInteractiveMode);
     alivcLivePlayer.setPlayInfoListener(new AlivcLivePlayInfoListener() {});
     
     mAlivcLivePlayer.setupWithConfig(config);
     mAlivcLivePlayer.setPlayView(frameLayout);
     
     alivcLivePlayer.startPlay(artc://live.aliyun.com/play/123?timestamp=1661596947&token=XXX&userId=userD&sdkAppId=XXX");  // The streaming URL for co-streaming of the other side. In this case, the streamer uses the streaming URL of the co-streamer to pull the stream of the co-streamer.

     Sample code for iOS

     AlivcLivePlayer *rtcPlayer = [[AlivcLivePlayer alloc] init];
     [rtcPlayer setLivePlayerDelegate:self];
     [rtcPlayer setPlayView:self.playerView  playCofig:self.rtcPlayConfig];
     [rtcPlayer startPlayWithURL:@"artc://live.aliyun.com/play/123?timestamp=1661596947&token=XXX&userId=userD&sdkAppId=XXX"];"];  // The streaming URL for co-streaming of the other side. In this case, the streamer uses the streaming URL of the co-streamer to pull the stream of the co-streamer.

   • At the same time, Co-streamer D obtains the streaming URL for co-streaming of Streamer A, and creates an AlivcLivePlayer object to pull the stream of Streamer A in real time.

     Sample code for Android

     AlivcLivePlayConfig config = new AlivcLivePlayConfig();
     config.isFullScreen = isAnchor;
     
     AlivcLivePlayer alivcLivePlayer = new AlivcLivePlayerImpl(context, AlivcLiveMode.AlivcLiveInteractiveMode);
     alivcLivePlayer.setPlayInfoListener(new AlivcLivePlayInfoListener() {});
     
     mAlivcLivePlayer.setupWithConfig(config);
     mAlivcLivePlayer.setPlayView(frameLayout);
     
     alivcLivePlayer.startPlay(artc://live.aliyun.com/play/123?timestamp=1661596947&token=XXX&userId=userA&sdkAppId=XXX"); // The streaming URL for co-streaming of the other side. In this case, the co-streamer uses the streaming URL of the streamer to pull the stream of the streamer.

     Sample code for iOS

     AlivcLivePlayer *rtcPlayer = [[AlivcLivePlayer alloc] init];
     [rtcPlayer setLivePlayerDelegate:self];
     [rtcPlayer setPlayView:self.playerView  playCofig:self.rtcPlayConfig];
     [rtcPlayer startPlayWithURL:@"artc://live.aliyun.com/play/123?timestamp=1661596947&token=XXX&userId=userA&sdkAppId=XXX"];"];  // The streaming URL for co-streaming of the other side. In this case, the co-streamer uses the streaming URL of the streamer to pull the stream of the streamer.

     Interactive co-streaming is implemented for Streamer A and Co-streamer D at an ultra-low latency.

4. Streamer A updates the mixed stream after co-streaming succeeds

   • For a video stream:

     In order to ensure that Viewer B and Viewer C (who are not co-streamers) can watch the stream of Co-streamer D, Streamer A must perform the stream mixing operation. This way, the streams of Streamer A and Co-streamer D can be mixed into a single stream, which the viewers can pull from CDN for playback. Specifically, Streamer A calls the setLiveMixTranscodingConfig method to start a mixed-stream transcoding task and specify the streams to be mixed. Parameters such as the resolution and frame rate of the mixed stream are those that Streamer A configured in AlivcLivePushConfig when creating the AlivcLivePusher object. For example, if Streamer A set the resolution to 720p in AlivcLivePushConfig when creating the AlivcLivePusher object, the resolution of the mixed stream is 720p.

     Sample code for Android

     AlivcLiveTranscodingConfig transcodingConfig = new AlivcLiveTranscodingConfig();
     
     AlivcLiveMixStream anchorMixStream = new AlivcLiveMixStream();
     anchorMixStream.setUserId(anchorId);
     anchorMixStream.setX(0);
     anchorMixStream.setY(0);
     anchorMixStream.setWidth(mAlivcLivePushConfig.getWidth());
     anchorMixStream.setHeight(mAlivcLivePushConfig.getHeight());
     anchorMixStream.setZOrder(1);
     
     AlivcLiveMixStream audienceMixStream = new AlivcLiveMixStream();
     audienceMixStream.setUserId(audience);
     audienceMixStream.setX((int) mAudienceFrameLayout.getX() / 3);
     audienceMixStream.setY((int) mAudienceFrameLayout.getY() / 3);
     audienceMixStream.setWidth(mAudienceFrameLayout.getWidth() / 2);
     audienceMixStream.setHeight(mAudienceFrameLayout.getHeight() / 2);
     audienceMixStream.setZOrder(2);
     
     ArrayList<AlivcLiveMixStream> mixStreams = new ArrayList<>();
     mixStreams.add(anchorMixStream);
     mixStreams.add(audienceMixStream);
     transcodingConfig.setMixStreams(mixStreams);
     
     alivcLivePusher.setLiveMixTranscodingConfig(transcodingConfig);

     Sample code for iOS

      AlivcLiveTranscodingConfig *liveTranscodingConfig = [[AlivcLiveTranscodingConfig alloc] init];
     
     AlivcLiveMixStream *anchorMixStream = [[AlivcLiveMixStream alloc] init];
     anchorMixStream.userId = userA;
     anchorMixStream.x = 0;
     anchorMixStream.y = 0;
     anchorMixStream.width = [self.rtcPushConfig getPushResolution].width;
     anchorMixStream.height = [self.rtcPushConfig getPushResolution].height;
     anchorMixStream.zOrder = 1;
     
     AlivcLiveMixStream *audienceMixStream = [[AlivcLiveMixStream alloc] init];
     audienceMixStream.userId = userD;
     audienceMixStream.x = 100;
     audienceMixStream.y = 200;
     audienceMixStream.width = 200;
     audienceMixStream.height = 300;
     audienceMixStream.zOrder = 2;
     
     liveTranscodingConfig.mixStreams = [NSArray arrayWithObjects:anchorMixStream, audienceMixStream, nil];
     [self.rtcPusher setLiveMixTranscodingConfig:liveTranscodingConfig];

     Viewers can watch the mixed stream of Streamer A and Co-streamer D.

   • For an audio-only stream:

     To update the mixed stream, the streamer only needs to specify the user ID of the co-streamer in AlivcLiveMixStream. The position information is not required.

     Sample code for Android

     AlivcLiveTranscodingConfig transcodingConfig = new AlivcLiveTranscodingConfig();
     
     AlivcLiveMixStream anchorMixStream = new AlivcLiveMixStream();
     anchorMixStream.setUserId(anchorId);
     
     AlivcLiveMixStream audienceMixStream = new AlivcLiveMixStream();
     audienceMixStream.setUserId(audience);
     
     ArrayList<AlivcLiveMixStream> mixStreams = new ArrayList<>();
     mixStreams.add(anchorMixStream);
     mixStreams.add(audienceMixStream);
     transcodingConfig.setMixStreams(mixStreams);
     
     alivcLivePusher.setLiveMixTranscodingConfig(transcodingConfig);

     Sample code for iOS

      AlivcLiveTranscodingConfig *liveTranscodingConfig = [[AlivcLiveTranscodingConfig alloc] init];
     
     AlivcLiveMixStream *anchorMixStream = [[AlivcLiveMixStream alloc] init];
     anchorMixStream.userId = userA;
     
     AlivcLiveMixStream *audienceMixStream = [[AlivcLiveMixStream alloc] init];
     audienceMixStream.userId = userD;
     
     liveTranscodingConfig.mixStreams = [NSArray arrayWithObjects:anchorMixStream, audienceMixStream, nil];
     [self.rtcPusher setLiveMixTranscodingConfig:liveTranscodingConfig];
                                     

5. Co-streamer D ends co-streaming

   • To stop co-streaming, Co-streamer D must call the method used to stop stream ingest in AlivcLivePusher and then destroy AlivcLivePusher.

     Sample code for Android

     alivcLivePusher.stopPush();
     alivcLivePusher.destroy();
     alivcLivePusher = null;

     Sample code for iOS

     [self.rtcPusher stopPush];
     [self.rtcPusher destory];
     self.rtcPusher = nil;

   • At the same time, Co-streamer D must also call the method used to stop playback in AlivcLivePlayer and then destroy AlivcLivePlayer.

     Sample code for Android

     alivcLivePlayer.stopPlay();
     alivcLivePlayer.destroy();
     alivcLivePlayer = null;

     Sample code for iOS

     [self.rtcPlayer stopPlay];
     self.rtcPlayer = nil;

   • Co-streamer D becomes an ordinary viewer, and can pull the stream from CDN. For more information, see Step 4: Create an AliPlayer object for CDN playback.

   • Streamer A needs to call the method used to stop playback in AlivcLivePlayer and then destroy AlivcLivePlayer.

     Sample code for Android

     alivcLivePlayer.stopPlay();
     alivcLivePlayer.destroy();
     alivcLivePlayer = null;

     Sample code for iOS

     [self.rtcPlayer stopPlay];
     self.rtcPlayer = nil;

   • At the same time, Streamer A needs to call the setLiveMixTranscodingConfig method and pass in an empty value. This way, stream mixing is stopped.

     Take note that whenever viewers only want to watch the stream of Streamer A, Streamer A can call the setLiveMixTranscodingConfig method and pass in an empty value to stop stream mixing.

     Warning

     If the streamer is still in the room but does not require stream mixing, make sure that stream mixing is stopped. If stream mixing is not stopped, the stream mixing module that continues to run will incur unnecessary fees.

     Sample code for Android

     alivcLivePusher.setLiveMixTranscodingConfig(null);

     Sample code for iOS

     [self.rtcPusher setLiveMixTranscodingConfig:nil];