ApsaraVideo Live:Implement screen sharing on Android

1. Set permissions

To implement stable and compatible screen sharing on Android, you must add the following permissions and service declaration to your AndroidManifest.xml file.

• FOREGROUND_SERVICE: Allows the app to run a foreground service, which prevents the screen sharing service from being terminated by the system when the app is in the background.

• FOREGROUND_SERVICE_MEDIA_PROJECTION: A granular permission introduced in Android 10 (API level 29) that identifies the foreground service as being used for screen capture (MediaProjection). This permission is mandatory for apps targeting Android 14 (API level 34) and later.

• POST_NOTIFICATIONS: Starting from Android 13 (API level 33), an app must declare this permission to post notifications, such as those for a foreground service. Without user authorization, the notification cannot be shown.

• Service declaration: To implement a foreground service, you must declare the service component in AndroidManifest.xml for the system to recognize and launch it. Failure to declare the service will result in an error or prevent it from starting.

<manifest ...>
    <!-- Declare foreground service permissions -->
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION" />
    <!-- Required for Android 13 (API level 33) and later -->
    <uses-permission android:name="android.permission.POST_NOTIFICATIONS"/> 
    
    <application ...>
          <!-- Declare the foreground service for screen sharing -->
          <service android:name="com.alivc.rtc.share.ForegroundService"
            android:enabled="true" />
    </application>
</manifest>

2. Configure camera and screen sharing streams

ARTC supports ingesting camera and screen sharing streams. Configure them based on your use case.

mAliRtcEngine.publishLocalVideoStream(false);

mAliRtcEngine.publishLocalVideoStream(true);

3. Configure the screen sharing stream encoder

To customize the encoding properties of the screen sharing stream, call the setScreenShareEncoderConfiguration method. You can configure properties such as resolution, frame rate, bitrate, keyframe interval (GOP), and video orientation.

The following table describes the configuration parameters:

Sample code:

AliRtcEngine.AliRtcScreenShareEncoderConfiguration screenShareEncoderConfiguration = new AliRtcEngine.AliRtcScreenShareEncoderConfiguration();
screenShareEncoderConfiguration.bitrate = bitrate;
screenShareEncoderConfiguration.dimensions = new AliRtcEngine.AliRtcVideoDimensions(width, height);
screenShareEncoderConfiguration.frameRate = fps;
screenShareEncoderConfiguration.keyFrameInterval = gop;
screenShareEncoderConfiguration.forceStrictKeyFrameInterval= mForceKeyFrameSwitch.isChecked();
mAliRtcEngine.setScreenShareEncoderConfiguration(screenShareEncoderConfiguration);

4. Start screen capture

Call the startScreenShare method to start the screen capture process, which captures the device's screen and ingests the stream into the channel. Configure the parameters based on your use case:

1. intent: The Activity used to start screen sharing. If you do not create a custom Activity, passing null is recommended.

2. screenShareMode: The screen sharing mode. You can choose to share only audio, only video, or both.

Button mStartScreenShareBtn = findViewById(R.id.start_screen_share_btn);
mStartScreenShareBtn.setOnClickListener(v -> {
    if(mAliRtcEngine != null) {
        // Configure the screen sharing stream encoder.
        getScreenShareEncoderConfiguration();
        mAliRtcEngine.setScreenShareEncoderConfiguration(screenShareEncoderConfiguration);
        // Start screen capture.
        mAliRtcEngine.startScreenShare(null, screenShareMode);
    }
});

5. Stop screen capture

Call the stopScreenShare method to stop screen capture. This terminates the ingestion of the screen sharing stream and releases the associated resources.

Button mStopScreenShareBtn = findViewById(R.id.stop_screen_share_btn);
mStopScreenShareBtn.setOnClickListener(v -> {
    if(mAliRtcEngine != null && mAliRtcEngine.isScreenSharePublished()) {
        mAliRtcEngine.stopScreenShare();
    }
});

6. View the shared screen

When a user starts screen sharing, other users in the channel receive the onRemoteTrackAvailableNotify callback. This event notifies the client when a remote user publishes or unpublishes their camera or screen sharing stream. Your application can use the parameters in this callback to dynamically create or destroy the render view associated with the stream.

The AliRtcVideoTrack enum in the onRemoteTrackAvailableNotify callback identifies the state of the remote video stream:

• AliRtcVideoTrackNo (0): The remote user is not publishing any video stream.

• AliRtcVideoTrackCamera (1): Only the camera stream is pushed.

• AliRtcVideoTrackScreen (2): Only the screen sharing stream is pushed.

• AliRtcVideoTrackBoth (3): Both the camera and screen sharing streams are pushed.

Based on this value, your application can adjust its UI logic accordingly, such as rendering camera and screen sharing streams in separate views or removing views for streams that are no longer available.

private AliRtcEngineNotify mRtcEngineNotify = new AliRtcEngineNotify() {
    // Set the render view for the remote video stream in the onRemoteTrackAvailableNotify callback.
    @Override
    public void onRemoteTrackAvailableNotify(String uid, AliRtcEngine.AliRtcAudioTrack audioTrack, AliRtcEngine.AliRtcVideoTrack videoTrack){
        handler.post(new Runnable() {
            @Override
            public void run() {
                if(videoTrack == AliRtcVideoTrackCamera || videoTrack == AliRtcVideoTrackScreen) {
                    viewRemoteVideo(uid, videoTrack);
                } else if (videoTrack == AliRtcVideoTrackBoth) {
                    viewRemoteVideo(uid, AliRtcVideoTrackCamera);
                    viewRemoteVideo(uid, AliRtcVideoTrackScreen);
                } else if(videoTrack == AliRtcVideoTrackNo) {
                    removeAllRemoteVideo(uid);
                }
            }
        });
    }
}

/**
 * Display the remote video stream (camera or screen sharing).
 * @param uid The remote user ID.
 * @param videoTrack The video stream type.
 */
private void viewRemoteVideo(String uid, AliRtcEngine.AliRtcVideoTrack videoTrack) {
    String streamKey = getStreamKey(uid, videoTrack);
    FrameLayout view;
    if (remoteViews.containsKey(streamKey)) {
        view = remoteViews.get(streamKey);
        if (view != null) {
            view.removeAllViews();
        } else {
            view = createVideoView(streamKey);
            gridVideoContainer.addView(view);
            remoteViews.put(streamKey, view);
        }
    } else {
        view = createVideoView(streamKey);
        gridVideoContainer.addView(view);
        remoteViews.put(streamKey, view);
    }
    // Create a SurfaceView and set it for rendering.
    SurfaceView surfaceView = mAliRtcEngine.createRenderSurfaceView(this);
    surfaceView.setZOrderMediaOverlay(true);
    view.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
    // Configure the canvas.
    AliRtcEngine.AliRtcVideoCanvas videoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
    videoCanvas.view = surfaceView;
    mAliRtcEngine.setRemoteViewConfig(videoCanvas, uid, videoTrack);
}

/**
 * Removes the view for a specified user's video stream.
 * @param uid The remote user ID.
 * @param videoTrack The video stream type.
 */
private void removeRemoteVideo(String uid, AliRtcEngine.AliRtcVideoTrack videoTrack) {
    String streamKey = getStreamKey(uid, videoTrack);

    // Find the corresponding FrameLayout container and remove the view. 
    FrameLayout frameLayout = remoteViews.remove(streamKey);
    if(frameLayout != null) {
        frameLayout.removeAllViews();
        gridVideoContainer.removeView(frameLayout);
        Log.d("RemoveRemoteVideo", "Removed video stream for: " + streamKey);
    }
}
/**
 * Removes all video views for a specified user.
 * @param uid The remote user ID. 
 */
private void removeAllRemoteVideo(String uid) {
    removeRemoteVideo(uid, AliRtcVideoTrackCamera);
    removeRemoteVideo(uid, AliRtcVideoTrackScreen);
}

7. (Optional) Configure the screen sharing audio volume

To share system audio, call the setAudioShareVolume method to control the volume of the screen sharing audio stream.

mAliRtcEngine.setAudioShareVolume(60);