ApsaraVideo Live:Use Push SDK for Android

Register the SDK

You must register the SDK before stream ingestion. Otherwise, you cannot use the SDK.

An SDK license is required for registration. To apply for and configure a license, see Integrate a Push SDK license.

Call the following methods before you use Push SDK for Android: 

AlivcLiveBase.setListener(new AlivcLiveBaseListener() {
  @Override
  public void onLicenceCheck(AlivcLivePushConstants.AlivcLiveLicenseCheckResultCode result, String reason) {
    Log.e(TAG, "onLicenceCheck: " + result + ", " + reason);
  }
});
AlivcLiveBase.registerSDK();

• AlivcLiveBase allows you to configure log levels, set local log paths, and retrieve the SDK version.

• Call the registerSDK method in AlivcLiveBase to register the SDK license.

• By implementing the onLicenceCheck method through AlivcLiveBase#setListener, you can asynchronously verify license configuration.

  Note

  This callback is only triggered after initializing the pusher instance.

Ingest a camera stream

1. Initialize the SDK.

   After you configure stream ingest parameters, call the init method to initialize the class. Sample code:

   AlivcLivePusher mAlivcLivePusher = new AlivcLivePusher();
   mAlivcLivePusher.init(mContext, mAlivcLivePushConfig);

   Note

   AlivcLivePusher does not support multiple instances. Therefore, each init method must correspond to a destroy method.

2. Register preview callbacks.

   Call the setLivePushInfoListener method to register preview callbacks:

   /**
    * Configure callbacks for stream ingest events.
    *
    * @param infoListener The listener.
    */
   mAlivcLivePusher.setLivePushInfoListener(new AlivcLivePushInfoListener() {
       @Override
       public void onPreviewStarted(AlivcLivePusher pusher) {
           // Notifies that preview starts.
       }
       // Other Override methods
       //....
       //....
   });

3. Start preview.

   To preview the camera feeds, specify the SurfaceView for the camera. Sample code:

   mAlivcLivePusher.startPreview(mSurfaceView)// Start preview. You can also call the startPreviewAysnc method (asynchronous).

4. Start stream ingest.

   Add the following code to the onPreviewStarted callback method: 

   mAlivcLivePusher.startPush(mPushUrl);

   Note

   • RTMP and RTS (artc://) ingest URLs are supported. To generate ingest URLs, see Generate ingest and streaming URLs.

   • ApsaraVideo Live does not support ingesting multiple streams to the same URL simultaneously. The second ingest request will be rejected.

/* Pause stream ingest. For a stream that is being ingested, the video preview and video stream ingest are paused at the last frame, and the audio stream continues to be ingested. */
mAlivcLivePusher.pause();
/* Resume stream ingest. The preview and ingest of audio and video streams are resumed. */
mAlivcLivePusher.resume();
/* Stop a stream that is being ingested. */
mAlivcLivePusher.stopPush();
/* Stop preview. This operation does not take effect for a stream that is being ingested. When preview is stopped, the preview window is frozen at the last frame. */
mAlivcLivePusher.stopPreview();
/* Restart stream ingest when the stream is being ingested or when an error callback is received. All resources in ALivcLivePusher are reinitialized, including preview and ingestion. If an error occurs, you can call this method or the reconnectPushAsync method to restart stream ingest. You can also call the destroy method to destroy the stream ingest instance. */
mAlivcLivePusher.restartPush();
/* Reconnect and repush the RTMP stream during streaming or network error state (errors related to AlivcLivePusherNetworkDelegate). In the error state, you can also call destroy to dispose the instance.*/
mAlivcLivePusher.reconnectPushAsync();
/* Dispose the stream ingest instance. After you call this method, stream ingest and preview are stopped, and the preview window is removed. All resources related to AlivcLivePusher are released. */
mAlivcLivePusher.destroy();

/* Switch between the front camera and the rear camera. */
mAlivcLivePusher.switchCamera();
/* Enable or disable the flash. You cannot enable the flash for the front camera. */
mAlivcLivePusher.setFlash(true); 
/* Adjust the focal length to zoom in or out. Valid values: [0,getMaxZoom()]. */
mAlivcLivePusher.setZoom(5);
/* Configure manual focus. Set the following parameters: point and autoFocus. The point parameter specifies the coordinates of the focus point. The autoFocus parameter specifies whether to enable autofocus. The autoFocus parameter takes effect only for this call. Whether autofocus is enabled otherwise depends on the setAutoFocus method. */
mAlivcLivePusher.focusCameraAtAdjustedPoint(x, y, true);
/* Specify whether to enable autofocus. */
mAlivcLivePusher.setAutoFocus(true);
/* Configure mirroring. The methods for mirroring are PushMirror and PreviewMirror. PushMirror takes effect only for stream playback, and PreviewMirror takes effect only for preview. */
mAlivcLivePusher.setPreviewMirror(false);
mAlivcLivePusher.setPushMirror(false);

Ingest a screen sharing stream

Push SDK supports ingesting screen sharing streams. You need to initialize the SDK and configure the preview before ingestion. After the stream is ingested, you can set advanced configurations. 

Android's MediaProjection API is used for capturing screen content, which requires user permission request. The data returned from the permission request must be passed to setMediaProjectionPermissionResultData in AlivcLivePushConfig to enable screen sharing. By default, the camera is disabled during screen sharing.

// resultData: the system intent for screen sharing
mAlivcLivePushConfig.setMediaProjectionPermissionResultData(resultData);

mAlivcLivePusher.setScreenOrientation(0);

mAlivcLivePusher.pauseScreenCapture();// Enable privacy protection.
mAlivcLivePusher.resumeScreenCapture();// Disable privacy protection.

Ingest both camera and screen sharing streams

When a streamer is sharing the screen with viewers, they can also ingest a camera stream. There are two modes in this scenario: 

1. The streamer can see the camera stream.

2. The streamer cannot see the camera stream.

   For example, during game streaming, the streamer does not want the camera view to block the game screen. However, the viewers can view the camera stream.

mAlivcLivePusher.startCamera(surfaceView);// Enable camera preview.
mAlivcLivePusher.stopCamera();// Disable camera preview.

mAlivcLivePusher.startCameraMix(x, y, w, h);// Enable camera stream mixing.
mAlivcLivePusher.stopCameraMix();// Disable camera stream mixing.

Configure preview display mode

Push SDK for Android supports the following preview modes. The preview mode does not affect stream ingest.

• ALIVC_LIVE_PUSHER_PREVIEW_SCALE_FILL: The video fills the preview window. If the aspect ratios of the video and preview window are inconsistent, video deformation occurs.

• ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FIT: The aspect ratio of the video is preserved. If aspect ratios differ, black bars appear on the preview window.

• ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FILL: The video is cropped to fit the preview window when aspect ratios differ.

Sample code: 

mAlivcLivePushConfig.setPreviewDisplayMode(AlivcPreviewDisplayMode.ALIVC_LIVE_PUSHER_PREVIEW_ASPECT_FIT);

Ingest an image

Push SDK for Android supports ingesting an image when the application is switched to the background or the bitrate is low. 

When the application is switched to the background, video stream ingest is paused by default, and only the audio stream is ingested. Streamers can display an image, informing viewers that the streamer is away and will be back soon.

mAlivcLivePushConfig.setPausePushImage("Image Path");// Specify the image that is ingested when the app is switched to the background.

You can also specify an image to be ingested in poor network conditions. When the bitrate is low, the image is displayed to prevent stuttering. Sample code:

mAlivcLivePushConfig.setNetworkPoorPushImage("Image Path");// Specify the image that is ingested in poor network conditions.

Ingest an audio-only stream

To transmit only audio to viewers, set setAudioOnly to true: 

mAlivcLivePushConfig.setAudioOnly(true);

Push external audio/video sources

Push SDK for Android supports ingesting external audio/video sources, such as a video file.

Before ingestion, enable custom audio and video input.

/**
* Other parameters, such as the output resolution, audio sample rate, and number of channels, are configured in config by using the setResolution, setAudioSampleRate, and setAudioChannels methods.
*/
mAlivcLivePushConfig.setExternMainStream(true,AlivcImageFormat.IMAGE_FORMAT_YUVNV12,AlivcSoundFormat.SOUND_FORMAT_S16);

Then, ingest external audio and video sources: 

/**
* This method does not configure the time sequence. You must manually configure the time sequence of input audio frames.
*/
mAlivcLivePusher.inputStreamAudioData(byte[] data, int size, int sampleRate, int channels, long pts);

/**
* This method does not configure the time sequence. You must manually configure the time sequence of input video frames.
*/
mAlivcLivePusher.inputStreamVideoData(byte[] data, int width, int height, int stride, int size, long pts, int rotation);

Configure watermarks

Push SDK for Android supports adding one or more watermarks in the PNG format. Sample code: 

mAlivcLivePushConfig.addWaterMark(waterPath,0.1,0.2,0.3);// Add a watermark.

Where:

• x and y are relative values that determine the watermark position. x=0.1 specifies that the left edge of the watermark is positioned at 10% of the stream width. When the resolution is 540 x 960, the x position is 540 x 0.1 = 54 pixels.

• width specifies the watermark width relative to stream width. The height is proportionally scaled.

Configure video quality

Push SDK for Android supports the following video quality modes: Resolution Priority, Fluency Priority, and custom.

mAlivcLivePushConfig.setQualityMode(AlivcQualityModeEnum.QM_RESOLUTION_FIRST);// Prioritize resolution

mAlivcLivePushConfig.setQualityMode(AlivcQualityModeEnum.QM_FLUENCY_FIRST);// Prioritize fluency

mAlivcLivePushConfig.setQualityMode(AlivcQualityModeEnum.QM_CUSTOM);// Custom mode
mAlivcLivePushConfig.setTargetVideoBitrate(1000); // The target bitrate is 1,000 Kbit/s.
mAlivcLivePushConfig.setMinVideoBitrate(300); // The minimum bitrate is 300 Kbit/s.
mAlivcLivePushConfig.setInitialVideoBitrate(800); // The initial bitrate is 800 Kbit/s.

Configure adaptive resolution

The SDK supports dynamically adjusting the resolution of an ingested stream. When the feature is enabled, the resolution is automatically reduced to ensure the smoothness and quality in poor network conditions. Sample code:

mAlivcLivePushConfig.setEnableAutoResolution(true); // Enable adaptive resolution. The default value is false.

Configure background music

Push SDK for Android allows you to manage background music. You can control the background music playback and configure audio mixing, denoising, in-ear monitoring, and muting. Sample code: 

/* Start the playback of background music. */
mAlivcLivePusher.startBGMAsync(mPath)；
/* Stop the playback of background music. If you want to change the background music, call the startBGMAsync method. You do not need to stop the current playback. */
mAlivcLivePusher.stopBGMAsync();
/* Pause the playback of background music. You can call this method only after the playback of background music starts. */
mAlivcLivePusher.pauseBGM();
/* Resume the playback of background music. You can call this method only after the playback of background music is paused. */
mAlivcLivePusher.resumeBGM();
/* Enable looping. */
mAlivcLivePusher.setBGMLoop(true);
/* Configure denoising. When enabled, the system filters out non-vocal parts from the collected audio. This feature may slightly reduce the volume of the human voice. We recommend that you allow your users to determine whether to enable this feature. By default, this feature is disabled. */
mAlivcLivePusher.setAudioDenoise(true);
/* Configure in-ear monitoring. In-ear monitoring is suitable for karaoke scenarios. When enabled, headphone users can hear their voice. When disabled, they cannot hear their voice on headphones. This parameter does not take effect if no headphones are detected. */
mAlivcLivePusher.setBGMEarsBack(true);
/* Configure audio mixing to adjust the volumes of the background music and human voice. */
mAlivcLivePusher.setBGMVolume(50);// Specify the volume of the background music.
mAlivcLivePusher.setCaptureVolume(50);// Specify the volume of the human voice.
/* Configure muting. When enabled, the background music and the human voice are both muted. To separately mute one of them, call the method that is used to configure audio mixing. */
mAlivcLivePusher.setMute(true);

Capture snapshots

Push SDK for Android supports capturing snapshots of local video streams. Sample code: 

// Capture snapshots of video streams. Parameters: the number of snapshots to capture, the capture interval, and the callback.
pusher.snapshot(1, 1, new AlivcSnapshotListener() {
    @Override
    public void onSnapshot(Bitmap bmp) {
            // You can save each snapshot as a local PNG file. Sample code:
        String dateFormat = new SimpleDateFormat("yyyy-MM-dd-hh-mm-ss-SS").format(new Date());
        File f = new File(context.getExternalFilesDir(Environment.DIRECTORY_PICTURES), "snapshot-" + dateFormat + ".png");
        if (f.exists()) {
            f.delete();
        }
        try {
            FileOutputStream out = new FileOutputStream(f);
            bmp.compress(Bitmap.CompressFormat.PNG, 90, out);
            out.flush();
            out.close();
        } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
});

Handle background running and screen locking

• When the app is switched to the background or the screen is locked, call the pause() or resume() method in the AlivcLivePusher class to pause or resume stream ingest.

• For non-system audio and video calls, the SDK continues capturing and ingesting the audio stream. You can call the mAlivcLivePusher.setMute() method to specify whether to enable audio capturing when the app is switched to the background or the screen is locked.

Callbacks

Common methods and interfaces

/* In custom mode, you can adjust the minimum and target bitrates in real time. */
mAlivcLivePusher.setTargetVideoBitrate(800);
mAlivcLivePusher.setMinVideoBitrate(400);
/* Specify whether autofocus is supported. */
mAlivcLivePusher.isCameraSupportAutoFocus();
/* Specify whether flash is supported. */
mAlivcLivePusher.isCameraSupportFlash();
/* Query whether the stream is being ingested. */
mAlivcLivePusher.isPushing()； 
/* Obtain the ingest URL. */
mAlivcLivePusher.getPushUrl()；
/* Query the stream ingest performance debugging information. For more information about the parameters of stream ingest performance, see the API references or comments in the code. */
mAlivcLivePusher.getLivePushStatsInfo();
/* Obtain the SDK version number. */
mAlivcLivePusher.getSDKVersion();
/* Specify the log level to filter debugging information. */
mAlivcLivePusher.setLogLevel(AlivcLivePushLogLevelAll);
/* Query the status of Push SDK for Android. */
mAlivcLivePusher.getCurrentStatus();
/* Query the last error code. If no error occurs, ALIVC_COMMON_RETURN_SUCCESS is returned. */
mAlivcLivePusher.getLastError();

Integrate the retouching feature

To use the retouching feature in Push SDK for Android, import a retouching library and configure callbacks.

1. Import the retouching library and panel by using Maven. Add the following code to the build.gradle file of your project. For the latest version of Queen SDK, see Download Queen SDK.

   implementation "com.aliyun.maliang.android:queen:2.5.0-official-full"
   implementation("com.aliyun.maliang.android:queen_menu:2.5.0-official-full") {
       exclude group: 'com.aliyun.maliang.android', module: 'queen'
   }

   You can also integrate the LiveBeauty module provided in the demo:

   File or folder	Description
   live_beauty	The abstract class for retouching.
   queen_beauty	The user interface (UI) widgets for retouching.

2. Obtain the LiveBeauty plug-in library.

   1. Use the clone command to download the plug-in library code from LiveBeautyto your local device.

      git clone https://github.com/MediaBox-Demos/amdemos-android-live.git

   2. Open the command line in the root directory of your Android Studio project. Run the following code, and then choose File > New > Import Module to import the LiveBeauty module to your Android project.

      git submodule add https://github.com/MediaBox-Demos/amdemos-android-live.git ***/***/***

      Where ***/***/*** specifies the path to save the LiveBeauty module.

   3. Add the path of the module to the settings.gradle file of your project:

      include ':app', ':LiveBeauty', ':LiveBeauty:live_queenbeauty'

      In the sample code, "app" is the main module.

   4. Add the dependency on the LiveBeauty module to the build.gradle file of the main module of your project:

      dependencies {
          implementation project(':LiveBeauty')
          implementation project(':LiveBeauty:live_queenbeauty')
      }

   5. Click File > Sync Project with Gradle Files and wait for Gradle to complete the synchronization. Then, you can use the LiveBeauty module.

3. Configure the UI module of the retouching plug-in.

   1. Add the QueenBeautyMenu widget to the layout XML file of your project. Example:

      <com.aliyunsdk.queen.menu.QueenBeautyMenu
          android:id="@+id/beauty_beauty_menuPanel"
          android:layout_width="match_parent"
          android:layout_height="wrap_content"
          android:layout_alignParentBottom="true"
          android:layout_centerHorizontal="true" />

   2. Initialize QueenBeautyMenu in the Activity. Example:

      // Initialize the rotouching panel.
      QueenMenuPanel beautyMenuPanel = QueenBeautyMenu.getPanel(context);
      beautyMenuPanel.onHideMenu(); 
      beautyMenuPanel.onHideValidFeatures(); 
      beautyMenuPanel.onHideCopyright(); 
      
      // Add the panel to the layout.
      QueenBeautyMenu beautyBeautyContainerView = findViewById(R.id.beauty_beauty_menuPanel);
      beautyBeautyContainerView.addView(beautyMenuPanel);

4. Configure the callbacks for facial recognition and retouching.

   If you want to access a third-party retouching library, register the setCustomDetect and setCustomFilter callbacks.

   • The data parameter returned by the customDetectProcess method of AlivcLivePushCustomDetect serves as the pointer for collected video frame data. The callback includes comprehensive parameters: long data, int width, int height, int rotation, int format, and long extra. These parameters provide detailed information about the video frame, enabling third-party retouching libraries to identify, analyze, or process the raw frame data with precision and flexibility.

   • The inputTexture parameter returned by the customFilterProcess method of AlivcLivePushCustomFilter represents the source image texture available for processing by third-party retouching libraries. The callback provides comprehensive parameters: int inputTexture, int textureWidth, int textureHeight, and long extra. If no custom processing is performed, return the original inputTexture. For processed textures, return the new texture ID.

   Sample code

   /**
    * The callback for facial recognition.
    */
   mAlivcLivePusher.setCustomDetect(new AlivcLivePushCustomDetect() {
       @Override
       public void customDetectCreate() {
   
       }
   
       @Override
       public long customDetectProcess(long dataPtr, int width, int height, int rotation, int format, long extra) {
           return 0;
       }
   
       @Override
       public void customDetectDestroy() {
   
       }
   });
   
   /**
    * The callback for retouching.
    */
    
    /**
    * Initialize BeautyManager
    */
   mAlivcLivePusher.setCustomFilter(new AlivcLivePushCustomFilter() {
       @Override
       public void customFilterCreate() {
           initBeautyManager();
       }
   
    /**
    * Process the video stream and add retouching effects
    */
       @Override
       public int customFilterProcess(int inputTexture, int textureWidth, int textureHeight, long extra) {
           if (mBeautyManager == null) {
               return inputTexture;
           }
   
           return mBeautyManager.onTextureInput(inputTexture, textureWidth, textureHeight);
       }
   
       @Override
       public void customFilterDestroy() {
           destroyBeautyManager();
       }
   });