ApsaraVideo Live:Implement audio/video communications on Android

Step 1: Import SDK

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        google()
        mavenCentral()
        // Add the Maven repository for the ARTC SDK
        maven { url 'https://maven.aliyun.com/repository/google' }
        maven { url 'https://maven.aliyun.com/repository/public' }
    }
}

allprojects {
    repositories {
        ...
        // Add the Maven repository for the ARTC SDK
        maven { url 'https://maven.aliyun.com/repository/google' }
        maven { url 'https://maven.aliyun.com/repository/public' }
    }
}

dependencies {
    // Add the dependencies for the ARTC SDK
    // Replace ${latest_version} with the specific version number.
    implementation 'com.aliyun.aio:AliVCSDK_ARTC:${latest_version}'
    // For versions 7.4.0 and below, add the keep dependency
    // implementation 'com.aliyun.aio.keep:keep:1.0.1'
}   

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        google()
        mavenCentral()
        // Add the relative directory where the ARTC SDK is located
        flatDir {
            dir 'app/libs'
        }
    }
}

allprojects {
    repositories {
        ...
        // Add the relative directory where the ARTC SDK is located
        flatDir {
            dir 'app/libs'
        }
    }
}

// Replace x.y.z with the specific version number.
implementation(name:'AliVCSDK_ARTC', version: 'x.y.z', ext:'aar')

Step 2: Configure supported CPU architectures

Open the app/build.gradle file and specify the CPU architectures supported by your project in defaultConfig, as shown in the following example. Available architectures include armeabi-v7a, arm64-v8a, x86, and x86_64.

android {
    defaultConfig {
        // ...other default configurations
        // Support armeabi-v7a and arm64-v8a architectures
        ndk {
             abiFilters "armeabi-v7a", "arm64-v8a"
        }
    }
}	

Step 3: Configure permissions

Configure the permissions required by your application:

Go to the app/src/main directory, open the AndroidManifest.xml file, and add the required permissions.

<uses-feature android:name="android.hardware.camera" android:required="false" /> 
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- Request legacy Bluetooth permissions on older devices. -->
<uses-permission
  android:name="android.permission.BLUETOOTH"
  android:maxSdkVersion="30" />
<uses-permission
  android:name="android.permission.BLUETOOTH_ADMIN"
  android:maxSdkVersion="30" />

<!-- Needed only if your app communicates with already-paired Bluetooth devices. -->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
<uses-permission android:name="android.permission.WRITE_SETTINGS"
  tools:ignore="ProtectedPermissions" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />

Note: For Android 6.0 (API level 23) and later, runtime permissions need to be requested dynamically. In addition to statically declaring them in the AndroidManifest.xml file, you also need to request permissions at runtime.

Runtime permissions include:

• Manifest.permission.CAMERA

• Manifest.permission.WRITE_EXTERNAL_STORAGE

• Manifest.permission.RECORD_AUDIO

• Manifest.permission.READ_EXTERNAL_STORAGE

• Manifest.permission.READ_PHONE_STATE

For Android 12 (API level 31) and later, the following permission also needs to be requested dynamically:

• Manifest.permission.BLUETOOTH_CONNECT

Permission descriptions:

Step 4: Prevent code obfuscation (optional)

In the app/proguard-rules.pro file, configure rules for the SDK to prevent the interfaces provided by the SDK from being obfuscated, which leads to improper function calls.

-keep class com.aliyun.allinone.** {
*;
}

-keep class com.aliyun.rts.network.AliHttpTool {
*;
}

-keep class com.aliyun.common.AlivcBase {
*;
}

-keep class com.huawei.multimedia.alivc.** {
*;
}

-keep class com.alivc.rtc.** {
*;
}

-keep class com.alivc.component.** {
*;
}

-keep class org.webrtc.** {
*;
}

This section explains how to use the ARTC SDK to build a basic real-time audio and video application. You can copy the complete code sample into your project to test the functionality. The steps below explain the core API calls.

The following diagram shows the basic workflow for implementing a video call:

1. Request permissions

When starting a video call, check if the required permissions have been granted in the app:

private static final int REQUEST_PERMISSION_CODE = 101;

private static final String[] PERMISSION_MANIFEST = {
    Manifest.permission.RECORD_AUDIO,
    Manifest.permission.READ_PHONE_STATE,
    Manifest.permission.WRITE_EXTERNAL_STORAGE,
    Manifest.permission.READ_EXTERNAL_STORAGE,
    Manifest.permission.CAMERA
};

private static final String[] PERMISSION_MANIFEST33 = {
    Manifest.permission.RECORD_AUDIO,
    Manifest.permission.READ_PHONE_STATE,
    Manifest.permission.CAMERA
};

private static String[] getPermissions() {
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.TIRAMISU) {
        return PERMISSION_MANIFEST;
    }
    return PERMISSION_MANIFEST33;
}

public boolean checkOrRequestPermission() {
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
        if (ContextCompat.checkSelfPermission(this, "android.permission.CAMERA") != PackageManager.PERMISSION_GRANTED
                || ContextCompat.checkSelfPermission(this, "android.permission.RECORD_AUDIO") != PackageManager.PERMISSION_GRANTED) {
            requestPermissions(getPermissions(), REQUEST_PERMISSION_CODE);
            return false;
        }
    }
    return true;
}

2. Get an authentication token

Joining an ARTC channel requires an authentication token to verify the user's identity. For details on how the token is generated, see Implement token-based authentication. A token can be generated using a single-parameter method or a multi-parameter method. The method you use determines which joinChannel API you need to call.

For production environments:

Because generating a token requires your AppKey, hardcoding it on the client side poses a security risk. In a production environment, we strongly recommend generating the token on your server and sending it to the client.

For development and debugging:

During development, if your business server does not yet have the logic to generate tokens, you can temporarily use the token generation logic from the APIExample to create a temporary token. The reference code is as follows:

public final class ARTCTokenHelper {
    /**
     * RTC AppId
     */
    public static String AppId = "";

    /**
     * RTC AppKey
     */
    public static String AppKey = "";

    /**
     * Generate a single-parameter token for joining a meeting based on channelId, userId, timestamp, and nonce.
     */
    public static String generateSingleParameterToken(String appId, String appKey, String channelId, String userId, long timestamp,  String nonce) {

        StringBuilder stringBuilder = new StringBuilder()
                .append(appId)
                .append(appKey)
                .append(channelId)
                .append(userId)
                .append(timestamp);
        String token =  getSHA256(stringBuilder.toString());
        try{
            JSONObject tokenJson = new JSONObject();
            tokenJson.put("appid", AppId);
            tokenJson.put("channelid", channelId);
            tokenJson.put("userid", userId);
            tokenJson.put("nonce", nonce);
            tokenJson.put("timestamp", timestamp);
            tokenJson.put("token", token);
            String base64Token = Base64.encodeToString(tokenJson.toString().getBytes(StandardCharsets.UTF_8), Base64.NO_WRAP);
            return base64Token;
        }catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

    /**
     * Generate a single-parameter token for joining a meeting based on channelId, userId, and timestamp.
     */
    public static String generateSingleParameterToken(String appId, String appKey, String channelId, String userId, long timestamp) {
        return generateSingleParameterToken(appId, appKey, channelId, userId, timestamp, "");
    }

    public static String getSHA256(String str) {
        try {
            MessageDigest messageDigest = MessageDigest.getInstance("SHA-256");
            byte[] hash = messageDigest.digest(str.getBytes(StandardCharsets.UTF_8));
            return byte2Hex(hash);
        } catch (NoSuchAlgorithmException e) {
            // Consider logging the exception and/or re-throwing as a RuntimeException
            e.printStackTrace();
        }
        return "";
    }

    private static String byte2Hex(byte[] bytes) {
        StringBuilder stringBuilder = new StringBuilder();
        for (byte b : bytes) {
            String hex = Integer.toHexString(0xff & b);
            if (hex.length() == 1) {
                // Use single quote for char
                stringBuilder.append('0');
            }
            stringBuilder.append(hex);
        }
        return stringBuilder.toString();
    }

    public static long getTimesTamp() {
        return System.currentTimeMillis() / 1000 + 60 * 60 * 24;
    }
}

3. Import ARTC SDK classes

Import the relevant classes and interfaces from the ARTC SDK:

// Import ARTC classes
import com.alivc.rtc.AliRtcEngine;
import com.alivc.rtc.AliRtcEngineEventListener;
import com.alivc.rtc.AliRtcEngineNotify;

4. Create and initialize the engine

• Create the RTC engine

  Call the getInstance[1/2] method to create an AliRtcEngine instance.

  private AliRtcEngine mAliRtcEngine = null;
  if(mAliRtcEngine == null) {
      mAliRtcEngine = AliRtcEngine.getInstance(this);
  }

• Initialize the engine

  • Call setChannelProfile to set the channel to AliRTCSdkInteractiveLive (interactive mode).

    Depending on your business needs, you can choose interactive mode, which is suitable for interactive entertainment scenarios, or communication mode, which is suitable for one-to-one or one-to-many broadcasting. Choosing the right mode ensures a smooth user experience and efficient use of network resources.

    Mode	Publishing	Subscribing	Description
    Interactive mode	Limited by role. Only users with the host role can publish streams. Participants can flexibly switch roles throughout the session.	No role restrictions. All participants have permission to subscribe to streams.	In interactive mode, events such as a host joining or leaving the channel, or starting to publish a stream, are notified to viewers in real time. Conversely, the viewers' activities are not notified to the hosts, ensuring uninterrupted streaming. In this mode, hosts are responsible for interaction, while viewers only consumes content. If your business needs might change, consider using interactive mode by default. Its flexibility allows you to adapt to different interaction requirements by adjusting user roles.
    Communication mode	No role restrictions. All participants have permission to publish streams.	No role restrictions. All participants have permission to subscribe to streams.	In communication mode, participants are aware of each other's presence. Although this mode does not differentiate user roles, it is functionally equivalent to the host role in interactive mode. The goal is to simplify operations, allowing users to achieve the desired functionality with fewer API calls.

  • Call setClientRole to set the user role to AliRTCSdkInteractive (host) or AliRTCSdkLive (viewer). Note: The host role publishes and subscribes by default. The viewer role only subscribes by default, with preview and publishing disabled.

    Note

    When a user switches roles in a channel, the system adjusts the audio and video publishing status accordingly:

    • Switching from host to viewer: The system stops publishing the local audio and video streams. Subscribed remote streams are not affected, and the user can continue to watch others.

    • Switching from viewer to host: The system starts publishing the local audio and video streams. Subscribed remote streams remain unchanged, and the user can continue to watch other participants.

    // Set the channel mode to interactive mode, use AliRTCSdkInteractiveLive for RTC
    mAliRtcEngine.setChannelProfile(AliRtcEngine.AliRTCSdkChannelProfile.AliRTCSdkInteractiveLive);
    // Set the user role, use AliRTCSdkInteractive to both ingest and pull streams, use AliRTCSdkLive if you only pull streams without ingesting
    mAliRtcEngine.setClientRole(AliRtcEngine.AliRTCSdkClientRole.AliRTCSdkInteractive);

• Set common callbacks

  If the SDK encounters an issue during operation, it will first attempt to recover automatically using its internal retry mechanisms. For errors that it cannot resolve on its own, the SDK will notify your application through predefined callback interfaces.

  The following are key callbacks for issues the SDK cannot handle, which your application must listen for and respond to:

  Cause of exception	Callback and parameters	Solution	Description
  Authentication failed	result in the onJoinChannelResult callback returns AliRtcErrJoinBadToken.	The app should check if the token is correct.	When a user calls an API, if authentication fails, the API's callback will return an authentication failure error.
  Token about to expire	onAuthInfoWillExpire	Retrieve a new token and call refreshAuthInfo to update the information.	A token expiration error can occur either when an API is called or during runtime. The error is reported through API callbacks or a separate error callback.
  Token expired	onAuthInfoExpired	The app must rejoin the channel.	A token expiration error can occur either when an API is called or during runtime. The error is reported through API callbacks or a separate error callback.
  Network connection issue	onConnectionStatusChange callback returns AliRtcConnectionStatusFailed.	The app must rejoin the channel.	The SDK can automatically recover from brief network disconnections. If the disconnection time exceeds a threshold, it will time out. The app must check the network status and guide the user to rejoin.
  Kicked from channel	onBye	AliRtcOnByeUserReplaced: Check if another user has joined with the same userId. AliRtcOnByeBeKickedOut: The user was kicked out of the channel and needs to rejoin. AliRtcOnByeChannelTerminated: The channel was terminated, and the user needs to rejoin.	The RTC service allows an administrator to remove participants.
  Local device exception	onLocalDeviceException	Check app permissions and whether the hardware is working correctly.	When a local device exception occurs that the SDK cannot resolve, it notifies the app via a callback. The app should then intervene to check the device status.

  private AliRtcEngineEventListener mRtcEngineEventListener = new AliRtcEngineEventListener() {
      @Override
      public void onJoinChannelResult(int result, String channel, String userId, int elapsed) {
          super.onJoinChannelResult(result, channel, userId, elapsed);
          handleJoinResult(result, channel, userId);
      }
  
      @Override
      public void onLeaveChannelResult(int result, AliRtcEngine.AliRtcStats stats){
          super.onLeaveChannelResult(result, stats);
      }
  
      @Override
      public void onConnectionStatusChange(AliRtcEngine.AliRtcConnectionStatus status, AliRtcEngine.AliRtcConnectionStatusChangeReason reason){
          super.onConnectionStatusChange(status, reason);
  
          handler.post(new Runnable() {
              @Override
              public void run() {
                  if(status == AliRtcEngine.AliRtcConnectionStatus.AliRtcConnectionStatusFailed) {
                      /* TODO: Must handle. We recommend notifying the user. This is reported only after the SDK's internal recovery strategies have failed. */
                      ToastHelper.showToast(VideoChatActivity.this, R.string.video_chat_connection_failed, Toast.LENGTH_SHORT);
                  } else {
                      /* TODO: Optional. Add business logic here, typically for data analytics or UI changes. */
                  }
              }
          });
      }
      @Override
      public void OnLocalDeviceException(AliRtcEngine.AliRtcEngineLocalDeviceType deviceType, AliRtcEngine.AliRtcEngineLocalDeviceExceptionType exceptionType, String msg){
          super.OnLocalDeviceException(deviceType, exceptionType, msg);
          /* TODO: Must handle. We recommend notifying the user of the device error. This is reported only after the SDK's internal recovery strategies have failed. */
          handler.post(new Runnable() {
              @Override
              public void run() {
                  String str = "OnLocalDeviceException deviceType: " + deviceType + " exceptionType: " + exceptionType + " msg: " + msg;
                  ToastHelper.showToast(VideoChatActivity.this, str, Toast.LENGTH_SHORT);
              }
          });
      }
  
  };
  
  private AliRtcEngineNotify mRtcEngineNotify = new AliRtcEngineNotify() {
      @Override
      public void onAuthInfoWillExpire() {
          super.onAuthInfoWillExpire();
          /* TODO: Must handle. When this callback is triggered, retrieve a new token for the current user and channel, then call refreshAuthInfo to update it. */
      }
  
      @Override
      public void onRemoteUserOnLineNotify(String uid, int elapsed){
          super.onRemoteUserOnLineNotify(uid, elapsed);
      }
  
      // Unset the remote video stream renderer in the onRemoteUserOffLineNotify callback.
      @Override
      public void onRemoteUserOffLineNotify(String uid, AliRtcEngine.AliRtcUserOfflineReason reason){
          super.onRemoteUserOffLineNotify(uid, reason);
      }
  
      // Set the remote video stream renderer in the onRemoteTrackAvailableNotify callback.
      @Override
      public void onRemoteTrackAvailableNotify(String uid, AliRtcEngine.AliRtcAudioTrack audioTrack, AliRtcEngine.AliRtcVideoTrack videoTrack){
          handler.post(new Runnable() {
              @Override
              public void run() {
                  if(videoTrack == AliRtcVideoTrackCamera) {
                      SurfaceView surfaceView = mAliRtcEngine.createRenderSurfaceView(VideoChatActivity.this);
                      surfaceView.setZOrderMediaOverlay(true);
                      FrameLayout view = getAvailableView();
                      if (view == null) {
                          return;
                      }
                      remoteViews.put(uid, view);
                      view.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
                      AliRtcEngine.AliRtcVideoCanvas remoteVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
                      remoteVideoCanvas.view = surfaceView;
                      mAliRtcEngine.setRemoteViewConfig(remoteVideoCanvas, uid, AliRtcVideoTrackCamera);
                  } else if(videoTrack == AliRtcVideoTrackNo) {
                      if(remoteViews.containsKey(uid)) {
                          ViewGroup view = remoteViews.get(uid);
                          if(view != null) {
                              view.removeAllViews();
                              remoteViews.remove(uid);
                              mAliRtcEngine.setRemoteViewConfig(null, uid, AliRtcVideoTrackCamera);
                          }
                      }
                  }
              }
          });
      }
  
      /* Your app must also handle cases where multiple devices attempt to join with the same UserID. */
      @Override
      public void onBye(int code){
          handler.post(new Runnable() {
              @Override
              public void run() {
                  String msg = "onBye code:" + code;
                  ToastHelper.showToast(VideoChatActivity.this, msg, Toast.LENGTH_SHORT);
              }
          });
      }
  };
  
  mAliRtcEngine.setRtcEngineEventListener(mRtcEngineEventListener);
  mAliRtcEngine.setRtcEngineNotify(mRtcEngineNotify);

5. Set audio and video properties

• Set audio properties

  Call setAudioProfile to set the audio encoding mode and scenario.

  mAliRtcEngine.setAudioProfile(AliRtcEngine.AliRtcAudioProfile.AliRtcEngineHighQualityMode, AliRtcEngine.AliRtcAudioScenario.AliRtcSceneMusicMode);

• Set video properties

  Set properties for the published video stream, such as resolution, bitrate, and frame rate.

  // Set video encoding parameters.
  AliRtcEngine.AliRtcVideoEncoderConfiguration aliRtcVideoEncoderConfiguration = new AliRtcEngine.AliRtcVideoEncoderConfiguration();
  aliRtcVideoEncoderConfiguration.dimensions = new AliRtcEngine.AliRtcVideoDimensions(
                  720, 1280);
  aliRtcVideoEncoderConfiguration.frameRate = 20;
  aliRtcVideoEncoderConfiguration.bitrate = 1200;
  aliRtcVideoEncoderConfiguration.keyFrameInterval = 2000;
  aliRtcVideoEncoderConfiguration.orientationMode = AliRtcVideoEncoderOrientationModeAdaptive;
  mAliRtcEngine.setVideoEncoderConfiguration(aliRtcVideoEncoderConfiguration);

6. Set publishing and subscribing properties

Configure the publishing of audio/video streams and set the default to subscribe to all users' streams:

• Call publishLocalAudioStream to publish a audio stream.

• Call publishLocalVideoStream to publish a video stream. For an audio-only call, you can set this to false.

// The SDK publishes audio by default, so you don't need to call publishLocalAudioStream.
mAliRtcEngine.publishLocalAudioStream(true);
// For video calls, you don't need to call publishLocalVideoStream(true) as the SDK publishes video by default.
// For audio-only calls, you need to call publishLocalVideoStream(false) to disable video publishing.
mAliRtcEngine.publishLocalVideoStream(true);

// Set default subscription to all remote audio and video streams.
mAliRtcEngine.setDefaultSubscribeAllRemoteAudioStreams(true);
mAliRtcEngine.subscribeAllRemoteAudioStreams(true);
mAliRtcEngine.setDefaultSubscribeAllRemoteVideoStreams(true);
mAliRtcEngine.subscribeAllRemoteVideoStreams(true);

7. Enable local preview

• Call setLocalViewConfig to set up the local render view and configure local video display properties.

• Call startPreview to start the local video preview.

mLocalVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
SurfaceView localSurfaceView = mAliRtcEngine.createRenderSurfaceView(VideoChatActivity.this);
localSurfaceView.setZOrderOnTop(true);
localSurfaceView.setZOrderMediaOverlay(true);
FrameLayout fl_local = findViewById(R.id.fl_local);
fl_local.addView(localSurfaceView, layoutParams);
mLocalVideoCanvas.view = localSurfaceView;
mAliRtcEngine.setLocalViewConfig(mLocalVideoCanvas, AliRtcVideoTrackCamera);
mAliRtcEngine.startPreview();

8. Join a channel

Call joinChannel to join the channel. If the token was generated using the single-parameter method, call the joinChannel[1/3] operation. If it was generated using the multi-parameter method, call the joinChannel[2/3] method. The result is returned in the onJoinChannelResult callback. A result of 0 indicates a successful join. A non-zero result may indicate an invalid token.

 mAliRtcEngine.joinChannel(token, null, null, null);

9. Set the remote view

When you initialize the engine, set the corresponding callback with mAliRtcEngine.setRtcEngineNotify. In the onRemoteTrackAvailableNotify callback, you can set up the remote view for the remote user. Example code:

@Override
public void onRemoteTrackAvailableNotify(String uid, AliRtcEngine.AliRtcAudioTrack audioTrack, AliRtcEngine.AliRtcVideoTrack videoTrack){
    handler.post(new Runnable() {
        @Override
        public void run() {
            if(videoTrack == AliRtcVideoTrackCamera) {
                SurfaceView surfaceView = mAliRtcEngine.createRenderSurfaceView(VideoChatActivity.this);
                surfaceView.setZOrderMediaOverlay(true);
                FrameLayout fl_remote = findViewById(R.id.fl_remote);
                if (fl_remote == null) {
                    return;
                }
                fl_remote.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
                AliRtcEngine.AliRtcVideoCanvas remoteVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
                remoteVideoCanvas.view = surfaceView;
                mAliRtcEngine.setRemoteViewConfig(remoteVideoCanvas, uid, AliRtcVideoTrackCamera);
            } else if(videoTrack == AliRtcVideoTrackNo) {
                FrameLayout fl_remote = findViewById(R.id.fl_remote);
                fl_remote.removeAllViews();
                mAliRtcEngine.setRemoteViewConfig(null, uid, AliRtcVideoTrackCamera);
            }
        }
    });
}

10. Leave the channel and destroy the engine

When the session is over, leave the channel and destroy the engine:

1. Call stopPreview to stop the video preview.

2. Call leaveChannel to leave the channel.

3. Call destroy to destroy the engine and release its resources.

private void destroyRtcEngine() {
    mAliRtcEngine.stopPreview();
    mAliRtcEngine.setLocalViewConfig(null, AliRtcVideoTrackCamera);
    mAliRtcEngine.leaveChannel();
    mAliRtcEngine.destroy();
    mAliRtcEngine = null;
}

11. Effect demonstration