custom audio playback - ApsaraVideo Live - Alibaba Cloud Documentation Center

For developers who need to implement specific features by controlling audio playback themselves, ARTC provides custom audio playback functionality. This allows you to take full control of the playback process.

Feature introduction

By default, ARTC includes a market-proven audio playback module that meets the needs of most use cases. However, in certain cases, you may already have a mature custom audio playback module or need to perform post-processing on audio data before playback. For this purpose, ARTC offers custom audio playback capabilities, allowing you to disable the internal playback logic and take over the reception and playback of audio data.

The core of this feature is to enable audio frame callbacks, which allows you to obtain remote or local PCM audio data and pass it to your playback device for processing and rendering.

Sample code

Android: Android/ARTCExample/AdvancedUsage/src/main/java/com/aliyun/artc/api/advancedusage/CustomAudioCaptureAndRender/CustomAudioRenderActivity.java

iOS: iOS/ARTCExample/AdvancedUsage/CustomAudioRender/CustomAudioRenderVC.swift

Before you begin

Before using custom audio playback, ensure you meet the following requirements:

You have the capability to develop a complete audio playback module or already have a mature third-party or self-developed audio player.
You understand basic audio processing concepts, including PCM data format, sample rate, and number of channels.
You have integrated the ARTC SDK and implemented basic audio/video call capabilities.
Your business requirements strictly necessitate bypassing ARTC's internal playback module. Otherwise, we recommend using the default playback solution to ensure stability and compatibility.

Implementation

1. Disable the SDK's internal playback module

To use custom audio playback, you must first disable the SDK's internal audio playback module. We recommend doing this when creating the engine instance by passing an extras parameter to the getInstance method.

The relevant parameter is user_specified_use_external_audio_player:

TRUE: Use an external audio player, which disables the SDK's internal playback.
FALSE: Do not use an external audio player, which enables the SDK's internal playback.

Android

String extras = "{\"user_specified_use_external_audio_player\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras);

iOS

// Create and initialize the engine
var customAudioPlayConfig: [String: String] = [:]
// Use an external player (disable the SDK's internal audio playback)
customAudioPlayConfig["user_specified_use_external_audio_player"] = "TRUE"
// Serialize to JSON
guard let jsonData = try? JSONSerialization.data(withJSONObject: customAudioPlayConfig, options: []),
let extras = String(data: jsonData, encoding: .utf8) else {
    print("JSON serialization failed")
    return
}
let engine = AliRtcEngine.sharedInstance(self, extras:extras)

2. Listen for audio callbacks

Call enableAudioFrameObserver to register for audio callbacks. You must pass the following parameters:

enable: A boolean that determines whether to enable data callbacks.
audioSource: The type of data source for the callback, which can include audio data from different stages. For custom audio playback, you typically need to observe the AliRtcAudioSourcePlayback source, which corresponds to the onPlaybackAudioFrame callback.
config: The desired audio parameters for the callback, including sample rate and number of channels.

Android

// Set the callback configuration
AliRtcEngine.AliRtcAudioFrameObserverConfig config = new AliRtcEngine.AliRtcAudioFrameObserverConfig();
config.sampleRate = AliRtcAudioSampleRate_48000;
config.channels = 1;
// Register to enable playback data callbacks
mAliRtcEngine.enableAudioFrameObserver(true, AliRtcAudioSourcePlayback, config);

iOS

// Set the callback configuration
var observerConfig: AliRtcAudioFrameObserverConfig = AliRtcAudioFrameObserverConfig()
observerConfig.sampleRate = ._Unknown
observerConfig.channels = .monoAudio
// Register to enable playback data callbacks
let audioSource: AliRtcAudioSource = .playback
engine.enableAudioFrameObserver(true, audioSource: audioSource, config: observerConfig)

3. Process the PCM data in the callback

Handle the received PCM data within the callback and submit it to your system's audio device for playback.

Android

@Override
public boolean onPlaybackAudioFrame(AliRtcEngine.AliRtcAudioFrame frame) {
    // Implement your custom logic to play the audio data from the callback here.

    return true;
}

iOS

func onPlaybackAudioFrame(_ frame: AliRtcAudioFrame) -> Bool {
    // Implement your custom logic to play the audio data from the callback here.
    
    return true
}

4. Disable the observer when leaving the channel

Android

mAliRtcEngine.enableAudioFrameObserver(false, AliRtcAudioSourcePlayback, config)

iOS

engine.enableAudioFrameObserver(false, audioSource: audioSource, config: observerConfig)

5. (Optional) Dynamically control internal playback

If your application requires dynamically enabling or disabling the SDK's internal playback during a call, use the setParameter interface.

Android

/* Dynamically disable internal playback */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}";
mAliRtcEngine.setParameter(parameter);

/* Dynamically enable internal playback */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"; 
mAliRtcEngine.setParameter(parameter);

iOS

// Dynamically disable internal playback
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}")
// Dynamically enable internal playback
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}")