ApsaraVideo Live:Custom audio playback

Overview

By default, ARTC includes a field-tested audio playback module that meets most playback requirements. However, if you have an existing custom audio playback module or need to post-process audio data before playback, ARTC provides the custom audio playback feature. This lets you disable the internal playback logic and manage receiving and playing audio data.

You can then pass this data to your playback device for processing and playback.

Sample code

Android: Android/ARTCExample/AdvancedUsage/src/main/java/com/aliyun/artc/api/advancedusage/CustomAudioCaptureAndRender/CustomAudioRenderActivity.java.

iOS: iOS/ARTCExample/AdvancedUsage/CustomAudioRender/CustomAudioRenderVC.swift.

Prerequisites

Before using custom audio playback, ensure you meet the following requirements:

• You can develop a complete audio playback module or already have an established third-party or in-house audio player.

• You understand the basic audio processing workflow, including concepts like PCM data format, sample rate, and number of channels.

• You have integrated the ARTC SDK and implemented basic audio and video call features.

• Your use case requires bypassing the ARTC internal playback module. Otherwise, we recommend using the default playback solution to ensure stability and compatibility.

Implementation

String extras = "{\"user_specified_use_external_audio_player\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras);

// Create and initialize the engine.
var customAudioPlayConfig: [String: String] = [:]
// Use an external player, which disables the SDK's internal playback.
customAudioPlayConfig["user_specified_use_external_audio_player"] = "TRUE"
// Serialize to JSON.
guard let jsonData = try? JSONSerialization.data(withJSONObject: customAudioPlayConfig, options: []),
let extras = String(data: jsonData, encoding: .utf8) else {
    print("JSON serialization failed")
    return
}
let engine = AliRtcEngine.sharedInstance(self, extras:extras)

NSString * extras = @"{\"user_specified_use_external_audio_player\":\"TRUE\"}";
self.engine = [AliRtcEngine sharedInstance:self extras:extras];

std::string extras = "{\"user_specified_use_external_audio_player\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras.c_str());

// Set the callback configuration.
AliRtcEngine.AliRtcAudioFrameObserverConfig config = new AliRtcEngine.AliRtcAudioFrameObserverConfig();
config.sampleRate = AliRtcAudioSampleRate_48000;
config.channels = 1;
// Register to enable the playback data callback.
mAliRtcEngine.enableAudioFrameObserver(true, AliRtcAudioSourcePlayback, config);

// Set the callback configuration.
var observerConfig: AliRtcAudioFrameObserverConfig = AliRtcAudioFrameObserverConfig()
observerConfig.sampleRate = ._Unknown
observerConfig.channels = .monoAudio
// Register to enable the playback data callback.
let audioSource: AliRtcAudioSource = .playback
engine.enableAudioFrameObserver(true, audioSource: audioSource, config: observerConfig)

AliRtcAudioFrameObserverConfig* config = [[AliRtcAudioFrameObserverConfig alloc] init];
config.channels = audioNumChannel;
config.sampleRate = AliRtcAudioSampleRate_48000;
config.mode = AliRtcAudioFrameObserverOperationModeReadOnly;
[self.engine enableAudioFrameObserver:TRUE audioSource:AliRtcAudioSourcePlayback config:config];

// Set the callback configuration.
AliEngineAudioFrameObserverConfig config ;
config.sampleRate = AliRtcAudioSampleRate_48000;
config.channels = AliEngineMonoAudio;
// Register to enable the playback data callback.
mAliRtcEngine->EnableAudioFrameObserver(true, AliEngineAudioSourcePlayback, config);

@Override
public boolean onPlaybackAudioFrame(AliRtcEngine.AliRtcAudioFrame frame) {
    // Implement your custom logic to play the audio data received in the callback.

    return true;
}

func onPlaybackAudioFrame(_ frame: AliRtcAudioFrame) -> Bool {
    // Implement your custom logic to play the audio data received in the callback.
    
    return true
}

- (BOOL)onPlaybackAudioFrame:(AliRtcAudioFrame* _Nonnull)frame {
    // Implement your custom logic to play the audio data received in the callback.
    return true;
}

bool CTutorialDlg::OnPlaybackAudioFrame(AliEngineAudioRawData audioRawData) {
	// Implement your custom logic to play the audio data received in the callback.
	return true;
}

mAliRtcEngine.enableAudioFrameObserver(false, AliRtcAudioSourcePlayback, config)

engine.enableAudioFrameObserver(false, audioSource: audioSource, config: observerConfig)

[self.engine enableAudioFrameObserver:false audioSource:AliRtcAudioSourcePlayback config:config];

mAliRtcEngine->EnableAudioFrameObserver(false, AliEngineAudioSourcePlayback, config);

/* Dynamically disable internal playback. */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}";
mAliRtcEngine.setParameter(parameter);

/* Dynamically enable internal playback. */
String parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"; 
mAliRtcEngine.setParameter(parameter);

// Dynamically disable internal playback.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}")
// Dynamically enable internal playback.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}")

// Dynamically disable internal playback.
[self.engine setParameter:"{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}"];
// Dynamically enable internal playback.
[self.engine setParameter:"{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"];

/* Dynamically disable internal playback. */
std::string parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"FALSE\"}}";
mAliRtcEngine->SetParameter(parameter.c_str());

/* Dynamically enable internal playback. */
std::string parameter = "{\"audio\":{\"enable_system_audio_device_play\":\"TRUE\"}}"; 
mAliRtcEngine->SetParameter(parameter.c_str());