ApsaraVideo Live:Common audio operations

1.1. Audio encoding modes (AliRtcAudioProfile)

1.2. Audio scenario modes (AliRtcAudioScenario)

1.3. Sample code

The following examples show how to configure the audio encoding and scenario modes for common use cases.

For Bluetooth capture

// Set the audio scenario to AliRtcSceneDefaultMode.
mAliRtcEngine.setAudioProfile(AliRtcEngineHighQualityMode, AliRtcSceneDefaultMode);

// Set the audio scenario to AliRtcSceneDefaultMode.
engine.setAudioProfile(AliRtcAudioProfile.engineHighQualityMode, audio_scene: AliRtcAudioScenario.sceneDefaultMode)

this.rtcEngine.setAudioProfile(AliRtcAudioProfile.AliRtcHighQualityMode, AliRtcAudioScenario.AliRtcSceneDefaultMode);

[self.engine setAudioProfile:AliRtcEngineHighQualityMode audio_scene:AliRtcSceneDefaultMode];

mAliRtcEngine->SetAudioProfile(AliEngineHighQualityMode, AliEngineSceneDefaultMode);

For interoperability with web clients

// Select an encoding mode with a 48 kHz sample rate, such as AliRtcEngineHighQualityMode.
mAliRtcEngine.setAudioProfile(AliRtcEngineHighQualityMode, AliRtcSceneMusicMode);

// Select an encoding mode with a 48 kHz sample rate, such as AliRtcEngineHighQualityMode.
engine.setAudioProfile(AliRtcAudioProfile.engineHighQualityMode, audio_scene: AliRtcAudioScenario.sceneMusicMode)

this.rtcEngine.setAudioProfile(AliRtcAudioProfile.AliRtcHighQualityMode, AliRtcAudioScenario.AliRtcSceneMusicMode);

[self.engine setAudioProfile:AliRtcEngineHighQualityMode audio_scene:AliRtcSceneMusicMode];

mAliRtcEngine->SetAudioProfile(AliEngineHighQualityMode, AliEngineSceneMusicMode);

For stereo sound

// Set the mode to one that supports stereo, such as AliRtcEngineStereoHighQualityMode.
mAliRtcEngine.setAudioProfile(AliRtcEngineStereoHighQualityMode, AliRtcSceneMusicMode);

// Set the mode to one that supports stereo, such as AliRtcEngineStereoHighQualityMode.
engine.setAudioProfile(AliRtcAudioProfile.engineStereoHighQualityMode, audio_scene: AliRtcAudioScenario.sceneMusicMode)

this.rtcEngine.setAudioProfile(AliRtcAudioProfile.AliRtcStereoHighQualityMode, AliRtcAudioScenario.AliRtcSceneMusicMode);

[self.engine setAudioProfile:AliRtcEngineStereoHighQualityMode audio_scene:AliRtcSceneMusicMode];

mAliRtcEngine->SetAudioProfile(AliEngineStereoHighQualityMode, AliEngineSceneMusicMode);

2.1. Mute local microphone

Use the muteLocalMic method to mute the local microphone and external audio input. You can call this method before or after joining a channel to set the mute status.

The following AliRtcMuteLocalAudioMode modes are supported:

Sample code:

// Mute all audio
mAliRtcEngine.muteLocalMic(true, AliRtcEngine.AliRtcMuteLocalAudioMode.AliRtcMuteAllAudioMode);
// Unmute all audio
mAliRtcEngine.muteLocalMic(false, AliRtcEngine.AliRtcMuteLocalAudioMode.AliRtcMuteAllAudioMode);
// Mute only the microphone
mAliRtcEngine.muteLocalMic(true, AliRtcEngine.AliRtcMuteLocalAudioMode.AliRtcMuteOnlyMicAudioMode);

// Mute all audio
self.rtcEngine?.muteLocalMic(true, mode: .allAudioMode)
// Unmute all audio
self.rtcEngine?.muteLocalMic(false, mode: .allAudioMode)
// Mute only the microphone
self.rtcEngine?.muteLocalMic(true, mode: .onlyMicAudioMode)

// Mute all audio
this.rtcEngine.muteLocalMic(true, AliRtcMuteLocalAudioMode.AliRtcMuteLocalAudioModeMuteAll);
// Unmute all audio
this.rtcEngine.muteLocalMic(false, AliRtcMuteLocalAudioMode.AliRtcMuteLocalAudioModeMuteAll);
// Mute only the microphone
this.rtcEngine.muteLocalMic(true, AliRtcMuteLocalAudioMode.AliRtcMuteLocalAudioModeMuteOnlyMic);

// Mute all audio
[self.engine muteLocalMic:TRUE mode:AliRtcMuteAllAudioMode];
// Unmute all audio
[self.engine muteLocalMic:FALSE mode:AliRtcMuteAllAudioMode];
// Mute only the microphone
[self.engine muteLocalMic:TRUE mode:AliRtcMuteOnlyMicAudioMode];

// Mute all audio
mAliRtcEngine->MuteLocalMic(true, AliEngineMuteLocalAudioModeMuteAll);
// Unmute all audio
mAliRtcEngine->MuteLocalMic(false, AliEngineMuteLocalAudioModeMuteAll);
// Mute only the microphone
mAliRtcEngine->MuteLocalMic(true, AliEngineMuteLocalAudioModeMuteOnlyMic);

2.2. Stop or resume microphone capture

The SDK enables microphone capture by default when it joins a channel. To stop microphone capture, call stopAudioCapture. This call stops audio capture and releases the microphone resource. To resume microphone capture, call startAudioCapture.

// Stop microphone capture.
mAliRtcEngine.stopAudioCapture();
// Resume microphone capture.
mAliRtcEngine.startAudioCapture();

// Stop microphone capture.
self.rtcEngine?.stopAudioCapture()
// Resume microphone capture.
self.rtcEngine?.startAudioCapture()

// Stop microphone capture.
this.rtcEngine.stopAudioCapture();
// Resume microphone capture.
this.rtcEngine.startAudioCapture();

// Stop microphone capture.
[self.engine stopAudioCapture];
// Resume microphone capture.
[self.engine startAudioCapture];

// Stop microphone capture.
mAliRtcEngine->StopAudioCapture();
// Resume microphone capture.
mAliRtcEngine->StartAudioCapture();

3.1. Mute a remote user

ARTC provides the muteRemoteAudioPlaying method to stop or resume audio playback for a specific remote user. The method is defined as follows:

public abstract int muteRemoteAudioPlaying(String uid, boolean mute);

• Muting does not affect receiving and decoding the audio stream. This setting can be configured before or after joining a session.

• Muting only affects local playback of the remote user's audio. It does not affect the remote user's audio capture.

3.2. Set the playback volume for a specific remote user

ARTC provides the setPlayoutVolume method to control the local playback volume.

/**
 * @brief Sets the playback volume.
 * @param volume The playback volume. The value ranges from 0 to 400.
 * - 0: Mute.
 * - <100: Decreases the volume.
 * - >100: Increases the volume.
 * @return
 * - 0: Success.
 * - A non-zero value: Failure.
 */
public abstract int setPlayoutVolume(int volume);

ARTC provides the setRemoteAudioVolume method to control the audio playback volume of a specific remote user. When the volume parameter is set to 0, this method has the same effect as the muteRemoteAudioPlaying method.

/**
 * @brief Adjusts the volume of a specific remote user for local playback.
 * @param uid The user ID. This is a unique identifier assigned by your app server.
 * @param volume The playback volume. The value ranges from 0 to 100, where 0 means mute and 100 means the original volume.
 * @return
 * - 0: Success.
 * - A non-zero value: Failure.
 */
public abstract int setRemoteAudioVolume(String uid, int volume);

4.1. Enable in-ear monitoring

Call the enableEarBack method before or after joining a channel to enable in-ear monitoring. To disable it, call enableEarBack again and set the parameter to false.

rtcEngine.enableEarBack(true);

engine.enableEarBack(true)

this.rtcEngine.enableEarBack(true);

[self.engine enableEarBack:YES];

mAliRtcEngine->EnableEarBack(TRUE);

4.2. Set the in-ear monitoring volume

Call the setEarBackVolume method to adjust the in-ear monitoring volume. The volume parameter sets the volume on a scale from 0 to 100, where 0 is muted and 100 is the original volume. The default value is 100.

rtcEngine.setEarBackVolume(60);

rtcEngine?.setEarBackVolume(volume)

this.rtcEngine.setEarBackVolume(value);

[self.engine setEarBackVolume:60];

mAliRtcEngine->SetEarBackVolume(volume);

5.1. Enable the callback feature

Call enableAudioVolumeIndication to enable the feature and configure its parameters. The parameters are described below:

• interval: The callback interval in milliseconds (ms). The recommended range is 300 ms to 500 ms. The minimum value is 10 ms. A negative value disables this feature.

• smooth: The smoothing factor. A higher value results in smoother volume changes, while a lower value provides better real-time performance. The recommended value is 3. The valid range is from 0 to 9.

• reportVad: The switch for active speaker detection. Set to 0 to disable this feature, or 1 to enable it.

mAliRtcEngine.enableAudioVolumeIndication(500, 3,1);

// User volume callback and active speaker detection
engine.enableAudioVolumeIndication(500, smooth: 3, reportVad: 1)

// User volume callback and active speaker detection
this.rtcEngine.enableAudioVolumeIndication(1000, 3, 1);

// User volume callback and active speaker detection
[self.engine enableAudioVolumeIndication:500 smooth:3 reportVad:1];

// User volume callback and active speaker detection
mAliRtcEngine->EnableAudioVolumeIndication(500, 3, 1);

5.2. Implement and register callbacks

Call the registerAudioVolumeObserver method to register the callbacks. The system then triggers the following callbacks at the specified interval:

• onAudioVolume: This callback periodically provides audio volume information so you can monitor the speaking intensity of each user in real time. The system reports the volume level of all detected users, including the local user and remote users. You can use this information to implement UI feedback, such as sound wave animations, volume indicators, or mute detection. If mUserId is "0", the entry is for the local captured volume. If mUserId is "1", the entry is for the mixed volume of all remote users. Other values indicate the volume of a specific user. totalVolume indicates the total mixed volume of all remote users.

• onActiveSpeaker: This is the active speaker callback triggered by voice activity detection (VAD). The system triggers this callback when it detects that a user has become the most active speaker. A user is considered the active speaker when their speaking volume and duration exceed a threshold. You can use this event to implement features such as automatically focusing on the active speaker. For example, in a conference, you can maximize the video window of the active speaker.

private final AliRtcEngine.AliRtcAudioVolumeObserver mAliRtcAudioVolumeObserver = new AliRtcEngine.AliRtcAudioVolumeObserver() {
    // User volume callback
    @Override
    public void onAudioVolume(List<AliRtcEngine.AliRtcAudioVolume> speakers, int totalVolume){
        handler.post(() -> {
            if(!speakers.isEmpty()) {
                for(AliRtcEngine.AliRtcAudioVolume volume : speakers) {
                    if("0".equals(volume.mUserId)) {
                        // Volume of the local user

                    } else if ("1".equals(volume.mUserId)) {
                        // Overall volume of remote users

                    } else {
                        // Volume of a remote user

                    }
                }
            }
        });
    }

    // Active speaker detection callback
    @Override
    public void onActiveSpeaker(String uid){
        // Active speaker
        handler.post(() -> {
            String mag = "onActiveSpeaker uid:" + uid;
            ToastHelper.showToast(AudioBasicUsageActivity.this, mag, Toast.LENGTH_SHORT);
        });
    }
};
// Register the callback
mAliRtcEngine.registerAudioVolumeObserver(mAliRtcAudioVolumeObserver);

func onAudioVolumeCallback(_ array: [AliRtcUserVolumeInfo]?, totalVolume: Int32) {
    // User volume callback
    "onAudioVolumeCallback, totalVolume: \(totalVolume)".printLog()
}

func onActiveSpeaker(_ uid: String) {
    // Active speaker callback
    "onActiveSpeaker, uid: \(uid)".printLog()
}

this.rtcEngineEventListener.onAudioVolumeCallback(
  (volumeInfo : AliRtcUserVolumeInfo[], volumeInfoCount : number, totalVolume : number) => {
    console.info(`Volume callback: volumeInfo=${volumeInfo}, volumeInfoCount=${volumeInfoCount}, totalVolume=${totalVolume}`);
  }
)

this.rtcEngineEventListener.onActiveSpeaker((uid: string) => {
  console.info(`Current active user: uid=${uid}`);
});

- (void)onAudioVolumeCallback:(NSArray <AliRtcUserVolumeInfo *> *)array totalVolume:(int)totalVolume {
    for(AliRtcUserVolumeInfo *info in array) {
        NSString *uid = info.uid;
        NSInteger volume = info.volume;
        dispatch_async(dispatch_get_main_queue(), ^{
            NSArray * volumeArray = [ _audioVolumeLabel.stringValue componentsSeparatedByString:@"\n"];

            if ([uid isEqualToString:@"1"]) { // Volume of remote users
                // Overall volume of remote users
            }else if ([uid isEqualToString:@"0"] ){ // Volume of the local user
                // Display the volume of the local user
            } else {

                // Overall volume of remote users
            }
        });
    }
}

- (void)onActiveSpeaker:(NSString *)uid {
    if ([uid isEqualToString:@"0"]) {
        [self log:[@"Active speaker: Self" UTF8String]];
    } else {
        NSDictionary *dic = [self.engine getUserInfo:uid];
        NSString *name = dic[@"displayName"];
        MyLog(@"Active speaker: %@",name);
    }
}

void CTutorialDlg::OnAudioVolumeCallback(const AliEngineUserVolumeInfo* volumeInfo, int volumeInfoCount, int totalVolume)
{
	CArray<AliEngineUserVolumeInfo*>* pArray = new CArray<AliEngineUserVolumeInfo*>;
	for (int i = 0; i < volumeInfoCount; i++)
	{
		AliEngineUserVolumeInfo* p = new AliEngineUserVolumeInfo;
		p->uid = volumeInfo[i].uid;
		p->volume = volumeInfo[i].volume;
		p->speechState = volumeInfo[i].speechState;
		p->sumVolume = volumeInfo[i].sumVolume;
		pArray->Add(p);
	}
    /* Notify the UI to refresh */
	PostMessage(MM_VOLUME_CALLBACK, (WPARAM)totalVolume, (LPARAM)pArray);
}

void CTutorialDlg::OnActiveSpeaker(const char *uid)
{
	m_strCurSpeaker = CString(uid);
    /* Notify the UI to refresh  */
	PostMessage(MM_ACTIVE_SPEAKER, NULL, NULL);
}

6.1. Default audio route

Set the default audio route to either the earpiece or the speaker before you join a meeting. If you do not set a default route, the speaker is used.

/**
* @brief Sets whether the default audio output is the speaker. The default is the speaker.
* @param defaultToSpeakerphone
* - true: Speaker mode.
* - false: Earpiece mode.
* @return
* - 0: Success.
* - <0: Failure.
*/
public int setDefaultAudioRoutetoSpeakerphone(boolean defaultToSpeakerphone);

6.2. Current audio route

Set the current audio route to either the earpiece or the speaker during a meeting. If you do not set a current route, the device specified by the default audio route is used.

When no external device is connected, call enableSpeakerphone to use the speaker. Set it to false to use the earpiece. To check whether the current audio device is the speaker or the earpiece, call the isSpeakerOn interface.

/**
 * @brief Sets the audio output to the earpiece or the speaker.
 * @param enable   true: Speaker mode. false: Earpiece mode.
 * @return
 * - 0: Success.
 * - <0: Failure.
 */
public int enableSpeakerphone(boolean enable);
/**
* @brief Gets whether the current audio output is the earpiece or the speaker.
* @return 
* - true: Speaker mode.
* - false: Earpiece mode.
*/
public boolean isSpeakerOn();

6.3. Audio route change callback

To receive a callback when the audio playback device changes, register and listen for the following callback.

public abstract class AliRtcEngineEventListener {
    /**
     * @brief Warning notification.
     * @details Notifies the app through this callback if a warning occurs in the engine.
     * @param warn The warning type.
     * @param message The warning message.
     */
    public void onOccurWarning(int warn, String message);
}

The following table shows the mapping between the warn return values and device types.