ApsaraVideo Live:Play raw PCM audio data and local files

Feature introduction

The ARTC SDK supports two methods for using external audio sources: injecting raw audio data in PCM format, or playing local audio files (such as MP3, WAV, or AAC).

Before you begin

Before you implement the feature, make sure you meet the following requirements:

• Create an ARTC application and obtain the AppID and AppKey from the ApsaraVideo Live console.

• Integrate the ARTC SDK into your project and implement basic audio and video call features.

Implementation

Play raw audio data

1. (Optional) Start the player in advance

If you need to play audio before joining a channel, call startAudioPlayer() to start the audio player ahead of time.

Android

mAliRtcEngine.startAudioPlayer();

iOS

self.rtcEngine?.startAudioPlayer()

2. Add an external audio stream (for playback only)

Call addExternalAudioStream to add an external audio stream to the AliRtcEngine object. A successful call returns a unique audioStreamID for the stream.

Note

To play the audio locally without publishing it to remote users, set publishVolume to 0.

Android

/* Set the parameters as needed. */
AliRtcEngine.AliRtcExternalAudioStreamConfig config = new AliRtcEngine.AliRtcExternalAudioStreamConfig();
/* Set the playout volume to 100 to enable local playback. */
config.playoutVolume = 100;
/* Set the publish volume to 0. This means the stream will not be published. */
config.publishVolume = 0;
config.channels = 1;
config.sampleRate = 48000;
// The return value is the external stream ID. Use this ID to push data to the SDK.
audioStreamID = mAliRtcEngine.addExternalAudioStream(config);

iOS

/* Set the parameters based on your service requirements. */
AliRtcExternalAudioStreamConfig *config = [AliRtcExternalAudioStreamConfig new];
config.channels = _pcmChannels;
config.sampleRate = _pcmSampleRate;
/* Set the playout volume to 0 to disable local playback. */
config.playoutVolume = 0;
/* Set the publish volume to 0 to disable stream ingest. */
config.publishVolume = 100;
// The return value is the external stream ID, which is used to send data to the SDK.
_externalPlayoutStreamId = [self.engine addExternalAudioStream:config];

3. Push audio data to the SDK

Push the raw audio data to the SDK for playback using pushExternalAudioStreamRawData.

Android

// Duration of audio data to push at one time (20-50ms recommended).
int duration = 30; 
// Get the next chunk of PCM data from your buffer.
byte[] dataToSend = ...
// Check dataToSend: If playback is active but data is insufficient, you can sleep and wait for more data.

// Calculate the number of samples in the data chunk.
int numOfSamples = dataToSend.length / (2 * frameInfo.audio_channels);

// Construct an AliRtcAudioFrame object.
AliRtcEngine.AliRtcAudioFrame rawData = new AliRtcEngine.AliRtcAudioFrame();
rawData.data = dataToSend;
rawData.numSamples = numOfSamples;
rawData.bytesPerSample = 2;
rawData.numChannels = mPcmChannels;
rawData.samplesPerSec = mPcmSampleRate;
// Push data to the SDK.
int ret = mAliRtcEngine.pushExternalAudioStreamRawData(audioStreamID, rawData);
if(ret == 0x01070101) {
    /*  The RTC buffer is full. The push failed and you must try again. */
} 
else if(ret < 0) {
    /* An error occurred. Check your parameters and stream state. */
}
// Sleep for a fixed duration based on the data sent.
sleep(duration - 10);

iOS

guard let rtc = self.rtcEngine, self.streamId > 0 else {
    return
}

// Duration of audio data to push at one time (20-50ms recommended).
let duration = 30
// Get the next chunk of PCM data from your buffer.
let dataToSend = ...
// Check dataToSend: If playback is active but data is insufficient, you can sleep and wait for more data.

// Calculate the number of samples for dataToSend.
let numOfSamples = dataToSend.count / (2 * frameInfo.audio_channels)

let pushFrame = AliRtcAudioFrame()
pushFrame.dataPtr = dataToSend
pushFrame.numOfSamples = numOfSamples  // Number of samples
pushFrame.bytesPerSample = 2
pushFrame.samplesPerSec = self.pcmSampleRate
pushFrame.numOfChannels = self.pcmChannels
let result = rtc.pushExternalAudioStream(self.pushStreamId, rawData: pushFrame)
if result == 0x01070101 {
    /* The RTC buffer is full. The push failed and you must try again. */
}
else if result < 0 {
    /* An error occurred. Check your parameters and stream state. */
}
// Sleep for a fixed duration.
Thread.sleep(forTimeInterval: Double(duration - 10) / 1000.0)

4. Remove the external audio stream

When playback ends, call the removeExternalAudioStream method to remove the external audio stream.

Android

mAliRtcEngine.removeExternalAudioStream(audioStreamID);
audioStreamID = -1;

iOS

self.rtcEngine?.removeExternalAudioStream(self.streamId)
self.streamId = -1

Play a local file

If your audio source is a local file, you can use the audio accompaniment APIs for playback, which seamlessly integrate with ARTC's audio processing capabilities (AEC, AGC, and ANS).

1. Start playback

Call the startAudioAccompany method to start playing the specified audio file.

Note

To play the file locally without publishing it to remote users, set publishVolume to 0.

Android

// Path to the audio file
private String mFilepath = "/assets/music.wav";

// Playback configuration
AliRtcEngine.AliRtcAudioAccompanyConfig config = new AliRtcEngine.AliRtcAudioAccompanyConfig();
config.loopCycles = 1;     // Number of loops
config.publishVolume = 0;   // PSet to 0 to prevent publishing
config.playoutVolume = 100;
config.startPosMs = 0;      // Start position for playback
// Start playback
mRtcEngine.startAudioAccompany(mFilepath, config);

iOS

// Path to the audio file
let filePath = Bundle.main.path(forResource: "music", ofType: "wav")

// Playback configuration
let config = AliRtcAudioAccompanyConfig()
config.loopCycles = 1   // Number of loops. -1 means infinite loop.
config.publishVolume = 0   // Set to 0 to prevent publishing
config.playoutVolume = 100
config.startPosMs = 0
// Start playback
let result = rtcEngine.startAudioAccompany(withFile: filePath, config: config)

You can control the playback using the following APIs:

• pauseAudioAccompany: Pauses the playback.

• resumeAudioAccompany: Resumes the playback.

• stopAudioAccompany: Stops the playback.

2. Listen for local playback state changes

After starting playback, you will receive state changes through the onAudioAccompanyStateChanged callback.

Android

@Override
public void onAudioAccompanyStateChanged(ARTCAICallEngine.ARTCAICallAudioAccompanyStateCode
                                                 playState, ARTCAICallEngine.ARTCAICallAudioAccompanyErrorCode
                                                 errorCode) {
    if (playState == ARTCAICallEngine.ARTCAICallAudioAccompanyStateCode.ARTCAICallAudioAccompanyEnded) {
        // Handle the end of playback.
    }
}

iOS

func onAudioAccompanyStateChanged(state: ARTCAICallAudioAccompanyState, errorCode: ARTCAICallAudioAccompanyErrorCode) {
    if state == .ARTCAICallAudioAccompanyEnded {
        // Handle the end of playback.
    }
}

References

• To publish raw audio data to remote users, see Custom audio capture.

• To play and publish accompaniment or sound effect files, see Play and publish external audio (sound effects and accompaniment).

FAQ

1. Can startAudioAccompany play multiple files at once?

   No, the accompaniment APIs only support playing one audio file at a time. To play multiple files simultaneously, use the sound effect APIs. For details, see Play and publish external audio (sound effects and accompaniment).

2. Why does pushExternalAudioStreamRawData return 0x01070101?

   This return code indicates that the internal RTC audio buffer is full and cannot accept more data. To resolve this, try the following:

   • Reduce the frequency at which you push data.

   • Increase the sleep duration in your push loop (the duration - 10 value in the example).

   • Check your application's buffer management to ensure you are not pushing data too quickly.