ApsaraVideo Live:Tangkapan audio kustom

1. Aktifkan atau nonaktifkan tangkapan internal

Untuk menggunakan tangkapan audio kustom, Anda harus terlebih dahulu menonaktifkan modul tangkapan internal SDK. Kami menyarankan melakukannya dengan meneruskan parameter extras saat memanggil getInstance untuk membuat engine. Gunakan parameter berikut:

user_specified_use_external_audio_record: Menonaktifkan tangkapan internal SDK untuk mengaktifkan tangkapan audio kustom.

• "TRUE": Gunakan tangkapan audio kustom (menonaktifkan tangkapan internal).

• "FALSE": Jangan gunakan tangkapan audio kustom (mengaktifkan tangkapan internal).

String extras = "{\"user_specified_use_external_audio_record\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.getInstance(this, extras);

// Create and initialize the engine.
var customAudioCaptureConfig: [String: String] = [:]
// Use custom audio capture.
customAudioCaptureConfig["user_specified_use_external_audio_record"] = "TRUE"
// Serialize to JSON.
guard let jsonData = try? JSONSerialization.data(withJSONObject: customAudioCaptureConfig, options: []),
let extras = String(data: jsonData, encoding: .utf8) else {
     print("JSON serialization failed")
     return
 }
let engine = AliRtcEngine.sharedInstance(self, extras:extras)

NSString * extras = @"{\"user_specified_use_external_audio_record\":\"TRUE\"}";
mAliRtcEngine = [AliRtcEngine sharedInstance:self extras:extras];

/* Windows supports enabling or disabling audio capture during engine creation. */
/* Disable internal capture. */
char* extra = "{\"user_specified_enable_use_virtual_audio_device\":\"TRUE\", \"user_specified_use_external_audio_record\":\"TRUE\"}";
mAliRtcEngine = AliRtcEngine.Create(extra);

/* Enable internal capture. */
char* extra = "{\"user_specified_enable_use_virtual_audio_device\":\"FALSE\", \"user_specified_use_external_audio_record\":\"FALSE\"}";
mAliRtcEngine = AliRtcEngine.Create(extra);

2. Tambahkan aliran audio eksternal

Panggil metode addExternalAudioStream untuk menambahkan aliran audio eksternal dan mendapatkan ID alirannya. Jika Anda memerlukan pemrosesan audio 3A—yang mencakup acoustic echo cancellation (AEC), automatic gain control (AGC), dan noise suppression (ANS)—atur parameter enable3A dalam objek AliRtcExternalAudioStreamConfig.

AliRtcEngine.AliRtcExternalAudioStreamConfig config = new AliRtcEngine.AliRtcExternalAudioStreamConfig();
config.sampleRate = SAMPLE_RATE; // Laju sampel
config.channels = CHANNEL; // Jumlah channel
// Volume publikasi
config.publishVolume = 100;
// Volume pemutaran lokal
config.playoutVolume = isLocalPlayout ? 100 : 0;
config.enable3A = true;

int result = mAliRtcEngine.addExternalAudioStream(config);
if (result <= 0) {
    return;
}
// Nilai kembali adalah ID aliran. Anda membutuhkannya untuk mendorong data ke SDK.
mExternalAudioStreamId = result;

/* Set parameters based on your application's needs. */
AliRtcExternalAudioStreamConfig *config = [AliRtcExternalAudioStreamConfig new];
// This must match the number of channels of the external PCM audio stream. Set to 1 for mono or 2 for stereo.
config.channels = _pcmChannels;
// This must match the sample rate of the external PCM audio stream.
config.sampleRate = _pcmSampleRate;
config.playoutVolume = 0;
config.publishVolume = 100;
_externalPlayoutStreamId = [self.engine addExternalAudioStream:config];

/* Set parameters based on your application's needs. */
AliRtcExternalAudioStreamConfig *config = [AliRtcExternalAudioStreamConfig new];
config.channels = pcmChannels;
/** Sample rate. Default: 48000. Supported values: 8000, 12000, 16000, 24000, 32000, 44100, 48000, 64000, 88200, 96000, 176400, 192000. */
config.sampleRate = pcmSampleRate;
config.playoutVolume = 0;
config.publishVolume = 100;
int ret = [self.engine addExternalAudioStream:config];

/* Get the media engine. */
IAliEngineMediaEngine* mAliRtcMediaEngine = nullptr;
    
mAliRtcEngine->QueryInterface(AliEngineInterfaceMediaEngine, (void **)&mAliRtcMediaEngine);
/* Set parameters based on your application's needs. */
AliEngineExternalAudioStreamConfig config;
config.playoutVolume = currentAudioPlayoutVolume;
config.publishVolume = currentAudioPublishVolume;
config.channels = 1;
config.sampleRate = 48000;
config.publishStream = 0;
audioStreamID = mAliRtcMediaEngine->AddExternalAudioStream(config);

mAliRtcMediaEngine->Release();

3. Implementasikan modul tangkapan audio kustom

Dengan tangkapan audio kustom, Anda bertanggung jawab untuk mengimplementasikan logika menangkap dan memproses data audio, lalu mengirim data tersebut ke SDK.

Alibaba Cloud menyediakan contoh tangkapan kustom yang menunjukkan cara membaca data berformat PCM dari file lokal atau mikrofon.

4. Dorong data audio ke SDK

Setelah aliran audio berhasil dipublikasikan (status dalam callback onAudioPublishStateChanged berubah menjadi AliRtcStatsPublished), panggil metode pushExternalAudioStreamRawData, atur parameter mExternalAudioStreamId ke ID aliran audio yang diperoleh pada Langkah 2, lalu teruskan data audio yang dikumpulkan ke SDK. Data audio yang dikumpulkan harus dikonversi menjadi objek AliRtcAudioFrame. Konfigurasi terkait adalah sebagai berikut:

• data: Data audio.

• numSamples: Jumlah titik sampel per channel dalam data yang disediakan.

• bytesPerSample: Byte per titik sampel (kedalaman bit / 8). Misalnya, nilai ini adalah 2 untuk audio 16-bit.

• numChannels: Jumlah channel audio.

• samplesPerSec: Laju sampel, dalam Hz (misalnya, 16.000 atau 48.000).

// Assume the captured audio data is in `audioData`, the size is `bytesRead` bytes, and it represents 10 ms of data.
if (mAliRtcEngine != null && bytesRead > 0) {
    // Construct an AliRtcAudioFrame object. `bitsPerSample` is the bit depth, which is typically 16.
    AliRtcEngine.AliRtcAudioFrame sample = new AliRtcEngine.AliRtcAudioFrame();
    sample.data = audioData;
    sample.numSamples = bytesRead / (channels * (bitsPerSample / 8)); // Calculate the number of samples based on the actual number of bytes read.
    sample.numChannels = channels;
    sample.samplesPerSec = sampleRate;
    sample.bytesPerSample = bitsPerSample / 8;

    int ret = 0;
    // Retry the push operation if it fails because the buffer is full.
    int retryCount = 0;
    final int MAX_RETRY_COUNT = 20;
    final int BUFFER_WAIT_MS = 10;
    do {
        // Push the captured data to the SDK.
        ret = mAliRtcEngine.pushExternalAudioStreamRawData(mExternalAudioStreamId, sample);
        if(ret == ErrorCodeEnum.ERR_SDK_AUDIO_INPUT_BUFFER_FULL) {
            // Handle the buffer full scenario. Wait for a short period and retry.
            retryCount++;
            if(mExternalAudioStreamId <= 0 || retryCount >= MAX_RETRY_COUNT) {
                // The stream has been stopped or the maximum retry count is reached. Exit the loop.
                break;
            }

            try {
                // Pause for a short interval.
                Thread.sleep(BUFFER_WAIT_MS);
            } catch (InterruptedException e) {
                e.printStackTrace();
                break;
            }
        } else {
            // Push succeeded or another error occurred. Exit the loop.
            break;
        }
    } while (retryCount < MAX_RETRY_COUNT);
}

// Construct an AliRtcAudioFrame object from the captured audio data.
let sample = AliRtcAudioFrame()
sample.dataPtr = UnsafeMutableRawPointer(mutating: pcmData)
sample.samplesPerSec = pcmSampleRate
sample.bytesPerSample = Int32(MemoryLayout<Int16>.size)
sample.numOfChannels = pcmChannels
sample.numOfSamples = numOfSamples

var retryCount = 0

while retryCount < 20 {
    if !(pcmInputThread?.isExecuting ?? false) {
        break
    }
    // Push the audio data to the SDK.
    let rc = rtcEngine?.pushExternalAudioStream(externalPublishStreamId, rawData: sample) ?? 0

    // Handle a full buffer.
    // 0x01070101 SDK_AUDIO_INPUT_BUFFER_FULL: The buffer is full. Retransmission is required.
    if rc == 0x01070101 && !(pcmInputThread?.isCancelled ?? true) {
        Thread.sleep(forTimeInterval: 0.03) // 30ms
        retryCount += 1;
    } else {
        if rc < 0 {
            "pushExternalAudioStream error, ret: \(rc)".printLog()
        }
        break
    }
}

while ( true ) {
    
    if (![pcmInputThread isExecuting]) {
        push_error = YES;
        break;
    }

    AliRtcAudioFrame *sample = [AliRtcAudioFrame new];
    sample.dataPtr = pcmData;
    sample.samplesPerSec = pcmSampleRate;
    sample.bytesPerSample = sizeof(int16_t);
    sample.numOfChannels = pcmChannels;
    sample.numOfSamples = numOfSamples;
    int rc = [self.engine pushExternalAudioStream:_externalPublishStreamId rawData:sample];
    count = count + 1;

    /* If the error is AliRtcErrAudioBufferFull, sleep for a moment and then continue pushing. */
    if ( rc == AliRtcErrAudioBufferFull && [pcmInputThread isCancelled ] == NO ) {
        [NSThread sleepForTimeInterval:0.04] ;
    }else {
        if ( rc < 0 ) {
            push_error = true ;
        }

        break ;
    }

}

/* Get the media engine. */
IAliEngineMediaEngine* mAliRtcMediaEngine = nullptr;   
mAliRtcEngine->QueryInterface(AliEngineInterfaceMediaEngine, (void **)&mAliRtcMediaEngine);

// Construct an audio frame from the data.
AliEngineAudioRawData rawData;
rawData.dataPtr = frameInfo.audio_data[0];
rawData.numOfSamples = (int) (frameInfo.audio_data[0].length / (2 * frameInfo.audio_channels));
rawData.bytesPerSample = 2;
rawData.numOfChannels = frameInfo.audio_channels;
rawData.samplesPerSec = frameInfo.audio_sample_rate;
// Push the data to the SDK.
int ret = mAliRtcMediaEngine->PushExternalAudioStreamRawData(audioStreamID, rawData);
// Handle buffer full and other errors.
if ( ret == AliEngineErrorAudioBufferFull ) {
    Sleep(40);
    continue ;
} 

// Release the media engine.
mAliRtcMediaEngine->Release();

5. Hapus aliran audio eksternal

Untuk berhenti mempublikasikan audio dari sumber kustom, panggil removeExternalAudioStream.

mAliRtcEngine.removeExternalAudioStream(mExternalAudioStreamId);

[self.engine removeExternalAudioStream:_externalPublishStreamId];

[self.engine removeExternalAudioStream:_externalPublishStreamId];

/* Get the media engine. */
IAliEngineMediaEngine* mAliRtcMediaEngine = nullptr;
    
mAliRtcEngine->QueryInterface(AliEngineInterfaceMediaEngine, (void **)&mAliRtcMediaEngine);

mAliRtcMediaEngine->RemoveExternalAudioStream(audioStreamID);
mAliRtcMediaEngine->Release();

6. (Opsional) Aktifkan atau nonaktifkan tangkapan internal secara dinamis

Untuk mengaktifkan atau menonaktifkan tangkapan internal SDK secara dinamis selama panggilan, gunakan metode setParameter.

/* Dynamically disable internal capture. */
String parameter = "{\"audio\":{\"enable_system_audio_device_record\":\"FALSE\"}}";
mAliRtcEngine.setParameter(parameter);

/* Dynamically enable internal capture. */
String parameter = "{\"audio\":{\"enable_system_audio_device_record\":\"TRUE\"}}"; 
mAliRtcEngine.setParameter(parameter);

// Dynamically disable internal capture.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_record\":\"FALSE\"}}")
// Dynamically enable internal capture.
engine.setParameter("{\"audio\":{\"enable_system_audio_device_record\":\"TRUE\"}}")

// Dynamically disable internal capture.
[self setParameter:@"{\"audio\":{\"enable_system_audio_device_record\":\"FALSE\"}}"];
// Dynamically enable internal capture.
[self setParameter:@"{\"audio\":{\"enable_system_audio_device_record\":\"TRUE\"}}"];

/* Dynamically disable internal capture. */
mAliRtcEngine->SetParameter("{\"audio\":{\"enable_system_audio_device_record\":\"FALSE\"}}");

/* Dynamically enable internal capture. */
mAliRtcEngine->SetParameter("{\"audio\":{\"enable_system_audio_device_record\":\"TRUE\"}}");