Overview
The ARTC SDK provides a flexible custom video capture feature that lets you manage video capture devices according to your business requirements.
We recommend using the SDK's internal capture by default. However, if it does not meet your specific requirements for video quality, device compatibility, or the capture workflow, custom video capture offers greater extensibility and customization.
Sample code
Android custom video capture: Android/ARTCExample/AdvancedUsage/src/main/java/com/aliyun/artc/api/advancedusage/CustomVideoCaptureAndRender/CustomVideoCaptureActivity.java.
iOS custom video capture: iOS/ARTCExample/AdvancedUsage/CustomVideoCapture/CustomVideoCaptureVC.swift.
Prerequisites
Before you begin, ensure you meet the following requirements:
You have an active Alibaba Cloud account, have created an ARTC application, and have obtained an AppID and AppKey. For instructions, see Create an application. Your credentials are available in the ApsaraVideo Live console.
You have integrated the ARTC SDK into your project and implemented basic audio and video call features. For more information, see Integrate the ARTC SDK and Implement an audio and video call.
Implementation
1. Disable internal capture
Call the enableLocalVideo method to disable the SDK's internal capture.
Android
/* Disable internal capture. */
mAliRtcEngine.enableLocalVideo(false);
/* Enable internal capture. This feature is enabled by default. */
mAliRtcEngine.enableLocalVideo(true);iOS
/* Disable internal capture. */
[_engine enableLocalVideo:NO];
/* Enable internal capture. This feature is enabled by default. */
[_engine enableLocalVideo:YES];Mac
/* Disable internal capture. */
[self.engine enableLocalVideo:NO];
/* Enable internal capture. This feature is enabled by default. */
[self.engine enableLocalVideo:YES];Windows
/* Disable internal capture. */
mAliRtcEngine->EnableLocalVideo(false);
/* Enable internal capture. This feature is enabled by default. */
mAliRtcEngine->EnableLocalVideo(true);2. Set an external video source
Call setExternalVideoSource to set an external video source. The key parameters are:
enable: Specifies whether to enable the external video source.
useTexture: Specifies whether to use texture input.
streamType: Specifies which stream the SDK should replace, such as a camera stream or a screen sharing stream.
renderMode: The render mode used if the aspect ratio of the external video source does not match the publishing profile. The SDK scales the video according to this mode.
Android
YUV input
/* YUV input */
/* Enable external capture. This example uses a camera stream. You can adjust the sourceType and renderMode as needed. */
mAliRtcEngine.setExternalVideoSource(true,false, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );
/* Disable external capture. This example uses a camera stream. You can adjust the sourceType as needed. */
mAliRtcEngine.setExternalVideoSource(false,false, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );Texture input
/* Texture input */
/* Enable external capture. This example uses a camera stream. You can adjust the sourceType and renderMode as needed. */
mAliRtcEngine.setExternalVideoSource(true,true, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );
/* Disable external capture. This example uses a camera stream. You can adjust the sourceType as needed. */
mAliRtcEngine.setExternalVideoSource(false,true, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );iOS
/* Enable external capture. This example uses a camera stream. You can adjust the sourceType and renderMode as needed. */
[_engine setExternalVideoSource:YES sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto];
/* Disable external capture. This example uses a camera stream. You can adjust the sourceType as needed. */
[_engine setExternalVideoSource:NO sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto]; Mac
/* Enable external capture. This example uses a camera stream. You can adjust the sourceType and renderMode as needed. */
[self.engine setExternalVideoSource:YES sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto];
/* Disable external capture. This example uses a camera stream. You can adjust the sourceType as needed. */
[self.engine setExternalVideoSource:NO sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto];Windows
/* Texture input */
/* Enable external capture. This example uses a camera stream. You can adjust the sourceType and renderMode as needed. */
mAliRtcEngine->SetExternalVideoSource(true,true, AliEngineVideoTrackCamera,AliEngineRenderModeAuto );
/* Disable external capture. This example uses a camera stream. You can adjust the sourceType as needed. */
mAliRtcEngine->SetExternalVideoSource(false,true, AliEngineVideoTrackCamera,AliEngineRenderModeAuto );3. Push video frames to the SDK
After capturing the video frames, push them to the ARTC SDK using the pushExternalVideoFrame method.
Android
YUV input example:
/* This example uses the YUV (I420) format. */
int width = 720;
int height = 1280;
AliRtcEngine.AliRtcVideoFormat videoformat = AliRtcEngine.AliRtcVideoFormat.AliRtcVideoFormatI420;
int[] lineSize = {width, width / 2, width / 2, 0};
int frameLength = width * height * 3 / 2;
byte[] buffer = new byte[frameLength];
/* Construct the data object to pass to the SDK. */
AliRtcEngine.AliRtcRawDataFrame rawDataFrame
= new AliRtcEngine.AliRtcRawDataFrame(buffer,
videoformat,
width,
height,
lineSize,
0,
buffer.length);
/* Call the API to push the data object. */
int ret = mAliRtcEngine.pushExternalVideoFrame(rawDataFrame, AliRtcVideoTrackCamera);
if (ret != 0) {
/* Handle the error. */
}Texture input example:
/* This example uses the texture format. */
/* Create the OpenGL environment. */
private static EglBase14 createEglBase14(EGLContext shareEglContext) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
EglBase14.Context eglBase14Context = shareEglContext == null ? null : new EglBase14.Context(shareEglContext);
EglBase14 eglBase14 = new EglBase14(eglBase14Context, EglBase.CONFIG_PIXEL_BUFFER);
try {
eglBase14.createDummyPbufferSurface();
eglBase14.makeCurrent();
} catch (RuntimeException e) {
Log.e(TAG, "CreateEGLBase14Context, failed, " + e.getMessage());
}
return eglBase14;
}
return null;
}
/* Construct the context for the input data. */
float[] transformMatrix = {
1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1
};
int frameWidth = 720; /* Image width. */
int frameHeight = 1280; /* Image height. */
int textureID = xxx; /* Your texture ID. */
AliRtcEngine.AliRtcRawDataFrame aliRawDataFrame
= new AliRtcEngine.AliRtcRawDataFrame(textureID,
AliRtcVideoFormatTexture2D,
frameWidth,
frameHeight,
transformMatrix,
0,
0,
frameWidth,
frameHeight,
mEglBase14.getEglContext());
/* Call the API to push the data object. */
int ret = mAliRtcEngine.pushExternalVideoFrame(aliRawDataFrame, AliRtcVideoTrackCamera);
if (ret != 0) {
/* Handle the error. */
}iOS
/* Specify the following parameters based on your data format. This example uses the I420 format. */
AliRtcVideoDataSample *dataSample = [[AliRtcVideoDataSample alloc] init];
dataSample.dataPtr = (long)yuv_read_data;
dataSample.format = AliRtcVideoFormat_I420;
dataSample.type = AliRtcBufferType_Raw_Data;
dataSample.width = width;
dataSample.height = height;
dataSample.strideY = width;
dataSample.strideU = width/2;
dataSample.strideV = width/2;
dataSample.dataLength = dataSample.strideY * dataSample.height * 3/2;
/* Call the API to push the data object. */
int ret = [self.engine pushExternalVideoFrame:dataSample sourceType:AliRtcVideosourceCameraType];
if (ret != 0) {
/* Handle the error. */
}Mac
/* Specify the following parameters based on your data format. This example uses the I420 format. */
AliRtcVideoDataSample *dataSample = [[AliRtcVideoDataSample alloc] init];
dataSample.dataPtr = (long)yuv_read_data;
dataSample.format = AliRtcVideoFormat_I420;
dataSample.type = AliRtcBufferType_Raw_Data;
dataSample.width = width;
dataSample.height = height;
dataSample.strideY = width;
dataSample.strideU = width/2;
dataSample.strideV = width/2;
dataSample.dataLength = dataSample.strideY * dataSample.height * 3/2;
/* Call the API to push the data object. */
int ret = [self.engine pushExternalVideoFrame:dataSample sourceType:AliRtcVideosourceCameraType];
if (ret != 0) {
/* Handle the error. */
if ( ret == AliRtcErrAudioBufferFull ) {
// The buffer is full. Wait 20 ms and retry.
[NSThread sleepForTimeInterval:0.02] ;
continue ;
}
break ;
}Windows
AliEngineVideoRawData sample;
sample.dataPtr = (unsigned char*)cache_buf;
sample.format = AliEngineVideoFormatI420;
sample.width = mIYuvWidth;
sample.height = mIYuvHeight;
sample.strideY = mIYuvWidth;
sample.strideU = mIYuvWidth / 2;
sample.strideV = mIYuvWidth / 2;
sample.dataLength = frame_length;
sample.rotation = 0;
/* Call the API to push the data object. */
int ret = mAliRtcEngine->PushExternalVideoFrame(sample, AliEngineVideoTrackScreen);
if ( ret != 0 ) {
if ( ret == AliEngineErrorAudioBufferFull ) {
Sleep(20);
continue;
}else {
// Handle the error.
break ;
}
} else {
}