Overview
The ARTC SDK provides a flexible custom video capture feature, giving you full control over your video capture devices.
We recommend using the SDK's built-in capture module as the primary option. However, if it does not meet your specific requirements for video quality, device compatibility, or the capture workflow, then custom video capture provides an alternative solution. It offers enhanced extensibility and customization。
Sample code
Android: Android/ARTCExample/AdvancedUsage/src/main/java/com/aliyun/artc/api/advancedusage/CustomVideoCaptureAndRender/CustomVideoCaptureActivity.java
iOS: iOS/ARTCExample/AdvancedUsage/CustomVideoCapture/CustomVideoCpatureVC.swift
Before you begin
Before you implement the feature, make sure you meet the following requirements:
Create an ARTC application and obtain the AppID and AppKey from the ApsaraVideo Live console.
Integrate the ARTC SDK into your project and implement basic audio and video call features.
Implementation
1. Disable internal SDK capture
Call the enableLocalVideo method to disable the SDK's internal video capture.
Android
/* Disable internal capture. */
mAliRtcEngine.enableLocalVideo(false);
/* Enable internal capture. Internal capture is enabled by default. You do not need to call this method unless you need to control this feature. */
mAliRtcEngine.enableLocalVideo(true);iOS
/* Disable internal capture. */
[_engine enableLocalVideo:NO];
/* Enable internal capture. Internal capture is enabled by default. You do not need to call this method unless you need to control this feature. */
[_engine enableLocalVideo:YES];2. Set the custom video source
Call setExternalVideoSource to set the custom video source. The main parameters are:
enable: Specifies whether to enable the feature.
useTexture: Specifies whether to use texture as the input.
streamType: Specifies the stream that the SDK replaces. This can be the camera stream or the screen sharing stream.
renderMode: The processing mode. If the aspect ratio of the custom video source does not match the publishing profile, the SDK processes it according to the specified render mode.
Android
YUV input
/* Input data in YUV format. */
/* Enable external capture. This example uses a camera stream. Specify the sourceType and renderMode as needed. */
mAliRtcEngine.setExternalVideoSource(true,false, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );
/* Disable external capture. This example uses a camera stream. Specify the sourceType as needed. */
mAliRtcEngine.setExternalVideoSource(false,false, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );Texture input
/* Input data as texture. */
/* Enable external capture. This example uses a camera stream. Specify the sourceType and renderMode as needed. */
mAliRtcEngine.setExternalVideoSource(true,true, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );
/* Disable external capture. This example uses a camera stream. Specify the sourceType as needed. */
mAliRtcEngine.setExternalVideoSource(false,true, AliRtcVideoTrackCamera,AliRtcRenderModeAuto );iOS
/* Enable external capture. This example uses a camera stream. Specify the streamType and renderMode as needed. */
[_engine setExternalVideoSource:YES sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto];
/* Disable external capture. This example uses a camera stream. Specify the streamType as needed. */
[_engine setExternalVideoSource:NO sourceType:AliRtcVideosourceCameraType renderMode:AliRtcRenderModeAuto]; 3. Send data to the SDK
After capturing the video data:
Android
YUV input
/* This example uses the YUV (I420) format. */
int width = 720;
int height = 1280;
AliRtcEngine.AliRtcVideoFormat videoformat = AliRtcEngine.AliRtcVideoFormat.AliRtcVideoFormatI420;
int[] lineSize = {width, width / 2, width / 2, 0};
int frameLength = width * height * 3 / 2;
byte[] buffer = new byte[frameLength];
/* Construct the data object to pass to the SDK. */
AliRtcEngine.AliRtcRawDataFrame rawDataFrame
= new AliRtcEngine.AliRtcRawDataFrame(buffer,
videoformat,
width,
height,
lineSize,
0,
buffer.length);
/* Call the API to pass the data object. */
int ret = mAliRtcEngine.pushExternalVideoFrame(rawDataFrame, AliRtcVideoTrackCamera);
if (ret != 0) {
/* Print the error message. */
}Texture input
/* This example uses the texture format. */
/* Construct the OpenGL environment. */
private static EglBase14 createEglBase14(EGLContext shareEglContext) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
EglBase14.Context eglBase14Context = shareEglContext == null ? null : new EglBase14.Context(shareEglContext);
EglBase14 eglBase14 = new EglBase14(eglBase14Context, EglBase.CONFIG_PIXEL_BUFFER);
try {
eglBase14.createDummyPbufferSurface();
eglBase14.makeCurrent();
} catch (RuntimeException e) {
eglBase14.release();
Log.e(TAG, "CreateEGLBase14Context, failed, " + e.getMessage());
}
return eglBase14;
}
return null;
}
/* Construct the context of the input data. */
float[] transformMatrix = {
1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1
};
int frameWidth = 720; /* The width of the image. */
int frameHeight = 1280; /* The height of the image. */
int textureID = xxx; /* Your texture ID. */
AliRtcEngine.AliRtcRawDataFrame aliRawDataFrame
= new AliRtcEngine.AliRtcRawDataFrame(textureID,
AliRtcVideoFormatTexture2D,
frameWidth,
frameHeight,
transformMatrix,
0,
0,
frameWidth,
frameHeight,
mEglBase14.getEglContext());
/* Call the API to pass the data object. */
int ret = mAliRtcEngine.pushExternalVideoFrame(aliRawDataFrame, AliRtcVideoTrackCamera);
if (ret != 0) {
/* Print the error message. */
}iOS
/* Specify the following parameters based on your data format. This example uses the I420 format. */
AliRtcVideoDataSample *dataSample = [[AliRtcVideoDataSample alloc] init];
dataSample.dataPtr = (long)yuv_read_data;
dataSample.format = AliRtcVideoFormat_I420;
dataSample.type = AliRtcBufferType_Raw_Data;
dataSample.width = width;
dataSample.height = height;
dataSample.strideY = width;
dataSample.strideU = width/2;
dataSample.strideV = width/2;
dataSample.dataLength = dataSample.strideY * dataSample.height * 3/2;
/* Call the API to pass the data object. */
int ret = [self.engine pushExternalVideoFrame:dataSample sourceType:AliRtcVideosourceCameraType];
if (ret != 0) {
/* Print the error message. */
}