This topic describes how to set video encoding properties by calling APIs.
Feature introduction
During video calls or interactive streaming, you can configure video settings such as video capture configuration, resolution, video frame rate, bitrate, mirror mode, and render mode.
Resolution:
Capture resolution: The resolution of the image provided by a capture device, such as a camera.
Encoding resolution: The resolution of the image after it has been encoded.
Bitrate: The number of bits transmitted per second, measured in bps (bits per second).
Frame rate: The number of frames displayed per second, measured in frames per second (fps).
Setting appropriate video resolution, frame rate, and bitrate can provide users with a better experience. Choosing the right mirror and render modes allows you to offer personalized video display styles.
Sample code
Android: Android/ARTCExample/BasicUsage/src/main/java/com/aliyun/artc/api/basicusage/VideoBasicUsage/VideoBasicUsageActivity.java
iOS: iOS/ARTCExample/BasicUsage/VideoBasicUsage/VideoBasicUsageVC.swift
Before you begin
Make sure you meet the following requirements:
Create an ARTC application and obtain the AppID and AppKey from the ApsaraVideo Live console.
Integrate the ARTC SDK into your project and implement basic audio and video call features.
Implementation
1. Set camera capture configurations
ARTC provides the setCameraCapturerConfiguration method to set the camera capture configuration, including camera direction and capture frame rate.
This method must be called before the camera is started, for example, before calling
startPrevieworjoinChannel(which automatically starts the camera).Calling
enableLocalVideo(false)to turn off camera capture releases camera resources. You can then reset the configuration.
The method is defined as follows:
/**
* @brief Sets capture preference.
* @param cameraCapturerConfiguration The preference settings.
* - preference:
* - {@link AliRtcCaptureOutputPreference#ALIRTC_CAPTURER_OUTPUT_PREFERENCE_PREVIEW} High-definition preview. Capture prioritizes video preview quality.
* - {@link AliRtcCaptureOutputPreference#ALIRTC_CAPTURER_OUTPUT_PREFERENCE_PERFORMANCE} Capture selects the resolution closest to the published stream, prioritizing device performance.
* - {@link AliRtcCaptureOutputPreference#ALIRTC_CAPTURER_OUTPUT_PREFERENCE_AUTO} Automatically adjusts the capture resolution.
* - cameraDirection: Sets the capture direction, either the front or rear camera.
* @return
* - 0: success
* - A non-zero value: failure
* @note You must set this parameter before you enable the camera, for example, before you call {@link #startPreview} or {@link #joinChannel}.
*/
public abstract int setCameraCapturerConfiguration(AliEngineCameraCapturerConfiguration cameraCapturerConfiguration);The following table describes the parameters.
Parameter | Type | Description |
preference | The capture preference.
| |
cameraDirection | The camera direction (front or rear). | |
fps | int | The capture frame rate. The default value is -1, which means the SDK uses the fps from the encoding configuration (internally 15). |
cameraCaptureProfile | Specifies a specific resolution for video capture.
| |
textureEncode | int | (Android only) Specifies whether to use texture encoding. |
cameraTextureCapture | int | (Android only) Specifies whether to enable texture capture for the camera. |
API call examples:
Android
AliRtcEngine.AliEngineCameraCapturerConfiguration config = new AliRtcEngine.AliEngineCameraCapturerConfiguration();
config.preference = AliRtcEngine.AliRtcCaptureOutputPreference.ALIRTC_CAPTURER_OUTPUT_PREFERENCE_AUTO;
config.cameraCaptureProfile = AliRtcEngine.AliRtcCameraCaptureProfile.ALIRTC_CAMERA_CAPTURER_PROFILE_DEFAULT;
config.cameraDirection = AliRtcEngine.AliRtcCameraDirection.CAMERA_FRONT;
config.fps = 30;
mAliRtcEngine.setCameraCapturerConfiguration(config);iOS
let cameraCaptureConfig: AliRtcCameraCapturerConfiguration = AliRtcCameraCapturerConfiguration()
cameraCaptureConfig.preference = .auto
cameraCaptureConfig.cameraCaptureProfile = .profileDefault
cameraCaptureConfig.cameraDirection = .front
cameraCaptureConfig.fps = 30
engine.setCameraCapturerConfiguration(cameraCaptureConfig)2. Set video encoding configurations
ARTC provides the setVideoEncoderConfiguration API to configure video encoding properties, including parameters that affect video quality such as resolution, frame rate, bitrate, and keyframe interval. By setting video encoding properties, developers can control how video streams are displayed under different network conditions.
setVideoEncoderConfiguration can be called both before and after joining a channel. If you only need to set it once during a call, it is recommended to set it before joining the channel.
The following table describes the parameters.
Parameter | Type | Description |
dimensions | Video resolution. Default value: 640x480. Maximum value: 1920x1080. | |
frameRate | int | Video encoding frame rate. Default value: 15. Maximum value: 30. |
bitrate | int | Video encoding bitrate (in kbps). Default value: 512. If set to 0, the SDK internally calculates a suitable encoding bitrate based on the video resolution and frame rate. The bitrate setting must be within a reasonable range corresponding to the resolution and frame rate. Otherwise, the SDK automatically adjusts the bitrate to a valid value. For the relationship between bitrate, resolution, and frame rate, refer to the code comments. |
keyFrameInterval | int | Keyframe interval, in milliseconds. Default value: 0, which means the SDK controls the keyframe interval internally. Note This must be set for interactive streaming and scenarios requiring interoperability with mini programs. |
forceStrictKeyFrameInterval | boolean | Specifies whether to force the encoder to strictly generate keyframes at the specified interval. The default value is false.
|
mirrorMode | The mirror mode for encoded video. This parameter controls whether the ingested video stream is mirrored. Note Use the | |
orientationMode | The rotation mode for encoded video.
| |
rotationMode | The video rotation angle (0, 90, 180, or 270). | |
codecType | The codec type.
| |
encodeCodecType | The codec. Supported: system default, H.264, H.265. | |
seiForceFrontIFrame | int | Forces an I-frame before sending Supplemental Enhancement Information (SEI). -1 indicates the default value (which is 1), 0 indicates not to force, and 1 indicates to force. |
API call examples:
Android
AliRtcEngine.AliRtcVideoEncoderConfiguration aliRtcVideoEncoderConfiguration = new AliRtcEngine.AliRtcVideoEncoderConfiguration();
aliRtcVideoEncoderConfiguration.dimensions = new AliRtcEngine.AliRtcVideoDimensions(720, 1280);
aliRtcVideoEncoderConfiguration.frameRate = 20;
aliRtcVideoEncoderConfiguration.bitrate = 1200;
aliRtcVideoEncoderConfiguration.keyFrameInterval = 2000;
aliRtcVideoEncoderConfiguration.orientationMode = AliRtcVideoEncoderOrientationModeAdaptive;
mAliRtcEngine.setVideoEncoderConfiguration(aliRtcVideoEncoderConfiguration);iOS
let config = AliRtcVideoEncoderConfiguration()
config.dimensions = CGSize(width: 720, height: 1280)
config.frameRate = 20
config.bitrate = 1200
config.keyFrameInterval = 2000
config.orientationMode = AliRtcVideoEncoderOrientationMode.adaptive
engine.setVideoEncoderConfiguration(config)3. Switch the camera direction
Android and iOS devices typically have front and rear cameras. The SDK uses the front camera by default. To change this, call the setCameraCaptureConfiguration API before the camera is started. If the camera is already on and you need to switch between the front and rear cameras, call the switchCamera API.
/**
* @brief Switches between the front and rear cameras.
* @return
* - 0: success
* - A non-zero value: failure
* @note This API is available only for iOS and Android.
*/
public abstract int switchCamera();API call examples:
Android
mSwitchCameraBtn.setOnClickListener(v -> {
if(mAliRtcEngine != null) {
mAliRtcEngine.switchCamera();
}
});iOS
@IBAction func onCameraDirectionChanged(_ sender: UISegmentedControl) {
rtcEngine?.switchCamera()
}4. Turn the camera on or off
ARTC provides two APIs to turn the camera on or off during an audio/video call: muteLocalCamera and enableLocalVideo.
API | muteLocalCamera | enableLocalVideo |
How it works | Replaces the sent data with black frames. | Stops the camera hardware from capturing video and releases related resources. |
Effect | The local preview remains normal, while remote users see a black screen. | Both local and remote views freeze on the last frame. |
Characteristics |
|
|
4.1. Stop or resume sending video data
ARTC provides the muteLocalCamera API to implement a video mute function. While keeping the video capture, encoding, and transmission channels running, it sends black video frames to remote users (the local preview remains normal). The API is defined as follows:
/**
* Stops or resumes sending local video data.
* @param mute true: sends black frames. false: resumes normal sending.
* @param track Only {@link AliRtcVideoTrack#AliRtcVideoTrackCamera} is supported.
* @return
* - 0: success
* - A non-zero value: failure
* @note Sends black video frames. The local preview is normal. The capture, encoding, and sending modules are still working, but the video content is replaced with black frames.
*/
public abstract int muteLocalCamera(boolean mute, AliRtcVideoTrack track);
/**
* @brief A notification that a remote user is sending black video frames.
* @param uid The ID of the user who called muteVideo.
* @param isMute
* - true: Black frames are ingested.
* - false: Normal stream ingest.
* @note This interface is a callback that is triggered when a remote user sends black video frames.
*/
public void onUserVideoMuted(String uid ,boolean isMute){}API call examples:
Android
Turn camera capture on or off:
if(!isMutedCamera) {
mAliRtcEngine.muteLocalCamera(true, AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera);
mPublishVideoBtn.setText(R.string.resume_pub_video);
isMutedCamera = true;
} else {
mAliRtcEngine.muteLocalCamera(false, AliRtcEngine.AliRtcVideoTrack.AliRtcVideoTrackCamera);
mPublishVideoBtn.setText(R.string.stop_pub_video);
isMutedCamera = false;
}Remote listener callback:
@Override
public void onUserVideoMuted(String uid ,boolean isMute){
handler.post(new Runnable() {
@Override
public void run() {
ToastHelper.showToast(VideoBasicUsageActivity.this, "remote user uid:" + uid + " camera mute:" + isMute, Toast.LENGTH_SHORT);
}
});
}iOS
Turn camera capture on or off:
@IBAction func onVideoMuteSwitched(_ sender: UISwitch) {
if sender.isOn {
// Black frames
rtcEngine?.muteLocalCamera(false, for: AliRtcVideoTrack.camera)
} else {
rtcEngine?.muteLocalCamera(true, for: AliRtcVideoTrack.camera)
}
}Remote listener callback:
extension VideoBasicUsageVC: AliRtcEngineDelegate {
func onUserVideoMuted(_ uid: String, videoMuted isMute: Bool) {
"onUserVideoMuted: user id \(uid) video muted: \(isMute)".printLog()
}
}4.2. Turn camera capture on or off
The enableLocalVideo API provides global control over the start and stop state of the local video capture device (such as the camera), affecting the generation and transmission of the video data stream.
/**
* @brief Disables or enables local video capture.
* @param enabled
* - true: Enables.
* - false: Disables.
* @return
* - 0: success
* - < 0: failure
* @note By default, this feature is enabled. You can listen for {@link AliRtcEngineNotify#onUserVideoEnabled} to obtain the status of local video capture.
*/
public abstract int enableLocalVideo(boolean enabled);
/**
* @brief A notification that a remote user has disabled camera stream capture.
* @param uid The ID of the user who called enableLocalVideo.
* @param isEnable
* - true: The camera stream capture is enabled.
* - false: The camera stream capture is disabled.
* @note This interface is a callback that is triggered when a remote user disables camera stream capture.
*/
public void onUserVideoEnabled(String uid, boolean isEnable){}API call examples:
Android
Turn the camera on or off:
mCameraSwitchBtn = findViewById(R.id.camera_control_btn);
mCameraSwitchBtn.setOnClickListener(v -> {
if(mAliRtcEngine != null) {
if(isEnableCamera) {
mAliRtcEngine.enableLocalVideo(false);
isEnableCamera = false;
mCameraSwitchBtn.setText(R.string.camera_on);
} else {
mAliRtcEngine.enableLocalVideo(true);
isEnableCamera = true;
mCameraSwitchBtn.setText(R.string.camera_off);
}
}
});Remote listener callback:
@Override
public void onUserVideoEnabled(String uid, boolean isEnable) {
handler.post(new Runnable() {
@Override
public void run() {
ToastHelper.showToast(VideoBasicUsageActivity.this, "remote user uid:" + uid + " camera enable:" + isEnable, Toast.LENGTH_SHORT);
}
});
}iOS
Turn the camera on or off:
@IBAction func onCameraSwitch(_ sender: UISwitch) {
if sender.isOn {
rtcEngine?.enableLocalVideo(true)
} else {
rtcEngine?.enableLocalVideo(false)
}
updateCaptureUIVisibility()
}Remote listener callback:
extension VideoBasicUsageVC: AliRtcEngineDelegate {
func onUserVideoEnabled(_ uid: String?, videoEnabled isEnable: Bool) {
"onUserVideoEnabled: user id \(uid ?? "invalid uid") video enable: \(isEnable)".printLog()
}
}5. Start or stop preview
ARTC provides the startPreview and stopPreview APIs to start and stop the local preview.
Note the following:
Before starting the preview, you need to call
setLocalViewConfigto set a render view for the local preview.By default, the SDK starts the preview when joining a channel. If you need to start the preview before joining, call
startPreviewin advance.After the preview is stopped, the local preview view freezes on the last frame.
Start the preview:
API call examples:
Android
Start the preview:
private void startPreview() {
if (mAliRtcEngine != null) {
ViewGroup.LayoutParams layoutParams = new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT);
// Set the local view.
if (mLocalVideoCanvas == null) {
mLocalVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
SurfaceView localSurfaceView = mAliRtcEngine.createRenderSurfaceView(VideoBasicUsageActivity.this);
if (localSurfaceView != null) {
localSurfaceView.setZOrderOnTop(true);
localSurfaceView.setZOrderMediaOverlay(true);
fl_local.addView(localSurfaceView, layoutParams);
mLocalVideoCanvas.view = localSurfaceView;
try {
mAliRtcEngine.setLocalViewConfig(mLocalVideoCanvas, AliRtcVideoTrackCamera);
} catch (Exception e) {
e.printStackTrace(); // Handle potential exceptions
}
}
}
// Start the preview.
mAliRtcEngine.startPreview();
}
}Stop the preview:
mAliRtcEngine.stopPreview();
mAliRtcEngine.setLocalViewConfig(null, AliRtcVideoTrackCamera);
mAliRtcEngine.leaveChannel();
mAliRtcEngine.destroy();
mAliRtcEngine = null;iOS
Start the preview:
func startPreview() {
let seatView = self.createSeatView(uid: self.userId)
let canvas = AliVideoCanvas()
canvas.view = seatView.canvasView
canvas.renderMode = .auto
canvas.mirrorMode = .onlyFrontCameraPreviewEnabled
canvas.rotationMode = ._0
self.rtcEngine?.setLocalViewConfig(canvas, for: AliRtcVideoTrack.camera)
self.rtcEngine?.startPreview()
}Stop the preview:
self.rtcEngine?.stopPreview()
self.rtcEngine?.leaveChannel()
AliRtcEngine.destroy()
self.rtcEngine = nil6. Set the mirror mode
ARTC provides the setVideoMirrorMode API to control the mirroring behavior of the local video preview and the published stream. It supports dynamic adjustment at runtime and is suitable for video calls and live streaming.
/**
* @brief Sets the preview and publishing mirror capabilities.
* @param mirrorMode The mirror mode.
* @return
* - 0: Success
* - <0: Failure.
* - AliRtcErrInner: Internal SDK state error. Check if the SDK instance was created successfully.
*
* @note
* - This API can be dynamically set both before and after joining a channel. The SDK records the state and applies it to the video when the preview and encoding are operational.
* - This API has higher priority than the mirror settings in setLocalViewConfig and setVideoEncoderConfiguration.
* - This API's functionality overlaps with the mirror settings in setLocalViewConfiguration and setVideoEncoderConfiguration. It is recommended to use only one method.
*/
public abstract int setVideoMirrorMode(AliRtcVideoPipelineMirrorMode mirrorMode);The following table describes the mirror modes.
Enum value | Description |
AliRtcVideoPipelineMirrorModeNoMirror | Mirroring is disabled for both preview and encoding. |
AliRtcVideoPipelineMirrorModeBothMirror | Mirroring is enabled for both preview and encoding (default). |
AliRtcVideoPipelineMirrorModeOnlyPreviewMirror | Mirroring is enabled only for the preview. |
AliRtcVideoPipelineMirrorModeOnlyPublishMirror | Mirroring is enabled only for the published stream. |
API call example
Android
mMirrorSpinner.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
@Override
public void onItemSelected(AdapterView<?> adapterView, View view, int position, long id) {
if(mAliRtcEngine != null) {
AliRtcEngine.AliRtcVideoPipelineMirrorMode mirrorMode = AliRtcEngine.AliRtcVideoPipelineMirrorMode.values()[position];
mAliRtcEngine.setVideoMirrorMode(mirrorMode);
}
}
@Override
public void onNothingSelected(AdapterView<?> adapterView) {
}
});iOS
let mirrorMode: AliRtcVideoPipelineMirrorMode = {
switch row {
case 0: return .bothMirror
case 1: return .noMirror
case 2: return .onlyPreviewMirror
case 3: return .onlyPublishMirror
default: return .bothMirror
}
}()
self.rtcEngine?.setVideoMirrorMode(mirrorMode)7. Set the render view
Before displaying the local preview and remote user's video, call the setLocalViewConfig or setRemoteViewConfig API to set a render view for the video to be displayed.
// Set the display view for the local preview.
public abstract int setLocalViewConfig(AliRtcVideoCanvas viewConfig, AliRtcVideoTrack track);
// Set the display view for a specified remote user.
public abstract int setRemoteViewConfig(AliRtcVideoCanvas canvas, String uid, AliRtcVideoTrack track);Common AliRtcVideoCanvas configurations:
Parameter | Type | Description |
view | View | The display view (required). |
renderMode | AliRtcRenderMode | The render mode.
|
mirrorMode | AliRtcRenderMirrorMode | The mirror mode.
|
rotationMode | AliRtcRotationMode | The rotation mode (0, 90, 180, or 270). |
backgroundColor | int | The background color in hexadecimal RGB format, such as 0x000000. |
textureId | int | (Android only) Supports third-party OpenGL ES texture display. The texture ID. |
textureWidth | int | (Android only) Supports third-party OpenGL ES texture display. The texture width. |
textureHeight | int | (Android only) Supports third-party OpenGL ES texture display. The texture height. |
sharedContext | long | (Android only) Supports third-party OpenGL ES texture display. The shared texture context. |
API call examples
7.1. Set the local render view
Android
mLocalVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
// Get and set the SurfaceView.
SurfaceView localSurfaceView = mAliRtcEngine.createRenderSurfaceView(VideoChatActivity.this);
localSurfaceView.setZOrderOnTop(true);
localSurfaceView.setZOrderMediaOverlay(true);
FrameLayout fl_local = findViewById(R.id.fl_local);
fl_local.addView(localSurfaceView, layoutParams);
mLocalVideoCanvas.view = localSurfaceView;
// Set the local preview view.
mAliRtcEngine.setLocalViewConfig(mLocalVideoCanvas, AliRtcVideoTrackCamera);
mAliRtcEngine.startPreview();iOS
let videoView = self.createVideoView(uid: self.userId)
let canvas = AliVideoCanvas()
canvas.view = videoView.canvasView
canvas.renderMode = .auto
canvas.mirrorMode = .onlyFrontCameraPreviewEnabled
canvas.rotationMode = ._0
self.rtcEngine?.setLocalViewConfig(canvas, for: AliRtcVideoTrack.camera)
self.rtcEngine?.startPreview()Windows
AliEngineVideoCanvas canvas;
/* Windows window handle */
canvas.view = mHWnd;
mAliRtcEngine.setLocalViewConfig(canvas, AliEngineVideoTrackCamera);7.2. Set the remote render view
Android
@Override
public void onRemoteTrackAvailableNotify(String uid, AliRtcEngine.AliRtcAudioTrack audioTrack, AliRtcEngine.AliRtcVideoTrack videoTrack){
handler.post(new Runnable() {
@Override
public void run() {
if(videoTrack == AliRtcVideoTrackCamera) {
SurfaceView surfaceView = mAliRtcEngine.createRenderSurfaceView(VideoChatActivity.this);
surfaceView.setZOrderMediaOverlay(true);
FrameLayout fl_remote = findViewById(R.id.fl_remote);
if (fl_remote == null) {
return;
}
fl_remote.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
AliRtcEngine.AliRtcVideoCanvas remoteVideoCanvas = new AliRtcEngine.AliRtcVideoCanvas();
remoteVideoCanvas.view = surfaceView;
mAliRtcEngine.setRemoteViewConfig(remoteVideoCanvas, uid, AliRtcVideoTrackCamera);
} else if(videoTrack == AliRtcVideoTrackNo) {
FrameLayout fl_remote = findViewById(R.id.fl_remote);
fl_remote.removeAllViews();
mAliRtcEngine.setRemoteViewConfig(null, uid, AliRtcVideoTrackCamera);
}
}
});
}iOS
func onRemoteTrackAvailableNotify(_ uid: String, audioTrack: AliRtcAudioTrack, videoTrack: AliRtcVideoTrack) {
"onRemoteTrackAvailableNotify uid: \(uid) audioTrack: \(audioTrack) videoTrack: \(videoTrack)".printLog()
// The stream status of the remote user.
if audioTrack != .no {
let videoView = self.videoViewList.first { $0.uidLabel.text == uid }
if videoView == nil {
_ = self.createVideoView(uid: uid)
}
}
if videoTrack != .no {
var videoView = self.videoViewList.first { $0.uidLabel.text == uid }
if videoView == nil {
videoView = self.createVideoView(uid: uid)
}
let canvas = AliVideoCanvas()
canvas.view = videoView!.canvasView
canvas.renderMode = .auto
canvas.mirrorMode = .onlyFrontCameraPreviewEnabled
canvas.rotationMode = ._0
self.rtcEngine?.setRemoteViewConfig(canvas, uid: uid, for: AliRtcVideoTrack.camera)
}
else {
self.rtcEngine?.setRemoteViewConfig(nil, uid: uid, for: AliRtcVideoTrack.camera)
}
if audioTrack == .no && videoTrack == .no {
self.removeVideoView(uid: uid)
self.rtcEngine?.setRemoteViewConfig(nil, uid: uid, for: AliRtcVideoTrack.camera)
}
}Windows
virtual void OnRemoteTrackAvailableNotify(const char *uid, AliEngineAudioTrack audioTrack, AliEngineVideoTrack videoTrack) {
AliEngineVideoCanvas remote_canvas;
if (videoTrack == AliEngineVideoTrackCamera
|| videoTrack == AliEngineVideoTrackBoth) {
RECT rect;
::GetWindowRect(mHWnd, &rect);
remote_canvas.displayView = remoteView;
remote_canvas.renderMode = AliEngineRenderModeAuto;
mAliRtcEngine->SetRemoteViewConfig(remote_canvas,uid,AliEngineVideoTrackCamera);
} else {
mAliRtcEngine->SetRemoteViewConfig(remote_canvas, uid, AliEngineVideoTrackCamera);
}
}