Intelligent Media Services:AliRtcEngine class

Creates an AliRtcEngine instance.

static AliEngine *Create(const char *extras);

Call timing

Make sure to call this method to create an AliRtcEngine instance before calling other ARTC SDK APIs.

Call restrictions

For each App, the SDK only supports creating one AliRtcEngine instance.

Parameter description

Response description

Returns a singleton instance of the AliEngine class.

Destroys an AliRtcEngine instance.

static void Destroy(AliEngineDestroyCompletionCallback *callback = nullptr);

Destroys the AliRtcEngine singleton object. After calling this method, all internally used resources will be released, and you will no longer be able to use other methods or any callbacks of AliRtcEngine. To use it again, you must call Create to create a new instance.

Call timing

It is recommended to call this method to release the instance after completing audio and video communication.

Call restrictions

To avoid deadlocks, it is not recommended to call this method in any SDK callbacks.

Parameter description

Queries a class instance.

virtual int QueryInterface(AliEngineInterfaceIdType iid, void** pInterface) = 0;

This method is used to obtain a specific type of feature interface instance. When developing audio and video related applications on Windows platform, many advanced features (such as audio device management AliEngineInterfaceAudioDeviceManger, video device management AliEngineInterfaceVideoDeviceManger, and media engine AliEngineInterfaceMediaEngine) are provided through corresponding interface classes. However, before calling related methods, you must obtain instances of these interfaces through the QueryInterface method.

Parameter description

Return description

• 0: The query is successful.

• <0: The query failed.

Specify whether to enable the HTML5 compatibility mode.

static void SetH5CompatibleMode(bool comp);

Parameter description

Queries whether the HTML5 compatibility mode is enabled.

static bool GetH5CompatibleMode();

Return description

A value of YES indicates that the HTML5 compatibility mode is enabled. A value of NO indicates that the HTML5 compatibility mode is disabled.

Sets the event listener.

virtual int SetEngineEventListener(AliEngineEventListener *listener) = 0;

This method sets an event listener for the SDK. By setting the listener, you can receive various notification events from the engine, such as when users join or leave a channel, the network status changes, or the audio and video stream status changes.

You must implement the `AliEngineEventListener` class to customize the event methods that you want to listen for. By default, all methods have empty implementations. You do not need to implement all methods. You can implement only the event methods that are relevant to your business.

Related callbacks

If an exception occurs during runtime, the SDK first attempts to automatically recover using its internal retry mechanism. For errors that the SDK cannot resolve on its own, the SDK notifies your application through predefined callback interfaces. The following table describes key callbacks for errors that the SDK cannot handle. Your application must listen for and respond to these callbacks.

Parameters

Response Description

• 0: Success.

• A non-zero value: The method call failed. This may be because the `listener` parameter is invalid or the SDK is not correctly initialized.

Query the current SDK version.

static const char *GetSDKVersion();

Return description

The current SDK version number in string format, for example: "2.5.0.x".

Set the channel mode.

virtual int SetChannelProfile(const AliEngineChannelProfile channelProfile) = 0;

This method is used to set the channel mode. The following modes are available:

• Video call mode: All users are streamers and can publish and subscribe to streams.

• Interactive live streaming mode: In this mode, you need to call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#c99c2b257dd8o" id="4e50f781a01e1">SetClientRole</a> to set the user role. Users who publish streams in the channel are set to the streamer role (AliRTCSdkInteractive). Users who only need to subscribe to streams and do not need to publish streams are set to the viewer role (AliRTCSdkLive). We recommend that you use this mode for RTC scenarios.

When to call

You can call this method only before you join a channel. You when you are in a channel. You can change the mode after you leave the channel.

Parameters

Response description

• 0: The method is called successfully.

• A value other than 0: The method call fails.

  • -1: The SDK is not initialized or has been destroyed.

Sets the audio profile.

virtual int SetAudioProfile(int audioProfile, int audioScene) = 0;

This method is used to set the audio encoding mode and audio scenario mode. For more information, see Setting audio encoding and scenario modes. By default, the ARTC SDK uses high-quality mode (AliEngineHighQualityMode) and music scenario mode (AliEngineSceneMusicMode). If the default settings do not meet your requirements, you need to call this method to configure them.

Call timing

This method can only be called before joining a channel. You cannot reconfigure it after joining a channel, but you can reconfigure it after leaving the channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Queries whether the audio-only mode is enabled.

virtual bool IsAudioOnlyMode() = 0;

Return description

A value of YES indicates that the audio-only mode is enabled. A value of NO indicates that the audio-video mode is enabled.

Enable the audio-only mode or the audio-video mode.

virtual int SetAudioOnlyMode(bool audioOnly) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Join a channel.

virtual int JoinChannel(const char *token, const char *channelId, const char *userId, const char *userName) = 0;

This method is used to join a channel. ARTC organizes users through channels, and users need to join a channel to "publish" or "subscribe" to audio and video streams. This method, along with JoinChannel[2/3] and JoinChannel[3/3], can be used to join a channel. The difference lies in the authentication method and the user information passed, as follows:

• This method is a single-parameter channel joining interface that requires only the Token generated by Token Authentication for single-parameter channel joining. This method is recommended for RTC scenarios.

• JoinChannel[2/3] is a multi-parameter channel joining interface that requires the multi-parameter Token generated by Token Authentication, and the user information used to generate the Token.

• JoinChannel[3/3] is used for AI real-time interaction scenarios, passing a single-parameter Token and setting the user attribute capabilityProfile according to the scenario.

Call timing

You need to call this method after creating the engine.

Call restrictions

• After successfully joining a channel, to join another channel, you must first call LeaveChannel to leave the current channel and ensure that you receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#b70203f9ea0mz" id="b2522e57e4vom">OnLeaveChannelResult</a> callback before calling this method again.

• This method only supports joining one channel at a time.

• Apps with different App IDs cannot communicate with each other.

• If joining a channel fails, you do not need to call this method again when retrying.

Related callbacks

After successfully calling this method, the following callbacks will be triggered:

• The result of the local client joining a channel is returned through the AliRtcEngineEventListener's <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#7cbf47a35bcel" id="cad6eb9b1bwx7">OnJoinChannelResult</a> callback.

• After successfully joining the channel, remote users will trigger the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#3333f6e9d344q" id="fab4a94264m28">OnRemoteUserOnLineNotify</a> callback.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Join a channel.

virtual int JoinChannel(const AliEngineAuthInfo &authInfo,
                                 const char *userName) = 0;

This method is used to join a channel. In ARTC, users are organized by channels. A user must join a channel to publish or subscribe to audio and video streams. This method, JoinChannel[1/3], and JoinChannel[3/3] can all be used to join a channel. They differ in the authentication method and the user information that is passed, as described in the following list:

• JoinChannel[1/3] is a single-parameter method. You can join a channel by passing a token that is generated for single-parameter authentication. For more information, see Token-based authentication. We recommend that you use this method to join a channel in RTC scenarios.

• This method is a multi-parameter method. You must pass a token that is generated for multi-parameter authentication and the user information that is used to generate the token. For more information, see Token-based authentication.

• JoinChannel[3/3] is used in AI-powered real-time interaction scenarios. You must pass a token for single-parameter authentication and set the capabilityProfile user property based on the scenario.

Usage limits

• After you successfully join a channel, to join another channel, you must first call LeaveChannel to leave the current channel. You can join the new channel only after you receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#b70203f9ea0mz" id="fcd2fac6abf0y">OnLeaveChannelResult</a> callback.

• A user can join only one channel at a time using this method.

• Applications that use different App IDs cannot interoperate with each other.

Related callbacks

After you call this method, the following callbacks are triggered:

• The result of the local client joining the channel is returned in the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#7cbf47a35bcel" id="b3db8c610ewua">OnJoinChannelResult</a> callback.

• After a user successfully joins the channel, the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#3333f6e9d344q" id="a1297d33bcd92">OnRemoteUserOnLineNotify</a> callback is triggered on the remote client.

Parameters

Join a channel.

virtual int JoinChannel(const char *token, const char *channelId, const char *userId, const AliEngineChannelParam &userParam) = 0;

This method is used to join a channel. ARTC organizes users through channels, and users need to join a channel to "publish" or "subscribe" to audio and video streams. This method, along with JoinChannel[1/3] and JoinChannel[2/3], can be used to join a channel. The difference lies in the authentication method and the user information passed, as follows:

• JoinChannel[1/3] is a single-parameter entry interface for RTC scenarios, which requires only the Token generated by Token Authentication for single-parameter entry. This interface is recommended for joining channels in RTC scenarios.

• JoinChannel[2/3] is a multi-parameter entry interface, which requires the Token generated by Token Authentication for multi-parameter entry, along with the user information used to generate the Token.

• This interface is used for AI real-time interaction scenarios, passing a single-parameter entry Token and setting the user attribute capabilityProfile according to the scenario. When communicating with an AI agent, set it to AliCapabilityProfileHuman.

Call restrictions

• After successfully joining a channel, to join another channel, you must first call leaveChannel to leave the current channel and ensure that you receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#b70203f9ea0mz" id="15ec544e3985m">OnLeaveChannelResult</a> callback before calling join channel again.

• This method only supports joining one channel at a time.

• Apps with different App IDs cannot communicate with each other.

• If joining a channel fails, there is no need to call this method again when retrying.

Related callbacks

After successfully calling this method, the following callbacks will be triggered:

• The result of the local user joining the channel will be notified through the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#7cbf47a35bcel" id="5d109191c950l">OnJoinChannelResult</a> callback.

• After successfully joining the channel, remote users will trigger the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#3333f6e9d344q" id="fa6f8d9357glx">OnRemoteUserOnLineNotify</a> callback.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call failed.

Leave a channel.

virtual int LeaveChannel() = 0;

After calling this method, the SDK terminates audio and video interaction and leaves the current channel.

Call timing

• You should call this method when you need to leave a channel after joining it.

• If you have already joined a channel and need to join another channel, you need to call this interface to leave the current channel first.

Related callbacks

• Local: After calling this method, the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#b70203f9ea0mz" id="9626ae24d89ue">OnLeaveChannelResult</a> callback will be triggered to notify the result of leaving the channel.

• Remote: After successfully calling this interface, remote users will trigger the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#44ce12a0ecj52" id="0dfcab6a6fylt">OnRemoteUserOffLineNotify</a> callback.

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Checks whether you are in a channel.

virtual bool IsInCall() = 0;

Return description

A value of true indicates that you are in a channel. A value of false indicates that you are not in a channel.

Specify the user role.

virtual int SetClientRole(const AliEngineClientRole clientRole) = 0;

This method is used to set the user role as a streamer or viewer.

In interactive mode, before joining a channel:

• When setting the user role as a streamer: The SDK automatically pushes local audio and video streams by default, and receives audio and video streams from other streamers.

• When setting the user role as a viewer: The SDK does not push local audio and video streams, but receives audio and video streams from other streamers.

Call timing

This method can be called both before and after joining a channel. You can set the user role before joining a meeting, or call it after joining to switch user roles.

Call restrictions

This method is only effective in interactive mode, which means it only works when the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#901e748242y3g" id="322c3da902jpt">SetChannelProfile</a> interface is set to AliEngineInteractiveLive.

In interactive mode, it is recommended to explicitly call this method to set the user role before joining a meeting.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Queries the current user role type (streamer or viewer).

virtual AliEngineClientRole GetClientRole() = 0;

Return description

After the method is called, an AliEngineClientRole object is returned. The object is an enumeration type that indicates the role type: streamer (0) or viewer (1).

Refresh the authentication information.

virtual int RefreshAuthInfo(const AliEngineAuthInfo &authInfo) = 0;

This method is used to update authentication information. The Token will expire after a certain period, at which point the SDK will not be able to establish a connection with the server.

Call timing

In the following situations:

• When you receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#9f8ebe177degw" id="f52a19bb78yoi">OnAuthInfoWillExpire</a> callback reporting that the authentication information is about to expire, it is recommended that you regenerate the Token on your server and then call this method to pass in the new Token.

• If the Token is not updated in time, the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#09c4ec9160eom" id="440c501877fxr">OnAuthInfoExpired</a> callback will be triggered to notify that the authentication has expired. In this case, you need to regenerate the Token and then call JoinChannel to rejoin the channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Refresh the authentication information.

virtual int RefreshAuthInfo(const char *token) = 0;

This method is used to update the Token. The Token will expire after a certain period, at which point the SDK will not be able to establish a connection with the server.

Call timing

In the following situations, we recommend that you regenerate a Token on your server and then call this method to pass in the new Token:

• When you receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#9f8ebe177degw" id="b21949148djrc">OnAuthInfoWillExpire</a> callback reporting that the authentication information is about to expire.

• If the Token is not updated in time, the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#09c4ec9160eom" id="62df9bf63786v">OnAuthInfoExpired</a> callback will be triggered to notify that the authentication has expired. In this case, you need to call JoinChannel to rejoin the channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Specifies whether to publish an audio track.

virtual int PublishLocalAudioStream(bool enabled) = 0;

This method is used to control whether to publish the locally captured audio stream. By default, the SDK publishes the audio stream. If you do not want to publish the audio stream by default, you can call PublishLocalAudioStream(false) before joining a channel to disable audio stream publishing.

Call timing

You can call this method before or after joining a channel. If you call this method before joining a channel, the default configuration is modified and takes effect when you join the channel.

Related callbacks

When the local audio stream publishing status changes, the local device triggers the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#eb8fab7f5ak7v" id="c584949653jn5">OnAudioPublishStateChanged</a> callback to notify the latest audio publishing status, and the remote device triggers the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#ff1f7c8581r3r" id="75f2096b1efxy">OnRemoteTrackAvailableNotify</a> callback to notify that the audio and video streams of the remote user have changed.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Query whether an audio track is published.

virtual bool IsLocalAudioStreamPublished() = 0;

Return description

If set to true, pushing is allowed. If set to false, pushing is prevented.

Specify whether to subscribe to the audio tracks of remote users.

virtual int SetDefaultSubscribeAllRemoteAudioStreams(bool sub) = 0;

This method is used to configure whether the system subscribes to the audio tracks of remote users by default. This setting affects the audio track subscription behavior for users who join the channel later. We recommend that you set this parameter to true unless you have special requirements.

Call timing

You can call this method before or after you join a channel.

• Before joining a channel:

  • By default, the SDK subscribes to the audio tracks of remote users when joining a channel. If you want to modify this behavior, you can call this method before joining the channel.

• After joining a channel:

  • If you want to stop the default subscription, you can call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#e51a461de2k9t" id="e95a7e2d6fjwt">SetDefaultSubscribeAllRemoteAudioStreams</a>(false), which prevents subscribing to the audio tracks of users who join the channel later.

  • After stopping the default subscription, if you want to resume subscribing to the audio track of a specific user, call the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#957e547813q56" id="1a391bcabc66e">SubscribeRemoteAudioStream</a> method. If you want to resume subscribing to multiple users, call this method multiple times.

  • After stopping the default subscription, calling <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#e51a461de2k9t" id="e79b5ad58awcw">SetDefaultSubscribeAllRemoteAudioStreams</a>(true) resumes the audio streams of users who join the channel afterward. However, it does not subscribe to the audio streams of remote users who joined while the subscription was stopped.

Call restrictions

If you have previously called <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#f1e98cf7b446c" id="c9b658a17evjt">SubscribeAllRemoteAudioStreams</a>(false) to turn off the master switch for audio subscription, this method call will be ineffective.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Subscribe to or stop subscribing to the audio tracks of all remote users.

virtual int SubscribeAllRemoteAudioStreams(bool sub) = 0;

This interface serves as the master switch for subscribing to remote audio tracks, and we recommend setting it to true. If set to false, it will result in:

• All remote audio in the current meeting will stop being subscribed to.

• New users who join later will not be subscribed to.

• You cannot individually control the audio stream of specific users through <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#957e547813q56" id="aface4ad3920l">SubscribeRemoteAudioStream</a>.

To resume subscribing, call this interface again and set it to true.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Subscribe to or stop subscribing to the audio track of a specific remote user.

virtual int SubscribeRemoteAudioStream(const char* uid, bool sub) = 0;

This API lets you stop or resume a subscription to the audio stream of a specific remote user. <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#f1e98cf7b446c" id="93db357f06pmq">SubscribeAllRemoteAudioStreams</a> provides global control, while SubscribeRemoteAudioStream provides fine-grained, individual control. We recommend that you set this parameter to true unless you have specific scenario requirements.

Call restrictions

If you have previously called <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#f1e98cf7b446c" id="1595cb66e5ty2">SubscribeAllRemoteAudioStreams</a>(false) to stop subscribing to all remote audio tracks, this method call will be ineffective.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Publish/stop publishing camera stream.

virtual int PublishLocalVideoStream(bool enabled) = 0;

This method is used to control whether to publish the locally captured video stream.

Call timing

You can call this method before or after joining a channel.

Calling before joining a channel modifies the default configuration, which takes effect when joining the channel.

Related callbacks

When the local video stream publishing status changes, the local end will trigger the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#6bf4efbd1cadl" id="d37709b09147d">OnVideoPublishStateChanged</a> callback to notify the latest audio publishing status, and the remote end will trigger the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#ff1f7c8581r3r" id="f16e4b51bchqb">OnRemoteTrackAvailableNotify</a> callback to notify changes in the remote user's audio and video streams.

Parameter description

Return description

• 0: The method call is successful.

• <0: The method call fails.

Queries whether a video track is published.

virtual bool IsLocalVideoStreamPublished() = 0;

Return description

A value of true indicates that stream ingest is enabled for the video track. A value of false indicates that stream ingest is disabled for the video track.

Specifies whether to subscribe to the video tracks of remote users.

virtual int SetDefaultSubscribeAllRemoteVideoStreams(bool sub) = 0;

This method is used to specify whether to subscribe to video tracks. By default, the SDK subscribes to video tracks.

Call timing

You can call this method before or after joining a channel.

• Before joining a channel:

  • You can use this method to cancel the default subscription setting.

• After joining a channel:

  • If you want to stop the default subscription, you can call SetDefaultSubscribeAllRemoteVideoStreams(false), which will prevent subscribing to the audio tracks of users who subsequently join the channel.

  • After stopping the default subscription, if you want to resume subscribing to the audio track of a specific user, call the SubscribeRemoteVideoStream method. If you want to resume subscribing to multiple users, call this method multiple times.

  • After stopping the default subscription, calling SetDefaultSubscribeAllRemoteVideoStreams(false) only resumes subscribing to the audio tracks of users who subsequently join the channel.

Call restrictions

If you have previously called <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#4f427cc2d9cjf" id="b9cacc3b8an8e">SubscribeAllRemoteVideoStreams</a>(false) to turn off the master switch for video subscription, this method call will be ineffective.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Subscribe to or stop subscribing to the video tracks of all remote users.

virtual int SubscribeAllRemoteVideoStreams(bool sub) = 0;

This interface serves as a master switch for subscribing to remote video streams. If set to false, it will result in:

• All remote video streams in the current meeting will stop being subscribed to.

• New users who join later will not be subscribed to (even if you have set <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#2cbc54790brn3" id="b7b0080f3cnoq">SetDefaultSubscribeAllRemoteVideoStreams</a>(true) for default subscription.

• You cannot control the audio stream of a specific user individually through <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#jyGTp" id="0e2edf97aa1g6">SubscribeRemoteVideoStream</a>.

To resume subscription, call this interface again and set it to true.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Subscribes to or stops subscribing to the media stream of a specific remote user.

virtual int SubscribeRemoteMediaStream(const char* uid, AliEngineVideoTrack videoTrack, bool subVideo,  bool subAudio) = 0;

This interface is used to combine subscription to remote audio and video streams.

Related interfaces

Compared to SubscribeRemoteMediaStream[2/2], this interface uses two boolean parameters subVideo and subAudio to determine whether to subscribe to remote audio and video streams, while videoTrack is used to control which video stream to pull.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Subscribe to or stop subscribing to the media stream of a specific remote user.

virtual int SubscribeRemoteMediaStream(const char* uid, AliEngineVideoTrack videoTrack, AliEngineAudioTrack audioTrack) = 0;

This interface is used to combine subscription to remote audio and video streams.

Related interfaces

Compared to SubscribeRemoteMediaStream[1/2], this interface uses two parameters, videoTrack and audioTrack, to inform the SDK about the subscription status in a single call, for example:

• To subscribe to both camera stream and microphone stream, set videoTrack to AliRtcVideoTrackCamera and audioTrack to AliRtcAudioTrackMic when calling.

• To unsubscribe from the camera stream but keep the microphone stream, set videoTrack to AliRtcVideoTrackNo and audioTrack to AliRtcAudioTrackMic when calling again.

• To unsubscribe from both streams, set videoTrack to AliRtcVideoTrackNo and audioTrack to AliRtcAudioTrackNo when calling again.

• To subscribe to both camera stream and screen sharing stream simultaneously, set videoTrack to AliRtcVideoTrackBoth. The same applies to audio.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call failed.

Subscribe to or stop subscribing to the media stream of a specific remote user across channels.

virtual int SubscribeRemoteDestChannelStream(const char* channelId, const char* uid, AliEngineVideoTrack track, bool sub_audio, bool sub) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop or resume subscribing to the video stream of a remote user.

virtual int SubscribeRemoteVideoStream(const char* uid, AliEngineVideoTrack track, bool sub) = 0;

Subscribe to or unsubscribe from the video stream of a specified user.

Call timing

You can call this method before or after joining a channel.

Call restrictions

The subscription behavior is controlled by the master switch SubscribeAllRemoteVideoStreams. If you have previously called SubscribeAllRemoteVideoStreams(false) to disable subscription to all remote videos, the subscription function will be turned off, and all other enabling behaviors will not take effect. Before calling this API, make sure you have called SubscribeAllRemoteVideoStreams(true) to re-enable the subscription function.

Parameter description

Return description

• true: The setting is successful.

• false: The setting failed.

Sets the local playback volume for the audio track of a specific remote user.

virtual int SetRemoteAudioVolume(const char *uid, int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed.

Specify whether to mute the local audio.

virtual int MuteLocalMic(bool mute, AliEngineMuteLocalAudioMode mode = AliEngineMuteLocalAudioModeDefault) = 0;

Call timing

You can call this method before or after joining a channel.

Related callbacks

After a successful call, remote users will receive the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#a15f0a6cf6id7" id="f4ae88f094vqy">OnUserAudioMuted</a> notification indicating whether the user is muted.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Specify whether to stop the playback of the audio track of a specific remote user.

virtual int MuteRemoteAudio(const char *uid,
                                    bool mute) = 0;

This interface is only used to stop or resume audio playback for a specified remote user. It does not affect the pulling or decoding of the remote audio stream. To unsubscribe from an audio stream, call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#957e547813q56" id="89841a0989t7y">SubscribeRemoteAudioStream</a> for a specific user, or call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#f1e98cf7b446c" id="37ccd4be636nf">SubscribeAllRemoteAudioStreams</a> for all users.

Call timing

You can call this method before or after joining a channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Specifies whether to stop the playback of the audio tracks of all remote users.

virtual int MuteAllRemoteAudio(bool mute) = 0;

This method is used to stop or resume the playback of all remote audio tracks.

Call timing

You can call this method before or after joining a channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Start audio collection.

virtual int StartAudioCapture() = 0;

This interface is used to control the start of audio collection. When called before joining a channel, you can control the audio collection device to open in advance. If not set, the SDK will automatically control the audio collection device. After calling stopAudioCapture to close audio collection, call this interface if you want to open it again.

Call timing

You can call this method before or after joining a channel.

Related interfaces

• The StartAudioCapture[2/2] interface lets you control whether the audio collection device remains on after leaving a channel through parameters.

Related callbacks

After calling this interface to modify the local audio collection status, you can get status changes through the OnLocalAudioStateChange callback.

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Start audio collection.

virtual int StartAudioCapture(bool keepAlive) = 0;

Close microphone collection after muting.

This interface is used to control audio collection. When called before joining a channel, it can control audio collection in advance. If not set, the SDK will automatically control the audio collection device.

Call timing

You can call this method before or after joining a channel.

Related interfaces

Compared to StartAudioCapture[1/2], this interface lets you control whether the collection device remains on after leaving a channel through the keepAlive parameter.

Related callbacks

After calling this interface to modify the local audio collection status, you can obtain status changes through the OnLocalAudioStateChange callback.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop audio collection.

virtual int StopAudioCapture() = 0;

After calling StartAudioCapture to start audio device collection, you can call this method to stop collection.

Related callbacks

After calling this method to modify the local audio collection status, you can obtain the status change through the OnLocalAudioStateChange callback.

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume callback interval and smoothing factor.

virtual int EnableAudioVolumeIndication(int interval, int smooth, int reportVad) = 0;

This method allows the SDK to regularly report volume-related information to the App about the local streaming user and the remote user with the highest instantaneous volume.

Call timing

You can call this method before or after joining a channel.

Related callbacks

After successfully calling this method, if there are users streaming in the channel, the SDK will trigger the following two callbacks according to the set time interval:

• The audio volume of the speaker will be notified through the OnAudioVolumeCallback callback, with the frequency determined by the interval parameter.

• For voice activation, when an active user is detected, the speaker's uid will be reported through the OnActiveSpeaker callback.

Call timing

You can call this method before or after joining a channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Starts the audio player.

virtual int StartAudioPlayer() = 0;

This interface can control opening the audio player in advance. If not set, the SDK will automatically open the audio player after subscribing to an audio stream.

Return description

• 0: The operation is successful.

• Non-0: The operation failed, returning an error code.

Stops the audio player.

virtual int StopAudioPlayer() = 0;

This interface can be used to control the shutdown of audio playback, corresponding to AliEngine::StartAudioPlayer.

Parameter description

None.

Return description

• 0: The operation is successful.

• Non-0: The operation failed, returning an error code.

Sets the playback volume.

virtual int SetPlayoutVolume(int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Other values: The operation failed and an error code is returned.

virtual int SetRecordingVolume(int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed, and an error code is returned.

Play an audio file.

virtual int PlayAudioFileTest(const char* filePath) = 0;

Parameter description

Return description

• 0: The operation is successful.

• <0: The operation failed.

Stop playing the audio file.

virtual int StopAudioFileTest() = 0;

Return description

• 0: The operation is successful.

• <0: The operation failed.

Start audio capture test before a call.

virtual int StartAudioCaptureTest() = 0;

This interface can control the opening of audio capture testing. You can determine whether audio capture is working properly through the AliEngineEventListener::OnTestAudioVolumeCallback callback result.

Return description

• 0: The operation is successful.

• <0: The operation failed.

Stops the audio capture test.

virtual int StopAudioCaptureTest() = 0;

Return description

• 0: The operation is successful.

• <0: The operation failed.

Enable/disable system audio recording and streaming.

virtual int EnableSystemAudioRecording(bool enable, const char *path = nullptr, const char *device_id = nullptr) = 0;

This interface is used to enable or disable system audio recording, such as sounds played by browsers, applications, etc.

Parameter description

Return description

• 0: The method call is successful.

• <0: The method call fails.

Queries whether system audio collection and pushing are enabled.

virtual bool IsSystemAudioRecording() = 0;

Return description

• true: enabling status.

• false: shutdown status.

Sets the volume of system audio that is collected and pushed.

virtual int SetSystemAudioRecordingVolume(int volume) = 0;

This interface is only available on Windows and macOS platforms.

This interface requires system audio collection and pushing to be enabled before it can be set, otherwise the setting will be ineffective.

Parameter description

Return description

• 0: The operation is successful.

• <0: The operation failed.

Queries the current volume of system audio that is collected and pushed.

virtual int GetSystemAudioRecordingVolume() = 0;

This interface is available only on Windows and macOS platforms.

Response description

The playback volume.

Sets the system audio playback volume.

virtual int SetSystemAudioPlayoutVolume(int volume) = 0;

This interface is only available on Windows and macOS platforms.

This interface requires system audio capture and pushing to be enabled before it can be set, otherwise the setting will not take effect.

Parameter description

Return description

• 0: The operation is successful.

• <0: The operation fails.

Gets the current system audio playback volume.

virtual int GetSystemAudioPlayoutVolume() = 0;

This interface is only available on Windows and macOS platforms.

Return description

The system audio playback volume.

Gets the list of audio capture devices in the system.

virtual AliEngineDeviceInfoList* GetAudioCaptureList() = 0;

Return value

AliEngineDeviceInfoList * Pointer to the list of audio capture devices.

Gets the name of the audio capture device in use.

virtual String GetCurrentAudioCaptureName() = 0;

Return value

String: The name of the current audio capture device.

Gets the ID of the audio capture device in use.

virtual String GetCurrentAudioCaptureID() = 0;

Return value

The ID of the audio capture device currently in use.

Select the audio capture device name.

virtual int SetCurrentAudioCaptureName(const char* captureName) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Select the audio recording device ID.

virtual int SetCurrentAudioCaptureID(const char* captureID) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Gets the list of speakers in the system.

virtual AliEngineDeviceInfoList* GetAudioPlayerList() = 0;

Return value

A pointer to the list of speakers in the system AliEngineDeviceInfoList *

Get the list of recordable speakers in the system (for system audio recording).

virtual AliEngineDeviceInfoList* GetSystemRecordAudioPlayerList() = 0;

Return value

The audio playback device list is mainly provided for EnableSystemAudioRecording to use as the last parameter, for capturing system audio playback.

Gets the name of the currently used speaker.

virtual String GetCurrentAudioPlayerName() = 0;

Return value

The name of the current audio playback device

Gets the ID of the currently used speaker.

virtual String GetCurrentAudioPlayerID() = 0;

Return value

The ID of the current audio playback device.

Select the speaker name.

virtual int SetCurrentAudioPlayerName(const char* playerName) = 0;

Return value

The name of the audio playback device.

Select speaker ID

virtual int SetCurrentAudioPlayerID(const char* playerID) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Sets the volume of the audio capture device. The volume range is [0, 100].

virtual int SetRecordingDeviceVolume(int volume) = 0;

Parameter list

Return value

0 indicates success, other values indicate failure.

Gets the volume of the audio recording device.

virtual int GetRecordingDeviceVolume() = 0;

Return value

>=0 returns the volume, other values indicate failure.

Sets the volume of the audio playback device. The volume range is [0, 100].

virtual int SetPlaybackDeviceVolume(int volume) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Gets the volume of the audio playback device.

virtual int GetPlaybackDeviceVolume() = 0;

Return value

>=0 indicates the system volume, other values indicate failure.

Specifies whether to mute an audio collection device.

virtual int SetRecordingDeviceMute(bool mute) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Gets the mute status of the audio recording device

virtual bool GetRecordingDeviceMute() = 0;

Return value

Whether the recording is muted.

Mute the audio playback device

virtual int SetPlaybackDeviceMute(bool mute) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Queries whether an audio player is muted.

virtual bool GetPlaybackDeviceMute() = 0;

Return description

Indicates whether the playback device is muted.

Set the voice change mode.

virtual int SetAudioEffectVoiceChangerMode(const AliEngineAudioEffectVoiceChangerMode &mode) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the audio pitch.

virtual int SetAudioEffectPitchValue(double value) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the reverberation mode.

virtual int SetAudioEffectReverbMode(const AliEngineAudioEffectReverbMode& mode) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the parameters of the reverberation mode.

virtual int SetAudioEffectReverbParamType(const AliEngineAudioEffectReverbParamType& type,
                                                  float value) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Add an external audio track.

virtual int AddExternalAudioStream(const AliEngineExternalAudioStreamConfig& config) = 0;

This method is used to add a new external audio track. The following are the related steps:

1. Call the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#991b862eb353w" id="08ee3d1ed35jj">AddExternalAudioStream</a> method to add an external audio track and obtain the external audio track ID.

2. Call the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#e6cf3f7cef0g6" id="fae817d0f00p3">PushExternalAudioStreamRawData</a> method to input audio data into the created audio track.

3. When the call ends, you need to call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#u27oe" id="3b9a3eb119umh">RemoveExternalAudioStream</a> to remove the external audio track.

To publish custom captured audio in a channel, you can refer to the document Custom Audio Capture.

Parameter description

Return description

A value greater than 0 indicates that the call is successful and is the ID of the external audio track. A value less than or equal to 0 indicates that the call fails.

Import external audio data.

virtual int PushExternalAudioStreamRawData(int streamId, AliEngineAudioRawData& data) = 0;

This interface is used to input data into a specified audio stream. The following are the related steps:

1. Call the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#991b862eb353w" id="3836ac109732c">AddExternalAudioStream</a> interface to add an external audio stream and obtain the external audio stream ID.

2. Call PushExternalAudioStreamRawData to input audio data into the created audio stream.

3. When ending the call, you need to call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#u27oe" id="398ea88d61fn0">RemoveExternalAudioStream</a> to remove the external audio stream.

To publish custom captured audio in a channel, you can refer to the document Custom audio capture.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of external audio for stream ingest.

virtual int SetExternalAudioStreamPublishVolume(int streamId, int vol) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of external audio for stream ingest.

virtual int SetExternalAudioStreamPlayoutVolume(int streamId, int vol) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Queries the volume of external audio for stream ingest.

virtual int GetExternalAudioStreamPublishVolume(int streamId) = 0;

Parameter description

Return description

The valid values of the volume range from 0 to 100. A value less than 0 indicates that the call fails.

Query the playback volume of external audio.

virtual int GetExternalAudioStreamPlayoutVolume(int streamId) = 0;

Parameter description

Return description

The valid values of the volume range from 0 to 100. A value less than 0 indicates that the call fails.

Removes an external audio stream.

virtual int RemoveExternalAudioStream(int streamId) = 0;

This method removes the external audio stream that corresponds to the specified streamId. This method is paired with the AddExternalAudioStream method.

When to call

If you want to use the custom audio input feature, you must call the AddExternalAudioStream method to add an audio stream and obtain the external audio stream ID. Then, call the <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#e6cf3f7cef0g6" id="bdab74f108i1p">PushExternalAudioStreamRawData</a> method to push your audio data to the SDK. When you want to stop using custom audio input, call this method to remove the corresponding external audio stream and release the resources.

Parameters

Return value

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Gets audio file information.

virtual int GetAudioFileInfo(const char *filePath) = 0;

This is an asynchronous interface. You can obtain audio file information through AliEngineEventListener::OnAudioFileInfo.

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed, and an error code is returned.

Starts accompaniment mixing.

virtual int StartAudioAccompany(const char *filePath, const AliEngineAudioAccompanyConfig& config) = 0;

This is an asynchronous interface. You can monitor the audio accompaniment player status through AliEngineEventListener::OnAudioAccompanyStateChanged.

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed and returned an error code.

Stops accompaniment mixing.

virtual int StopAudioAccompany() = 0;

Return description

• 0: The operation is successful.

• Non-0: The operation failed, returning an error code.

Sets the accompaniment volume.

 virtual int SetAudioAccompanyVolume(int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Other values: The operation failed and an error code is returned.

Sets the volume of audio accompaniment for stream ingest.

virtual int SetAudioAccompanyPublishVolume(int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed and returned an error code.

Queries the volume of audio accompaniment for stream ingest.

virtual int GetAudioAccompanyPublishVolume() = 0;

Return description

The volume of audio accompaniment for stream ingest. Valid values: [0-100].

Sets the local playback volume of the audio accompaniment.

virtual int SetAudioAccompanyPlayoutVolume(int volume) = 0;

Parameter description

Return description

• 0: The operation is successful.

• Non-0: The operation failed and an error code is returned.

Queries the local playback volume of the accompaniment.

virtual int GetAudioAccompanyPlayoutVolume() = 0;

Return description

The local playback volume of the accompaniment. Valid values: [0-100].

Pauses accompaniment mixing.

virtual int PauseAudioAccompany() = 0;

Return description

• 0: The operation is successful.

• Non-0: The operation failed and returns an error code.

Resumes accompaniment mixing.

virtual int ResumeAudioAccompany() = 0;

Return description

• 0: The operation is successful.

• Non-0: The operation failed, and an error code is returned.

Gets the duration of the accompaniment file in milliseconds.

virtual int GetAudioAccompanyDuration() = 0;

Return description

The duration of the audio accompaniment file in milliseconds.

Gets the playback progress of the accompaniment file in milliseconds.

virtual int GetAudioAccompanyCurrentPosition() = 0;

Return description

The current playback position of the audio accompaniment file in milliseconds.

Sets the playback position of the accompaniment file.

virtual int SetAudioAccompanyPosition(int pos) = 0;

Return description

• 0: The operation is successful.

• Non-0: The operation failed and an error code is returned.

Preload a sound effect.

virtual int PreloadAudioEffect(unsigned int soundId,
                                       const char *filePath) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Delete a preloaded sound effect.

virtual int UnloadAudioEffect(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Start the playback of a sound effect.

virtual int PlayAudioEffect(unsigned int soundId, const char *filePath, const AliEngineAudioEffectConfig& config) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop the playback of a sound effect.

virtual int StopAudioEffect(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop the playback of all sound effects.

- (int)StopAllAudioEffects;

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Pause the playback of a sound effect.

virtual int PauseAudioEffect(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Pause the playback of all sound effects.

virtual int PauseAllAudioEffects() = 0;

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Resume the playback of a sound effect.

virtual int ResumeAudioEffect(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Resume the playback of all sound effects.

virtual int ResumeAllAudioEffects() = 0;

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of a sound effect for stream ingest.

virtual int SetAudioEffectPublishVolume(unsigned int soundId, int volume) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Queries the volume of a sound effect for stream ingest.

virtual int GetAudioEffectPublishVolume(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of all sound effects for local playback.

virtual int SetAllAudioEffectsPublishVolume(int volume) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of a sound effect for local playback.

virtual int SetAudioEffectPlayoutVolume(unsigned int soundId, int volume) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Query the volume of a sound effect for local playback.

virtual int GetAudioEffectPlayoutVolume(unsigned int soundId) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of all sound effects for local playback.

virtual int SetAllAudioEffectsPlayoutVolume(int volume) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the volume of all sound effects for stream ingest.

virtual int SetAllAudioEffectsPublishVolume(int volume) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the rendering view and drawing parameters for local preview.

virtual int SetLocalViewConfig(AliEngineVideoCanvas renderConfig,
                                         AliEngineVideoTrack track) = 0;

This method is used to set up the local preview view. Calling this method binds the display window (view) of the local video stream and sets the rendering mode, mirror mode, rotation angle, and other parameters for the local user's view. It only affects the local user's preview and does not affect the streamed video. To set up the view for a remote user, call SetRemoteViewConfig.

Call timing

This method can be called both before and after joining a channel.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the capture preferences of the camera.

virtual int SetCameraCapturerConfiguration(const AliEngineCameraCapturerConfiguration& config) = 0;

This interface is used to configure camera capture preferences, such as camera orientation, capture frame rate, etc.

Call timing

Must be set before turning on the camera, for example, call before the following operations:

• StartPreview (start preview)

• JoinChannel (join channel)

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Disable or re-enable local video collection.

virtual int EnableLocalVideo(bool enabled) = 0;

This method is used to control the enabling and disabling of local video collection. When local video collection is disabled, there is no video data for local preview and streaming, but it does not affect receiving remote video. If you call this interface to turn off local camera collection, both local preview and remote streaming will remain on the last frame.

Call timing

This method can be called both before and after joining a channel.

Related callbacks

After successfully calling this interface, remote users will be notified through the OnUserVideoEnabled callback.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Specify whether to stop publishing a local video track.

virtual int MuteLocalCamera(bool mute, AliEngineVideoTrack track) = 0;

When streaming, you can call this interface to push black video frames while maintaining normal local preview. The collection, encoding, and sending modules continue to work, but the video content consists of black frames.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Set the rendering view and drawing parameters for the video track of a remote user.

virtual int SetRemoteViewConfig(AliEngineVideoCanvas renderConfig,
                                          const char *uid,
                                          AliEngineVideoTrack track) = 0;

This method binds the display view for a specified video stream of a remote user and sets the rendering mode, mirror mode, rotation angle, and other parameters for displaying the remote user's view locally. It only affects the video image seen by the local user. To set up a local preview view, call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#fbc8ab10f5e03" id="c0f59b8c26y7o">SetLocalViewConfig</a>.

• If the view parameter in AliEngineVideoCanvas is empty, rendering stops.

• To reset the renderMode parameter of AliEngineVideoCanvas during playback, keep other parameters unchanged and only modify renderMode.

• To reset the mirrorMode parameter of AliEngineVideoCanvas during playback, keep other parameters unchanged and only modify mirrorMode.

Call timing

We recommend that you call this method when you receive the OnRemoteTrackAvailableNotify callback, which indicates that the remote user's video is available.

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Queries whether the camera is turned on.

virtual bool IsCameraOn() = 0;

Return description

A value of YES indicates that the camera is turned on. A value of NO indicates that the camera is turned off.

Set video encoding properties.

virtual void SetVideoEncoderConfiguration(const AliEngineVideoEncoderConfiguration& config) = 0;

This method is used to set video parameters corresponding to video stream encoding properties, such as resolution, frame rate, bitrate, video orientation, etc. It is recommended to call this interface for all video scenarios.

Call timing

This method can be called both before and after joining a channel. If you only need to set the camera stream video encoding properties once per channel entry, it is recommended to call this before joining the channel.

Call restrictions

Both the mirrorMode parameter in this method and setVideoMirrorMode can set video stream mirroring. It is recommended to use only one method, as using multiple methods simultaneously may cause mirroring effects to stack, resulting in mirror setting failures or errors.

Parameter description

Sets video decoding properties. This method is used to set camera stream video decoding properties, including hardware/software decoding settings, whether to enable B frames, etc.

virtual void SetVideoDecoderConfiguration(const AliEngineVideoDecoderConfiguration& config) = 0;

Call timing

It is recommended to call this method before joining a channel and pulling streams.

Parameter description

Start a local preview. The camera is automatically turned on.

virtual int StartPreview(int width = 0, int height = 0) = 0;

This method is used to start a local video preview and automatically turn on the camera. To stop the local preview, you can call the StopPreview method.

Call timing

• Before calling this method, you need to set a view for local preview through SetLocalViewConfig, otherwise you cannot preview, but stream ingest is not affected.

• If needed, you can call this method before JoinChannel to start the preview, which will automatically turn on the camera.

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop a local preview.

virtual int StopPreview() = 0;

This method is used to close the local video preview and shut down the camera. After stopping the preview, the local end will remain on the last frame without affecting the stream ingest.

Call timing

You can call this method to stop the preview after it has been started.

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Sets the preview and stream ingest mirroring capability.

virtual int setVideoMirrorMode(AliEngineVideoPipelineMirrorMode mirrorMode) = 0;

Sets whether to enable mirror mode for local preview video and pushed video streams.

Call timing

This interface can be dynamically set both before and after joining a channel. The SDK internally records the state and operates on the video when it can handle preview and encoding (stream ingest).

Call restrictions

This interface overlaps with the mirrorMode parameter in SetLocalViewConfig&SetVideoEncoderConfiguration. We recommend that you only use this method.

Parameter description

Return description

• 0: The setting is successful.

• <0: The setting failed.

  • AliRtcErrInner: SDK internal state error. Check whether the SDK instance was created successfully.

Sets the capture scaling timing, determining whether video data is scaled immediately during capture or only during encoding.

virtual void SetCapturePipelineScaleMode(const AliEngineCapturePipelineScaleMode mode) = 0;

Sets the timing for video data capture scaling, determining whether to scale immediately during capture or only during encoding. For example, when the capture resolution differs from the encoding resolution, you can decide whether preview data and streaming data are consistent by setting the scaling timing.

Call timing

This method needs to be set before opening the camera, for example, before calling StartPreview to start preview or JoinChannel to join a channel.

Parameter description

Gets the camera list.

virtual AliEngineDeviceInfoList* GetCameraList() = 0;

Return value

A pointer to the camera list object AliEngineDeviceInfoList*.

Gets the name of the currently used camera.

virtual String GetCurrentCameraName() = 0;

Return value

The name of the currently used camera device.

Gets the ID of the current camera

virtual String GetCurrentCameraID() = 0;

Return value

The ID of the current camera.

Select camera name.

virtual int SetCurrentCameraName(const char* cameraName) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Select a camera ID.

virtual int SetCurrentCameraID(const char* cameraID) = 0;

Parameter description

Return value

0 indicates success, other values indicate failure.

Gets the supported resolutions.

virtual AliEngineVideoResolutionList * GetCurrentCameraSupportedResolutionList(int source) = 0 ;

Parameter description

Return value

!=NULL indicates that the list of supported resolutions is obtained successfully. NULL indicates that the operation fails.

Specifies whether to enable an external video source.

virtual int SetExternalVideoSource(bool enable,
        AliEngineVideoTrack type,
        AliEngineRenderMode renderMode) = 0;

Parameter description

Import external video data.

virtual int PushExternalVideoFrame(const AliEngineVideoRawData &frame,
        AliEngineVideoTrack type) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Starts relayed live streaming.

virtual int StartPublishLiveStream(const String& streamURL, const AliEngineLiveTranscodingParam &transcoding) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Update the parameters for relayed live streaming.

virtual int UpdatePublishLiveStream(const String& streamURL, const AliEngineLiveTranscodingParam &transcoding) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Stop relayed live streaming.

virtual int StopPublishLiveStream(const String& streamURL) = 0;

Parameter description

Return description

A value of 0 indicates that the call is successful. A value other than 0 indicates that the call fails.

Query the status of relayed live streaming.

virtual AliEngineLiveTranscodingState GetPublishLiveStreamState(const String& streamURL) = 0;

Parameter description

Return description

The status of relayed live streaming is returned.

Starts network quality monitoring. Some local network issues may cause audio and video calls to fail. By calling this method, you can obtain information about the uplink and downlink networks, such as bandwidth, packet loss rate, jitter, and link RTT, which helps locate and resolve related network issues.

 virtual int StartLastmileDetect(bool uplink, bool downlink, int uplinkBandWidth, int downlinkBandWidth) = 0;

Call timing

You must call this method before you call JoinChannel. The detection results are returned in callbacks. A network detection process may take a long time. You can call StopLastmileDetect to stop the network detection process as required.

Call restrictions

The SDK can only execute one network detection at a time. Repeated calls before the previous detection ends will be ineffective.

Related callbacks

After successfully calling this method, two callbacks will be triggered:

• OnLastmileDetectResultWithQuality: This callback is triggered after about 3 seconds to roughly determine the network quality level.

• OnLastmileDetectResultWithBandWidth: This callback is triggered after about 30 seconds to provide detailed detection results.

Parameter description

Return description

• 0: The method call is successful.

• <0: The method call fails, for example, if you have already joined a channel.

Stops network quality testing. You can call this method to stop network quality testing.

virtual int StopLastmileDetect() = 0;

Call timing

You need to call this method after creating the engine and before joining a channel.

Return description

• 0: The call to this method is successful.

• <0: The call to this method fails.

Sends media extension information, implemented internally using SEI.

virtual int SendMediaExtensionMsg(const int8_t * message, uint32_t length, int32_t repeatCount, uint32_t delay, bool isKeyFrame) = 0;

The SDK provides functionality to send and receive media extension information. For the receiver side, refer to AliEngineEventListener::OnMediaExtensionMsgReceived. You need to reuse audio and video data tunnels when you use SEI. Therefore, you must control the frequency at which custom messages are sent and the length of the messages. Take note of the following points:

• You can send a maximum of fps (set in the profile) messages per second.

• To avoid affecting the transmission quality of media data, the custom message body is limited to 4K bytes, which can be used to transmit small amounts of data.

• The repeatCount parameter in the sendMediaExtensionMsg function represents the redundancy of custom messages. If it is greater than 1, the message will be sent multiple times to prevent message loss due to network packet loss. In this case, others in the channel will also receive the same message multiple times and need to remove duplicates.

• For custom messages that are sent, subscribers in the channel will also receive them during bypass live streaming. Setting to -1 will permanently send the data until sendMediaExtensionMsg is set again.

• Only one sendMediaExtensionMsg will be sent at a time, meaning that calling sendMediaExtensionMsg will overwrite the previous call to sendMediaExtensionMsg if the previous call has not been sent or completed.

Parameter description

Return description

• 0: The operation is successful.

• <0: The operation failed, and an error code is returned.

  • ERR_INNER(-1): Internal SDK error, which may occur if the SDK is not initialized or if the method is called after the SDK is destroyed.

Sends media extension information, implemented internally using SEI.

virtual int SendMediaExtensionMsgEx(const int8_t * message, uint32_t length, int32_t repeatCount, uint32_t delay, bool isKeyFrame, int32_t payloadType) = 0;

The SDK provides functionality for sending and receiving media extension information. For the receiver side, refer to AliEngineEventListener::OnMediaExtensionMsgReceived. You need to reuse audio and video data tunnels when you use SEI. Therefore, you must control the frequency at which custom messages are sent and the length of the messages. Take note of the following points:

• You can send a maximum of fps (set in the profile) messages per second.

• To avoid affecting the transmission quality of media data, the custom message body is limited to 4K bytes, which can be used to transmit small amounts of data.

• The repeatCount parameter in the sendMediaExtensionMsg function represents the redundancy of custom messages. If the value is greater than 1, the message will be sent multiple times to prevent message loss due to network packet loss. In this case, other users in the channel will also receive the same message multiple times, requiring deduplication.

• For custom messages that are sent, subscribers in the channel will also receive them during bypass live streaming. Setting to -1 means permanently sending data until sendMediaExtensionMsg is set again.

• Only one sendMediaExtensionMsg will be sent at a time, meaning that calling sendMediaExtensionMsg will overwrite the previous call if the previous call has not been sent or completed.

Parameter description

Return description

• 0: The operation is successful.

• <0: The operation failed, and an error code is returned.

  • ERR_INNER(-1): Internal SDK error, possibly because the SDK is not initialized or the method is called after the SDK is destroyed.

The callback that is invoked when the network connection status changes.

virtual void OnConnectionStatusChange(int status, int reason) {};

Parameter description

The callback that is invoked when a local device exception occurs. Pay attention to this callback.

virtual void OnLocalDeviceException(AliEngineLocalDeviceType deviceType, AliEngineLocalDeviceExceptionType exceptionType, const char* msg){};

Parameter description

The callback for authentication information that is about to expire. The authentication expires 30 seconds after you receive this callback. Pay attention to this callback.

virtual void OnAuthInfoWillExpire() {};

This callback indicates that the user's authentication information is about to expire. The authentication expires 30 seconds after you receive this callback. You need to obtain a new Token and update the authentication information using one of the following methods:

• Call the RefreshAuthInfo method to update the authentication information.

• Call the LeaveChannel method to leave the current channel, and then call the JoinChannel method to rejoin the channel.

Call timing

The SDK triggers this callback 30 seconds before the user authentication information expires. You should update the authentication information promptly after receiving this callback.

The callback for expired authentication. The expiration information is returned by the server when users call a method that requires authentication.

virtual void OnAuthInfoExpired() {};

This callback indicates that the user's authentication information has expired. If you want to continue in the meeting, you need to generate a new Token on the server side and then update the authentication information using one of the following methods:

• Call LeaveChannel to leave the current channel, and then call JoinChannel to rejoin the channel.

Trigger timing

This callback is triggered when the user's authentication information expires.

The callback that is invoked to return the result of joining a channel (this callback is equivalent to the block operation of calling the JoinChannel method, which handles events after joining a channel, and you can choose either one).

virtual void OnJoinChannelResult(int result, const char *channel, const char *userId, int elapsed) {}

Trigger conditions

When the application calls the JoinChannel method, this callback indicates whether the channel was successfully joined and returns information about the channel and the time taken to join the channel.

Parameter description

The callback that is invoked to return the result of leaving a channel. This callback is returned after you call the LeaveChannel method. If you call Destroy immediately after calling LeaveChannel, you will not receive this callback.

virtual void OnLeaveChannelResult(int result, AliEngineStats stats) {}

Trigger conditions

When the application successfully calls LeaveChannel to leave the channel, this callback is triggered to return the result of leaving the channel and the statistics for the channel session.

Parameters

The callback that is invoked when a remote user goes offline.

virtual void OnRemoteUserOffLineNotify(const char *uid, AliEngineUserOfflineReason reason) {}

This callback notifies the local user that a remote user has left the channel for various reasons. This interface will be triggered when a remote user goes offline.

Trigger conditions

• When a remote user actively leaves the channel, this callback is triggered.

• When a remote streamer calls <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#c99c2b257dd8o" id="03e45f655fcxx">SetClientRole</a> to switch to the viewer role (set to AliEngineClientRoleLive), this callback is triggered.

• When no data is received from a remote streamer for a long time and they are considered disconnected, this callback is triggered.

Parameter description

The callback that is invoked when a remote user goes online.

virtual void OnRemoteUserOnLineNotify(const char *uid, int elapsed) {}

This interface is used to notify the local client that a remote user has joined the channel.

Trigger conditions

• A remote user successfully joins the channel.

• After the current user joins the channel, they will receive join callbacks for users who are already in the channel, to display previously joined users.

Parameter description

The callback that is invoked when the stream of a remote user changes.

virtual void OnRemoteTrackAvailableNotify(const char *uid,
                                                  AliEngineAudioTrack audioTrack,
                                                  AliEngineVideoTrack videoTrack) {}

This callback is triggered when the streaming status of a remote user changes. Through this callback, developers can monitor in real-time whether remote users are publishing audio and video streams, and display or hide remote users' audio and video information on the interface accordingly.

Trigger conditions

This callback is triggered in the following scenarios:

• When a remote user changes from not streaming to streaming (including audio and video)

• When a remote user changes from streaming to not streaming (including audio and video)

• In interactive mode, when a remote user calls SetClientRole to switch from viewer to streamer role, and sets up streaming, this callback will be triggered

Taking video as an example, if a remote user sets not to stream, this callback will not be triggered:

• When a remote user starts publishing a camera stream (streaming status: no video stream -> camera stream only), the callback returns AliEngineVideoTrackCamera indicating that the remote user's camera stream is available.

• When the remote user also publishes a screen sharing stream (streaming status: camera stream only -> both camera stream and screen sharing stream), the callback returns AliEngineVideoTrackBoth indicating that both the remote user's camera stream and screen sharing stream are available.

• When the remote user stops publishing the camera stream and only keeps the screen sharing stream (streaming status: both camera stream and screen sharing stream -> screen sharing stream only), the callback returns AliEngineVideoTrackScreen indicating that only the screen sharing stream is currently available.

• When the remote user also stops publishing the screen sharing stream (streaming status: screen sharing stream only -> no video stream), the callback returns AliEngineVideoTrackNo indicating that no video stream is currently available.

Parameter description

The callback that is invoked when a user is forced to leave a channel.

virtual void OnBye(int code) {}

This callback is triggered when a user is disconnected for some reason or when the meeting ends. Developers can determine the reason for disconnection based on the callback parameter code and take appropriate action.

Trigger conditions

• The current user is kicked out by the server.

• The meeting ends (the server actively removes the channel).

• Passive departure, requiring the client to attempt to recover the session or reconnect.

Parameter description

The callback that is invoked when the status of stream ingest for an audio track changes.

virtual void OnAudioPublishStateChanged(AliEnginePublishState oldState, AliEnginePublishState newState, int elapseSinceLastState, const char *channel) {};

This callback is used to monitor changes in the stream ingest state of the local user's audio stream.

Trigger conditions

When the audio stream ingest state of the user changes, for example:

• Stop stream ingest.

• Call <a baseurl="t2713962_v4_0_0.xdita" data-node="4904760" data-root="16090" data-tag="xref" href="#c99c2b257dd8o" id="904fc4e83a35n">SetClientRole</a> to switch to viewer role.

Parameter description

The callback that is invoked when the status of subscription to an audio track changes.

virtual void OnAudioSubscribeStateChanged(const char *uid,
                                                  AliEngineSubscribeState oldState, 
                                                  AliEngineSubscribeState newState,
                                                  int elapseSinceLastState,
                                                  const char *channel) {};

This callback notifies the local user that the subscription status of a remote user's audio stream has changed. Through this callback, you can learn about the changes in the subscription status of a remote user's audio stream, along with the time interval from the previous state to the current state.

Parameter description

Notification when a user mutes their audio.

virtual void OnUserAudioMuted(const char* uid, bool isMute) {}

Parameter description

A notification that the user's audio was interrupted. This typically occurs when another application, such as a phone call app, takes control of the audio.

virtual void OnUserAudioInterruptedBegin(const char* uid) {}

Parameter description

Notification that a user's audio interruption has ended (corresponds to OnUserAudioInterruptedBegin).

virtual void OnUserAudioInterruptedEnded(const char* uid) {}

Parameter description

The callback that is invoked when the first video frame from a remote user is displayed.

virtual void OnVideoPublishStateChanged(AliEnginePublishState oldState, AliEnginePublishState newState, int elapseSinceLastState, const char *channel) {};

This callback is used to monitor changes in the video stream ingest status of the local user.

Parameter description

Callback for camera stream subscription changes.

virtual void OnVideoSubscribeStateChanged(const char *uid, 
                                                  AliEngineSubscribeState oldState, 
                                                  AliEngineSubscribeState newState,
                                                  int elapseSinceLastState,
                                                  const char *channel) {};

This callback notifies the local user that the subscription status of a remote user's camera track has changed. Through this callback, you can learn about the subscription status change of a specific remote user's camera track, along with the time interval from the previous state to the current state.

Related callbacks

Video streams mainly include camera tracks and screen sharing tracks. This interface is for the subscription status change of camera tracks, while the related callback interface for screen sharing tracks is OnScreenShareSubscribeStateChanged.

Parameter description

User MuteVideo notification.

virtual void OnUserVideoMuted(const char* uid, bool isMute) {}

Parameter description

Notification for disabling or re-enabling local video capture.

virtual void OnUserVideoEnabled(const char* uid, bool isEnable) {}

Parameter description

The callback that is triggered when a remote user's application moves to the background.

virtual void OnUserWillResignActive(const char* uid) {}

Parameter description

Callback triggered when a remote user's application returns to the foreground.

virtual void OnUserWillBecomeActive(const char* uid) {}

Parameter description

Real-time data callback (triggered once every 2 seconds).

virtual void OnStats(const AliEngineStats& stats) {}

Parameter description

Local video statistics information (triggered every 2 seconds).

virtual void OnLocalVideoStats(const AliEngineLocalVideoStats& localVideoStats);

Parameter information