Intelligent Media Services:API operation details

Retrieves the user ID of the current call.

var userId: String? {get}

Indicates whether a call is in progress. The value is true from the time the call is connected until it is hung up or an error occurs. Otherwise, the value is false.

var isOnCall: Bool { get }

Retrieves information about the current agent. This includes the agent type, channel ID, UID of the agent in the channel, and the running instance ID of the agent. For more information, see ARTCAICallAgentInfo.

var agentInfo: ARTCAICallAgentInfo? { get }

Retrieves the current state of the agent. States include listening, thinking, and speaking. For more information, see ARTCAICallAgentState.

var agentState: ARTCAICallAgentState { get }

Sets and retrieves callback events. For more information about callback events, see ARTCAICallEngineDelegate.

weak var delegate: ARTCAICallEngineDelegate? { get set }

Starts a call.

func call(userId: String, token: String, agentInfo: ARTCAICallAgentInfo, completed:((_ error: NSError?) -> Void)?)

Parameter details:

Starts a call using call configurations.

func call(config: ARTCAICallConfig) -> Bool

Parameter details:

Exits the call.

 func handup(_ stopAIAgent: Bool)

Parameter details:

Specifies the audio configurations. This includes encoding settings, such as sample rate, the number of sound channels, and bitrate, and scenario settings, such as default and music scenarios. For more information, see ARTCAICallAudioConfig. This parameter takes effect only when it is set before the call.

var audioConfig: ARTCAICallAudioConfig { set get }

Specifies the video configurations. This parameter takes effect only when it is set before the call. This is valid only for VisionAgent and VideoAgent. For more information, see ARTCAICallVideoConfig.

var videoConfig: ARTCAICallVideoConfig { set get }

Sets the rendering view and configurations for the local camera. This is valid only for VisionAgent and VideoAgent. For more information, see ARTCAICallViewConfig.

func setLocalViewConfig(viewConfig: ARTCAICallViewConfig?)

Parameter details:

Sets the rendering view configurations for the agent. This is required when the agent performs image rendering. This is valid only for VisionAgent and VideoAgent. For more information, see ARTCAICallViewConfig.

func setAgentViewConfig(viewConfig: ARTCAICallViewConfig?)

Parameter details:

Interrupts the agent's speech.

func interruptSpeaking() -> Bool

Enables or disables smart interruption.

func enableVoiceInterrupt(enable: Bool) -> Bool

Parameter details:

Switches the voice.

func switchVoiceId(voiceId: String) -> Bool

Parameter details:

Enables or disables the speaker.

func enableSpeaker(enable: Bool) -> Bool

Parameter details:

Enables or disables push-to-talk mode. In push-to-talk mode, the agent reports the result only after finishPushToTalk is called.

func enablePushToTalk(enable: Bool) -> Bool

Parameter details:

In push-to-talk mode, starts speaking.

func startPushToTalk() -> Bool

In push-to-talk mode, finishes speaking.

func finishPushToTalk() -> Bool

In push-to-talk mode, cancels the current speech.

func cancelPushToTalk() -> Bool

Mutes or unmutes the microphone.

func muteMicrophone(mute: Bool) -> Bool

Parameter details:

Stops or resumes the playback of the agent's audio stream.

func muteAgentAudioPlaying(mute: Bool) -> Bool

Parameter details:

This must be set when you use a vision agent and takes effect only when it is set before the call. The visual configurations include resolution and frame rate. For more information, see ARTCAICallVisionConfig.

var visionConfig: ARTCAICallVisionConfig { set get }

Disables or enables the camera.

func muteLocalCamera(mute: Bool) -> Bool

Parameter details:

Switches between the front and rear cameras.

func switchCamera() -> Bool

Parses shared agent information. If the parsing is successful, an instance of the ARTCAICallAgentShareConfig type is returned. You can use this instance to start a shared agent call.

func parseShareAgentCall(shareInfo: String) -> ARTCAICallAgentShareConfig?

Parameter details:

Starts a shared agent call.

func generateShareAgentCall(shareConfig: ARTCAICallAgentShareConfig, userId: String, completed: ((_ rsp: ARTCAICallAgentInfo?, _ token: String?, _ error: NSError?, _ reqId: String) -> Void)?)

Parameter details:

Retrieves the ARTC engine.

func getRTCInstance() -> AnyObject?

Sends a text message to the agent.

func sendTextToAgent(req: ARTCAICallSendTextToAgentRequest) -> Bool

Parameter details:

Sends a custom message to the server. This method must be called after the call is connected.

func sendCustomMessageToServer(msg: String) -> Bool

Parameter details:

Updates the system prompt for the Large Language Model (LLM). This method must be called after the call is connected.

func updateLlmSystemPrompt(prompt: String) -> Bool

Parameter details:

Updates Model Studio application parameters.

func updateBailianAppParams(params: [String: Any]) -> Bool

Parameter details:

Updates the Video Cassette Recording (VCR) configuration.

 func updateVcrConfig(vcrConfig: ARTCAICallAgentVcrConfig) -> Bool

Parameter details:

For a visual understanding agent, starts custom frame capture. After this is started, you cannot talk to the agent using voice. This method must be called after the call is connected.

func startVisionCustomCapture(req: ARTCAICallVisionCustomCaptureRequest) -> Bool

Parameter details:

For a visual understanding agent, ends custom frame capture. This method must be called after the call is connected.

func stopVisionCustomCapture() -> Bool

Releases resources.

func destroy()

An error occurred during the current call.

@objc optional func onErrorOccurs(code: ARTCAICallErrorCode)

Parameter details:

The call agent has started.

@objc optional func onAgentStarted()

The call has started.

@objc optional func onCallBegin()

The call has ended.

@objc optional func onCallEnd()

Indicates whether the agent's video is available.

@objc optional func onAgentVideoAvailable(available: Bool)

Parameter details:

Indicates whether the agent's audio is available.

@objc optional func onAgentAudioAvailable(available: Bool)

Parameter details:

The ARTC engine is created. You can call getRTCInstance in this callback to retrieve the ARTC engine instance.

@objc optional func onRTCEngineCreated()

Indicates whether push-to-talk mode is enabled for the current call.

@objc optional func onPushToTalk(enable: Bool)

Parameter details:

The current agent is about to leave, which ends the current call.

@objc optional func onAgentWillLeave(reason: Int32, message: String)

Parameter details:

A custom message is received from the current agent.

@objc optional func onReceivedAgentCustomMessage(data: [String: Any]?)

Parameter details:

The agent state has changed.

@objc optional func onAgentStateChanged(state: ARTCAICallAgentState)

Parameter details:

The network status has changed.

@objc optional func onNetworkStatusChanged(uid: String, quality: ARTCAICallNetworkQuality)

Parameter details:

A notification is sent when the volume changes.

@objc optional func onVoiceVolumeChanged(uid: String, volume: Int32)

Parameter details:

A notification that contains the result of the agent's recognition of the user's question.

@objc optional func onUserSubtitleNotify(text: String, isSentenceEnd: Bool, sentenceId: Int)

Parameter details:

A notification that contains the agent's answer.

@objc optional func onVoiceAgentSubtitleNotify(text: String, isSentenceEnd: Bool, userAsrSentenceId: Int)

Parameter details:

The LLM has finished replying in the current call.

@objc optional func onLLMReplyCompleted(text: String, userAsrSentenceId: Int)

Parameter details:

The voice for the current call has changed.

@objc optional func onVoiceIdChanged(voiceId: String)

Parameter details:

Indicates whether voice interruption is enabled for the current call.

@objc optional func onVoiceInterrupted(enable: Bool)

Parameter details:

The agent's current speech is interrupted.

@objc optional func onSpeakingInterrupted(reason: ARTCAICallSpeakingInterruptedReason)

Parameter details:

Indicates whether custom frame capture mode is enabled for the current visual understanding call.

@objc optional func onVisionCustomCapture(enable: Bool)

Parameter details:

The first frame of the agent's digital human is rendered.

@objc optional func onAgentAvatarFirstFrameDrawn()

A human is about to take over from the current agent.

@objc optional func onHumanTakeoverWillStart(takeoverUid: String, takeoverMode: Int)

Parameter details:

The human takeover is connected.

@objc optional func onHumanTakeoverConnected(takeoverUid: String)

Parameter details:

A notification that contains the agent's emotion result.

@objc optional func onAgentEmotionNotify(emotion: String, userAsrSentenceId: Int)

Parameter details:

A callback that indicates the availability of the agent's message channel. After this callback is triggered, you can send messages to the agent.

@objc optional func onAgentDataChannelAvailable()

A VCR result is received from the current agent. For more information, see ARTCAICallAgentVcrResult.

@objc optional func onReceivedAgentVcrResult(result: ARTCAICallAgentVcrResult)

The connection status has changed during the call.

@objc optional func onConnectionStatusChange(status: ARTCAICallConnectionStatus, reason: Int32)

Parameter details:

Provides the audio loopback latency.

@objc optional func onAudioDelayInfo(sentenceId: Int32, delayMs: Int64)

Parameter details:

If music accompaniment is played through the RTC instance during the current call, this playback callback is triggered.

 @objc optional func onAudioAccompanyStateChanged(state: ARTCAICallAudioAccompanyState, errorCode: ARTCAICallAudioAccompanyErrorCode)

Parameter details:

Creates a default engine instance.

public static func createEngine() -> ARTCAICallEngineInterface