Intelligent Media Services:Integrate into iOS apps

Environment requirements

• Xcode 16.0 or later. We recommend that you use the latest official version.

• CocoaPods 1.9.3 or later.

• A physical device that runs iOS 10.0 or later.

Integrate the SDK

target 'Your target' do

  # Integrate ApsaraVideo MediaBox SDK for Alibaba Real-Time Communication (ARTC). You can integrate AliVCSDK_ARTC, AliVCSDK_Standard, or AliVCSDK_InteractiveLive.
  pod 'AliVCSDK_ARTC', '~> x.x.x'
  # Integrate AICallKit SDK.
  pod 'ARTCAICallKit', '~> 1.6.0'

  ...

end

Configure the project

• Open the info.Plist file of your project and add the NSMicrophoneUsageDescription and NSCameraUsageDescription permissions.

• In the project settings, enable Background Modes on the Signing & Capabilities tab. We recommend that you enable Background Modes. Otherwise, calls that are switched to the background cannot continue. In this case, you must call the handup() method to end the calls.

Sample code for using the SDK

// Integrate the SDK.
import ARTCAICallKit

// Create an engine instance.
var engine: ARTCAICallEngineInterface = {
    return ARTCAICallEngineFactory.createEngine()
}()

// Configure callback events.
self.engine.delegate = self

// Start a call after an intelligent agent is started.
let agentInfo = ARTCAICallAgentInfo(agentType: workflow_type, channelId: channel_id, uid: ai_agent_user_id, instanceId: agent_instance_id)
self.engine.call(userId: self.userId, token: rtc_auth_token, agentInfo: agentInfo) { [weak self] error in
    if let error = error {
        // Handle the error.
    }
    else {
        // The call is successful.
    }
}

// End the call.
self.engine.handup()

// For more information about how to call other methods, see the "SDK reference" section of this topic.

// The callback events. The following examples include only callback events that do not require specialized main code.
public func onErrorOccurs(code: ARTCAICallErrorCode) {
    // An error occurred.
    self.engine.handup()
}

public func onCallBegin() {
    // The call starts.
}

public func onCallEnd() {
    // The call ends.
}

public func onAgentStateChanged(state: ARTCAICallAgentState) {
    // The status of the intelligent agent changes.
}

public func onUserSubtitleNotify(text: String, isSentenceEnd: Bool, sentenceId: Int) {
    // The question of the user is recognized by the intelligent agent.
}

public func onVoiceAgentSubtitleNotify(text: String, isSentenceEnd: Bool, userAsrSentenceId: Int) {
    // The intelligent agent returns an answer.
}

public func onVoiceIdChanged(voiceId: String) {
    // The voice of the current call changes.
}

public func onVoiceInterrupted(enable: Bool) {
    // Indicate whether voice interruption is enabled for the current call.
}

API reference

API overview

ARTCAICallEngineInterface

userId

Queries the ID of the user on the current call.

var userId: String?  {get}

isOnCall

Indicates whether a call is in progress. Valid values: true and false.

var isOnCall: Bool { get }

agentInfo

Queries the information about the current intelligent agent. The information includes the type, channel ID, UID of the intelligent agent in the channel, and instance ID of the intelligent agent.

var agentInfo: ARTCAICallAgentInfo?  { get }

agentState

The status of the intelligent agent. The agent can be listening, thinking, or speaking.

var agentState: ARTCAICallAgentState { get }

delegate

Configures and queries callback events.

weak var delegate: ARTCAICallEngineDelegate?  { get set }

call

Starts a call.

func call(userId: String, token: String, agentInfo: ARTCAICallAgentInfo, completed:((_ error: NSError?) -> Void)?)

Parameters

handup

Ends a call.

 func handup(_ stopAIAgent: Bool)

Parameters

setAgentViewConfig

Configures the rendering view for the intelligent agent. This setting is valid only for the digital human agent.

func setAgentViewConfig(viewConfig: ARTCAICallViewConfig?)

Parameters

interruptSpeaking

Interrupts the speech of the intelligent agent.

func interruptSpeaking() -> Bool

enableVoiceInterrupt

Enables or disables intelligent interruption.

func enableVoiceInterrupt(enable: Bool) -> Bool

Parameters

switchVoiceId

Changes the voice.

func switchVoiceId(voiceId: String) -> Bool

Parameters

enableSpeaker

Enables or disables the speaker.

func enableSpeaker(enable: Bool) -> Bool

Parameters

enablePushToTalk

Enables or disables intercom mode. In intercom mode, the intelligent agent returns a result only after the finishPushToTalk operation is called.

func enablePushToTalk(enable: Bool) -> Bool

Parameters

startPushToTalk

Starts speaking in intercom mode.

func startPushToTalk() -> Bool

finishPushToTalk

Finishes speaking in intercom mode.

func finishPushToTalk() -> Bool

cancelPushToTalk

Cancels speaking in intercom mode.

func cancelPushToTalk() -> Bool

muteMicrophone

Mutes or unmutes the microphone.

func muteMicrophone(mute: Bool) -> Bool

Parameters

visionConfig

The visual configuration, including the resolution and frame rate. For more information, see ARTCAICallVisionConfig. The configuration is required if you use a visual intelligent agent. The configuration takes effect only if you set it before the call.

var visionConfig: ARTCAICallVisionConfig { set get }

muteLocalCamera

Turns on or off a camera.

func muteLocalCamera(mute: Bool) -> Bool

Parameters

switchCamera

Switches between the front and rear cameras.

func switchCamera() -> Bool

parseShareAgentCall

Parses the information about a shared intelligent agent. If the parsing is successful, an instance of the ARTCAICallAgentShareConfig type is returned, which can be used to start a call with the intelligent agent.

func parseShareAgentCall(shareInfo: String) -> ARTCAICallAgentShareConfig?

Parameters

generateShareAgentCall

Starts a call with a shared intelligent agent.

func generateShareAgentCall(shareConfig: ARTCAICallAgentShareConfig, userId: String, completed: ((_ rsp: ARTCAICallAgentInfo?, _ token: String?, _ error: NSError?, _ reqId: String) -> Void)?)

Parameters

getRTCInstance

Queries the information about an RTC engine instance.

func getRTCInstance() -> AnyObject?

destroy

Releases resources.

func destroy()

ARTCAICallEngineDelegate

onErrorOccurs

An error occurred during the current call.

@objc optional func onErrorOccurs(code: ARTCAICallErrorCode)

Parameters

onCallBegin

A call starts.

@objc optional func onCallBegin()

onCallEnd

A call ends.

@objc optional func onCallEnd()

onAgentVideoAvailable

Indicates whether the ingested video stream of the intelligent agent is available.

@objc optional func onAgentVideoAvailable(available: Bool)

Parameters

onAgentAudioAvailable

Indicates whether the ingested audio stream of the intelligent agent is available.

@objc optional func onAgentAudioAvailable(available: Bool)

Parameters

onPushToTalk

Indicates whether intercom mode is enabled for the current call.

@objc optional func onPushToTalk(enable: Bool)

Parameters

onAgentWillLeave

The intelligent agent is about to end the current call.

@objc optional func onAgentWillLeave(reason: Int32, message: String)

Parameters

onReceivedAgentCustomMessage

A custom message is received from the current intelligent agent.

@objc optional func onReceivedAgentCustomMessage(data: [String: Any]?)

Parameters

onAgentStateChanged

The status of the intelligent agent changes.

@objc optional func onAgentStateChanged(state: ARTCAICallAgentState)

Parameters

onNetworkStatusChanged

The network status changes.

@objc optional func onNetworkStatusChanged(uid: String, quality: ARTCAICallNetworkQuality)

Parameters

onVoiceVolumeChanged

The volume changes.

@objc optional func onVoiceVolumeChanged(uid: String, volume: Int32)

Parameters

onUserSubtitleNotify

The intelligent agent recognizes the question of the user.

@objc optional func onUserSubtitleNotify(text: String, isSentenceEnd: Bool, sentenceId: Int)

Parameters

onVoiceAgentSubtitleNotify

The intelligent agent returns an answer.

@objc optional func onVoiceAgentSubtitleNotify(text: String, isSentenceEnd: Bool, userAsrSentenceId: Int)

Parameters

onVoiceIdChanged

The voice of the intelligent agent changes.

@objc optional func onVoiceIdChanged(voiceId: String)

Parameters

onVoiceInterrupted

Indicates whether voice interruption is enabled for the current call.

@objc optional func onVoiceInterrupted(enable: Bool)

Parameters

onAgentAvatarFirstFrameDrawn

The first video frame of the digital human agent is rendered.

@objc optional func onAgentAvatarFirstFrameDrawn()

onHumanTakeoverWillStart

A human agent is stepping in to take over from the current intelligent agent.

@objc optional func onHumanTakeoverWillStart(takeoverUid: String, takeoverMode: Int)

Parameters

onHumanTakeoverConnected

A human agent has taken over from the current intelligent agent.

@objc optional func onHumanTakeoverConnected(takeoverUid: String)

Parameters

onAgentEmotionNotify

The intelligent agent detects an emotion.

@objc optional func onAgentEmotionNotify(emotion: String, userAsrSentenceId: Int)

Parameters

ARTCAICallEngineFactory

createEngine

Creates a default engine instance.

public static func createEngine() -> ARTCAICallEngineInterface