Intelligent Media Services:Integration overview

Step 1: Check microphone and camera permissions for the app

Check the microphone and camera permissions for the app. If the permissions are not granted, display a dialog box to prompt the user for authorization. You need to implement this feature in your app. For sample code, see AVDeviceAuth.h.

Step 2: Create and initialize the engine

Create and initialize an ARTCAICallEngine instance. The following code provides an example:

// Create an engine instance.
let engine = ARTCAICallEngineFactory.createEngine()
let agentType: ARTCAICallAgentType

// Initialize the engine instance.
public func setup() {
    // Set the callback.
    self.engine.delegate = self

    // If the agent is a digital human, configure the view for the digital human.
    if self.agentType == .AvatarAgent {
        let agentViewConfig = ARTCAICallViewConfig(view: self.avatarAgentView)
        self.engine.setAgentViewConfig(viewConfig: agentViewConfig)
    }
    // If the agent is for visual understanding, configure the local video preview.
    else if self.agentType == .VisionAgent {
        let cameraViewConfig = ARTCAICallViewConfig(view: self.cameraView)
        self.engine.setLocalViewConfig(viewConfig: cameraViewConfig)
    }
    // If the agent is for a video call, configure the view for the digital human and the local video preview.
    else if self.agentType == .VideoAgent {
        let agentViewConfig = ARTCAICallViewConfig(view: self.avatarAgentView)
        self.engine.setAgentViewConfig(viewConfig: agentViewConfig)

        let cameraViewConfig = ARTCAICallViewConfig(view: self.cameraView)
        self.engine.setLocalViewConfig(viewConfig: cameraViewConfig)
    }
}

Step 3: Implement callback methods

Implement the required engine callback methods. For more information about engine callback operations, see API reference.

// Callback processing (This example shows only non-core callback operations).
public func onErrorOccurs(code: ARTCAICallErrorCode) {
    // An error occurred.
    self.engine.handup()
}

public func onCallBegin() {
    // The call started.
}

public func onCallEnd() {
    // The call ended.
}

public func onAgentStateChanged(state: ARTCAICallAgentState) {
    // The agent status changed.
}

public func onUserSubtitleNotify(text: String, isSentenceEnd: Bool, sentenceId: Int) {
    // Notification for the result of the agent's recognition of the user's question.
}

public func onVoiceAgentSubtitleNotify(text: String, isSentenceEnd: Bool, userAsrSentenceId: Int) {
    // Notification for the agent's answer.
}

public func onVoiceIdChanged(voiceId: String) {
    // The timbre of the current call changed.
}

public func onVoiceInterrupted(enable: Bool) {
    // Whether voice interruption is enabled for the current call.
}

Step 4: Create and initialize ARTCAICallConfig

For more information about ARTCAICallConfig, see ARTCAICallConfig.

let callConfig = ARTCAICallConfig()
callConfig.agentId = "xxx"             // The agent ID.
callConfig.agentType = self.agentType  // The agent type.
callConfig.userId = "xxx"              // Use the ID of the user who has logged on to your app.
callConfig.region = "xx-xxx"           // The region where the agent service is located.
callConfig.userJoinToken = "xxxxxxxxx" // The RTC token.

// For visual understanding and video calls, you must set the video configuration.
// In this example, frameRate is set to 5. Adjust this value based on the agent's frame sampling rate, which is configured in the console and is typically 2. Do not set the frame rate to a value greater than 15 fps.
// bitrate: If frameRate is greater than 10, you can set the bitrate to 512.
if self.config.agentType == .VisionAgent{
    callConfig.videoConfig = ARTCAICallVideoConfig(frameRate: 5, bitrate: 340, useFrontCameraDefault: false)
}
if self.config.agentType == .VideoAgent {
    callConfig.videoConfig = ARTCAICallVideoConfig(frameRate: 5, bitrate: 340, useFrontCameraDefault: true)
}
                

For the userJoinToken parameter, you must obtain an RTC token. For more information, see Generate an ARTC authentication token.

Step 5: Initiate a call to the agent

Call the call(config) method to initiate a call to the agent.

// After the agent starts, start the call.
public func start() {
    let callConfig = ...   // Create a callConfig object based on the preceding code.
    if self.engine.call(config: callConfig) {
        // The API operation is successfully called.
    }
}


// After the call is connected, the following callback is triggered.
public func onCallBegin() {
    // The call started.
}

Step 6: Implement in-call features

After the call starts, you can process captions, interrupt the agent, and perform other operations as needed. For more information, see Implement features.

Step 7: End the call

Call the hangup() method to end the call with the agent.

public func handup() {
    self.engine.handup()
}