Intelligent Media Services:Integration

Step 1: Create and initialize the engine

Create and initialize an instance of ARTCAICallEngine. The following code shows an example:

// Specify the agent type: audio-only, digital human, or visual understanding.
let agentType: AICallAgentType;

// Initialization parameters.
const config = {
  muteMicrophone: false, // Specifies whether to mute the microphone. Set as needed.
  // Other parameters are omitted.
}

// If the agent type is visual understanding, configure the local video preview.
if (agentType === AICallAgentType.VisionAgent) {
  config.previewElement = 'example-preview-element-id';
  config.cameraConfig = {
    width: 450,
    height: 800,
    frameRate: 5,
  };
}

const engine = new ARTCAICallEngine(config);

// If the agent type is digital human, configure the view for displaying the digital human.
if (agentType === AICallAgentType.AvatarAgent) {
  engine.setAgentView('example-agent-element-id');
}

Step 2: Implement callback methods

Implement the engine callback methods as needed. For more information about the engine callback API, see ARTCAICallEngine events.

// This callback is triggered if the call fails to start.
engine.on('errorOccurred', (code: number) => {
  // An error occurred during the process. Hang up the call.
  engine.hangup();
});

// This callback is triggered if the agent starts the call successfully.
engine.on('agentStarted', () => {
  // The agent instance has started.
});

// This callback is triggered when the current call is connected.
engine.on('callBegin', () => {
  // The call starts.
});

engine.on('callEnd', () => {
  // The call ends.
});

engine.on('agentStateChange', (newState: AICallAgentState) => {
  // The agent state changes.
});

engine.on('agentSubtitleNotify', (subtitle: AICallSubtitleData) => {
  // Notification for the agent's response result.
});

engine.on('userSubtitleNotify', (subtitle: AICallSubtitleData, voiceprintResult: AICallVoiceprintResult) => {
  // Notification for the user's speech recognition result.
});

engine.on('voiceIdChanged', (voiceId: string) => {
  // The voice timbre for the current call has changed.
});

engine.on('voiceInterruptChanged', (enable: boolean) => {
  // The status of the smart voice interruption for the current call has changed.
});

Step 3: Create and initialize AICallConfig

For more information about the structure, see AICallConfig.

const callConfig = {
  agentId: "xxx",             // The agent ID.
  agentType: agentType,       // The agent type.
  userId: "xxx",              // Use the user ID from your app logon.
  region: "xx-xxx",           // The region where the agent service is located.
  userJoinToken: "xxxxxxxxx", // The RTC token.
};

For the userJoinToken parameter, obtain an RTC token. For more information, see Obtain an RTC token.

Step 4: Initiate an agent call

Invoke the callWithConfig(config) method to initiate an agent call.

// After the agent starts, begin the call.
engine.callWithConfig(callConfig);

// This callback is triggered when the current call is connected.
engine.on('callBegin', () => {
  // The call starts.
});

Step 5: Implement in-call features

After the call starts, you can implement features such as captions and agent interruption as needed. For more information, see Feature implementation.

Step 6: End the call

Invoke the hangup() method to end the agent call.

engine.hangup();