Intelligent Media Services:Implement query-based agent activation

Introduction

When activated, an AI agent typically says a welcome message to inform users that it is ready to provide services. However, sometimes users want the agent to answer their questions right away. Query-based activation meets this need.

Implementation

This topic uses the offline wake-up scenario as an example.

Offline wake-up is a mechanism by which a device or system can be activated by a specific command without an Internet connection. There are two methods of offline wake-up:

• Regular activation: The agent is activated by the wake word, such as "Tmall Genie" or "Siri."

• Query-based activation: The agent recognizes the wake word and a specific query, for example, "Tmall Genie, what's the weather like in Singapore today?" The question is input as the initial command, and the welcome message is disabled. Then, the agent immediately responds to the question.

Prerequisites

Wake-up word recognition is implemented on the client side.

Procedure

1. When the client side recognizes the wake word and a specific query, convert the query as the new wake word.

2. Call StartAIAgentInstance.

   • Set WakeUpQuery in the AIAgentConfig parameter to the converted query.

   • Set Greeting in the AIAgentConfig parameter to an empty string to disable the welcome message.

Sample code

//Create ARTCAICallConfig and configure ARTCAICallConfig.
ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig(); 
 
// Configure parameters of agentConfig.
artcaiCallConfig.agentConfig.wakeUpQuery = "What's the weather like today?";

// Continue with the rest of the call initiation process.

// Create an ARTCAICallAgentConfig object and set the wakeUpQuery parameter.
let agentConfig = ARTCAICallAgentConfig()
agentConfig.wakeUpQuery = "What's the weather like today?"

// If you use the integration solution with UI, set config.agentConfig when you initiate a call.
let controller = AUIAICallStandardController(userId: userId)
controller.config.agentConfig = agentConfig
...

// If you use the integration solution without UI, convert agentConfig into a JSON string and use it as a request parameter for calling StartAIAgentInstance or GenerateAIAgentCall.
let jsonString = agentConfig.getJsonString(agentType)
...

// Create an AICallAgentConfig object and set the wakeUpQuery parameter.
const agentConfig = new AICallAgentConfig();
agentConfig.wakeUpQuery = 'What\'s the weather like today?';

// If you use the integration solution with UI, modify src/runConfig.ts.
const runConfig: AICallRunConfig = { 
  // ...
  callAgentConfig: agentConfig,
  // ...
};

// If you use the integration solution without UI, convert agentConfig into a JSON string and use it as a request parameter for calling StartAIAgentInstance or GenerateAIAgentCall.
const jsonString = agentConfig.getJsonString(agentType)
...