Intelligent Media Services:FAQ about Real-time Conversational AI

Do I need to deploy an AI agent on the origin server?

No, you do not need to deploy it on the cloud. The AI agent is a public cloud service provided by Alibaba Cloud. Simply create an agent in the console and access it through OpenAPI.

How does an agent access a large language model (LLM) deployed in Alibaba Cloud Model Studio?

Real-time Conversational AI has been integrated with Alibaba Cloud Model Studio. Configure the required parameters in the console to access the LLMs provided by Model Studio. For more information, see LLM. 

How do I handle errors that occur when starting a call?

Error 1: Could not resolve placeholder 'biz.live_mic.gslb' in value "${biz.live_mic.gslb}

• Cause: A configuration is missing or deleted in the server-side application.yaml file.

• Solution: Variables injected through the @value annotation must exist in the file. You can set the placeholders to a default value or as an empty string "" if applicable.

Error 2: User not authorized to operate on the specified resource

• Solution: When deploying the app server, ensure the configured AccessKey pair is correct and grant the AliyunICEFullAccess permission to the RAM user. For more information, see Deploy through source code.

Error 3: generateAIAgentCall Tea error. e:code: 404, Specified access key is not found

• Solution: Check if the AccessKey ID and AccessKey Secret are correctly configured in the server-side application.yaml file. Then, grant the AliyunICEFullAccess permission to the RAM user. For more information, see Deploy through source code.

Error 4: generateAIAgentCall Tea error. e:code: 400, The specified agentId "123456" is not found. request id: xxxxxxx

• Solution: Check if the agent ID and region are correctly configured.

Error 5: The token is returned, but the client still cannot connect.

• Solution: Use the RTC token verification tool to check the accuracy of token generation. The relevant parameters are in the server request or return values. Specify nonce as null and check if the generated token is correct.

The client reports the "AgentNotFound" error when starting a messaging conversation

Check if the agent ID and region are correctly configured in the client code.

String mAgentId = "XXX";          // The ID of the AI agent created in the console for messaging conversations.
String mRegion = "cn-shanghai";   // The region where the AI agent is deployed in the console.
ARTCAIChatAgentInfo agentInfo = new ARTCAIChatEngine.ARTCAIChatAgentInfo(mAgentId, mRegion)

// The ID of the AI agent created in the console for messaging conversations.
let agentInfo = ARTCAIChatAgentInfo(agentId: "xxxx")
// The region where the AI agent is deployed in the console.
agentInfo.region = "cn-shanghai"

The client reports "UnsupportedWorkflowType" error when starting a messaging conversation

Error message: The specified workflowType \"VoiceChat\" is not supported by this interface. Please use a compatible workflowType: [\"MessageChat\"]

Solution: Check if the agent is associated with a messaging workflow.

How do I adjust the client audio sample rate?

Real-time Conversational AI supports the following settings for audio capture on a client:

• Sample rate: 48 kHz, mono channel

• Sample rate: 16 kHz, mono channel

AICallKit SDK uses a sample rate of 48 kHz by default. To switch to 16 kHz, refer to the following example code:

self.engine.audioConfig = ARTCAICallAudioConfig(audioProfile: .BasicQualityMode, audioScenario: .MusicMode)

// Call other APIs
...  

// Initiate a call
self.engine.call(...)

ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();
artcaiCallConfig.audioConfig.audioProfile = ARTCAICallAudioBasicQualityMode;
engine.init(artcaiCallConfig);

// AudioQualityMode: Integer    
// Sample rate for RTC output.
// 0: 48 kHz mono 
// 1: 48 kHz stereo
// 2: 16 kHz mono
// The value of ExperimentalConfig must be a JSON string
{
    "ExperimentalConfig":"{\"AudioQualityMode\":2}"
}

// AudioQualityMode: Integer    
// Sample rate for RTC output.
// 0: 48 kHz mono 
// 1: 48 kHz stereo
// 2: 16 kHz mono
// The value of ExperimentalConfig must be a JSON string
{
    "ExperimentalConfig":"{\"AudioQualityMode\":2}"
}

let callConfig = ...   // Create and initialize ARTCAICallConfig.

let agentConfig = ARTCAICallAgentConfig()         // Create ARTCAICallAgentConfig object.
agentConfig.experimentalConfig = [
    "AudioQualityMode": 2
]
...                                                  // Set parameters according to business needs
callConfig.agentConfig = agentConfig
self.engine.call(config: callConfig)                 // Initiate a call

ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();
artcaiCallConfig.agentConfig.experimentalConfig = new JSONObject();
try {
    artcaiCallConfig.agentConfig.experimentalConfig.put("AudioQualityMode", 2);
} catch (JSONException e) {
    e.printStackTrace();
}
engine.init(artcaiCallConfig);