Use the Real-time Conversational AI API to push text to an AI agent, which synthesizes it into speech and plays it back to users during audio and video calls.
How it works
Call the SendAIAgentSpeech operation with the text you want the agent to speak. The agent receives the text, synthesizes it into audio, and delivers the audio to users in the ongoing call.
Use cases
This feature suits scenarios that require immediate, audible delivery of system-generated information. For example:
-
Smart appliances
A user sends a command through a mobile application or voice assistant — for example, adjusting the air conditioner temperature or turning on a light. The system calls SendAIAgentSpeech to pass the result text to the agent, which reads it out to confirm whether the operation succeeded.
-
Automotive systems
When an automotive system detects a condition such as low tire pressure or insufficient fuel, it sends the alert text through SendAIAgentSpeech to the agent, which reads it aloud to the driver.
Implementation
Server-side
Call SendAIAgentSpeech to pass the text to the agent.
The following Java example shows how to call SendAIAgentSpeech. Replace <your-instance-id> with the ID of your AI agent instance and <text-to-speak> with the notification text.
package com.aliyun.rtc;
import com.aliyun.tea.*;
public class SampleSendAIAgentSpeech {
/**
* Initialize the client using your AccessKey ID and AccessKey secret.
* Store credentials in environment variables — do not hardcode them.
*/
public static com.aliyun.ice20201109.Client createClient() throws Exception {
com.aliyun.teaopenapi.models.Config config = new com.aliyun.teaopenapi.models.Config()
// Required. Set the ALIBABA_CLOUD_ACCESS_KEY_ID environment variable.
.setAccessKeyId(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"))
// Required. Set the ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variable.
.setAccessKeySecret(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
config.endpoint = "ice.cn-shanghai.aliyuncs.com";
return new com.aliyun.ice20201109.Client(config);
}
private static void sendAIAgentSpeech() throws Exception {
com.aliyun.ice20201109.Client client = createClient();
com.aliyun.ice20201109.models.SendAIAgentSpeechRequest request =
new com.aliyun.ice20201109.models.SendAIAgentSpeechRequest()
.setInstanceId("<your-instance-id>") // AI agent instance ID
.setText("<text-to-speak>"); // Text the agent will read aloud
try {
com.aliyun.teautil.models.RuntimeOptions runtime =
new com.aliyun.teautil.models.RuntimeOptions();
client.sendAIAgentSpeechWithOptions(request, runtime);
} catch (TeaException error) {
System.out.println(error.getMessage());
com.aliyun.teautil.Common.assertAsString(error.message);
} catch (Exception _error) {
TeaException error = new TeaException(_error.getMessage(), _error);
System.out.println(error.getMessage());
com.aliyun.teautil.Common.assertAsString(error.message);
}
}
public static void main(String[] args) throws Exception {
sendAIAgentSpeech();
}
}