Vocalize notifications from AI agent - Intelligent Media Services

This topic describes how to enable an AI agent to vocalize the input text for notifications during audio and video calls.

Feature description

Real-time Conversational AI provides an API where you can pass the text content to be vocalized as a parameter. After the client receives the input text, it initiates speech synthesis to convert the text into speech and deliver it to users.

Use cases

This feature is suitable for scenarios that require immediate, clear, and audible communication of information. For example:

Smart appliances
Users send commands to devices via a mobile application or voice assistant, such as adjusting air conditioner temperature or turning on lights. Then, the system calls an OpenAPI operation to pass the result text to the agent. The agent reads out the received text to inform users whether the device operation is successful.
Automotive systems
When an automotive system detects issues such as low tire pressure or insufficient fuel, it sends relevant text through OpenAPI to the agent. The agent reads out the received text to alert the driver.

Implementation

Server-side

Call SendAIAgentSpeech to pass the text content to be vocalized to the agent.

package com.aliyun.rtc;

import com.aliyun.tea.*;

public class SampleSendAIAgentSpeech {

    /**
     * <b>description</b> :
     * <p>Use your AccessKey ID and AccessKey secret to initialize the client.</p>
     * @return Client
     * 
     * @throws Exception
     */
    public static com.aliyun.ice20201109.Client createClient() throws Exception {
        // Important: Exposing credentials in code poses security risks. Use this code example as a reference only.
        com.aliyun.teaopenapi.models.Config config = new com.aliyun.teaopenapi.models.Config()
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_ID is configured.
                .setAccessKeyId("yourak")
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET is configured.
                .setAccessKeySecret("yoursk");
        // The endpoint. For details, refer to https://api.alibabacloud.com/product/ICE
        config.endpoint = "ice.cn-shanghai.aliyuncs.com";
        return new com.aliyun.ice20201109.Client(config);
    }

    private static void sendAIAgentSpeech() throws Exception {
        com.aliyun.ice20201109.Client client = createClient();

        com.aliyun.ice20201109.models.SendAIAgentSpeechRequest request = new com.aliyun.ice20201109.models.SendAIAgentSpeechRequest()
            .setInstanceId("xxx")
            .setText("yourtext");

        try {
            com.aliyun.teautil.models.RuntimeOptions runtime = new com.aliyun.teautil.models.RuntimeOptions();
            
            client.sendAIAgentSpeechWithOptions(request, runtime);
        } catch (TeaException error) {
            System.out.println(error.getMessage());
            com.aliyun.teautil.Common.assertAsString(error.message);
        } catch (Exception _error) {
            TeaException error = new TeaException(_error.getMessage(), _error);
            System.out.println(error.getMessage());
            com.aliyun.teautil.Common.assertAsString(error.message);
        }   
    }
    
    public static void main(String[] args) throws Exception {
        sendAIAgentSpeech();
    }
}