Send Custom Messages to Clients via AI Agent API - Intelligent Media Services - Alibaba Cloud - Intelligent Media Services

You can send custom messages from your server to a client during a real-time audio or video call to deliver key business updates.

Feature introduction

A proactive message is sent automatically by the server based on predefined conditions. Users receive key business updates while focusing on the call with an AI agent, without having to search for information afterward.

Proactive messages help ensure that critical information reaches users, improving engagement and understanding while boosting operational efficiency.

Audio/Video call agent

Use cases

Proactive messages apply to audio and video calls across fields such as business matchmaking, virtual teaching, and psychological counseling. The following examples use virtual teaching:

Course material updates

The server sends notifications about new documents during a video call so students can download and view them immediately.
Homework reminders

As the course nears its end, the server pushes reminders about homework submission deadlines.
Exam preparations

Before an exam, the server sends relevant materials such as exam schedules and formats.

Implementation

Send proactive messages from the server

Use the SendAIAgentDataChannelMessage API to send a message via the data channel during an agent call.

// This file is auto-generated, don't edit it.
package com.aliyun.rtc;

import java.util.Arrays;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.aliyun.ice20201109.models.StartAIAgentInstanceResponse;
import com.aliyun.tea.*;

public class Sample {

    /**
     * <b>description</b> :
     * <p>Initialize the client using AccessKey ID and Secret.</p>
     * @return Client
     * 
     * @throws Exception
     */
    public static com.aliyun.ice20201109.Client createClient() throws Exception {
        // Important: Exposing credentials in code poses security risks. Use this code example as a reference only.
        com.aliyun.teaopenapi.models.Config config = new com.aliyun.teaopenapi.models.Config()
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_ID is configured.
                .setAccessKeyId("yourak")
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET is configured.
                .setAccessKeySecret("yoursk");
        // The endpoint. For details, refer to https://api.alibabacloud.com/product/ICE
        config.endpoint = "ice.cn-shanghai.aliyuncs.com";
        return new com.aliyun.ice20201109.Client(config);
    }


    private static void sendAIAgentDataChannelMessage() throws Exception {
        com.aliyun.ice20201109.Client client =  createClient();

        com.aliyun.ice20201109.models.SendAIAgentDataChannelMessageRequest request = new com.aliyun.ice20201109.models.SendAIAgentDataChannelMessageRequest()
            .setInstanceId("yourinstanceid")
            .setMessage("{\"key\":\"value\"}");
            try {
                com.aliyun.teautil.models.RuntimeOptions runtime = new com.aliyun.teautil.models.RuntimeOptions();
                
                client.sendAIAgentDataChannelMessageWithOptions(request, runtime);
            } catch (TeaException error) {
                System.out.println(error.getMessage());
                com.aliyun.teautil.Common.assertAsString(error.message);
            } catch (Exception _error) {
                TeaException error = new TeaException(_error.getMessage(), _error);
                System.out.println(error.getMessage());
                com.aliyun.teautil.Common.assertAsString(error.message);
            }   
    }
    
    public static void main(String[] args) throws Exception {
        sendAIAgentDataChannelMessage();
    }
}

Handle proactive messages on the client

After receiving a proactive message through the data channel, the client parses the message payload.

Android

// Implement the callback in ARTCAICallEngine.IARTCAICallEngineCallback.

@Override
public void onReceivedAgentCustomMessage(String data) {
// Process the received proactive message from the agent.
}

iOS

// Create an engine instance for the audio or video call.
var engine: ARTCAICallEngineInterface = {
    return ARTCAICallEngineFactory.createEngine()
}()

// Set the delegate for callback handling.
self.engine.delegate = self

// Implement the callback.
public func onReceivedAgentCustomMessage(data: [String : Any]?) {
  // Process the custom message received from the current agent.
    debugPrint("onReceivedAgentCustomMessage:\(data ?? [:])")
}

Web

engine.on('receivedAgentCustomMessage', (data) => {
  // Process the received proactive message from the agent.
  console.log('receivedAgentCustomMessage', data);
})

Chatbot agent

Use cases

Server-side message pushing is useful in scenarios such as e-commerce, education, and finance.

Example: E-commerce
During a chat with a customer service agent, if a user is browsing a product but has not purchased it, the server can push a limited-time discount or a low-stock alert. For instance, while a user asks about a phone's features, the server sends a promotion notice along with an alert that only three units remain, creating urgency. During a shopping festival, the server can also push exclusive coupons or popular product recommendations to drive sales.

Implementation

Send proactive messages from the server

Use the SendMessageChatText API to send a message to the client.

package com.aliyun.rtc;

import com.aliyun.tea.*;

public class SampleSendMessagetChatText {

    /**
     * <b>description</b> :
     * <p>Initialize the client using AccessKey ID and Secret.</p>
     * @return Client
     * 
     * @throws Exception
     */
    public static com.aliyun.ice20201109.Client createClient() throws Exception {
        // Important: Exposing credentials in code poses security risks. Use this code example as a reference only.
        com.aliyun.teaopenapi.models.Config config = new com.aliyun.teaopenapi.models.Config()
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_ID is configured.
                .setAccessKeyId("yourak")
                // Required. Make sure the environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET is configured.
                .setAccessKeySecret("yoursk");
        // The endpoint. For details, refer to https://api.alibabacloud.com/product/ICE
        config.endpoint = "ice.cn-shanghai.aliyuncs.com";
        return new com.aliyun.ice20201109.Client(config);
    }


    private static void sendMessageChatText() throws Exception {
        com.aliyun.ice20201109.Client client =  createClient();

        com.aliyun.ice20201109.models.SendMessageChatTextRequest request = new com.aliyun.ice20201109.models.SendMessageChatTextRequest()
            .setAIAgentId("youragentid")
            .setSessionId("yoursessionid")
            .setText("yourtext")
            .setReceiverId("receiverid")
            .setType("announcement") // announcement or custom
            .setNeedArchiving(true);
            try {
                com.aliyun.teautil.models.RuntimeOptions runtime = new com.aliyun.teautil.models.RuntimeOptions();
                
                client.sendMessageChatTextWithOptions(request, runtime);
            } catch (TeaException error) {
                System.out.println(error.getMessage());
                com.aliyun.teautil.Common.assertAsString(error.message);
            } catch (Exception _error) {
                TeaException error = new TeaException(_error.getMessage(), _error);
                System.out.println(error.getMessage());
                com.aliyun.teautil.Common.assertAsString(error.message);
            }   
    }
    
    public static void main(String[] args) throws Exception {
        sendMessageChatText();
    }
}

Handle proactive messages on the client

Implement a callback on the client side to receive and process custom messages.

Android

// Implement the onReceivedCustomMessage callback in ARTCAIChatEngine.IARTCAIChatEngineCallback.

 @Override
public void onReceivedCustomMessage(String  text) {
    // Handle the custom message received from the agent here.
}

iOS

// Create an instance of the chatbot engine.
let engine: ARTCAIChatEngineInterface = {
    return ARTCAICallEngineFactory.createChatEngine()
}()

// Set the delegate.
self.engine.delegate = self

// Implement the delegate method.
public func onReceivedCustomMessage(text: String) {
    // Handle the custom message here.
    debugPrint("onReceivedAgentCustomMessage:\(text)")
}

Web

engine.on('receivedCustomMessage', (text) => {
  // Handle the custom message here.
  console.log('receivedCustomMessage', text);
});