Send text input to an LLM via API - Intelligent Media Services

This topic describes how to send text input to a large language model (LLM) via API.

Function introduction

During a conversation between a user and an AI agent, you can send text content to the LLM via an API call. This enhances the agent's functionality and flexibility, allowing it to dynamically adjust its responses based on the latest real-time input.

Use case

In a vision call, you might need to send both the phone's camera feed and corresponding text to a multimodal LLM. To handle cases where there is no audio input from the phone, the system automatically generates a default text string to accompany the image data. This feature allows you to override that default text with your own custom content.

Implementation

In Real-time Conversational AI, you can call an API from either the client-side or the server-side to send text input to the LLM.

Sample code

Client-side

Use the following platform-specific code snippets to send text to the LLM via the client-side API.

Android

// Create a request object containing the text message for the agent.
ARTCAICallSendTextToAgentRequest req = ARTCAICallSendTextToAgentRequest("xxx");
// Send the text to the agent. Note that this must be called after the connection is established. engine is the ARTCAICallEngine instance.
engine.sendTextToAgent(req)

iOS

// Create a request object containing the text message for the agent.
let req = ARTCAICallSendTextToAgentRequest(text: "xxx")
// Send the text to the agent. Note that this must be called after the connection is established.
_ = self.engine.sendTextToAgent(req: req)

Web

// Send the text to the agent. Note that this must be called after the connection is established. engine is the ARTCAICallEngine instance.
engine.sendTextToAgent(new AICallSendTextToAgentRequest('xxx'));

Server-side

On the server-side, call the SendAIAgentText API to send text input to the LLM.

// This file is auto-generated, don't edit it. Thanks.
package com.aliyun.rtc;

import com.aliyun.tea.*;

public class Sample {

    /**
     * <b>description</b> :
     * <p>Use your AccessKey pair to initialize the client.</p>
     * @return Client
     * 
     * @throws Exception
     */
    public static com.aliyun.ice20201109.Client createClient() throws Exception {
        // A leaked AccessKey pair can expose your account and all resources within the account to serious risks. The following sample code is provided for reference only.
        com.aliyun.teaopenapi.models.Config config = new com.aliyun.teaopenapi.models.Config()
                // Required. Make sure that the ALIBABA_CLOUD_ACCESS_KEY_ID environment variable is configured.
                .setAccessKeyId("yourak")
                // Required. Make sure that the ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variable is configured.
                .setAccessKeySecret("yoursk");
        // Refer to https://api.alibabacloud.com/product/ICE
        config.endpoint = "ice.cn-shanghai.aliyuncs.com";
        return new com.aliyun.ice20201109.Client(config);
    }

    private static void sendAIAgentText() throws Exception {
        com.aliyun.ice20201109.Client client =  createClient();

        com.aliyun.ice20201109.models.SendAIAgentTextRequest request = new com.aliyun.ice20201109.models.SendAIAgentTextRequest()
            .setInstanceId("yourinstanceid")
            .setText("yourtext");
            try {
                // When running this code, you should print the API response to see the result.
                com.aliyun.teautil.models.RuntimeOptions runtime = new com.aliyun.teautil.models.RuntimeOptions();
                
                client.sendAIAgentTextWithOptions(request, runtime);
            } catch (TeaException error) {
                // Handle exceptions with caution in actual business scenarios, and do not ignore exceptions in your project. In this example, exceptions are  printed for demonstration.
                // Obtain an error message.
                System.out.println(error.getMessage());
                // The URL for troubleshooting.
                com.aliyun.teautil.Common.assertAsString(error.message);
            } catch (Exception _error) {
                TeaException error = new TeaException(_error.getMessage(), _error);
                // Handle exceptions with caution in actual business scenarios, and do not ignore exceptions in your project. In this example, exceptions are  printed for demonstration.
                // Obtain an error message.
                System.out.println(error.getMessage());
                // The URL for troubleshooting.
                com.aliyun.teautil.Common.assertAsString(error.message);
            }   
    }

    public static void main(String[] args_) throws Exception {
        sendAIAgentText();
    }
}