Pass user information to Model Studio or a self-developed model - Intelligent Media Services

This topic explains how to pass business parameters through to Alibaba Cloud Model Studio or your self-developed large language model (LLM).

Feature introduction

Parameter passthrough is a mechanism that allows you to send custom business data, such as user context, environment tags, user IDs, or membership information, along with requests in a service call chain. The primary goal is to provide the LLM with additional context to maintain business logic, improve response accuracy, and enhance observability, ensuring data integrity throughout the process.

Use cases

In AI agent applications, you can pass parameters to either applications created in Alibaba Cloud Model Studio or your self-developed model. For instance, in a customer service scenario, passing parameters such as the user's issue type, business segment, and past inquiry history helps the LLM understand the context and provide more accurate and relevant answers. This improves support efficiency and user satisfaction.

Example: In e-commerce support, passing information about a user's purchased product allows the LLM to quickly resolve issues related to product usage or after-sales service.

Implementation

Audio/video call agents

Pass to Model Studio

Server-side initiation: When you call the GenerateAIAgentCall or StartAIAgentInstance API, configure the AIAgentConfig parameter. For more information, see AIAgentConfig.
Client-side initiation: In the ARTCAICallConfig.agentConfig.llmConfig parameter, configure the BailianAppParams parameter to pass parameters to Alibaba Cloud Model Studio. For more information, see Model Studio parameter details.

Client-side sample code

Android

// Create and configure an ARTCAICallConfig object.
ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();
 
// Configure the bailianAppParams parameter of agentConfig. This is a JSON string.
artcaiCallConfig.agentConfig.llmConfig.bailianAppParams = "XXX";

// Omit the call initiation process.

iOS

// Configure the bailianAppParams parameter of agentConfig.
let agentConfig = ARTCAICallAgentConfig()         // Create an ARTCAICallAgentConfig object.
agentConfig.llmConfig.bailianAppParams = ["xxx", "xxxx"]
// Omit the call initiation process.
...

Web

// Configure the aiAgentBailianAppParams parameter of agentConfig.
const agentConfig = new AICallAgentConfig(); // Create an ARTCAICallAgentConfig object.
agentConfig.llmConfig.bailianAppParams = {
  // Configure the bailianAppParams parameter.
};

// Omit the call initiation process.

Pass to a self-developed model

Server-side initiation: When you call the GenerateAIAgentCall or StartAIAgentInstance API, configure the userData parameter.
Client-side initiation: When you start the agent, set the userData property.

For more information about how to retrieve the custom userData that you passed after the agent starts, see Access LLMs.

Client-side sample code

Android

ARTCAICallEngine.ARTCAICallConfig artcaiCallConfig = new ARTCAICallEngine.ARTCAICallConfig();  // Create an ARTCAICallConfig object. For more information, see the user guide. 
artcaiCallConfig.userData = "{\"xxx\": \"xxx\"}"  // Set business parameters for the model.
...                                               // Set other parameters as needed.
    
engine.init(artcaiCallConfig);                    // Initialize the engine.
engine.call(token);                               // Initiate a call.

iOS

let callConfig = ARTCAICallConfig()        // Create an ARTCAICallConfig object. For more information, see the user guide.
callConfig.userData = ["xxx": "xxxx"]      // Set business parameters for the model.
...                                        // Set other parameters as needed.
    
self.engine.call(config: callConfig)       // Initiate a call.

Web

const callConfig = {
  // Create an AICallConfig object. For more information, see the user guide.
};

callConfig.userData = JSON.stringify({xxx: 'xxx'})

 // Initiate a call.
engine.callWithConfig(callConfig);

Chatbot agents

Pass to Model Studio

Before you call the APIs provided by the AICallKit SDK to configure business parameters, ensure that the AICallKit SDK library is correctly imported.
ARTCAIChatTemplateConfig is a class used to create a template configuration for messaging conversations. The bailianParam parameter is a JSON string that contains the business parameters to pass to Alibaba Cloud Model Studio.
ARTCAIChatEngine is the engine class for managing messaging conversations. Use the setTemplateConfig method to apply the configured templateConfig to the engine. This ensures that the business parameters are passed during subsequent interactions with Alibaba Cloud Model Studio.

Sample code

Android

// Parameters for the Alibaba Cloud Model Studio application center. This is a JSON string.
String bailianParam = "XXX";
// Create the TemplateConfig parameter for the messaging conversation.
ARTCAIChatEngine.ARTCAIChatTemplateConfig templateConfig= new ARTCAIChatEngine.ARTCAIChatTemplateConfig(bailianParam, "");
// Set the configuration parameters for the messaging conversation. mChatEngine is an ARTCAIChatEngine object.               
mChatEngine.setTemplateConfig(templateConfig);

iOS

// Create the TemplateConfig parameter for the messaging conversation.
let templateConfig = ARTCAIChatTemplateConfig()
// Set parameters for the Alibaba Cloud Model Studio application center.
templateConfig.bailianAppParams = ["xxx":"xxxxx"]

// Set templateConfig for the messaging conversation engine.              
self.engine.templateConfig = templateConfig

Web

const engine = new AIChatEngine();

// Create the TemplateConfig parameter for the messaging conversation.
const templateConfig = new AIChatTemplateConfig();
// Set parameters for the Alibaba Cloud Model Studio application center.
templateConfig.bailianAppParams = {
  xxx: 'xxxxx',
};

// Set templateConfig for the messaging conversation engine.
engine.templateConfig = templateConfig;

Pass to a self-developed large model

To pass business parameters to a self-developed large model, call the setUserData method of the AICallKit SDK. Before you call this method, ensure that the SDK library is correctly imported.
The userData parameter is a JSON string that contains the business parameters to pass to a self-developed model.
Use the mChatEngine.setUserData() method to set the business parameters in the engine.

Sample code

Android

// Custom business parameters (as a JSON string).
String userData = "XXX";
// Set the parameters on the engine (mChatEngine).
mChatEngine.setUserData(userData);

iOS

// Custom business parameters.
let userData = ["xxx":"xxxxx"]

// Set the userData on the chat engine.      
self.engine.userData = userData

Web

// Custom business parameters.
const userData = { xxx: 'xxxxx' };

// Set the userData on the chat engine.
engine.userData = userData;

Model Studio parameter details

Parameter	Type	Description	Example
biz_params	Object	The custom parameters passed to a workflow application or agent orchestration application. For details, see Pass-through of application parameters Note Alibaba Cloud passes the following agent-related parameters. You do not need to configure them. instanceId: the ID of the agent instance. channelId: the ID of the RTC channel. When passing parameters to an Alibaba Cloud Model Studio plugin, use `user_defined_params`. This parameter is not required for Model Studio workflows.	Alibaba Cloud Model Studio plugin `{ "user_defined_params": { "city":"Hangzhou" } }` Alibaba Cloud Model Studio workflow `{ "city":"Hangzhou" }`
memory_id	String	A unique identifier for long-term memory. This parameter enables efficient preservation and recall of historical information.	YOUR_MEMORY_ID
image_list	Array	The URLs to associated images that provide visual information related to the current operation.	["https://example.com/images/example.jpg"]
rag_options	Object	Parameters related to Retrieval-Augmented Generation (RAG). This includes but is not limited to retrieving specified knowledge bases or documents.
pipeline_ids	Array	The knowledge base ID. When specified, all documents within the specified knowledge base will be retrieved.	["KnowledgeBaseID1", "KnowledgeBaseID2"]
file_ids	Array	The ID of the unstructured document to be processed.	["DocumentID1", "DocumentID2"]
metadata_filter	Object	The metadata of unstructured documents. When specified, documents that contain the metadata are retrieved.	{ "name": "Zhang San" }
structured_filter	Object	The column names and the corresponding values that are used to filter structured documents.	{ "key1": "value1", "key2": "value2" }
tags	Array	The tags associated with unstructured documents. When specified, unstructured documents that contain the tags are retrieved.	["Tag1", "Tag2"]

For more information, see Application API. Examples:

Pass to an Alibaba Cloud Model Studio plugin

aliyunBailianParamsJson = {
    "biz_params": {
        "user_defined_params": 
        {
            "your_plugin_id":
            {
                "article_index": 2
            }
        }
    },
    "memory_id": "your_memory_id",
    "image_list": [
        "https://your_image_url"  # Example image URL
    ],
    "rag_options": {
        "pipeline_ids": [
            "your_id",  # Replace with the actual knowledge base ID
        ],
        "file_ids": [
            "DocumentID1",  # Replace with the actual document ID
            "DocumentID2"   # Replace with the actual document ID
        ],
        "metadata_filter": {
            "name": "John Doe"  # Replace with the required metadata filter condition
        },
        "structured_filter": {
            "key1": "value1",  # Example filter condition
            "key2": "value2"   # Example filter condition
        },
        "tags": [
            "Tag1",  # Replace with actual tags
            "Tag2"   # Replace with actual tags
        ]
    }
}
bailianParams = json.dumps(aliyunBailianParamsJson)

Pass to an Alibaba Cloud Model Studio workflow


aliyunBailianParamsJson = {
    "biz_params": {
        "key1": "value1"
    },
    "memory_id": "your_memory_id",
    "image_list": [
        "https://your_image_url"  # Example image URL
    ],
    "rag_options": {
        "pipeline_ids": [
            "your_id",  # Replace with the actual knowledge base ID
        ],
        "file_ids": [
            "DocumentID1",  # Replace with the actual document ID
            "DocumentID2"  # Replace with the actual document ID
        ],
        "metadata_filter": {
            "name": "John Doe"  # Replace with the required metadata filter condition
        },
        "structured_filter": {
            "key1": "value1",  # Example filter condition
            "key2": "value2"  # Example filter condition
        },
        "tags": [
            "Tag1",  # Replace with actual tags
            "Tag2"  # Replace with actual tags
        ]
    }
}
bailianParams = json.dumps(aliyunBailianParamsJson)