Agent callbacks - Intelligent Media Services - Alibaba Cloud Documentation Center

Agent callbacks enable your application automatically trigger preset actions or responses when specific events occur. This topic explains how to use them.

Overview

When an AI agent is running and a specific event occurs, Alibaba Cloud automatically sends a request to your server, enabling you to process the event using your custom business logic.

Configure agent callbacks

Go to the AI Agents page in the console, find the agent to configure, and click Manage in the Actions column.
On the Callback Configurations tab, click Enable to configure agent callbacks. Select the event types and set the Callback URL and an optional Authentication Token.
Note
The token is placed in the Authorization header for authentication. Your server should verify this token to ensure the request is from a trusted source.
Click OK to save the configuration.

Callback payload fields

Name	Type	Required	Description	Example
aiAgentId	String	Yes	The agent ID.	xxxxx
instanceId	String	Yes	The unique ID of the agent instance.	39f8e0bc005e4f309379701645f4****
event	String	Yes	The event type. Agent status callback: agent_start: The agent task starts. session_start: The call session is established. agent_stop: The agent task stops. error: An error occurred. Workflow status callback: intent_detected: The agent starts to detect user intent. intent_recognized: A complete user intent is recognized. llm_data_received: The first packet of a streamed response is received from the large language model (LLM). tts_data_received: The first packet of a streamed response is received from the Text-to-Speech (TTS) service. Real-time chat record callback: chat_record: Real-time chat record. Audio recording callback: audio_record: User audio (role is user): Sent when a complete user intent is recognized. The content includes the user's audio data and the corresponding STT result. Agent audio (role is agent): Sent when the agent finishes speaking or is interrupted by the user. The content includes the agent's audio data and the corresponding text. full_audio_record: If full call recording is enabled, the entire call audio is mixed and recorded into a single audio file. Client-side proactive message callback client_defined_data: Callback for custom messages sent by the client. We recommend including identifiers to distinguish the start and end of a message. Instruction callback: instruction: Callback for agent action instructions.	agent_start
data	Json	No	The data payload, such as chat records.	chat_record Text message chat record: { 'requestId': 'abcd', 'code': 'Success', 'message': 'Success', 'dialogues': [ { 'roundId': 'xxxxxxx', 'producer': 'agent', 'text': '1+1=2', 'reasoningText': 'Okay, the user is asking what 1 plus 1 equals. It seems simple, but I need to think about it carefully.', 'time': 1739445458025, 'source': 'chat', 'dialogueId': 'xxxxxxxxxx', 'type': 'normal' }, { 'roundId': 'xxxxxxxxxxx', 'producer': 'user', 'text': 'Just answer 1+1=?', 'time': 1739445436218, 'source': 'chat', 'dialogueId': 'xxxxxxxxxxx', 'type': 'normal' } ] } Audio/video chat record: `{ 'role': 'user', 'type': 'normal', 'text': 'Tell a longer story', 'sentence_id': 1 }` audio_record `{ 'role': 'user', 'sentence_id': 1, 'start_timestamp': 1743151532.33012, 'text': 'Tell a longer story', 'audio_url': '<file oss address>' }` full_audio_record `{ 'audio_url': '<file oss address>'', 'start_timestamp': '2025-11-06T09:33:48.776253+00:00', 'end_timestamp': '2025-11-06T09:34:34.550809+00:00' }` video_frame `{ 'sentence_id': 1, 'start_timestamp': 1743151532.33012, 'text': 'What am I holding in my hand?', 'url_list': [<file oss address>, <file oss address>] }`
code	String	Yes	The status code for the callback event. For details, see Callback event status code.	1001
message	String	Yes	The callback message.	User has been kicked from the room
timestamp	String	Yes	The timestamp for when the callback event occurred.	2023-10-01T12:00:00Z
userData	String	No	User-defined information.
extendData	Json	No	Custom extended information.

Callback examples

Workflow status callback

For workflow status events, the extendData fields are as follows:

Name	Type	Description
channelId	String	The channel ID.
sentenceId	Int	A unique ID for a conversational turn. Note Multiple agent responses to a single user question share the same sentenceId.
requestTimestamp	String	When event is llm_data_received or tts_data_received, this is the timestamp when the request was sent to the LLM or the TTS service. When event is intent_recognized, this is the timestamp when the agent detected the end of the user's speech. This value is `None` if the agent determines that the end-of-speech condition has not been met.
responseTimestamp	String	When event is llm_data_received or tts_data_received, this is the timestamp of the first response packet from the LLM or the TTS service. When event is intent_recognized, this is the timestamp when the ASR result was obtained after the user finished speaking.

Server example

Python

from aiohttp import web
import json
from loguru import logger

async def handle_post(request):
    """
    Handles POST requests and logs the received data.
    """
    # Get the Authorization header from the request.
    authorization_header = request.headers.get('Authorization')
    if authorization_header is None or not authorization_header.startswith('Bearer fixed-token'):
        logger.error("Unauthorized request")
        return web.Response(status=401, text='Unauthorized')

    try:
        # Get data from the request body.
        callback_data = await request.json()

        logger.info("Parsed JSON data:")
        logger.info(json.dumps(callback_data, indent=4))

        return web.Response(text='Callback received successfully', status=200)
    except json.JSONDecodeError:
        # If JSON parsing fails, return an error response.
        return web.Response(text='Invalid JSON', status=400)

app = web.Application()
app.add_routes([web.post('/', handle_post)])

if __name__ == '__main__':
    web.run_app(app, host='localhost', port=8081)

Callback event status codes

Status code	Callback event	Callback message
1001	Agent starts	AI Agent starts.
1002	Agent stops	AI Agent ends.
1003	Session starts	Session starts
4001	Concurrent agent routes exhausted	Concurrent routes exhausted
4002	Agent kicked from the channel	User has been kicked from the room
4003	Invalid agent token	Invalid token for the AI agent
4004	Agent stream subscription failed	Failed to pull stream for the AI agent
4005	Third-party ASR failed	error description
4006	Avatar service is unavailable	Avatar service is not available
8001	Intent recognized	Intent recognized event
8002	LLM data received	LLM data received event
8003	TTS data received	TTS data received event