Agent callbacks - Intelligent Media Services - Alibaba Cloud Documentation Center

Use agent callbacks to automatically trigger preset actions or responses in your application when specific events occur.

Overview

When an AI agent triggers certain runtime events, Alibaba Cloud sends a callback request to your server. You can then add business logic to process the request.

Configure agent callbacks

Log on to the Intelligent Media Service console. Select the agent that you want to configure and click Manage in the Actions column.
On the Callback Configuration tab, enable agent callbacks, select the callback types, and enter the Callback URL and an optional Authentication Token.

Note
This token is sent in the Authorization field of the request header. Your server should verify this token to ensure request security.

Available callback types include: Agent status callbacks, Workflow status callbacks, Real-time chat history callbacks, Hang-up intent detection callbacks, Outbound call status callbacks, Inbound call status callbacks, Custom client message callbacks, and Instruction callbacks. If your callback URL supports both HTTP and HTTPS, HTTPS is strongly recommended for enhanced security.
Click OK to complete the callback configuration.

Callback payload fields

Parameter	Type	Required	Description	Example
aiAgentId	String	Yes	The agent ID.	xxxxx
instanceId	String	Yes	The unique ID of the agent instance.	39f8e0bc005e4f309379701645f4****
event	String	Yes	The event type. Agent status callback: agent_start: Triggered when the agent task starts. session_start: Triggered when the call session is established. agent_stop: Triggered when the agent task stops. error: Triggered when an error occurs. Workflow status callback: intent_detected: Triggered when the agent starts to detect user intent. intent_recognized: Triggered when the agent recognizes a more complete user intent. llm_data_received: Triggered when response data is received from the large language model (LLM). For a streamed response, this event is triggered upon receiving the first data packet. tts_data_received: Triggered when response data is received from the Text-to-Speech (TTS) service. For a streamed response, this event is triggered upon receiving the first data packet. Real-time chat record callback: chat_record: Provides a real-time transcript of the conversation. Audio recording callback: audio_record: User audio (role="user"): Triggered when the agent recognizes a more complete user intent. The payload contains the user's audio data and the corresponding speech-to-text (STT) result. Agent audio (role="agent"): Triggered when the agent finishes playing audio or when the user interrupts it. The payload contains the agent's audio data and the corresponding text. full_audio_record: Triggered after the call ends if full call recording is enabled. The payload contains a link to the complete, mixed-channel audio file. Call status callback: outbound_call: Triggered for status changes or hang-up events in an outbound call. inbound_call: Triggered for status changes or hang-up events in an inbound call. Callback for custom messages from the client: client_defined_data: Callback for custom messages sent by the client. It is recommended to include an identifier to distinguish the start and end of a message. Agent action instruction callback: instruction: Contains an action instruction for the agent to execute.	agent_start
data	Json	No	The data payload. The structure of this field depends on the event type.	chat_record Text message chat record: { 'requestId': 'abcd', 'code': 'Success', 'message': 'Success', 'dialogues': [ { 'roundId': 'xxxxxxx', 'producer': 'agent', 'text': '1+1=2', 'reasoningText': 'The user is asking what 1+1 is. It seems simple, but I need to think carefully.', 'time': 1739445458025, 'source': 'chat', 'dialogueId': 'xxxxxxxxxx', 'type': 'normal' }, { 'roundId': 'xxxxxxxxxxx', 'producer': 'user', 'text': 'Just answer, what is 1+1?', 'time': 1739445436218, 'source': 'chat', 'dialogueId': 'xxxxxxxxxxx', 'type': 'normal' } ] } `{ 'role': 'user', 'type': 'normal', 'text': 'Tell me a long story.', 'sentence_id': 1 }` audio_record `{ 'role': 'user', 'sentence_id': 1, 'start_timestamp': 1743151532.33012, 'text': 'Tell me a long story.', 'audio_url': '<file oss address>' }` full_audio_record `{ 'audio_url': '<file oss address>', 'start_timestamp': '2025-11-06T09:33:48.776253+00:00', 'end_timestamp': '2025-11-06T09:34:34.550809+00:00' }`
code	String	Yes	The callback event status code.	1001
message	String	Yes	The callback message.	User has been kicked from the room
timestamp	String	Yes	The time the event occurred, formatted as an ISO 8601 string (UTC).	2023-10-01T12:00:00Z
userData	String	No	User-defined information.
extendData	Json	No	Custom extension data.

Callback examples

Outbound call status

When the event is outbound_call, this callback reports the status of an outbound call. The following table describes the fields in the extendData object for this event.

Parameter	Type	Description
aiAgentId	String	The AI agent ID.
channelId	String	The channel ID.
instanceId	String	The unique ID of the AI agent instance.
callerNumber	String	The phone number of the caller (the AI agent).
calleeNumber	String	The phone number of the callee.
failReason	Int	The failure reason. This field is returned only when the outbound call fails.
status	Int	The current call status of the AI agent. Possible values are: `2`: The outbound call or call transfer failed. `3`: The outbound call or call transfer was successfully connected. `4`: The call was hung up.
callStartTime	String	The time the call was connected. This field is returned only upon hangup.
callEndTime	String	The time the call was hung up. This field is returned only upon hangup.
hangupRole	Int	The party that hung up the call. This field is returned only upon hangup. Possible values are: `0`: The caller (the AI agent). `1`: The callee. `2`: The party that received the transferred call.
forwardInfo	JSON	Information about the call transfer. This field is returned only for call transfer-related callbacks. It contains the following subfields: `callerNumber`: The phone number of the party initiating the transfer. Data type: String. `calleeNumber`: The phone number of the party receiving the transfer. Data type: String. `callStartTime`: The time the transferred call was connected. Data type: String. This field is returned only if the transfer is successful or a hangup occurs.

Outbound call failure

This callback indicates that an outbound call failed because the callee's number was invalid, the call was rejected, or the callee was unreachable.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10002,
  "message":"Dial status failed",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "failReason": -6,
      "status": 2
  }
}

Outbound call connected

This callback is sent when the callee answers an outbound call.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10003,
  "message":"Dial status connected",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 3
  }
}

Callee hangup

This callback is sent when the callee hangs up after an outbound call connects.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10004,
  "message":"Hangup",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 4,
      "callStartTime": "2023-10-01T12:00:00.135045+00:00",
      "callEndTime": "2023-10-01T12:01:00.135045+00:00",
      "hangupRole": 1
  }
}

Agent hangup

This callback is sent when the AI agent hangs up after an outbound call connects.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10004,
  "message":"Hangup",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 4,
      "callStartTime": "2023-10-01T12:00:00.135045+00:00",
      "callEndTime": "2023-10-01T12:01:00.135045+00:00",
      "hangupRole": 0
  }
}

Successful call transfer

This callback is sent when a call transfer connects successfully.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10006,
  "message":"Forward call connected",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 3,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX",
        "callStartTime": "2023-10-01T12:00:59Z"
      }
  }
}

Call transfer failed

This callback is sent when a call transfer fails.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10005,
  "message":"Forward call failed",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "failReason": 480,
      "status": 2,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX"
      }
  }
}

Hangup after call transfer

This callback is sent when the party who received the transfer hangs up.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"outbound_call",
  "code":10004,
  "message":"Hangup",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 4,
      "callStartTime": "2023-10-01T12:00:00.135045+00:00",
      "callEndTime": "2023-10-01T12:01:00.135045+00:00",
      "hangupRole": 2,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX",
        "callStartTime": "2023-10-01T12:00:59Z"
      }
  }
}

Inbound call status

When the event is inbound_call, this callback reports the status of an inbound call. The following table describes the fields in the extendData object for this event.

Parameter	Type	Description
aiAgentId	String	This value is the same as the top-level `aiAgentId` field in the callback.
channelId	String	The channel ID.
instanceId	String	This value is the same as the top-level `instanceId` field in the callback.
callerNumber	String	The phone number of the caller (the party making the inbound call).
calleeNumber	String	The phone number of the callee (the AI agent).
failReason	Int	The failure reason. This field is returned only when the inbound call fails.
status	Int	The current call status of the AI agent. Possible values are: `2`: The inbound call or call transfer failed. `3`: The inbound call or call transfer was successfully connected. `4`: The call was hung up.
callStartTime	String	The time the call was connected. This field is returned only upon hangup.
callEndTime	String	The time the call was hung up. This field is returned only upon hangup.
hangupRole	Int	The party that hung up the call. This field is returned only upon hangup. Possible values are: `0`: The callee (the AI agent). `1`: The caller (the party making the inbound call). `2`: The party that received the transferred call.
forwardInfo	JSON	Information about the call transfer. This field is returned only for call transfer-related callbacks. It contains the following subfields: `callerNumber`: The phone number of the party initiating the transfer. Data type: String. `calleeNumber`: The phone number of the party receiving the transfer. Data type: String. `callStartTime`: The time the transferred call was connected. Data type: String. This field is returned only if the transfer is successful or a hangup occurs.

Inbound call connected

This callback is sent when the AI agent successfully answers an inbound call.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10003,
  "message":"Dial status connected",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 3
  }
}

Inbound call failed

This callback is sent when the AI agent fails to answer an inbound call.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10002,
  "message":"Dial status failed",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "failReason": -6,
      "status": 2
  }
}

Inbound call hangup

This callback is sent when the AI agent hangs up after an inbound call connects.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10004,
  "message":"Hangup",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 4,
      "callStartTime": "2023-10-01T12:00:00.135045+00:00",
      "callEndTime": "2023-10-01T12:01:00.135045+00:00",
      "hangupRole": 0
  }
}

Successful call transfer

This callback is sent when a call transfer from an inbound call connects successfully.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10006,
  "message":"Forward call connected",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 3,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX",
        "callStartTime": "2023-10-01T12:00:59Z"
      }
  }
}

Call transfer failed

This callback is sent when a call transfer from an inbound call fails.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10005,
  "message":"Forward call failed",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "failReason": 480,
      "status": 2,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX"
      }
  }
}

Hangup after call transfer

This callback is sent when the party who received the transfer hangs up after a transfer from an inbound call.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"inbound_call",
  "code":10004,
  "message":"Hangup",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "aiAgentId": "0d31c************b3c787",
      "channelId": "XXX",
      "instanceId": "39f8e0bc005e4f309379*********",
      "callerNumber": "XXX",
      "calleeNumber": "XXX",
      "status": 4,
      "callStartTime": "2023-10-01T12:00:00.135045+00:00",
      "callEndTime": "2023-10-01T12:01:00.135045+00:00",
      "hangupRole": 2,
      "forwardInfo": {
        "callerNumber": "XXX",
        "calleeNumber": "XXX",
        "callStartTime": "2023-10-01T12:00:59Z"
      }
  }
}

Workflow status

For workflow status events, the extendData object contains the following fields:

Parameter	Type	Description
channelId	String	The channel ID.
sentenceId	Int	A unique ID for a conversational turn. Note The AI agent's responses to a single user query share the same `sentenceId`.
requestTimestamp	String	For the llm_data_received event, this is the timestamp when the request was sent to the LLM. For the tts_data_received event, this is the timestamp when the request was sent to the TTS service. For the intent_recognized event, this is the timestamp when the AI agent detected the end of the user's speech. If the agent has not yet determined the end of speech, this value is `None`.
responseTimestamp	String	For the llm_data_received event, this is the timestamp of the first response from the LLM. For the tts_data_received event, this is the timestamp of the first response from the TTS service. For the intent_recognized event, this is the timestamp when the ASR result was returned after the user finished speaking.

Instruction callback

When the event type is instruction, the callback indicates that a specific action instruction tag has been triggered. Supported instruction callbacks include:

Call transfer instruction callback

This callback is sent when the AI agent triggers a call transfer action tag.

{
  "aiAgentId":"0d31c************b3c787",
  "instanceId":"39f8e0bc005e4f309379*********",
  "event":"instruction",
  "code":11001,
  "message":"Forward call triggered",
  "timestamp":"2023-10-01T12:00:00Z",
  "extendData":{
      "triggerTime": "2023-10-01T12:00:00Z"
  }
}

Server example

Python

from aiohttp import web
import json
from loguru import logger
async def handle_post(request):
    """
    Handle POST requests and log the received data.
    """
    # Get the Authorization header from the request.
    authorization_header = request.headers.get('Authorization')
    if authorization_header is None or not authorization_header.startswith('Bearer fixed-token'):
        logger.error("Unauthorized request")
        return web.Response(status=401, text='Unauthorized')
    try:
        # Parse the request body as JSON.
        callback_data = await request.json()
        logger.info("Parsed JSON data:")
        logger.info(json.dumps(callback_data, indent=4))
        return web.Response(text='Callback received successfully', status=200)
    except json.JSONDecodeError:
        # Return an error if JSON parsing fails.
        return web.Response(text='Invalid JSON', status=400)
app = web.Application()
app.add_routes([web.post('/', handle_post)])
if __name__ == '__main__':
    web.run_app(app, host='localhost', port=8081)

Callback event status codes

Status code	Callback event	Description
1001	Agent starts	Agent starts.
1002	Agent stops	Agent stops.
1003	Session starts	Session starts.
4001	Concurrent agent routes exhausted	The maximum number of concurrent agent routes has been reached.
4002	Agent kicked from channel	The system kicked the agent from the channel.
4003	Invalid agent token	The agent's token is invalid.
4004	Agent stream subscription failed	The agent failed to subscribe to the stream.
4005	Third-party ASR failed	The third-party ASR service failed.
4006	Avatar service unavailable	The avatar service is unavailable.
8001	Intent recognized	Intent recognized.
8002	LLM data received	LLM data received.
8003	TTS data received	TTS data received.
10002	Dial status failed	The call failed to connect.
10003	Dial status connected	The call connected successfully.
10004	Hangup	The call was hung up.
10005	Forward call failed	The call forwarding attempt failed.
10006	Forward call connected	The forwarded call connected successfully.
11001	Forward call triggered	The system triggered a call forward.