Client event reference for the Qwen-Audio Realtime API.
User guide: Realtime Audio Chat (Qwen-Audio-Realtime). For event interaction sequences, see WebSocket API.
session.update
Description: After a connection is established, send this event to update the default session configuration. Include only the fields you want to change; omitted fields retain their current values. If any parameter is invalid, the server returns an error. If all parameters are valid, the server applies the changes and returns the full configuration.
Note
turn_detection can only be modified before the first audio is sent (IDLE state).
|
type string (required)
Event type. Fixed value: session.update.
|
{
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"voice": "longanqian",
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
}
}
}
Function Calling: {
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"voice": "longanqian",
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for a specified city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"title": "City"
}
},
"required": ["city"]
}
}
}
],
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
}
}
}
Voiceprint registration (voiceprint_audio_urls): {
"type": "session.update",
"session": {
"turn_detection": {
"type": "smart_turn",
"voiceprint_audio_urls": [
"https://example.com/speaker1.pcm",
"https://example.com/speaker2.wav"
]
}
}
}
|
|
session object (optional)
Session configuration.
Properties
modalities array (optional)
Output modalities for the model. Valid values:
voice string (optional)
TTS voice name. Default: longanqian. Two types are supported. Can only be set in the first session.update; ignored in subsequent calls.
System voices: Available values: longanqian, longanlingxin, longanlingxi, longanxiaoxin, longanlufeng. Cloned voices: created via the Voice Cloning API. Pass the returned voice_id as this parameter's value. For details, see Voice configuration.
instructions string (optional)
System instructions that define the model's role, response style, and behavioral preferences. Applies to the entire session.
input_audio_format string (optional)
Input audio format. Currently only pcm (16 kHz, 16-bit, mono) is supported and is the default. Can only be modified before the first audio is sent (IDLE state).
output_audio_format string (optional)
Output audio format. Currently only pcm (24 kHz, 16-bit, mono) is supported and is the default.
max_history_turns integer (optional)
Maximum number of conversation turns (question-answer pairs) included in a single request. Valid values: 1 to 50. Default: 20.
tools array (optional)
Tool definitions for Function Calling. Once configured, the model decides whether to call a tool based on user input.
Properties
type string (required)
Fixed value: function.
function.name string (required)
Name of the tool function.
function.description string (optional)
Description of the tool function. The model uses this to determine whether to call the tool.
function.parameters object (optional)
Description of the tool function's input parameters. The model uses this to extract the required parameters. Omit this field if the function takes no parameters.
Properties
type string (required)
Fixed value: object.
properties object (optional)
Describes each parameter's name, data type, and description.
required array (optional)
Specifies which parameters are required.
turn_detection object|null (optional)
Turn detection configuration. To switch to push-to-talk mode, set this field to null. In push-to-talk mode, audio must be committed manually and inference triggered manually. If this field isn't provided, VAD is enabled with its default parameters.
Properties
type string (optional)
VAD type. Valid values:
-
server_vad (default): Detects speech onset and offset based on acoustic features and automatically triggers inference.
-
smart_turn: An intelligent turn detection mode that combines acoustic perception and semantic understanding to determine turn boundaries. Filler sounds such as "um" or "uh" don't trigger a new turn or interrupt model playback.
threshold float (optional)
VAD sensitivity. Only effective in server_vad mode (ignored in smart_turn mode). Lower values increase VAD sensitivity, making it easier to detect faint sounds (including background noise) as speech. Higher values decrease sensitivity, requiring clearer and louder speech to trigger detection.
Range: [-1.0, 1.0]. Default: 0.5.
silence_duration_ms integer (optional)
Minimum silence duration (in milliseconds) after speech ends before triggering a model response. Only effective in server_vad mode (ignored in smart_turn mode). Lower values produce faster responses but may cause false triggers during brief pauses.
Range: [200, 6000]. Default: 800. Recommended range for conversations: 400-800.
voiceprint_audio_urls array (optional)
Only effective in smart_turn mode. A list of publicly accessible URLs pointing to pre-recorded audio samples from the target user, used for speaker enhancement. Once registered, the model locks onto the target speaker during duplex conversations, effectively ignoring other voices and background noise. Maximum 5 URLs. Audio format requirements: 16kHz PCM or WAV.
Important
This parameter can only be configured in the first session.update event. It is ignored in subsequent session.update events.
|
input_audio_buffer.append
Description: Appends audio data to the input buffer. Send this event continuously at a high frequency — for example, one chunk every 20–40 ms. The server sends no acknowledgment for this event.
|
type string (required)
Event type. Fixed value: input_audio_buffer.append.
|
{
"type": "input_audio_buffer.append",
"audio": "<Base64-encoded audio data>"
}
|
|
audio string (required)
Base64-encoded audio data.
|
input_audio_buffer.commit
Description: Push-to-talk mode only. Commits the buffered audio as a user message. This doesn't automatically trigger inference. Send response.create to trigger inference manually.
This event is ignored in server_vad and smart_turn modes.
|
type string (required)
Event type. Fixed value: input_audio_buffer.commit.
|
{
"type": "input_audio_buffer.commit"
}
|
input_audio_buffer.clear
Description: Push-to-talk mode only. Clears uncommitted audio from the buffer. This event is ignored in server_vad and smart_turn modes. The server responds with an input_audio_buffer.cleared event.
|
type string (required)
Event type. Fixed value: input_audio_buffer.clear.
|
{
"type": "input_audio_buffer.clear"
}
|
conversation.item.create
Description: Inserts a conversation item into the conversation context. Use this event to inject historical context or add text content, or to return Function Calling results.
Note
If item.id already exists in the conversation, an error is returned and the item isn't created.
|
type string (required)
Event type. Fixed value: conversation.item.create.
|
Inject a user text message: {
"type": "conversation.item.create",
"previous_item_id": "item_xxx",
"item": {
"id": "my_item_001",
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "Please summarize our last conversation"
}
]
}
}
Return a Function Calling result: {
"type": "conversation.item.create",
"item": {
"type": "function_call_output",
"call_id": "call_xxx",
"output": "{\"temperature\":18,\"condition\":\"sunny\"}"
}
}
|
|
previous_item_id string (optional)
Specifies the conversation item after which the new item is inserted. If not provided, the item is appended to the end of the conversation.
|
|
item object (required)
The conversation item to create.
Properties
id string (optional)
Unique identifier for the conversation item. If not provided, the server generates one automatically. An error is returned if the specified ID already exists in the conversation.
type string (required)
Conversation item type. Valid values:
-
message: A regular conversation message.
-
function_call: A function call request. Typically generated by the server, but clients can also use it to inject historical context.
-
function_call_output: A tool execution result. After receiving a function_call, the client executes the tool and returns the result using this type.
role string (required for message type)
Message role. Valid values: system, user, assistant.
content array (required for message type)
List of message content elements. Each element contains a type field and corresponding data fields.
Supported content types by role
system
input_text: System message. Required field: text.
assistant
output_text: Assistant text output. Required field: text.
call_id string (required for function_call / function_call_output type)
Unique identifier for the function call, used to correlate requests and results.
name string (required for function_call type)
Name of the function to call.
arguments string (required for function_call type)
Function call parameters in JSON string format.
output string (required for function_call_output type)
Tool execution result in JSON string format.
|
conversation.item.retrieve
Description: Retrieves a conversation item stored on the server. Audio-type content in the response contains only the transcript (transcript), not the original audio data.
|
type string (required)
Event type. Fixed value: conversation.item.retrieve.
|
{
"type": "conversation.item.retrieve",
"item_id": "item_xxx"
}
|
|
item_id string (required)
ID of the conversation item to retrieve. The server returns the result in a conversation.item.retrieved event.
|
conversation.item.delete
Description: Deletes a conversation item from the conversation context. The server confirms the deletion with a conversation.item.deleted event.
|
type string (required)
Event type. Fixed value: conversation.item.delete.
|
{
"type": "conversation.item.delete",
"item_id": "item_xxx"
}
|
|
item_id string (required)
ID of the conversation item to delete.
|
response.create
Description: Triggers model inference. Behavior varies by mode:
-
Push-to-talk mode: Must be called manually. Commit the buffered audio with input_audio_buffer.commit first, or return a function_call_output result before triggering. Can't be called while a response is being generated.
-
server_vad mode: Typically triggered automatically by the server. Clients can also call it manually when no response is being generated. Can't be called while a response is being generated.
-
smart_turn mode: Can be called while waiting for the next user turn. Can't be called during an active turn (between input_audio_buffer.speech_started and response.done).
The optional response field overrides the session defaults for the current inference round. In Function Calling scenarios, after the client returns a function_call_output, this event triggers the second inference round.
Note
In server_vad and smart_turn modes, manually triggered inferences can still be interrupted by new speech.
|
type string (required)
Event type. Fixed value: response.create.
|
{
"type": "response.create",
"response": {
"modalities": ["audio", "text"]
}
}
|
|
response object (optional)
Overrides the session defaults for the current inference round. If not provided, the current session configuration is used.
Properties
modalities array (optional)
Overrides the output modalities for the current round. Valid values are the same as session.update modalities.
voice string (optional)
Overrides the TTS voice for the current round.
|
response.cancel
Description: Cancels the current inference. Any text generated so far is saved to the item list. The server then returns a response.done event with status=cancelled.
An error is returned if no inference is in progress.
|
type string (required)
Event type. Fixed value: response.cancel.
|
{
"type": "response.cancel"
}
|