Qwen-Audio-ASR-Message real-time speech recognition client events - Alibaba Cloud Model Studio

This topic describes the client events that the client sends to the server over WebSocket in the Qwen-Audio-3.1-ASR-Flash-Message real-time speech recognition service, including the data structures and field definitions for run-task (start a task), and finish-task (end a task).

run-task

Starts a speech recognition task and sets parameters such as the model, audio format, and sample rate.

When to send: Send immediately after the WebSocket connection is established.

Response event: You can send audio only after the server returns the task-started event.

headerobject(Required)

Properties

actionstring(Required)

The command type. Set to run-task.

task_idstring(Required)

A client-generated task ID (UUID format) that links subsequent events to this task.

streamingstring(Required)

Set to duplex.

payloadobject(Required)

Properties

task_groupstring(Required)

The task group. Set to audio.

taskstring(Required)

The task type. Set to asr.

functionstring(Required)

The function type. Set to recognition.

modelstring(Required)

The model name. Set to qwen-audio-3.1-asr-flash-message. See Qwen-Audio-3.1-ASR-Flash-Message.

inputobject(Required)

The input object. Pass {} when no context is provided.

Properties

contextarray(object)(Optional)

The conversation context, which improves recognition accuracy for domain-specific vocabulary. For usage details, see Context enhancement.

ImportantLimits: You can provide at most 5 context messages of each type (input_text and text). If you exceed this limit, only the 5 most recent messages are kept. The total text length per turn (the combined length of the text fields of the user and assistant messages) cannot exceed 400 characters (counted by character, where each character counts as 1). Text beyond this limit is truncated from the end.

ImportantWhen you provide context, the messages in context must follow a specific order. Context messages must be arranged by conversation turn, and within each turn the user message (of type input_text) must come before its corresponding assistant message (of type text).

Properties

rolestring(Required)

The message role. Valid values:

user: The recognition results from previous turns of user speech, or a domain-specific word list.
assistant: The responses from the large language model in previous turns.

contentarray(object)(Required)

The list of message contents.

Properties

typestring(Required)

The content type. Valid values:

input_text: The recognition results from previous turns of user speech, or a domain-specific word list (used when role is user). You must also provide the text field.
text: The responses from the large language model in previous turns (used when role is assistant). You must also provide the text field.

textstring(Required)

The text content. When type is input_text, enter the recognition results from previous turns of user speech or a domain-specific word list. When type is text, enter the responses from the large language model in previous turns.

parametersobject(Required)

The speech recognition parameters.

Properties

disfluency_removal_enabled boolean (Optional)

Whether to filter filler words and polish the output. Defaults to false. Set to true to enable this feature.

intermediate_result_enabled boolean (Optional)

Whether to return intermediate streaming results. Defaults to false. Set to true to return intermediate streaming results.

keep_dialect boolean (Optional)

Specifies whether to preserve dialect expressions. Default: false. Set to false to transcribe dialect speech as standard Mandarin text, or true to preserve dialect expressions.

vad_model string (Optional)

The VAD model. Default: far_field_meeting_16k.

near_meeting_16k: near-field scenarios.
far_field_meeting_16k: far-field scenarios.

formatstring(Required)

The audio format.

Valid values:

pcm
wav
mp3
opus
speex
aac
amr

Importantopus/speex: Must use Ogg encapsulation.

wav: Must use PCM encoding.

amr: Only the AMR-NB type is supported.

sample_rateinteger(Required)

The sample rate, in Hz.

Only 16000 Hz is supported.

vocabulary_idstring(Optional)

The ID of a precompiled hot word list.

Generate this ID in advance by calling the create hot word list API. Pass the ID during recognition to use the hot words in the list.

Suitable for scenarios where the vocabulary is known and relatively stable, and where you need to reuse the same word list across requests.

For usage details, see Precompiled hotwords.

vocabularyobject(Optional)

Instant hot words.

Passed as key-value pairs, where the key is the hot word text (string) and the value is the hot word weight (integer). No hot word list needs to be created in advance. The weight ranges from [1, 5] or is set to 50: a value in [1, 5] makes the model more likely to output the word as the value increases; a value of 50 designates a super hot word, which greatly improves recall, but the number of super hot words cannot exceed 50.

Suitable for temporary, session-level hot word optimization.

When instant and precompiled hotwords are configured together, the system merges both sets. If the merged set contains more than 2000 hotwords, the system randomly selects 2000 to use. For usage details, see Instant hotwords.

max_sentence_silenceinteger(Optional)

The VAD silence threshold for segmentation, in ms. When the silence after a segment of speech exceeds this threshold, the system determines that the sentence has ended.

Default value: 1300.

Valid values: [200, 6000].

heartbeatboolean(Optional)

Whether to enable heartbeat packets.

Default value: false.

true: Keeps the connection to the server alive even when silent audio is sent continuously.
false (default): Even when silent audio is continuously sent, the connection times out and closes after a period of time.

Silent audio refers to content in an audio file or data stream that contains no sound signal. You can generate silent audio in several ways, such as using audio editing software like Audacity or Adobe Audition, or using a command-line tool like FFmpeg.

speech_noise_thresholdfloat(Optional)

The threshold for distinguishing speech from noise, used to adjust the sensitivity of Voice Activity Detection (VAD).

Valid values: [-1.0, 1.0].

Value descriptions:

The closer the value is to -1: The noise threshold decreases, so noise is more likely to be recognized as speech, which may cause more noise to be transcribed.
The closer the value is to +1: The noise threshold increases, so speech is more likely to be misjudged as noise, which may cause some speech to be filtered out.

This is an advanced configuration parameter. Adjusting it can significantly affect recognition results. Recommendations:

Thoroughly test and verify the results before adjusting.
Adjust in small increments based on the actual audio environment (a step of 0.1 is recommended).

Basic request

{
    "header": {
        "action": "run-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "task_group": "audio",
        "task": "asr",
        "function": "recognition",
        "model": "qwen-audio-3.1-asr-flash-message",
        "parameters": {
            "format": "pcm",
            "sample_rate": 16000
        },
        "input": {}
    }
}

With context

{
    "header": {
        "action": "run-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "task_group": "audio",
        "task": "asr",
        "function": "recognition",
        "model": "qwen-audio-3.1-asr-flash-message",
        "parameters": {
            "format": "pcm",
            "sample_rate": 16000
        },
        "input": {
            "context": [
                {
                    "role": "user",
                    "content": [
                        {
                            "type": "input_text",
                            "text": "Hello there"
                        }
                    ]
                },
                {
                    "role": "assistant",
                    "content": [
                        {
                            "type": "text",
                            "text": "Hello, I am Qwen. How can I help you?"
                        }
                    ]
                }
            ]
        }
    }
}

Instant hot words

{
    "header": {
        "action": "run-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "task_group": "audio",
        "task": "asr",
        "function": "recognition",
        "model": "qwen-audio-3.1-asr-flash-message",
        "parameters": {
            "format": "pcm",
            "sample_rate": 16000,
            "vocabulary": {"John Smith": 5, "Jane Doe": 5}
        },
        "input": {}
    }
}

continue-task

Updates the conversation context during task execution to improve recognition.

When to send: Send during task execution when you need to update the conversation context.

headerobject(Required)

Properties

actionstring(Required)

The command type. Set to continue-task.

task_idstring(Required)

A client-generated task ID (UUID format) that must match the task_id in the run-task event.

streamingstring(Required)

Set to duplex.

payloadobject(Required)

Properties

inputobject(Required)

The input object.

Properties

context _array(object)___ (Optional) The conversation context, which improves recognition accuracy for domain-specific vocabulary. For usage details, see Improve recognition accuracy.

ImportantLimits: You can provide at most 5 context messages of each type (input_text and text). If you exceed this limit, only the 5 most recent messages are kept. The total text length per turn (the combined length of the text fields of the user and assistant messages) cannot exceed 400 characters (counted by character, where each character counts as 1). Text beyond this limit is truncated from the end.

Properties role _string_ __(Required) The message role. Valid values:

user: The recognition results from previous turns of user speech, or a domain-specific word list.

assistant: The responses from the large language model in previous turns.

content _array(object)___(Required) The list of message contents. Properties type _string_ __(Required) The content type. Valid values:

input_text: The recognition results from previous turns of user speech, or a domain-specific word list (used when role is user). You must also provide the text field.

text: The responses from the large language model in previous turns (used when role is assistant). You must also provide the text field.

text _string_ __(Required) The text content. When type is input_text, enter the recognition results from previous turns of user speech or a domain-specific word list. When type is text, enter the responses from the large language model in previous turns.

{
    "header": {
        "action": "continue-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "input": {
            "context": [
                {
                    "role": "user",
                    "content": [
                        {
                            "type": "input_text",
                            "text": "Hello there"
                        }
                    ]
                },
                {
                    "role": "assistant",
                    "content": [
                        {
                            "type": "text",
                            "text": "Hello, I am Qwen. How can I help you?"
                        }
                    ]
                }
            ]
        }
    }
}

finish-task

Notifies the server that all audio has been sent and requests to end the task.

When to send: Send after all audio data has been sent.

Response event: The server returns the task-finished event.

headerobject(Required)

Properties

actionstring(Required)

The command type. Set to finish-task.

task_idstring(Required)

A client-generated task ID (UUID format) that must match the task_id in the run-task event.

streamingstring(Required)

Set to duplex.

payloadobject(Required)

Properties

inputobject(Required)

Set to {}.

{
    "header": {
        "action": "finish-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "input": {}
    }
}