Qwen-Audio-ASR-Message real-time speech recognition server-side events - Alibaba Cloud Model Studio

The Qwen-Audio-3.1-ASR-Flash-Message real-time speech recognition service pushes server-side events to the client over WebSocket. This topic describes the data structures and field descriptions of the four event types: task-started, result-generated, task-finished, and task-failed.

task-started

The task has started successfully. The client can begin sending audio data.

headerobject

Properties

task_idstring

The task ID generated by the client (in UUID format).

eventstring

The event type. This value is always task-started.

attributesobject

Additional attributes. This value is usually empty.

payloadobject

This value is always {}.

{
    "header": {
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "event": "task-started",
        "attributes": {}
    },
    "payload": {}
}

result-generated

Recognition results. By default, final sentence text is returned (sentence_end=true). Set intermediate_result_enabled=true to also receive intermediate text (sentence_end=false). Intermediate text may be revised: update the displayed text by sentence_id instead of concatenating it. A sentence-start event may contain sentence_begin=true and empty text.

headerobject

Properties

task_idstring

The task ID generated by the client (in UUID format).

eventstring

The event type. This value is always result-generated.

payloadobject

Properties

outputobject

Properties

sentenceobject

Properties

begin_timeinteger

The start time of the sentence, in milliseconds.

end_timeinteger

The end time of the sentence, in milliseconds.

textstring

The recognized text.

heartbeatboolean

If this value is true, the result is a heartbeat packet and can be ignored.

sentence_beginboolean

Indicates the start of a sentence.

sentence_endboolean

Whether the sentence has ended (true = final result, false = intermediate result).

sentence_idinteger

The sequence identifier of the sentence. In normal recognition results, sentence_id increments from 1. When heartbeat is true (that is, a heartbeat packet), sentence_id is always 0.

wordsarray[object]

Word-level timestamp information.

Properties

begin_timeinteger

The start time of the word, in milliseconds.

end_timeinteger

The end time of the word, in milliseconds.

textstring

The recognized text.

punctuationstring

The punctuation mark.

usageobject

Cumulative usage for the current task as of this event.

Properties

durationinteger

Cumulative audio duration in seconds, retained for compatibility.

input_tokensinteger

Cumulative number of input tokens.

output_tokensinteger

Cumulative number of output tokens.

total_tokensinteger

Cumulative total number of tokens: the sum of input_tokens and output_tokens.

Sentence-start result:

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "result-generated",
    "attributes": {}
  },
  "payload": {
    "output": {
      "sentence": {
        "sentence_id": 1,
        "begin_time": 0,
        "end_time": null,
        "text": "",
        "channel_id": 0,
        "speaker_id": null,
        "sentence_end": false,
        "sentence_begin": true,
        "words": []
      }
    }
  }
}

Final result:

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "result-generated",
    "attributes": {}
  },
  "payload": {
    "output": {
      "sentence": {
        "sentence_id": 1,
        "begin_time": 120,
        "end_time": 1700,
        "text": "欢迎使用阿里云。",
        "channel_id": 0,
        "speaker_id": null,
        "sentence_end": true,
        "words": [
          {
            "begin_time": 120,
            "end_time": 520,
            "text": "欢迎",
            "punctuation": "",
            "fixed": false,
            "speaker_id": null
          },
          {
            "begin_time": 520,
            "end_time": 880,
            "text": "使用",
            "punctuation": "",
            "fixed": false,
            "speaker_id": null
          },
          {
            "begin_time": 880,
            "end_time": 1700,
            "text": "阿里云",
            "punctuation": "。",
            "fixed": false,
            "speaker_id": null
          }
        ],
        "stash": {
          "sentence_id": 2,
          "text": "",
          "begin_time": 1700,
          "current_time": 1700,
          "words": []
        }
      }
    },
    "usage": {
      "duration": 2,
      "input_tokens": 84,
      "output_tokens": 4,
      "total_tokens": 88
    }
  }
}

task-finished

The task has ended normally. You can close the connection or reuse it.

headerobject

Properties

task_idstring

The task ID generated by the client (in UUID format).

eventstring

The event type. This value is always task-finished.

attributesobject

Additional attributes. This value is usually empty.

payloadobject

Contains output (an empty object) and usage (the final cumulative usage for this task). The usage fields have the same definitions as in result-generated.

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "task-finished",
    "attributes": {}
  },
  "payload": {
    "output": {},
    "usage": {
      "duration": 2,
      "input_tokens": 84,
      "output_tokens": 4,
      "total_tokens": 88
    }
  }
}

task-failed

The task has failed. The connection is closed and cannot be reused.

headerobject

Properties

task_idstring

The task ID generated by the client (in UUID format).

eventstring

The event type. This value is always task-failed.

error_codestring

A description of the error type.

error_messagestring

The specific cause of the error.

attributesobject

Additional attributes. This value is usually empty.

payloadobject

This value is always {}.

{
    "header": {
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "event": "task-failed",
        "error_code": "CLIENT_ERROR",
        "error_message": "request timeout after 23 seconds.",
        "attributes": {}
    },
    "payload": {}
}