Qwen-Audio-TTS/CosyVoice server events - Alibaba Cloud Model Studio

User guide: For model introduction and selection recommendations, see Speech synthesis.

task-started

After the client sends the run-task command, the server returns a task-started event to signal that the task has started. The client can send subsequent commands only after receiving this event.

header.task_idstring

The task ID generated by the client.

header.eventstring

Event type. Fixed value: task-started.

payloadobject

Empty object.

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "task-started",
    "attributes": {}
  },
  "payload": {}
}

result-generated

After the client sends text, the server continuously returns result-generated events. Each event carries sentence-level metadata.

header.task_idstring

The task ID generated by the client.

header.eventstring

Event type. Fixed value: result-generated.

payload.outputobject

Output information.

Properties

typestring

Sub-event type. Valid values:

sentence-begin: Sentence start. Returns the text to be synthesized.
sentence-synthesis: Marks an audio frame. One audio frame is transmitted over the WebSocket binary channel immediately after each event.
sentence-end: Sentence end. Returns the text content and cumulative character count.

sentence.indexinteger

Sentence index, starting from 0.

sentence.wordsarray

Word-level timestamp array.

words element properties

textstring

Text content of the word.

begin_indexinteger

Start character index of the word within the sentence. Starts at 0.

end_indexinteger

End character index of the word within the sentence. Starts at 1.

begin_timeinteger

Start time of the word's corresponding audio, in milliseconds.

end_timeinteger

End time of the word's corresponding audio, in milliseconds.

original_textstring

Text of the sentence as segmented for synthesis.

payload.usageobject

Billing information, returned in the sentence-end event.

Properties

charactersinteger

Cumulative number of billed characters.

sentence-begin

{
  "header": {
    "task_id": "3f2d5c86-0550-45c0-801f-xxxxxxxxxx",
    "event": "result-generated",
    "attributes": {}
  },
  "payload": {
    "output": {
      "sentence": {
        "index": 0,
        "words": []
      },
      "type": "sentence-begin",
      "original_text": "Before my bed, moonlight shines bright,"
    }
  }
}

sentence-synthesis

{
    "header": {
        "task_id": "3f2d5c86-0550-45c0-801f-xxxxxxxxxx",
        "event": "result-generated",
        "attributes": {}
    },
    "payload": {
        "output": {
            "sentence": {
                "index": 0,
                "words": []
            },
            "type": "sentence-synthesis"
        }
    }
}

sentence-end

{
  "header": {
    "task_id": "3f2d5c86-0550-45c0-801f-xxxxxxxxxx",
    "event": "result-generated",
    "attributes": {}
  },
  "payload": {
    "output": {
      "sentence": {
        "index": 0,
        "words": [
          {
            "text": "Before",
            "begin_index": 0,
            "end_index": 1,
            "begin_time": 0,
            "end_time": 263
          }
        ]
      },
      "type": "sentence-end",
      "original_text": "Before my bed, moonlight shines bright,"
    },
    "usage": {
      "characters": 6
    }
  }
}

task-finished

The server returns a task-finished event when the task completes. The client can then close the WebSocket connection or reuse it to start a new task.

header.task_idstring

The task ID generated by the client.

header.eventstring

Event type. Fixed value: task-finished.

payload.usage.charactersinteger

Cumulative number of billed characters.

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "task-finished",
    "attributes": {
      "request_uuid": "0a9dba9e-d3a6-45a4-be6d-xxxxxxxxxxxx"
    }
  },
  "payload": {
    "usage": {
      "characters": 13
    }
  }
}

task-failed

The server returns a task-failed event when the task fails. On receiving this event, the client must close the WebSocket connection and handle the error.

header.task_idstring

The task ID generated by the client.

header.eventstring

Event type. Fixed value: task-failed.

header.error_codestring

Error code.

header.error_messagestring

Detailed error message.

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "task-failed",
    "error_code": "InvalidParameter",
    "error_message": "[tts:]Engine return error code: 418",
    "attributes": {}
  },
  "payload": {}
}