Qwen-Audio-TTS/CosyVoice WebSocket client events - Alibaba Cloud Model Studio

User guide: For model introduction and selection recommendations, see Speech synthesis.

run-task

Description: Starts a speech synthesis task and configures the model, voice, sample rate, and other parameters.

When to send: Immediately after the WebSocket connection is established.

Response event: The server returns a task-started event. Wait for this event before sending subsequent commands.

headerobject(required)

Properties

actionstring(required)

The command type. Set to run-task.

task_idstring(required)

A client-generated task ID in UUID format. This ID correlates subsequent events and must match the task_id in the continue-task and finish-task commands.

streamingstring(required)

Set to duplex.

payloadobject(required)

Properties

task_groupstring(required)

The task group. Set to audio.

taskstring(required)

The task type. Set to tts.

functionstring(required)

The function type. Set to SpeechSynthesizer.

modelstring(required)

The model name.

inputobject(required)

Set to an empty object {}. Send the text to synthesize through the continue-task command.

parametersobject(required)

Speech synthesis parameters.

Properties

text_typestring(required)

Set to PlainText.

voicestring(required)

The voice used for speech synthesis.

System voices: See Qwen-Audio-TTS voice list, CosyVoice Voice list
Cloned voices: Custom voices created through voice cloning
Custom voices: Custom voices created through voice design

formatstring(optional)

The audio encoding format.

Valid values:

pcm
wav
mp3 (default)
opus

sample_rateinteger(optional)

The audio sample rate in Hz.

Valid values: 8000, 16000, 22050 (default), 24000, 44100, 48000.

volumeinteger(optional)

The volume level.

Default value: 50.

Valid values: [0, 100].

ratefloat(optional)

The speech rate.

Default value: 1.0.

Valid values: [0.5, 2.0].

pitchfloat(optional)

The pitch.

Default value: 1.0.

Valid values: [0.5, 2.0].

bit_rateinteger(optional)

The audio bit rate in kbps. When the audio format is mp3 or opus, use bit_rate to adjust the bit rate.

Default value: 32.

Valid values: [6, 510].

enable_ssmlboolean(optional)

Specifies whether to enable SSML.

Default value: false.

When set to true, only one continue-task command is allowed.

For the SSML usage restrictions (supported models, voices, and APIs), see Limitations.

word_timestamp_enabledboolean(optional)

Specifies whether to enable word-level timestamps.

Default value: false.

Available only in streaming output mode. Supported voices: cloned voices of qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2, and system voices marked as supported in Qwen-Audio-TTS voice list, CosyVoice Voice list. Cloned voices of other models do not support this feature.

seedinteger(optional)

A random seed for controlling variation in the synthesis output. When the model version, text, voice, and other parameters are unchanged, using the same seed produces identical results.

Default value: 0.

Valid values: [0, 65535].

language_hintsarray[string](optional)

Important

This parameter is an array, but the current version only processes the first element. Pass a single value.
This parameter specifies the target language for speech synthesis. It's unrelated to the language of the audio sample used in voice cloning. To set the source language for a cloning task, see the voice cloning API reference.

Specifies the target language for speech synthesis to improve output quality.

When digit pronunciation, abbreviation expansion, symbol reading, or minority-language synthesis doesn't meet expectations, use this parameter. For example:

Unexpected digit pronunciation: "hello, this is 110" is read as "hello, this is one zero" instead of the expected Chinese pronunciation
Inaccurate symbol pronunciation: "@" is read as the Chinese equivalent instead of "at"
Poor minor language synthesis quality with unnatural results

Valid values:

zh: Chinese
en: English
fr: French
de: German
ja: Japanese
ko: Korean
ru: Russian
pt: Portuguese
th: Thai
id: Indonesian
vi: Vietnamese
es: Spanish
it: Italian
ms: Malaysian
fil: Filipino
ar: Arabic

instructionstring(optional)

Sets an instruction to control dialect, emotion, or voice character during synthesis. For detailed usage, see Instruction control.

enable_aigc_tagboolean(optional)

Specifies whether to embed an AIGC watermark in the generated audio. When set to true, the watermark is embedded in audio files of supported formats (wav/mp3/opus).

Default value: false.

Only qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.

aigc_propagatorstring(optional)

Sets the ContentPropagator field in the AIGC watermark, identifying the content propagator. Takes effect only when enable_aigc_tag is true.

Default value: Alibaba Cloud UID.

Only qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.

aigc_propagate_idstring(optional)

Sets the PropagateID field in the AIGC watermark, uniquely identifying a specific propagation action. Takes effect only when enable_aigc_tag is true.

Default value: The request ID of the current speech synthesis request.

Only qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.

hot_fixobject(optional)

Configures pronunciation corrections and text replacements applied before synthesis.

This feature isn't supported by qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, or cosyvoice-v2.

Parameters:

pronunciation: Custom pronunciation. Specifies pinyin annotations for words to correct inaccurate default pronunciations.
replace: Text replacement. Replaces specified words with target text before synthesis. The replaced text is used as the actual synthesis input.

Example:

"hot_fix": {
  "pronunciation": [
    {"weather": "tian1 qi4"}
  ],
  "replace": [
    {"today": "gold day"}
  ]
}

enable_markdown_filterboolean(optional)

ImportantOnly cloned voices of cosyvoice-v3-flash support this feature.

Specifies whether to enable Markdown filtering. When enabled, the system automatically strips Markdown markup symbols from the input text before synthesis, preventing them from being read aloud.

Default value: false.

Valid values:

true: Enable Markdown filtering
false: Disable Markdown filtering

{
    "header": {
        "action": "run-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "task_group": "audio",
        "task": "tts",
        "function": "SpeechSynthesizer",
        "model": "qwen-audio-3.0-tts-flash",
        "parameters": {
            "text_type": "PlainText",
            "voice": "longanlingxi",
            "format": "mp3",
            "sample_rate": 22050,
            "volume": 50,
            "rate": 1.0,
            "pitch": 1.0,
            "enable_ssml": false
        },
        "input": {}
    }
}

continue-task

Description: Sends the text to synthesize. The text can be sent all at once or in multiple segments.

When to send: After receiving the task-started event from the server.

Limits:

Maximum of 20,000 characters per message
Maximum of 200,000 characters cumulatively
The send interval must not exceed 23 seconds; otherwise, the connection times out.

headerobject(required)

Properties

actionstring(required)

The command type. Set to continue-task.

task_idstring(required)

The task ID in UUID format. Must match the task_id in run-task.

streamingstring(required)

Set to duplex.

payloadobject(required)

Properties

inputobject(required)

Contains the text to synthesize.

textstring(required)

The text to synthesize. Maximum of 20,000 characters per message and 200,000 characters cumulatively.

{
    "header": {
        "action": "continue-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "input": {
            "text": "Before my bed, moonlight shines bright, I suspect it's frost upon the ground."
        }
    }
}

finish-task

Description: Notifies the server that all text has been sent and requests task completion.

When to send: Immediately after all text has been sent.

Response event: The server returns a task-finished event.

headerobject(required)

Properties

actionstring(required)

The command type. Set to finish-task.

task_idstring(required)

The task ID in UUID format. Must match the task_id in run-task.

streamingstring(required)

Set to duplex.

payloadobject(required)

Properties

inputobject(required)

Set to {} for normal task completion. Include directive to cancel the current synthesis turn.

directivestring(optional)

Controls how the task ends. Currently only cancel is supported. When set to cancel, the current synthesis turn is canceled and the server immediately returns a task-finished event without producing further audio.

After cancellation, you can start a new synthesis task on the same WebSocket connection by sending a new run-task event without reconnecting.

ImportantModel limitations:

China (Beijing): All Qwen-Audio-TTS models support this feature. CosyVoice models require v2 or later.
Singapore: All Qwen-Audio-TTS models support this feature. CosyVoice models do not support this feature.

{
    "header": {
        "action": "finish-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "input": {}
    }
}

Cancel task example:

{
    "header": {
        "action": "finish-task",
        "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
        "streaming": "duplex"
    },
    "payload": {
        "input": {
            "directive": "cancel"
        }
    }
}