Real-time speech synthesis - Qwen TTS, Qwen-Audio-TTS/CosyVoice streaming - Alibaba Cloud Model Studio

Learn about real-time speech synthesis features, streaming calls, and ways to improve synthesis quality.

Features

Real-time speech synthesis accepts text segments and streams audio back, enabling speech output as a large language model generates text.

Languages and dialects: Select a model and voice for the target language, or use an instruction to specify a dialect. See Dialects.
Expressive speech: Adjust delivery with natural-language instructions, or insert emotion and nonverbal tags into the text. See Instruction control and Emotion and nonverbal tags.
Custom voices: Create a voice from an audio sample with voice cloning, or from a text description with voice design.
Fine-grained control and formulas: Use SSML and LaTeX to control pauses and pronunciation, or convert formulas to speech.
Audio controls: Adjust speed, pitch, volume, and bitrate, and output common audio formats such as PCM, WAV, MP3, and Opus at sample rates up to 48 kHz. See the corresponding API parameters for supported values and conditions.

NoteSupported features vary by model and voice. Check the scope in the corresponding feature documentation.

To submit complete text in one HTTP request, see Non-real-time speech synthesis.

Get started

Install the DashScope SDK, then get an API key in the console and set the DASHSCOPE_API_KEY environment variable.

Qwen-Audio-TTS (recommended)

This example uses qwen-audio-3.1-tts-flash and the longanhuan_v3.1 system voice in Beijing. It sends text in segments and saves the streamed audio to output.mp3.

Java

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-audio-3.1-tts-flash")
                .voice("longanhuan_v3.1")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"Hello, welcome to Alibaba Cloud Model Studio.", "Text can be sent in segments, and audio is returned progressively."}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

Python

import os
import dashscope

# If you have not set an environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx"
# The following is the configuration for the China (Beijing) region. Replace "{WorkspaceId}" with your actual workspace ID.
client = dashscope.Client(
    api_key=os.environ["DASHSCOPE_API_KEY"],
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

with open("output.mp3", "wb") as audio:
    # Send the text in segments and receive the audio segment by segment
    with client.audio.tts.connect(
        model="qwen-audio-3.1-tts-flash",
        voice="longanhuan_v3.1",
        format="mp3",
        sample_rate=24000,
    ) as session:
        for text in ["Hello, welcome to Alibaba Cloud Model Studio.", "Text can be sent in segments, and audio is returned progressively."]:
            session.send_text(text)
        session.flush()  # Trigger synthesis of the buffered text (does not end the input)
        session.stop()   # End the input and wait for the remaining audio to be returned
        for chunk in session.audio_chunks():
            audio.write(chunk)

CosyVoice

This example uses cosyvoice-v3-flash and the longanyang system voice in Beijing. It sends text in segments and saves the streamed audio to output.mp3.

Java

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("cosyvoice-v3-flash")
                .voice("longanyang")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"Hello, welcome to Alibaba Cloud Model Studio.", "Text can be sent in segments, and audio is returned progressively."}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

Python

import os
import dashscope

# If you have not set an environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx"
# The following is the configuration for the China (Beijing) region. Replace "{WorkspaceId}" with your actual workspace ID.
client = dashscope.Client(
    api_key=os.environ["DASHSCOPE_API_KEY"],
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

with open("output.mp3", "wb") as audio:
    # Send the text in segments and receive the audio segment by segment
    with client.audio.tts.connect(
        model="cosyvoice-v3-flash",
        voice="longanyang",
        format="mp3",
        sample_rate=24000,
    ) as session:
        for text in ["Hello, welcome to Alibaba Cloud Model Studio.", "Text can be sent in segments, and audio is returned progressively."]:
            session.send_text(text)
        session.flush()  # Trigger synthesis of the buffered text (does not end the input)
        session.stop()   # End the input and wait for the remaining audio to be returned
        for chunk in session.audio_chunks():
            audio.write(chunk)

Improve synthesis quality

Languages and pronunciation

Choose an approach based on what you need to control:

Requirement	Approach
Set the target language of synthesized speech	Use `language_hints` to specify the overall synthesis language, guide the pronunciation of numbers, abbreviations, and symbols, and improve synthesis in less widely used languages.
Correct specific words or replace text	Use `hot_fix` to assign pinyin to specific words or replace words before synthesis. This is useful for consistent corrections to names and other local text.
Control pauses, number reading, or pronunciation precisely	Mark the relevant text with SSML tags to control pauses, reading rules, and pronunciation.
Read mathematical formulas	Use the CosyVoice LaTeX feature to convert supported mathematical formulas into Chinese speech.

For supported models and parameter configuration, see the Qwen-Audio-TTS API or CosyVoice API. For the models, voices, interfaces, and usage of SSML and LaTeX, see SSML and LaTeX.

NoteWhen using SSML, submit the complete SSML text in a single input and receive audio incrementally. Fragmented text input is not supported.

Instruction control

Use instruction to specify the synthesis language and describe the speech rate, emotion, or speaking style in natural language to control the synthesized speech.

NoteTo create a new voice with a specific gender, age, or vocal character, or for guidance and examples on writing voice descriptions, see voice design.

Qwen-Audio-TTS

System, cloned, and designed voices for qwen-audio-3.1-tts-plus, qwen-audio-3.1-tts-flash, qwen-audio-3.0-tts-plus, and qwen-audio-3.0-tts-flash all support natural-language instructions through instruction in synthesis requests.

We recommend specifying the synthesis language and desired effect in instruction to improve synthesis quality. The language specified in the instruction must match the language of the text to be synthesized. For example, when synthesizing Dutch text, set:

Speak in Dutch, slowly and clearly.

CosyVoice

NoteFor CosyVoice, instruction must not exceed 100 characters. Chinese characters, including simplified and traditional characters and Chinese characters used in Japanese and Korean, count as two characters each. Other characters, including punctuation, letters, digits, kana, and Hangul, count as one each.

Model	Applicable voices	Instruction requirements	Instruction text languages
cosyvoice-v3.5-plus, cosyvoice-v3.5-flash	Cloned and designed voices No system voices.	Natural-language instructions.	Chinese, English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese.
cosyvoice-v3-plus	System voices	Use the fixed format and content in the voice list. Cloned and designed voices do not support instruction control.	Use the language specified in the voice list.
cosyvoice-v3-flash	System voices	Use the fixed format and content in the voice list.	Chinese.
cosyvoice-v3-flash	Cloned and designed voices	Natural-language instructions.	Chinese, English, French, German, Japanese, Korean, and Russian.

Dialects

Use dialect synthesis to generate speech in Chinese dialects such as Henan or Sichuan dialect. How you select a dialect depends on the model and voice type.

Qwen-Audio-TTS

System voices: Select a voice from the Qwen-Audio-TTS voice list that supports the dialect. A voice that directly supports the dialect requires no additional setting. For voices that support dialect instructions, use the instruction control parameter instruction.
Cloned voices: Specify a dialect with the instruction control parameter instruction.

For voices that support dialect instructions, pass instruction="请讲上海话。" when initializing SpeechSynthesizer to synthesize Chinese text in Shanghainese.

CosyVoice

System voices: Select a voice that directly supports the dialect, such as longshange_v3, or a voice that supports dialect instructions, such as longanhuan_v3. Follow the format in the voice list.
Cloned voices: Use instruction with a model that supports instruction control to specify a dialect.
Designed voices: Dialects are not supported.

In the CosyVoice Python getting-started example, replace the initialization and text submission code with the following to synthesize Henan dialect speech using longanhuan_v3:

with client.audio.tts.connect(
    model="cosyvoice-v3-flash",
    voice="longanhuan_v3",
    parameters={"instruction": "请用河南话表达。"},  # Instruction control
    format="mp3",
    sample_rate=24000,
) as session:
    session.send_text("叫你去买盐，你买回来一袋面，这不是弄啥嘞吗！")
    session.flush()

For Qwen3-TTS, select a voice that supports the target dialect from the Qwen3-TTS voice list.

Emotion and nonverbal tags

NoteApplies to qwen-audio-3.1-tts-plus, qwen-audio-3.1-tts-flash, qwen-audio-3.0-tts-plus, and qwen-audio-3.0-tts-flash.

Emotion and paralinguistic tags control the emotion and style of speech and insert sounds such as laughter or sighs at specific positions. Write the tags directly in the text to synthesize, text, without additional parameters.

Usage examples

This text starts with an excited delivery, inserts laughter, and then continues speaking:

[excited]What a beautiful day![laughing]Let us go outside and have some fun!

This text switches to an excited delivery in the second sentence:

[serious]Please pay attention to the safety instructions.[excited]Now, let us get started!

Supported tags

Control tags

A control tag sets the emotion or style of the following text until the next control tag. When long sentences are automatically segmented, the effect is bounded by the resulting text segment.

Tag	Description
`[sad]`	Sad
`[amazed]`	Amazed
`[deep and loud shouting]`	Deep, loud shouting
`[trembling]`	Trembling
`[angry]`	Angry
`[excited]`	Excited
`[sarcastic]`	Sarcastic
`[curious]`	Curious
`[like dracula]`	Dracula-like delivery
`[bored]`	Bored
`[tired]`	Tired
`[scornful]`	Scornful
`[shouting]`	Shouting
`[asmr]`	Soft ASMR delivery
`[panicked]`	Panicked
`[mischievously]`	Mischievous
`[empathetic]`	Empathetic
`[whispers]`	Whispering
`[reluctantly]`	Reluctant
`[crying]`	Crying
`[serious]`	Serious
`[very slowly]`	Very slow
`[very fast]`	Very fast

Nonverbal tags

A nonverbal tag inserts a sound at its position without changing the emotional style of the surrounding text.

Tag	Description
`[gasp]`	Gasp
`[sighing]`	Sigh
`[clears throat]`	Clear the throat
`[giggles]`	Giggle
`[laughing]`	Laugh
`[cough]`	Cough
`[snorts]`	Snort

Apply in production

Cancel task

With Qwen-Audio-TTS or CosyVoice, send a cancellation instruction to interrupt the current synthesis task. The server ends the task and returns task-finished.

Method	Version requirement	Usage
Python SDK	2.0.1 or later	Call `session.cancel()` on the session object returned by `client.audio.tts.connect`.
Java SDK	2.22.26 or later	Call `SpeechSynthesizer.streamingCancel()`.
Raw WebSocket protocol	No SDK required	Send `finish-task` with `directive=cancel` in `input`.

WebSocket connection reuse

A WebSocket connection can handle successive synthesis rounds. Start the next round as follows for each model series:

Model series	End-of-round event	Start the next round
Qwen-Audio-TTS, CosyVoice	`task-finished`	Send `run-task` with a new `task_id`, then send text using the series-specific flow. A new `task_id` is required for every round, including after cancellation. NoteWhen calling Qwen-Audio-TTS or CosyVoice with the Python SDK, the session created by `client.audio.tts.connect` closes the underlying connection when the task completes or is canceled. To reuse a connection, first create a shared WebSocket connection with `client.ws_connection()` and pass it to `client.audio.tts.connect` via `connection=`. You can then run multiple synthesis rounds in sequence over the same connection. Close the connection with `WsConnection.close` when you are done.
Qwen3-TTS	`response.done`	Continue sending `input_text_buffer.append` in the same session. Synthesis starts automatically or after `input_text_buffer.commit`, depending on the session mode. Send `session.finish` only after all text has been submitted.

For SDK object pool configuration and complete examples, see Speech synthesis best practices. For the Qwen3-TTS session flow, see the WebSocket API.

For Qwen-Audio-TTS / CosyVoice, a connection closes automatically if no new task starts within 60 seconds after the previous task ends. Reconnect if the connection has closed or the task state is invalid.

Rate limits

Control request frequency and concurrency according to the model’s rate limits. For limits, common errors, and handling instructions, see Rate limits.

High-concurrency best practices

Use object pools to reuse synthesis objects and connections, and configure concurrency, warm-up, and error handling for your workload. See Speech synthesis best practices.

Supported models

Singapore

Qwen-Audio-TTS: qwen-audio-3.1-tts-plus, qwen-audio-3.1-tts-flash, qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash
CosyVoice: cosyvoice-v3-plus, cosyvoice-v3-flash
Qwen3-TTS: qwen3-tts-flash-realtime

China (Beijing)

Qwen-Audio-TTS: qwen-audio-3.1-tts-plus, qwen-audio-3.1-tts-flash, qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash
CosyVoice: cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-plus, cosyvoice-v3-flash, cosyvoice-v2
Qwen3-TTS: qwen3-tts-flash-realtime

For system voices and audio samples, see the Qwen-Audio-TTS, CosyVoice, and Qwen3-TTS voice lists. A cloned or designed voice must be used with the target model specified when the voice was created.

API reference

Mobile applications that require low latency and stable audio transmission on weak networks can use AOQ. For AOQ access, protocol differences, and model support across AOQ, WebRTC, and WebSocket, see Realtime API overview.

FAQ

How do I troubleshoot silent audio when using a cloned voice?

Check that voice is the ID of a successfully created voice and that its target_model matches the synthesis model. For voice status, source audio requirements, and quality guidance, see Voice cloning.

Why does the actual duration of synthesized audio differ from the duration shown in the WAV file?

For streamed WAV output, the file header may not reflect the final audio length. To obtain accurate duration metadata, set format to pcm, receive the complete audio, and add a WAV header containing the actual length.

Why can't the audio file play?

Troubleshoot based on your scenario:

Audio saved as a complete file (such as xx.mp3)
1. Audio format consistency: The audio format in the request parameters must match the file extension (for example, if the parameter is wav, the file must be .wav).
2. Player compatibility: Confirm that the player supports the audio format and sample rate.
Streaming audio playback
1. Save the audio stream as a complete file and try playing it with a media player. If the file doesn't play, refer to scenario 1 above.
2. If the file plays correctly, the issue is in the streaming playback implementation. Confirm that the player supports streaming playback (such as ffmpeg, pyaudio, AudioFormat, or MediaSource).

Why is audio playback stuttering?

Troubleshoot with the following steps:

Check the text sending rate: Make sure the sending interval is reasonable to avoid the previous audio segment finishing before the next text arrives.
Check callback function performance:
- Confirm that no blocking logic exists in the callback function.
- Callbacks run on the WebSocket thread. Blocking operations affect data reception. Write audio data to a separate buffer and process it in another thread.
Check network stability: Network fluctuations can cause audio transmission interruptions or delays.

Why is speech synthesis taking a long time?

With segmented input, long gaps between text submissions increase total synthesis time. The first call also includes WebSocket connection overhead. For warm-up and timing in high-concurrency scenarios, see Speech synthesis best practices.

How do I restrict an API key to only the speech synthesis service (permission isolation)?

Create a new workspace and grant access to specific models only. This limits the scope of the API key. For details, see Manage workspaces.