Stream text-to-speech conversion over WebSocket with low first-packet latency. Real-time speech synthesis supports streaming input and output, voice cloning, voice design, and fine-grained audio controls for voice assistants, audiobooks, and intelligent customer service.
Overview
Convert text to speech in real time through a bidirectional WebSocket streaming protocol with low latency.
Streaming input and output with low first-packet latency
Adjustable speech rate, pitch, volume, and bitrate for fine-grained audio control
Compatible with mainstream audio formats (PCM, WAV, MP3, Opus) with up to 48 kHz sample rate output
Supports Instruction control, which lets you control speech expressiveness through natural language instructions
Supports Voice cloning and Voice Design for custom voice creation
Supports Emotion and rich language tags, which lets you embed emotion or sound effect tags within text
For batch scenarios such as audiobooks and courseware voiceover, use Non-real-time speech synthesis. For model selection guidance, see Speech synthesis.
Prerequisites
If you call the API through the DashScope SDK, install the latest SDK.
Quick start
The following examples demonstrate speech synthesis for each model. For more examples and parameter details, see the API reference.
Qwen-Audio-TTS
The following example synthesizes speech using a system voice.
To use the Instruction control feature, set instructions through the instruction parameter.
Python
# coding=utf-8
import os
import dashscope
from dashscope.audio.tts_v2 import *
# The API Key differs between the Singapore and Beijing regions. Get your API Key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you have not configured the environment variable, replace the next line with your Chinese Model Studio API Key: dashscope.api_key = "sk-xxx"
dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY')
# The following is the configuration for the Singapore region. Replace "{WorkspaceId}" with your actual workspace ID. The configuration varies by region.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference'
# Model
# qwen-audio-3.0-tts-flash/qwen-audio-3.0-tts-plus: Use voices such as longanlingxi.
# Each voice supports different languages. To synthesize non-Chinese languages such as Japanese or Korean, select a voice that supports the target language. See the voice list for details.
model = "qwen-audio-3.0-tts-flash"
# Voice
voice = "longanlingxi"
# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor
synthesizer = SpeechSynthesizer(model=model, voice=voice)
# Send the text to be synthesized and get the binary audio
audio = synthesizer.call("How is the weather today?")
# The first text submission requires establishing a WebSocket connection, so the first-packet latency includes connection setup time
print('[Metric] requestId: {}, first-packet latency: {} ms'.format(
synthesizer.get_last_request_id(),
synthesizer.get_first_package_delay()))
# Save the audio to a local file
with open('output.mp3', 'wb') as f:
f.write(audio)Java
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.utils.Constants;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
public class Main {
// Model
// qwen-audio-3.0-tts-flash/qwen-audio-3.0-tts-plus: Use voices such as longanlingxi.
// Each voice supports different languages. To synthesize non-Chinese languages such as Japanese or Korean, select a voice that supports the target language. See the voice list for details.
private static String model = "qwen-audio-3.0-tts-flash";
// Voice
private static String voice = "longanlingxi";
public static void streamAudioDataToSpeaker() {
// Request parameters
SpeechSynthesisParam param =
SpeechSynthesisParam.builder()
// The API Key differs between the Singapore and Beijing regions. Get your API Key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you have not configured the environment variable, replace the next line with your Chinese Model Studio API Key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model(model) // Model
.voice(voice) // Voice
.build();
// Synchronous mode: disable callback (second parameter is null)
SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, null);
ByteBuffer audio = null;
try {
// Block until audio is returned
audio = synthesizer.call("How is the weather today?");
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
// Close the WebSocket connection when the task is done
synthesizer.getDuplexApi().close(1000, "bye");
}
if (audio != null) {
// Save the audio data to the local file "output.mp3"
File file = new File("output.mp3");
// The first text submission requires establishing a WebSocket connection, so the first-packet latency includes connection setup time
// Note: getFirstPackageDelay() requires dashscope-sdk-java 2.18.0 or later
System.out.println(
"[Metric] requestId: "
+ synthesizer.getLastRequestId()
+ ", first-packet latency (ms): "
+ synthesizer.getFirstPackageDelay());
try (FileOutputStream fos = new FileOutputStream(file)) {
fos.write(audio.array());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
public static void main(String[] args) {
// The following is the configuration for the Singapore region. Replace "{WorkspaceId}" with your actual workspace ID. The configuration varies by region.
Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference";
streamAudioDataToSpeaker();
System.exit(0);
}
}CosyVoice
cosyvoice-v3.5-plus and cosyvoice-v3.5-flash are available only in the Beijing region and support only voice design and voice cloning scenarios (no system voices). Before use, create a custom voice through Voice cloning or Voice Design, then set voice to the voice ID and model to the corresponding model name in your code.
The following example synthesizes speech using a system voice (see CosyVoice Voice list).
To use the Instruction control feature, set instructions through the instruction parameter.
Python
# coding=utf-8
import os
import dashscope
from dashscope.audio.tts_v2 import *
# The API Key differs between the Singapore and Beijing regions. Get your API Key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you have not configured the environment variable, replace the next line with your Chinese Model Studio API Key: dashscope.api_key = "sk-xxx"
dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY')
# The following is the configuration for the Singapore region. Replace "{WorkspaceId}" with your actual workspace ID. The configuration varies by region.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference'
# Model
# Different model versions require their corresponding voices:
# cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang.
# cosyvoice-v2: Use voices such as longxiaochun_v2.
# Each voice supports different languages. To synthesize non-Chinese languages such as Japanese or Korean, select a voice that supports the target language. See the Qwen-Audio-TTS/CosyVoice voice list for details.
model = "cosyvoice-v3-flash"
# Voice
voice = "longanyang"
# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor
synthesizer = SpeechSynthesizer(model=model, voice=voice)
# Send the text to be synthesized and get the binary audio
audio = synthesizer.call("How is the weather today?")
# The first text submission requires establishing a WebSocket connection, so the first-packet latency includes connection setup time
print('[Metric] requestId: {}, first-packet latency: {} ms'.format(
synthesizer.get_last_request_id(),
synthesizer.get_first_package_delay()))
# Save the audio to a local file
with open('output.mp3', 'wb') as f:
f.write(audio)Java
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.utils.Constants;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
public class Main {
// Model
// Different model versions require their corresponding voices:
// cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang.
// cosyvoice-v2: Use voices such as longxiaochun_v2.
// Each voice supports different languages. To synthesize non-Chinese languages such as Japanese or Korean, select a voice that supports the target language. See the Qwen-Audio-TTS/CosyVoice voice list for details.
private static String model = "cosyvoice-v3-flash";
// Voice
private static String voice = "longanyang";
public static void streamAudioDataToSpeaker() {
// Request parameters
SpeechSynthesisParam param =
SpeechSynthesisParam.builder()
// The API Key differs between the Singapore and Beijing regions. Get your API Key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you have not configured the environment variable, replace the next line with your Chinese Model Studio API Key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model(model) // Model
.voice(voice) // Voice
.build();
// Synchronous mode: disable callback (second parameter is null)
SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, null);
ByteBuffer audio = null;
try {
// Block until audio is returned
audio = synthesizer.call("How is the weather today?");
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
// Close the WebSocket connection when the task is done
synthesizer.getDuplexApi().close(1000, "bye");
}
if (audio != null) {
// Save the audio data to the local file "output.mp3"
File file = new File("output.mp3");
// The first text submission requires establishing a WebSocket connection, so the first-packet latency includes connection setup time
// Note: getFirstPackageDelay() requires dashscope-sdk-java 2.18.0 or later
System.out.println(
"[Metric] requestId: "
+ synthesizer.getLastRequestId()
+ ", first-packet latency (ms): "
+ synthesizer.getFirstPackageDelay());
try (FileOutputStream fos = new FileOutputStream(file)) {
fos.write(audio.array());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
public static void main(String[] args) {
// The following is the configuration for the Singapore region. Replace "{WorkspaceId}" with your actual workspace ID. The configuration varies by region.
Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference";
streamAudioDataToSpeaker();
System.exit(0);
}
}Qwen-TTS
The following example synthesizes speech using a system voice (see Supported voices).
To use the Instruction control feature, replace model with qwen3-tts-instruct-flash-realtime and set instructions through the instructions parameter.
Python
Server commit mode
import os
import base64
import threading
import time
import dashscope
from dashscope.audio.qwen_tts_realtime import *
qwen_tts_realtime: QwenTtsRealtime = None
text_to_synthesize = [
'Right? I love supermarkets like this.',
'Especially during Chinese New Year,',
'I go shopping at supermarkets.',
'And I feel',
'absolutely thrilled!',
'I want to buy so many things!'
]
DO_VIDEO_TEST = False
def init_dashscope_api_key():
"""
Set your DashScope API key. More information:
https://github.com/aliyun/alibabacloud-bailian-speech-demo/blob/master/PREREQUISITES.md
"""
# API keys differ between the Singapore and Beijing regions. Get an API key: https://www.alibabacloud.com/help/en/model-studio/get-api-key
if 'DASHSCOPE_API_KEY' in os.environ:
dashscope.api_key = os.environ[
'DASHSCOPE_API_KEY'] # Load API key from environment variable DASHSCOPE_API_KEY
else:
dashscope.api_key = 'your-dashscope-api-key' # Set API key manually
class MyCallback(QwenTtsRealtimeCallback):
def __init__(self):
self.complete_event = threading.Event()
self.file = open('result_24k.pcm', 'wb')
def on_open(self) -> None:
print('connection opened, init player')
def on_close(self, close_status_code, close_msg) -> None:
self.file.close()
print('connection closed with code: {}, msg: {}, destroy player'.format(close_status_code, close_msg))
def on_event(self, response: str) -> None:
try:
global qwen_tts_realtime
type = response['type']
if 'session.created' == type:
print('start session: {}'.format(response['session']['id']))
if 'response.audio.delta' == type:
recv_audio_b64 = response['delta']
self.file.write(base64.b64decode(recv_audio_b64))
if 'response.done' == type:
print(f'response {qwen_tts_realtime.get_last_response_id()} done')
if 'session.finished' == type:
print('session finished')
self.complete_event.set()
except Exception as e:
print('[Error] {}'.format(e))
return
def wait_for_finished(self):
self.complete_event.wait()
if __name__ == '__main__':
init_dashscope_api_key()
print('Initializing ...')
callback = MyCallback()
qwen_tts_realtime = QwenTtsRealtime(
# To use instruction control, replace the model with qwen3-tts-instruct-flash-realtime
model='qwen3-tts-flash-realtime',
callback=callback,
# Singapore region
url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime'
)
qwen_tts_realtime.connect()
qwen_tts_realtime.update_session(
voice = 'Cherry',
response_format = AudioFormat.PCM_24000HZ_MONO_16BIT,
# To use instruction control, uncomment the following lines and replace the model with qwen3-tts-instruct-flash-realtime
# instructions='Speak quickly with a rising intonation, suitable for introducing fashion products.',
# optimize_instructions=True,
mode = 'server_commit'
)
for text_chunk in text_to_synthesize:
print(f'send text: {text_chunk}')
qwen_tts_realtime.append_text(text_chunk)
time.sleep(0.1)
qwen_tts_realtime.finish()
callback.wait_for_finished()
print('[Metric] session: {}, first audio delay: {}'.format(
qwen_tts_realtime.get_session_id(),
qwen_tts_realtime.get_first_audio_delay(),
))
Commit mode
import base64
import os
import threading
import dashscope
from dashscope.audio.qwen_tts_realtime import *
qwen_tts_realtime: QwenTtsRealtime = None
text_to_synthesize = [
'This is the first sentence.',
'This is the second sentence.',
'This is the third sentence.',
]
DO_VIDEO_TEST = False
def init_dashscope_api_key():
"""
Set your DashScope API key. More information:
https://github.com/aliyun/alibabacloud-bailian-speech-demo/blob/master/PREREQUISITES.md
"""
# API keys differ between the Singapore and Beijing regions. Get an API key: https://www.alibabacloud.com/help/en/model-studio/get-api-key
if 'DASHSCOPE_API_KEY' in os.environ:
dashscope.api_key = os.environ[
'DASHSCOPE_API_KEY'] # Load API key from environment variable DASHSCOPE_API_KEY
else:
dashscope.api_key = 'your-dashscope-api-key' # Set API key manually
class MyCallback(QwenTtsRealtimeCallback):
def __init__(self):
super().__init__()
self.response_counter = 0
self.complete_event = threading.Event()
self.file = open(f'result_{self.response_counter}_24k.pcm', 'wb')
def reset_event(self):
self.response_counter += 1
self.file = open(f'result_{self.response_counter}_24k.pcm', 'wb')
self.complete_event = threading.Event()
def on_open(self) -> None:
print('connection opened, init player')
def on_close(self, close_status_code, close_msg) -> None:
print('connection closed with code: {}, msg: {}, destroy player'.format(close_status_code, close_msg))
def on_event(self, response: str) -> None:
try:
global qwen_tts_realtime
type = response['type']
if 'session.created' == type:
print('start session: {}'.format(response['session']['id']))
if 'response.audio.delta' == type:
recv_audio_b64 = response['delta']
self.file.write(base64.b64decode(recv_audio_b64))
if 'response.done' == type:
print(f'response {qwen_tts_realtime.get_last_response_id()} done')
self.complete_event.set()
self.file.close()
if 'session.finished' == type:
print('session finished')
self.complete_event.set()
except Exception as e:
print('[Error] {}'.format(e))
return
def wait_for_response_done(self):
self.complete_event.wait()
if __name__ == '__main__':
init_dashscope_api_key()
print('Initializing ...')
callback = MyCallback()
qwen_tts_realtime = QwenTtsRealtime(
# To use instruction control, replace the model with qwen3-tts-instruct-flash-realtime
model='qwen3-tts-flash-realtime',
callback=callback,
# Singapore region
url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime'
)
qwen_tts_realtime.connect()
qwen_tts_realtime.update_session(
voice = 'Cherry',
response_format = AudioFormat.PCM_24000HZ_MONO_16BIT,
# To use instruction control, uncomment the following lines and replace the model with qwen3-tts-instruct-flash-realtime
# instructions='Speak quickly with a rising intonation, suitable for introducing fashion products.',
# optimize_instructions=True,
mode = 'commit'
)
print(f'send text: {text_to_synthesize[0]}')
qwen_tts_realtime.append_text(text_to_synthesize[0])
qwen_tts_realtime.commit()
callback.wait_for_response_done()
callback.reset_event()
print(f'send text: {text_to_synthesize[1]}')
qwen_tts_realtime.append_text(text_to_synthesize[1])
qwen_tts_realtime.commit()
callback.wait_for_response_done()
callback.reset_event()
print(f'send text: {text_to_synthesize[2]}')
qwen_tts_realtime.append_text(text_to_synthesize[2])
qwen_tts_realtime.commit()
callback.wait_for_response_done()
qwen_tts_realtime.finish()
print('[Metric] session: {}, first audio delay: {}'.format(
qwen_tts_realtime.get_session_id(),
qwen_tts_realtime.get_first_audio_delay(),
))
Java
Server commit mode
appendText()
import com.alibaba.dashscope.audio.qwen_tts_realtime.*;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.google.gson.JsonObject;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.AudioSystem;
import java.io.*;
import java.util.Base64;
import java.util.Queue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicBoolean;
public class Main {
static String[] textToSynthesize = {
"Right? I really love this kind of supermarket.",
"Especially during the Chinese New Year.",
"Going to the supermarket.",
"It just makes me feel.",
"Super, super happy!",
"I want to buy so many things!"
};
public static QwenTtsRealtimeAudioFormat ttsFormat = QwenTtsRealtimeAudioFormat.PCM_24000HZ_MONO_16BIT;
// Real-time PCM audio player
public static class RealtimePcmPlayer {
private int sampleRate;
private SourceDataLine line;
private AudioFormat audioFormat;
private Thread decoderThread;
private Thread playerThread;
private AtomicBoolean stopped = new AtomicBoolean(false);
private Queue<String> b64AudioBuffer = new ConcurrentLinkedQueue<>();
private Queue<byte[]> RawAudioBuffer = new ConcurrentLinkedQueue<>();
private ByteArrayOutputStream totalAudioStream = new ByteArrayOutputStream();
// Initialize the audio format and audio line.
public RealtimePcmPlayer(int sampleRate) throws LineUnavailableException {
this.sampleRate = sampleRate;
this.audioFormat = new AudioFormat(this.sampleRate, 16, 1, true, false);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, audioFormat);
line = (SourceDataLine) AudioSystem.getLine(info);
line.open(audioFormat);
line.start();
decoderThread = new Thread(new Runnable() {
@Override
public void run() {
while (!stopped.get()) {
String b64Audio = b64AudioBuffer.poll();
if (b64Audio != null) {
byte[] rawAudio = Base64.getDecoder().decode(b64Audio);
RawAudioBuffer.add(rawAudio);
// Write audio data to totalAudioStream.
try {
totalAudioStream.write(rawAudio);
} catch (IOException e) {
throw new RuntimeException(e);
}
} else {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
}
});
playerThread = new Thread(new Runnable() {
@Override
public void run() {
while (!stopped.get()) {
byte[] rawAudio = RawAudioBuffer.poll();
if (rawAudio != null) {
try {
playChunk(rawAudio);
} catch (IOException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
} else {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
}
});
decoderThread.start();
playerThread.start();
}
// Play an audio chunk and block until playback completes.
private void playChunk(byte[] chunk) throws IOException, InterruptedException {
if (chunk == null || chunk.length == 0) return;
int bytesWritten = 0;
while (bytesWritten < chunk.length) {
bytesWritten += line.write(chunk, bytesWritten, chunk.length - bytesWritten);
}
int audioLength = chunk.length / (this.sampleRate*2/1000);
// Wait for the buffered audio to finish playing.
Thread.sleep(audioLength - 10);
}
public void write(String b64Audio) {
b64AudioBuffer.add(b64Audio);
}
public void cancel() {
b64AudioBuffer.clear();
RawAudioBuffer.clear();
}
public void waitForComplete() throws InterruptedException {
while (!b64AudioBuffer.isEmpty() || !RawAudioBuffer.isEmpty()) {
Thread.sleep(100);
}
line.drain();
}
public void shutdown() throws InterruptedException, IOException {
stopped.set(true);
decoderThread.join();
playerThread.join();
// Save the complete audio file.
File file = new File("TotalAudio_"+ttsFormat.getSampleRate()+"."+ttsFormat.getFormat());
try (FileOutputStream fos = new FileOutputStream(file)) {
fos.write(totalAudioStream.toByteArray());
}
if (line != null && line.isRunning()) {
line.drain();
line.close();
}
}
}
public static void main(String[] args) throws InterruptedException, LineUnavailableException, IOException {
QwenTtsRealtimeParam param = QwenTtsRealtimeParam.builder()
// To use instruction control, replace the model with qwen3-tts-instruct-flash-realtime.
.model("qwen3-tts-flash-realtime")
// China (Beijing) region
.url("wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime")
// API keys differ between Singapore and China (Beijing). See https://www.alibabacloud.com/help/en/model-studio/get-api-key.
.apikey(System.getenv("DASHSCOPE_API_KEY"))
.build();
AtomicReference<CountDownLatch> completeLatch = new AtomicReference<>(new CountDownLatch(1));
final AtomicReference<QwenTtsRealtime> qwenTtsRef = new AtomicReference<>(null);
// Create a real-time audio player instance.
RealtimePcmPlayer audioPlayer = new RealtimePcmPlayer(24000);
QwenTtsRealtime qwenTtsRealtime = new QwenTtsRealtime(param, new QwenTtsRealtimeCallback() {
@Override
public void onOpen() {
// Handle connection establishment.
}
@Override
public void onEvent(JsonObject message) {
String type = message.get("type").getAsString();
switch(type) {
case "session.created":
// Handle session creation.
if (message.has("session")) {
String eventId = message.get("event_id").getAsString();
String sessionId = message.get("session").getAsJsonObject().get("id").getAsString();
System.out.println("[onEvent] session.created, session_id: "
+ sessionId + ", event_id: " + eventId);
}
break;
case "response.audio.delta":
String recvAudioB64 = message.get("delta").getAsString();
// Play audio in real time.
audioPlayer.write(recvAudioB64);
break;
case "response.done":
// Handle response completion.
break;
case "session.finished":
// Handle session termination.
completeLatch.get().countDown();
default:
break;
}
}
@Override
public void onClose(int code, String reason) {
// Handle connection closure.
}
});
qwenTtsRef.set(qwenTtsRealtime);
try {
qwenTtsRealtime.connect();
} catch (NoApiKeyException e) {
throw new RuntimeException(e);
}
QwenTtsRealtimeConfig config = QwenTtsRealtimeConfig.builder()
.voice("Cherry")
.responseFormat(ttsFormat)
.mode("server_commit")
// To use instruction control, uncomment the following lines and replace the model with qwen3-tts-instruct-flash-realtime.
// .instructions("")
// .optimizeInstructions(true)
.build();
qwenTtsRealtime.updateSession(config);
for (String text:textToSynthesize) {
qwenTtsRealtime.appendText(text);
Thread.sleep(100);
}
qwenTtsRealtime.finish();
completeLatch.get().await();
qwenTtsRealtime.close();
// Wait for audio playback to complete, then shut down the player.
audioPlayer.waitForComplete();
audioPlayer.shutdown();
System.exit(0);
}
}
Commit mode
commit()
import com.alibaba.dashscope.audio.qwen_tts_realtime.*;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.google.gson.JsonObject;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.AudioSystem;
import java.io.*;
import java.util.Base64;
import java.util.Queue;
import java.util.Scanner;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicBoolean;
public class Main {
public static QwenTtsRealtimeAudioFormat ttsFormat = QwenTtsRealtimeAudioFormat.PCM_24000HZ_MONO_16BIT;
// Real-time PCM audio player
public static class RealtimePcmPlayer {
private int sampleRate;
private SourceDataLine line;
private AudioFormat audioFormat;
private Thread decoderThread;
private Thread playerThread;
private AtomicBoolean stopped = new AtomicBoolean(false);
private Queue<String> b64AudioBuffer = new ConcurrentLinkedQueue<>();
private Queue<byte[]> RawAudioBuffer = new ConcurrentLinkedQueue<>();
private ByteArrayOutputStream totalAudioStream = new ByteArrayOutputStream();
// Initialize the audio format and audio line.
public RealtimePcmPlayer(int sampleRate) throws LineUnavailableException {
this.sampleRate = sampleRate;
this.audioFormat = new AudioFormat(this.sampleRate, 16, 1, true, false);
DataLine.Info info = new DataLine.Info(SourceDataLine.class, audioFormat);
line = (SourceDataLine) AudioSystem.getLine(info);
line.open(audioFormat);
line.start();
decoderThread = new Thread(new Runnable() {
@Override
public void run() {
while (!stopped.get()) {
String b64Audio = b64AudioBuffer.poll();
if (b64Audio != null) {
byte[] rawAudio = Base64.getDecoder().decode(b64Audio);
RawAudioBuffer.add(rawAudio);
// Write audio data to totalAudioStream.
try {
totalAudioStream.write(rawAudio);
} catch (IOException e) {
throw new RuntimeException(e);
}
} else {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
}
});
playerThread = new Thread(new Runnable() {
@Override
public void run() {
while (!stopped.get()) {
byte[] rawAudio = RawAudioBuffer.poll();
if (rawAudio != null) {
try {
playChunk(rawAudio);
} catch (IOException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
} else {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
}
});
decoderThread.start();
playerThread.start();
}
// Play an audio chunk and block until playback completes.
private void playChunk(byte[] chunk) throws IOException, InterruptedException {
if (chunk == null || chunk.length == 0) return;
int bytesWritten = 0;
while (bytesWritten < chunk.length) {
bytesWritten += line.write(chunk, bytesWritten, chunk.length - bytesWritten);
}
int audioLength = chunk.length / (this.sampleRate*2/1000);
// Wait for the buffered audio to finish playing.
Thread.sleep(audioLength - 10);
}
public void write(String b64Audio) {
b64AudioBuffer.add(b64Audio);
}
public void cancel() {
b64AudioBuffer.clear();
RawAudioBuffer.clear();
}
public void waitForComplete() throws InterruptedException {
// Wait for all buffered audio data to finish playing.
while (!b64AudioBuffer.isEmpty() || !RawAudioBuffer.isEmpty()) {
Thread.sleep(100);
}
// Wait for the audio line to drain.
line.drain();
}
public void shutdown() throws InterruptedException {
stopped.set(true);
decoderThread.join();
playerThread.join();
// Save the complete audio file.
File file = new File("TotalAudio_"+ttsFormat.getSampleRate()+"."+ttsFormat.getFormat());
try (FileOutputStream fos = new FileOutputStream(file)) {
fos.write(totalAudioStream.toByteArray());
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
if (line != null && line.isRunning()) {
line.drain();
line.close();
}
}
}
public static void main(String[] args) throws InterruptedException, LineUnavailableException, FileNotFoundException {
Scanner scanner = new Scanner(System.in);
QwenTtsRealtimeParam param = QwenTtsRealtimeParam.builder()
// To use instruction control, replace the model with qwen3-tts-instruct-flash-realtime.
.model("qwen3-tts-flash-realtime")
// China (Beijing) region
.url("wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime")
// API keys differ between Singapore and China (Beijing). See https://www.alibabacloud.com/help/en/model-studio/get-api-key.
.apikey(System.getenv("DASHSCOPE_API_KEY"))
.build();
AtomicReference<CountDownLatch> completeLatch = new AtomicReference<>(new CountDownLatch(1));
// Create a real-time player instance.
RealtimePcmPlayer audioPlayer = new RealtimePcmPlayer(24000);
final AtomicReference<QwenTtsRealtime> qwenTtsRef = new AtomicReference<>(null);
QwenTtsRealtime qwenTtsRealtime = new QwenTtsRealtime(param, new QwenTtsRealtimeCallback() {
@Override
public void onOpen() {
System.out.println("connection opened");
System.out.println("Enter text and press Enter to send. Enter 'quit' to exit the program.");
}
@Override
public void onEvent(JsonObject message) {
String type = message.get("type").getAsString();
switch(type) {
case "session.created":
System.out.println("start session: " + message.get("session").getAsJsonObject().get("id").getAsString());
break;
case "response.audio.delta":
String recvAudioB64 = message.get("delta").getAsString();
byte[] rawAudio = Base64.getDecoder().decode(recvAudioB64);
// Play audio in real time.
audioPlayer.write(recvAudioB64);
break;
case "response.done":
System.out.println("response done");
// Wait for audio playback to complete.
try {
audioPlayer.waitForComplete();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
// Prepare for the next input.
completeLatch.get().countDown();
break;
case "session.finished":
System.out.println("session finished");
if (qwenTtsRef.get() != null) {
System.out.println("[Metric] response: " + qwenTtsRef.get().getResponseId() +
", first audio delay: " + qwenTtsRef.get().getFirstAudioDelay() + " ms");
}
completeLatch.get().countDown();
default:
break;
}
}
@Override
public void onClose(int code, String reason) {
System.out.println("connection closed code: " + code + ", reason: " + reason);
try {
// Wait for playback to complete, then shut down the player.
audioPlayer.waitForComplete();
audioPlayer.shutdown();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
});
qwenTtsRef.set(qwenTtsRealtime);
try {
qwenTtsRealtime.connect();
} catch (NoApiKeyException e) {
throw new RuntimeException(e);
}
QwenTtsRealtimeConfig config = QwenTtsRealtimeConfig.builder()
.voice("Cherry")
.responseFormat(ttsFormat)
.mode("commit")
// To use instruction control, uncomment the following lines and replace the model with qwen3-tts-instruct-flash-realtime.
// .instructions("")
// .optimizeInstructions(true)
.build();
qwenTtsRealtime.updateSession(config);
// Read user input in a loop.
while (true) {
System.out.print("Enter the text to synthesize: ");
String text = scanner.nextLine();
// Exit when the user enters 'quit'.
if ("quit".equalsIgnoreCase(text.trim())) {
System.out.println("Closing the connection...");
qwenTtsRealtime.finish();
completeLatch.get().await();
break;
}
// Skip empty input.
if (text.trim().isEmpty()) {
continue;
}
// Re-initialize the countdown latch.
completeLatch.set(new CountDownLatch(1));
// Send the text.
qwenTtsRealtime.appendText(text);
qwenTtsRealtime.commit();
// Wait for the current synthesis to complete.
completeLatch.get().await();
}
// Clean up resources.
audioPlayer.waitForComplete();
audioPlayer.shutdown();
scanner.close();
System.exit(0);
}
}
Session configuration
Qwen-TTS interaction modes
Qwen-TTS Realtime API provides two interaction modes:
server_commit mode: The server handles text segmentation and synthesis timing automatically. Suited for continuous synthesis of large text blocks. The client appends text without managing segmentation or submission.
commit mode: The client explicitly submits the text buffer to trigger synthesis. Suited for scenarios that require precise control over synthesis timing, such as per-turn synthesis in conversational AI.
Switch interaction modes:
WebSocket: Set the
session.updateevent'smodefield.{ "type": "session.update", "session": { "mode": "server_commit" } }Python SDK: Set the
modeparameter in theupdate_sessionmethod.qwen_tts_realtime.update_session( voice='Cherry', response_format=AudioFormat.PCM_24000HZ_MONO_16BIT, mode='server_commit' )Java SDK: Set the
modeparameter throughQwenTtsRealtimeConfig.builder().QwenTtsRealtimeConfig config = QwenTtsRealtimeConfig.builder() .voice("Cherry") .responseFormat(ttsFormat) .mode("server_commit") .build(); qwenTtsRealtime.updateSession(config);
For complete SDK code examples, see Python SDK and Java SDK. For WebSocket event lifecycle and connection reuse details, see WebSocket API reference.
Advanced features
Instruction control
Instruction control uses natural language descriptions to adjust speech tone, speed, emotion, and timbre characteristics without configuring complex audio parameters.
Instruction specifications by model:
Qwen-Audio-TTS
Supported models: qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash
System voices and voice cloning voices: accept any instruction.
CosyVoice
Supported models: cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-plus, cosyvoice-v3-flash
Instruction format requirements vary by model:
cosyvoice-v3.5-plus, cosyvoice-v3.5-flash:
Voice cloning/design voices: accept any instruction.
System voices: v3.5 doesn't support system voices.
cosyvoice-v3-plus:
Voice cloning/design voices: don't support instruction control.
System voices: instructions must use a fixed format and content. See CosyVoice Voice list.
cosyvoice-v3-flash:
Voice cloning/design voices: accept any instruction.
System voices: instructions must use a fixed format and content. See CosyVoice Voice list.
Usage: Specify instruction content through the instruction parameter.
Supported languages for instruction text:
cosyvoice-v3.5-plus, cosyvoice-v3.5-flash:
Voice cloning/design voices: Chinese, English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese.
System voices: v3.5 doesn't support system voices.
cosyvoice-v3-plus:
Voice cloning/design voices: Chinese, English, French, German, Japanese, Korean, and Russian.
System voices: instructions must use a fixed format and content. See CosyVoice Voice list.
cosyvoice-v3-flash:
Voice cloning/design voices: Chinese, English, French, German, Japanese, Korean, and Russian.
System voices: Chinese only.
Instruction text length limit: 100 characters maximum. Chinese characters (including simplified/traditional Chinese, Japanese kanji, and Korean hanja) count as 2 characters each. All other characters (such as punctuation, letters, digits, Japanese kana, and Korean hangul) count as 1 character each.
Qwen-TTS
Supported models: Qwen3-TTS-Instruct-Flash-Realtime series models only.
Usage: Specify instruction content through the instructions parameter.
Supported languages for instruction text: Chinese and English only.
Instruction text length limit: 1,600 tokens maximum.
Use cases:
Audiobook and radio drama voiceover
Advertising and promotional video voiceover
Game character and animation voiceover
Emotionally expressive voice assistants
Documentary and news broadcast narration
Writing effective voice descriptions:
Core principles:
Be specific, not vague: Use words that depict vocal qualities, such as "deep", "crisp", or "slightly fast pace". Avoid subjective or vague terms like "nice" or "normal".
Be multi-dimensional, not one-dimensional: A good description typically covers multiple dimensions (such as gender, age, and emotion). Writing only "female voice" is too broad to generate a distinctive timbre.
Be objective, not subjective: Focus on the physical and perceptual characteristics of the voice. For example, use "higher pitch with an energetic tone" instead of "my favorite voice".
Be original, not imitative: Describe the vocal qualities rather than requesting imitation of specific individuals (such as celebrities or actors). The model doesn't support imitation, and it may pose copyright risks.
Be concise, not redundant: Make sure every word serves a purpose. Avoid repetitive synonyms or meaningless modifiers.
Description dimension reference:
Combine the following dimensions to describe a voice. The more dimensions you include, the more accurate the output.
Dimension
Example descriptions
Gender
Male, female, androgynous
Age
Child (5-12), teenager (13-18), young adult (19-35), middle-aged (36-55), elderly (55+)
Pitch
High, mid-range, low, slightly high, slightly low
Speed
Fast, moderate, slow, slightly fast, slightly slow
Emotion
Cheerful, calm, gentle, serious, lively, composed, soothing
Characteristics
Magnetic, crisp, husky, mellow, sweet, rich, powerful
Use case
News broadcast, ad voiceover, audiobook, animation character, voice assistant, documentary narration
Examples:
Standard broadcast style: clear and precise articulation with perfect pronunciation
A young, lively female voice with a faster pace and noticeable upward intonation, suited for fashion product introductions
A calm middle-aged male, slow pace, deep and magnetic voice, suited for news reading or documentary narration
A gentle, intellectual female, around 30 years old, with an even tone, suited for audiobook narration
A cute child's voice, approximately an 8-year-old girl, speaking with a slightly childish quality, suited for animation character voiceover
Dialects
This section describes how to produce speech in Chinese dialects (such as Henan dialect, Sichuan dialect, and Cantonese). Configuration methods vary by model and voice type.
Dialect configuration by model:
Qwen-Audio-TTS
System voices: Select one of the following voice types:
A system voice with built-in dialect support, which outputs the corresponding dialect without additional configuration.
A voice that supports Instruction control and can be configured to output a specific dialect through instruction text.
Voice cloning voices: Configure through the Instruction control feature. For example, set the instruction text to
请用河南话表达.
Supported dialects: See the "Supported languages" column for each model in Qwen-Audio-TTS.
CosyVoice
System voices: Select one of the following voice types from CosyVoice Voice list:
A system voice with built-in dialect support (such as
longshange_v3), which outputs the corresponding dialect without additional configuration.A voice that supports Instruction control and can be configured to output a specific dialect (such as
longanhuan_v3), specified through instruction text.
Voice cloning voices: Configure through the Instruction control feature. For example, set the instruction text to
请用河南话表达.Voice design voices: don't support dialects.
Supported dialects: See the "Supported languages" column for each model in CosyVoice.
Example: Use cosyvoice-v3-flash with the longanhuan_v3 voice, and set the instruction text to "请用河南话表达。" to output speech in Henan dialect.
# coding=utf-8
import os
import dashscope
from dashscope.audio.tts_v2 import *
# The API Key differs between the Singapore and Beijing regions. Get your API Key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you have not configured the environment variable, replace the next line with your Chinese Model Studio API Key: dashscope.api_key = "sk-xxx"
dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY')
# The following is the configuration for the Singapore region. Replace "{WorkspaceId}" with your actual workspace ID. The configuration varies by region.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference'
# Model
# Different model versions require their corresponding voices:
# cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang.
# cosyvoice-v2: Use voices such as longxiaochun_v2.
# Select the appropriate voice for your target language
model = "cosyvoice-v3-flash"
# Voice
voice = "longanhuan_v3"
# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor
synthesizer = SpeechSynthesizer(model=model, voice=voice, instruction="请用河南话表达。")
# Send the text to be synthesized and get the binary audio
audio = synthesizer.call("叫你去买盐,你买回来一袋面,这不是弄啥嘞吗!")
# The first text submission requires establishing a WebSocket connection, so the first-packet latency includes connection setup time
print('[Metric] requestId: {}, first-packet latency: {} ms'.format(
synthesizer.get_last_request_id(),
synthesizer.get_first_package_delay()))
# Save the audio to a local file
with open('output.mp3', 'wb') as f:
f.write(audio)Qwen-TTS
System voices: Use a system voice with built-in dialect support. See the Qwen-TTS voice list in Supported voices.
Voice cloning voices: don't support dialects.
Voice design voices: don't support dialects.
Supported dialects: See the "Supported languages" column for each model in Qwen3-TTS.
Emotion and rich language tags
Qwen-Audio-TTS series models support embedding emotion and rich language tags directly in the text to synthesize (the text parameter). These tags control emotional expression or insert vocal effects (such as laughter and sighs) at specified positions, producing more expressive speech without configuring complex audio parameters.
Supported models: qwen-audio-3.0-tts-plus and qwen-audio-3.0-tts-flash only.
Limitation: Only unidirectional streaming mode is supported.
Control tags
Control tags set the emotion or style of the speech. Place a tag in the text to affect all subsequent text until the next control tag appears or the sentence is automatically segmented due to length.
Tag | Description |
| Sad |
| Amazed |
| Deep, loud shouting |
| Trembling |
| Angry |
| Excited |
| Sarcastic |
| Curious |
| Dracula style (deep, eerie) |
| Bored |
| Tired |
| Singing |
| Scornful |
| Shouting |
| ASMR soft whisper |
| Panicked |
| Mischievous |
| Empathetic |
| Whisper |
| Reluctant |
| Crying |
| Serious |
| Very slow speech |
| Very fast speech |
Rich language tags
Rich language tags insert a vocal effect at the current position in the text without affecting the emotional style of surrounding text.
Tag | Description |
| Gasp |
| Sigh |
| Throat clearing |
| Giggle |
| Laughter |
| Cough |
| Snort |
Usage examples
The following example shows how to combine control tags and rich language tags in the text parameter:
[excited]What a beautiful day today![laughing]Let's go out and have fun together!
In this text, [excited] is a control tag that applies an excited emotion to all subsequent text. [laughing] is a rich language tag that inserts a laugh at that position before continuing to synthesize the remaining text.
You can also switch between different emotions within the same text:
[serious]Please pay attention to the safety precautions.[excited]Alright, let's get started now!
Here, [serious] sets the first sentence to a serious tone, and [excited] switches to an excited tone starting from the second sentence.
WebSocket raw protocol calls
The following examples demonstrate how to connect directly to the server through the WebSocket raw protocol, suited for scenarios without the DashScope SDK. These are minimal runnable implementations. For WebSocket protocol details, see the API reference for each model.
Apply in production
Connection reuse (WebSocket)
WebSocket connections are reusable: after a synthesis task completes, you can start the next task on the same connection without re-establishing it.
Reuse process:
Qwen-Audio-TTS / Qwen-Audio-TTS/CosyVoice: The client sends
finish-task, and after the server returnstask-finished, the client can sendrun-taskto start a new task.Qwen-TTS: The client sends
session.finish, and after the server returnssession.finished, the client can create a new session to start the next task.
Wait for the server to return the completion event (
task-finishedorsession.finished) before starting a new task.Qwen-Audio-TTS, Qwen-Audio-TTS/CosyVoice require a different
task_idfor each task on a reused connection.If a task fails, the server returns an error event and closes the connection. The connection can't be reused.
If no new task is started within 60 seconds after the previous task ends, the connection is automatically closed.
For the event details of each model, see the corresponding API reference.
High-concurrency best practices
The DashScope SDK has built-in pooling that reuses WebSocket connections and synthesizer objects, eliminating the overhead of repeatedly creating and destroying them.
Supported models and regions
Singapore
To call the following models, select an API Key from the Singapore region:
Qwen-Audio-TTS: qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash
Qwen-Audio-TTS/CosyVoice: cosyvoice-v3-plus, cosyvoice-v3-flash
Qwen-TTS:
Qwen3-TTS-Instruct-Flash-Realtime: qwen3-tts-instruct-flash-realtime (stable, currently equivalent to qwen3-tts-instruct-flash-realtime-2026-01-22), qwen3-tts-instruct-flash-realtime-2026-01-22 (latest snapshot)
Qwen3-TTS-VD-Realtime: qwen3-tts-vd-realtime-2026-01-15 (latest snapshot), qwen3-tts-vd-realtime-2025-12-16 (snapshot)
Qwen3-TTS-VC-Realtime: qwen3-tts-vc-realtime-2026-01-15 (latest snapshot), qwen3-tts-vc-realtime-2025-11-27 (snapshot)
Qwen3-TTS-Flash-Realtime: qwen3-tts-flash-realtime (stable, currently equivalent to qwen3-tts-flash-realtime-2025-11-27), qwen3-tts-flash-realtime-2025-11-27 (latest snapshot), qwen3-tts-flash-realtime-2025-09-18 (snapshot)
China (Beijing)
To call the following models, select an API Key from the Beijing region:
Qwen-Audio-TTS: qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash
Qwen-Audio-TTS/CosyVoice: cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-plus, cosyvoice-v3-flash, cosyvoice-v2
Qwen-TTS:
Qwen3-TTS-Instruct-Flash-Realtime: qwen3-tts-instruct-flash-realtime (stable, currently equivalent to qwen3-tts-instruct-flash-realtime-2026-01-22), qwen3-tts-instruct-flash-realtime-2026-01-22 (latest snapshot)
Qwen3-TTS-VD-Realtime: qwen3-tts-vd-realtime-2026-01-15 (latest snapshot), qwen3-tts-vd-realtime-2025-12-16 (snapshot)
Qwen3-TTS-VC-Realtime: qwen3-tts-vc-realtime-2026-01-15 (latest snapshot), qwen3-tts-vc-realtime-2025-11-27 (snapshot)
Qwen3-TTS-Flash-Realtime: qwen3-tts-flash-realtime (stable, currently equivalent to qwen3-tts-flash-realtime-2025-11-27), qwen3-tts-flash-realtime-2025-11-27 (latest snapshot), qwen3-tts-flash-realtime-2025-09-18 (snapshot)
Qwen-TTS-Realtime: qwen-tts-realtime (stable, currently equivalent to qwen-tts-realtime-2025-07-15), qwen-tts-realtime-latest (latest, currently equivalent to qwen-tts-realtime-2025-07-15), qwen-tts-realtime-2025-07-15 (snapshot)
Supported voices
Different models support different voices. Set the voice request parameter to the value in the voice parameter column of the corresponding voice list.
API reference
FAQ
Q: How do I fix incorrect pronunciation in speech synthesis? How do I control the pronunciation of polyphonic characters?
Replace the polyphonic character with a homophone to quickly fix the pronunciation issue.
Use SSML markup to control pronunciation .
Q: How do I troubleshoot silent audio when using a cloned voice?
Verify the voice status
Call the Voice cloning/design API interface and confirm that the voice
statusisOK.Check model version consistency
Make sure the
target_modelparameter used during voice cloning matches themodelparameter used for speech synthesis. For example:Cloning used
cosyvoice-v3-plusSynthesis must also use
cosyvoice-v3-plus
Verify the source audio quality
Check whether the source audio used for voice cloning meets the requirements in Voice cloning/design API:
Audio duration: 10-20 seconds
Clear audio quality
No background noise
Check the request parameters
Confirm that the
voiceparameter in the speech synthesis request is set to the cloned voice ID.
Q: What should I do if the synthesized audio from a cloned voice is unstable or incomplete?
If the synthesized audio from a cloned voice has any of the following issues:
Audio playback is incomplete, with only part of the text spoken
Synthesis quality is inconsistent
Audio contains abnormal pauses or silent segments
Possible cause: The source audio doesn't meet the quality requirements.
Solution: Check whether the source audio meets the requirements in Recording guide for voice cloning. Re-record the audio following the recording guidelines.
Q: Why does the actual duration of synthesized audio differ from the duration shown in the WAV file?
Speech synthesis uses a streaming mechanism that returns data as it's generated. The duration in the saved WAV file header is an estimate and may be inaccurate. For precise duration, set the format to pcm, wait for the full synthesis result, and then add the WAV file header manually.
Q: Why can't the audio file play?
Troubleshoot based on your scenario:
Audio saved as a complete file (such as xx.mp3)
Audio format consistency: The audio format in the request parameters must match the file extension (for example, if the parameter is wav, the file must be .wav).
Player compatibility: Confirm that the player supports the audio format and sample rate.
Streaming audio playback
Save the audio stream as a complete file and try playing it with a media player. If the file doesn't play, refer to scenario 1 above.
If the file plays correctly, the issue is in the streaming playback implementation. Confirm that the player supports streaming playback (such as ffmpeg, pyaudio, AudioFormat, or MediaSource).
Q: Why is audio playback stuttering?
Troubleshoot with the following steps:
Check the text sending rate: Make sure the sending interval is reasonable to avoid the previous audio segment finishing before the next text arrives.
Check callback function performance:
Confirm that no blocking logic exists in the callback function.
Callbacks run on the WebSocket thread. Blocking operations affect data reception. Write audio data to a separate buffer and process it in another thread.
Check network stability: Network fluctuations can cause audio transmission interruptions or delays.
Q: Why is speech synthesis taking a long time?
Troubleshoot with the following steps:
Check input intervals
For streaming synthesis, check whether the text sending interval is too long. Long intervals increase the total synthesis time.
Analyze performance metrics
First-packet latency: normally around 500 ms.
RTF (Real-Time Factor = total synthesis time / audio duration): should be less than 1.0.
Q: How do I restrict an API key to only the speech synthesis service (permission isolation)?
Create a new workspace and grant access to specific models only. This limits the scope of the API key. For details, see Manage workspaces.