Alibaba Cloud Model Studio:Qwen-Audio-TTS/CosyVoice Real-time Speech Synthesis HarmonyOS SDK

Learn about HarmonyOS SDK integration, parameters, APIs, callbacks, and sample code for real-time speech synthesis.

For model descriptions and selection guidance, see Speech synthesis.

NativeNui

The HarmonyOS SDK provides streaming text-to-speech through NativeNui.

• Create a streaming text-to-speech instance by calling new NativeNui(Constants.ModeType.MODE_STREAM_INPUT_TTS). NativeNui.GetInstance() returns a MODE_DIALOG singleton and cannot be used for streaming text-to-speech.
• Do not call initialize() in streaming text-to-speech mode. Pass credentials and synthesis parameters directly to startStreamInputTts(), playStreamInputTts(), or asyncPlayStreamInputTts().
• Receive synthesis events and audio through INativeStreamInputTtsCallback.
• The SDK returns STREAM_INPUT_TTS_EVENT_SYNTHESIS_STARTED when the task starts, audio through onStreamInputTtsDataCallback, STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE when the task ends, and STREAM_INPUT_TTS_EVENT_TASK_FAILED when synthesis fails.

import { Constants, INativeStreamInputTtsCallback, NativeNui, StreamInputTtsEvent } from 'neonui';

const nuiInstance = new NativeNui(Constants.ModeType.MODE_STREAM_INPUT_TTS);

Call flows

Qwen-Audio-TTS/CosyVoice supports one-shot input and streaming input.

One-shot input is suitable for short text or scenarios that require SSML.

1. Call playStreamInputTts or asyncPlayStreamInputTts to pass the complete text and start synthesis. The former blocks until synthesis finishes. The latter returns immediately and synthesizes in the background. Do not call startStreamInputTts first, and do not call a stop method afterward.
2. Receive audio in onStreamInputTtsDataCallback.
3. Synthesis ends when STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE is received.

Streaming input is suitable for real-time conversations or incremental synthesis of long text. SSML is not supported in this mode.

1. Call startStreamInputTts to open the connection and configure the callback and parameters.
2. Call sendStreamInputTts to send text fragments.
3. Receive audio in onStreamInputTtsDataCallback.
4. After all text is sent, call stopStreamInputTts.
5. Synthesis ends when STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE is received.

When text-to-speech is no longer needed, call releaseStreamInputTts to release resources.

Both the length of a single text input and the cumulative length of multiple inputs are limited. See the CosyVoice WebSocket API.

startStreamInputTts

Starts bidirectional streaming synthesis, opens the connection, and registers the callback. This method can block. Do not call it on the UI thread.

startStreamInputTts(
  callback: INativeStreamInputTtsCallback,
  ticket: string,
  parameters: string,
  session_id: string,
  log_level: number,
  save_log: boolean
): number

Parameter	Type	Description
callback	INativeStreamInputTtsCallback	Event and audio callback.
ticket	string	JSON string that contains authentication, connection, and debugging settings.
parameters	string	JSON string that contains synthesis settings.
session_id	string	Client-specified session ID. Pass an empty string to let the server generate one.
log_level	number	SDK log level. Use a Constants.LogLevel value: 0 (VERBOSE), 1 (DEBUG), 2 (INFO), 3 (WARNING), 4 (ERROR), or 5 (NONE).
save_log	boolean	Whether to save logs locally. If set to true, set debug_path in ticket.

The method returns an error code. Constants.NuiResultCode.SUCCESS (0) indicates success.

ticket fields

{
  "url": "wss://dashscope.aliyuncs.com/api-ws/v1/inference",
  "apikey": "st-****",
  "device_id": "my_device_id"
}

Field	Type	Required	Description
url	string	Yes	Service endpoint. Use the public endpoint wss://dashscope.aliyuncs.com/api-ws/v1/inference, or a workspace-specific endpoint: wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference for Beijing or wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/inference for Singapore. Replace {WorkspaceId} with your workspace ID.
apikey	string	Yes	API key. Use a temporary API key to reduce the risk of exposing a long-lived key.
device_id	string	Yes	A unique end-user identifier, such as an in-app user ID or a client-generated device ID. It is used mainly for log tracing and troubleshooting.
complete_waiting_ms	number	No	Time in milliseconds to wait for the synthesis-complete event after stopStreamInputTts(false). Default: 10000.
debug_path	string	No	Log directory. This field is required when save_log is true. The SDK keeps at most two log files.
max_log_file_size	number	No	Maximum size of a single log file in bytes. Default: 104857600 (100 MiB). This field takes effect only when save_log is true.
log_track_level	number	No	Internal trace log filter level. Default: 2. Valid values are the same as log_level. The HarmonyOS callback interface does not currently expose a streaming TTS log callback, so the filtered logs are written only by the SDK.

parameters fields

{
  "model": "qwen-audio-3.0-tts-flash",
  "voice": "longanlingxi",
  "format": "mp3",
  "sample_rate": 24000,
  "volume": 50,
  "rate": 1.0,
  "pitch": 1.0,
  "enable_audio_decoder": true
}

Field	Type	Required	Description
model	string	Yes	Model name. See Speech synthesis models.
voice	string	Yes	Voice. For system voices, see Qwen-Audio-TTS voices and CosyVoice voices. You can also use a voice created through voice cloning or Voice Design.
format	string	No	Audio encoding format: pcm, wav, mp3 (default), or opus. cosyvoice-v1 does not support Opus.
enable_audio_decoder	boolean	No	Whether to enable the SDK decoder. Default: false. For MP3 or Opus, set this to true to decode the audio to PCM before it is returned through the data callback.
volume	number	No	Volume. Default: 50. Valid range: [0, 100].
sample_rate	number	No	Sample rate in Hz. Valid values: 8000, 16000, 22050 (default), 24000, 44100, and 48000.
rate	number	No	Speech rate. Default: 1.0. Valid range: [0.5, 2.0].
pitch	number	No	Pitch. Default: 1.0. Valid range: [0.5, 2.0].
bit_rate	number	No	MP3 or Opus bitrate in kbps. Default: 32. Valid range: [6, 510]. cosyvoice-v1 does not support this field.
enable_ssml	boolean	No	Whether to enable SSML. Default: false. See SSML limits.
word_timestamp_enabled	boolean	No	Whether to return word-level timestamps. Default: false. This field is available only for streaming output. It supports cloned voices of qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2, and system voices marked as supported in the Qwen-Audio-TTS voice list and CosyVoice voice list. Cloned voices of other models are not supported. Timestamp results are included in all_response of INativeStreamInputTtsCallback.
seed	number	No	Random seed used to vary the synthesized result. If the model version, text, voice, and all other parameters are the same, the same seed reproduces the same result. Default: 0. Valid range: [0, 65535]. cosyvoice-v1 does not support this field.
language_hints	string[]	No	Target synthesis language. This setting improves synthesis and is independent of the language of the sample audio used for voice cloning. To set the source language of a voice cloning task, see the Voice Cloning API reference. The current version uses only the first array element, so pass one value. Use this field when the reading of numbers, abbreviations, or symbols is unexpected, or when synthesis in a less common language is unnatural. For example, it can make "hello, this is 110" read as “one one zero” in English instead of the Chinese reading, or make @ read as “at”. Supported values: zh, en, fr, de, ja, ko, ru, pt, th, id, vi, es, it, ms, fil, and ar. cosyvoice-v1 does not support this field.
instruction	string	No	Instruction that controls dialect, emotion, or role. See Instruction control.
enable_aigc_tag	boolean	No	Whether to embed an invisible AIGC tag. Default: false. Supported by qwen-audio-3.0-tts-plus, qwen-audio-3.0-tts-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2.
aigc_propagator	string	No	ContentPropagator field in the AIGC tag. Takes effect only when enable_aigc_tag is true. Default: the Alibaba Cloud UID. The supported models are the same as for enable_aigc_tag.
aigc_propagate_id	string	No	PropagateID field in the AIGC tag. Takes effect only when enable_aigc_tag is true. Default: the current request ID. The supported models are the same as for enable_aigc_tag.
hot_fix	object	No	Text hot-fix settings for custom pronunciation and text replacement. cosyvoice-v2 and cosyvoice-v1 do not support this field. Qwen-Audio-3.0-TTS and other supported models do. For the schema, see Client events.

sendStreamInputTts

sendStreamInputTts(text: string): number

Sends a text fragment after startStreamInputTts succeeds. This method does not parse SSML tags. After all text is sent, call stopStreamInputTts().

Parameter	Type	Description
text	string	Text to synthesize. SSML is not supported. SSML tags are read as ordinary text.

The method returns an error code.

stopStreamInputTts

stopStreamInputTts(flag_async: boolean = true): number

Ends streaming input.

• true (default): End asynchronously and return immediately. Wait for STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE to determine that synthesis has completed.
• false: Block until all audio and the synthesis-complete event are received. The timeout is controlled by complete_waiting_ms.

Calling a cancel method after a synchronous stop can block. We recommend the default asynchronous mode.

Parameter	Type	Description
flag_async	boolean	Whether to end asynchronously. Default: true. true returns without waiting for a server response. false blocks until synthesis is complete.

The method returns an error code.

cancelStreamInputTts

cancelStreamInputTts(): number

Immediately closes the connection and terminates the current task. No more audio callbacks are received after this call. The method returns an error code.

cancelStreamInputTtsKeepConnection

cancelStreamInputTtsKeepConnection(): number

Sends a protocol-level cancel command to terminate the current task but keep the WebSocket connection open. Use this method when another synthesis round must start immediately, avoiding the cost of reopening the connection. The method returns an error code.

playStreamInputTts

playStreamInputTts(
  callback: INativeStreamInputTtsCallback,
  ticket: string,
  parameters: string,
  text: string,
  session_id: string,
  log_level: number,
  save_log: boolean
): number

Synchronous one-shot synthesis. This method initializes the task, sends the text, receives all audio, and then returns. Do not call startStreamInputTts first, and do not call a stop method afterward. SSML is enabled by default. If enable_ssml is explicitly set, that value takes precedence. Do not call this method on the UI thread.

callback, ticket, parameters, session_id, log_level, and save_log are defined in startStreamInputTts. text is the text to synthesize and supports SSML. The method returns an error code.

asyncPlayStreamInputTts

asyncPlayStreamInputTts(
  callback: INativeStreamInputTtsCallback,
  ticket: string,
  parameters: string,
  text: string,
  session_id: string,
  log_level: number,
  save_log: boolean
): number

Asynchronous one-shot synthesis. This method returns immediately and delivers results through callbacks. Do not call startStreamInputTts first, and do not call a stop method afterward. SSML is enabled by default. If enable_ssml is explicitly set, that value takes precedence.

callback, ticket, parameters, session_id, log_level, and save_log are defined in startStreamInputTts. text is the text to synthesize and supports SSML. The method returns an error code.

releaseStreamInputTts

releaseStreamInputTts(): number

Releases the streaming TTS instance and its resources. Call this method when the page is destroyed or text-to-speech is no longer needed. The method returns an error code.

INativeStreamInputTtsCallback

export interface INativeStreamInputTtsCallback {
  onStreamInputTtsEventCallback(
    event: StreamInputTtsEvent,
    task_id: string,
    session_id: string,
    ret_code: number,
    error_msg: string,
    timestamp: string,
    all_response: string
  ): void;

  onStreamInputTtsDataCallback(data: ArrayBuffer | null): void;
}

onStreamInputTtsEventCallback

Parameter	Type	Description
event	StreamInputTtsEvent	Synthesis event.
task_id	string	Synthesis task ID.
session_id	string	Session ID. A client-specified value is returned unchanged. Otherwise, the server generates one.
ret_code	number	Error code. Valid only for a task-failed event.
error_msg	string	Error message. Valid only for a task-failed event.
timestamp	string	Timestamp result.
all_response	string	Complete server response as a JSON string. Parse it for usage, timestamp, and error details.

onStreamInputTtsDataCallback

onStreamInputTtsDataCallback(data: ArrayBuffer | null): void;

Continuously returns audio fragments. Note the following:

• MP3 and Opus data requires a streaming decoder. Alternatively, set enable_audio_decoder to true to let the SDK return PCM.
• To assemble a complete file, append callback data in order.
• For WAV and MP3, only the first callback contains the file header. Each Opus frame is an independent Ogg page and can be concatenated in order.

StreamInputTtsEvent

Event	Description
STREAM_INPUT_TTS_EVENT_SYNTHESIS_STARTED	The server has accepted the request and started processing. The first audio data usually arrives through onStreamInputTtsDataCallback soon after this event.
STREAM_INPUT_TTS_EVENT_SENTENCE_BEGIN	The server has started synthesizing an utterance.
STREAM_INPUT_TTS_EVENT_SENTENCE_SYNTHESIS	Synthesis progress information, including billing information and timestamps.
STREAM_INPUT_TTS_EVENT_SENTENCE_END	The server has completed synthesis of an utterance.
STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE	The server has returned all audio data. onStreamInputTtsEventCallback is not called after this event, which is the explicit end-of-stream signal. This event does not indicate that local playback has finished.
STREAM_INPUT_TTS_EVENT_TASK_FAILED	Synthesis failed. Retrieve task_id, error_code, and error_message from all_response, or use the ret_code and error_msg callback arguments.

Example task-failed response:

{
  "header": {
    "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx",
    "event": "task-failed",
    "error_code": "InvalidParameter",
    "error_message": "[tts:]Engine return error code: 418",
    "attributes": {}
  },
  "payload": {}
}

Sample code

1. Obtain an API key. Do not hard-code a long-lived API key in a client application. We recommend that your application server obtain a temporary API key and send it to the client.
2. Download the latest SDK package. Extract the package, copy entry/libs/neonui.har to the entry/libs directory of your application, and add the dependency to entry/oh-package.json5:

{
  "dependencies": {
    "neonui": "file:libs/neonui.har"
  }
}

3. Open the sample project from the SDK package in DevEco Studio. The sample page is entry/src/main/ets/pages/dashscope/DashCosyVoiceStreamTtsPage.ets. Configure an API key and run the project.

The following code shows the core streaming-input flow. For complete audio playback, parameter selection, and task state management, see DashCosyVoiceStreamTtsPage.ets in the SDK package.

import { Constants, INativeStreamInputTtsCallback, NativeNui, StreamInputTtsEvent } from 'neonui';

const callback: INativeStreamInputTtsCallback = {
  onStreamInputTtsEventCallback: (event: StreamInputTtsEvent, taskId: string,
    sessionId: string, retCode: number, errorMsg: string,
    timestamp: string, allResponse: string): void => {
    if (event == StreamInputTtsEvent.STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE) {
      // Synthesis is complete.
    } else if (event == StreamInputTtsEvent.STREAM_INPUT_TTS_EVENT_TASK_FAILED) {
      // Handle the error based on retCode, errorMsg, or allResponse.
    }
  },
  onStreamInputTtsDataCallback: (data: ArrayBuffer | null): void => {
    if (data != null) {
      // Write PCM to AudioRenderer, or append encoded audio data in order.
    }
  }
};

const nuiInstance = new NativeNui(Constants.ModeType.MODE_STREAM_INPUT_TTS);
const ticket: Record<string, Object> = {
  'url': 'wss://dashscope.aliyuncs.com/api-ws/v1/inference',
  'apikey': 'st-****',
  'device_id': 'my_device_id'
};
const parameters: Record<string, Object> = {
  'model': 'qwen-audio-3.0-tts-flash',
  'voice': 'longanlingxi',
  'format': 'mp3',
  'sample_rate': 24000,
  'enable_audio_decoder': true
};

const result = nuiInstance.startStreamInputTts(
  callback,
  JSON.stringify(ticket),
  JSON.stringify(parameters),
  '',
  Constants.LogLevel.LOG_LEVEL_INFO,
  false
);

if (result == Constants.NuiResultCode.SUCCESS) {
  nuiInstance.sendStreamInputTts('Hello, ');
  nuiInstance.sendStreamInputTts('welcome to real-time speech synthesis.');
  nuiInstance.stopStreamInputTts(true);
}

// Call nuiInstance.releaseStreamInputTts() from the SYNTHESIS_COMPLETE handler.

For one-shot input, directly call playStreamInputTts or asyncPlayStreamInputTts:

const oneShotInstance = new NativeNui(Constants.ModeType.MODE_STREAM_INPUT_TTS);
oneShotInstance.asyncPlayStreamInputTts(
  callback,
  JSON.stringify(ticket),
  JSON.stringify(parameters),
  'Hello, welcome to real-time speech synthesis.',
  '',
  Constants.LogLevel.LOG_LEVEL_INFO,
  false
);
// Call oneShotInstance.releaseStreamInputTts() from the SYNTHESIS_COMPLETE handler.

Advanced features

SSML

Purpose: Embed XML tags in text to control pronunciation, speech rate, pauses, and other synthesis details.

Limits: Only the one-shot APIs playStreamInputTts and asyncPlayStreamInputTts support SSML. The streaming-input API sendStreamInputTts does not.

Usage: The SDK enables SSML by default for playStreamInputTts and asyncPlayStreamInputTts. Pass SSML text in text. For more information, see SSML and LaTeX.

Mathematical expressions

Purpose: Make the model correctly read common mathematical formulas and expressions.

Usage: Pass text that contains a LaTeX-formatted mathematical expression in text. For supported syntax, see LaTeX text-to-speech.