Qwen-Audio-3.0-Realtime real-time voice conversation Android SDK - Alibaba Cloud Model Studio

Use the Android SDK for Qwen-Audio-3.0-Realtime to build real-time voice conversations with audio input and audio or text output.

User guide: For model introductions and selection advice, see Real-time voice conversation.

Quick start

Obtain and configure an API key.
Download the SDK and run the sample code:
- Download the latest SDK package.
- Extract the ZIP package. Obtain the AAR SDK from app/libs and add it to your project dependencies. For Android C++ integration, use android_libs and android_include in the ZIP package to obtain the dynamic libraries and header files.
- Open the project in Android Studio. The sample code is in DashQwenAudioChatActivity.java. Replace the API key to try the feature.

Call procedure

Initialize the SDK.
Set parameters for your use case. Use the parameters argument of initialize to set the connection and control parameters, and use setParams to set the voice conversation parameters.
Call startDialog to start the conversation.
In onNuiAudioStateChanged, start the recording device based on the audio state.
Continuously supply recording data in onNuiNeedAudioData, or call updateAudio to actively push recording data.
Continuously receive the audio returned by the model in onNuiAssistEventCallback.
Listen for events and obtain event information in onNuiEventCallback.
Call stopDialog to stop the conversation, and listen for EVENT_TRANSCRIBER_COMPLETE to confirm that it has ended.
When the conversation feature is no longer needed, call release to release the SDK resources.

Request parameters

Connection and control parameters

Pass a JSON string in the parameters argument of initialize.

Example: The following JSON string does not list every parameter. Add parameters as needed for your use case.

{
  "url": "wss://dashscope.aliyuncs.com/api-ws/v1/inference",
  "apikey": "st-****",
  "device_id": "my_device_id",
  "service_mode": "1"
}

Parameters

Parameter	Type	Required	Description
`url`	`String`	Yes	Service endpoint: `wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` China (Beijing): `wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` Singapore: `wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>` Replace `{WorkspaceId}` with your actual Workspace ID.
`apikey`	`String`	Yes	API key.
`service_mode`	`String`	Yes	Runtime mode. Set this parameter to `"1"` for real-time voice conversation.
`device_id`	`String`	Yes	A unique string that identifies the end user. You can use an in-app user ID or a client-generated device identifier. This ID is mainly used for log tracing and troubleshooting.
`audio_update_manually`	`String`	No	Whether to actively push audio data. Default: `"false"`. If set to `"true"` and the SDK supports on-device audio capabilities such as AEC and VAD, those capabilities are enabled by default.
`workspace`	`String`	No	The storage path for on-device resource files. This parameter is required when `audio_update_manually` is `"true"` and an on-device audio capability such as AEC or VAD is enabled.
`debug_path`	`String`	No	The log file storage path. This parameter takes effect only when `save_log` is `true` in `initialize`. In this case, the path is required. The SDK retains at most two log files locally.
`save_wav`	`String`	No	Whether to save debug audio under `debug_path`. Default: `"false"`. Valid values are `"true"` and `"false"`. This parameter takes effect only when `save_log` is `true`, and `debug_path` must also be set.
`max_log_file_size`	`int`	No	The maximum log file size in bytes. This parameter takes effect only when `save_log` is `true`. Default: `104857600` (100 × 1024 × 1024 bytes, or 100 MiB).
`log_track_level`	`int`	No	The filter level for logs sent through `onNuiLogTrackCallback`. Default: `2`. Valid values: `0` (VERBOSE), `1` (DEBUG), `2` (INFO), `3` (WARNING), `4` (ERROR), and `5` (NONE). A log is returned only when its level is greater than or equal to both `log_track_level` and the `level` passed to `initialize`. For example, if `log_track_level` is `2` (INFO) and `level` is `3` (WARNING), only WARNING and higher-level logs (values greater than or equal to `3`) are returned.
`aec_params`	`object`	No	Advanced on-device AEC configuration. This object takes effect only when `audio_update_manually` is `"true"`.
`aec_params.enable_aec`	`boolean`	No	Whether to enable on-device AEC. If `audio_update_manually` is `"true"` and the SDK supports on-device AEC, AEC is enabled by default.
`aec_params.save_audio`	`boolean`	No	Whether to save audio processed by the on-device AEC module. If `save_wav` is `"true"` and `debug_path` is set, this feature is enabled by default and the audio is saved under `debug_path`.
`aec_params.enable_aec_data_callback`	`boolean`	No	Whether to return AEC-processed audio through `EVENT_AEC_DATA` in `onNuiAssistEventCallback`. Default: `false`.
`vad_params`	`object`	No	Advanced on-device VAD configuration. This object takes effect only when `audio_update_manually` is `"true"`.
`vad_params.enable_vad`	`boolean`	No	Whether to enable on-device VAD. If `audio_update_manually` is `"true"` and the SDK supports on-device VAD, VAD is enabled by default.
`vad_params.save_audio`	`boolean`	No	Whether to save audio processed by the on-device VAD module. If `save_wav` is `"true"` and `debug_path` is set, this feature is enabled by default and the audio is saved under `debug_path`.

Voice conversation parameters

Pass a JSON string in the params argument of setParams.

Example: The following JSON string does not list every parameter. Add parameters as needed for your use case.

{
  "service_type": 4,
  "nls_config": {
    "model": "qwen-audio-3.0-realtime-plus",
    "sr_format": "pcm"
  }
}

Parameters

Top-level parameter	Type	Required	Description
`service_type`	`int`	Yes	Voice service type. Set this parameter to `4` for real-time voice conversation.
`nls_config`	`object`	Yes	Core voice conversation configuration, including model selection and conversation behavior.
`nls_config.model`	`string`	Yes	Model name. Supports the `qwen-audio-3.0-realtime-plus` and `qwen-audio-3.0-realtime-flash` model series.
`nls_config.sr_format`	`string`	Yes	Input audio format. Only `pcm` is supported. The default format is 16 kHz, 16-bit, mono PCM.
`nls_config.modalities`	`string`	No	A string containing an array of output modalities. Valid values: `["text"]`: returns text only. `["audio", "text"]` (default): returns both audio and text.
`nls_config.voice`	`string`	No	TTS voice. Default: `longanqian`. This parameter can be set only in the first `session.update`; later values are ignored. System voices: `longanqian`, `longanlingxin`, `longanlingxi`, `longanxiaoxin`, and `longanlufeng`. You can also specify a cloned `voice_id` created through the voice cloning API. See Voice configuration.
`nls_config.enable_speech_emotion`	`boolean`	No	Whether to enable enhanced emotional expression. When enabled, the response voice has more noticeable emotional variation. Default: `true`. Valid values: `true` and `false`.
`nls_config.instructions`	`string`	No	System instructions that define the model's role, response style, and behavioral preferences for the entire session.
`nls_config.max_history_turns`	`int`	No	Maximum number of historical question-answer turns allowed in a request. Valid values: 1 to 50. Default: `20`.
`nls_config.tools`	`string`	No	A string containing an array of Function Calling tool definitions. After you configure this parameter, the model decides whether to call a tool based on the user input. Each definition uses `type` (required and fixed to `function`) and a `function` object containing `name` (required), `description` (optional), and `parameters` (optional). If a function has no arguments, omit `parameters`. Example: `[ { "type": "function", "function": { "name": "get_weather", "description": "Queries weather information for a specified city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City" } }, "required": [ "city" ] } } } ]`
`nls_config.turn_detection`	`string`	No	A string containing a JSON object for turn detection. If omitted, the session uses push-to-talk mode, in which audio is committed and inference is triggered manually. If set, duplex conversation mode is enabled.
`nls_config.turn_detection.type`	`string`	No	VAD type. `server_vad` (default) detects speech boundaries from acoustic features and automatically triggers inference. `smart_turn` combines acoustic and semantic signals; sounds without semantic content, such as filler sounds, do not start a turn or interrupt model playback.
`nls_config.turn_detection.threshold`	`float`	No	VAD sensitivity. This parameter applies only to `server_vad` and has no effect on `smart_turn`. Lower values make VAD more sensitive to quiet sounds and background noise; higher values require clearer and louder speech. Valid range: `[-1.0, 1.0]`. Default: `0.5`.
`nls_config.turn_detection.silence_duration_ms`	`int`	No	Minimum silence duration after speech, in milliseconds, before the model response is triggered. This parameter applies only to `server_vad` and has no effect on `smart_turn`. Lower values reduce latency but can trigger on brief pauses. Valid range: `[200, 6000]`. Default: `800`. Recommended for conversation: `400` to `800`.
`nls_config.turn_detection.voiceprint_audio_urls`	`string`	No	A string containing an array of publicly accessible prerecorded audio URLs for the target speaker. This parameter applies only to `smart_turn`. In duplex conversations, it helps the model focus on the target speaker and ignore other speakers and background noise. Up to five URLs are supported. Audio must be 16 kHz PCM or WAV.

Key APIs

NativeNui

initialize

Initializes a voice conversation SDK instance. The SDK is a singleton. Do not initialize it more than once before you call release.

This method blocks, so call it on a non-UI thread.

Method signature

public synchronized int initialize(final INativeNuiCallback callback,
                                   String parameters,
                                   final Constants.LogLevel level,
                                   final boolean save_log)

Parameters

Parameter	Type	Description
`callback`	INativeNuiCallback	The implementation of the event and data callback interface.
`parameters`	`String`	A JSON string that contains the authentication, connection, and debug parameters. See Connection and control parameters.
`level`	`Constants.LogLevel`	Controls the print level of the SDK's own logs.
`save_log`	`boolean`	Whether to save local logs. If set to `true`, specify the path through `debug_path` in the Connection and control parameters, and optionally set the file size through `max_log_file_size`.

Return value

An error code. See Error code reference.

setParams

Sets the Voice conversation parameters in JSON format. Call this method before startDialog.

Method signature

public synchronized int setParams(String params)

Parameters

Parameter	Type	Description
`params`	`String`	Voice conversation parameters.

Return value

An error code. See Error code reference.

startDialog

Starts the conversation.

Method signature

public synchronized int startDialog(VadMode vad_mode, String dialog_params)

Parameters

Parameter Type Description

Parameter	Type	Description
`vad_mode`	`VadMode`	VAD mode. Fixed to `VadMode.TYPE_P2T`.
`dialog_params`	`String`	If `apikey` in the connection and control parameters is a temporary API key, update it here after it expires. JSON format: `{ "apikey": "st-****" }`

vad_mode

VadMode

VAD mode. Fixed to VadMode.TYPE_P2T.

dialog_params

String

If apikey in the connection and control parameters is a temporary API key, update it here after it expires.
JSON format:

{
  "apikey": "st-****"
}

Return value

An error code. See Error code reference.

stopDialog

Ends the conversation. After you call this method, the server returns the final conversation result and ends the task.

Method signature

public synchronized int stopDialog();

Return value

An error code. See Error code reference.

cancelDialog

Ends the conversation immediately. After you call this method, the task ends at once without waiting for the server to return the final conversation result.

Method signature

public synchronized int cancelDialog();

Return value

An error code. See Error code reference.

updateAction

Sends a conversation action command during an interaction to update runtime behavior such as the conversation context.

Method signature

public synchronized int updateAction(String params);

Parameters

Parameter	Type	Description
`params`	`String`	A JSON string used to update runtime behavior such as the conversation context.
`params.type`	`String`	Set to `"action"`.
`params.command`	`String`	Runtime command. Valid values: `function_call`: updates a function call request. `play_start`: when on-device AEC is used, notifies the SDK that audio playback has started. `play_over`: when on-device AEC is used, notifies the SDK that audio playback has ended.
`params.context`	`String`	Function call request update. Used when `command` is `"function_call"`.
`params.context.type`	`String`	Event type. This parameter is required when `command` is `"function_call"`. `conversation.item.create` inserts a conversation item for historical context, supplemental text, or a tool result. After sending `conversation.item.create`, use `response.create` to trigger another inference.
`params.context.item`	`object`	Required when `params.context.type` is `conversation.item.create`. The conversation item to create.
`params.context.response`	`object`	Optional when `params.context.type` is `response.create`. Overrides the session defaults for this inference. If omitted, the current session configuration is used.

context.item parameters:

Parameter	Type	Description
`id`	`String`	Optional unique conversation item ID. If omitted, the server generates one. An error is returned if the specified ID already exists.
`type`	`String`	Required item type. Valid values: `message`: a regular message. `function_call`: a function call request. This type is typically generated by the server. The client can also use it to add historical context. `function_call_output`: a tool execution result. After receiving a `function_call`, the client runs the tool and uses this type to write back the result.
`role`	`String`	Required for `message`. Valid values: `system`, `user`, and `assistant`.
`content`	`array`	Required for `message`. Each item contains a `type` and the associated data field. `system` supports `input_text` with `text`; `user` supports `input_text` with `text` and `input_audio` with Base64-encoded `audio`; `assistant` supports `output_text` with `text`.
`call_id`	`String`	Required for `function_call` and `function_call_output`. The unique ID that associates a function call request with its result.
`name`	`String`	Required for `function_call`. The name of the function to call.
`arguments`	`String`	Required for `function_call`. Function arguments as a JSON string.
`output`	`String`	Required for `function_call_output`. Tool execution result as a JSON string.

context.response parameters:

Parameter	Type	Description
`modalities`	`array`	Output modalities. `["text"]` returns text only. `["audio", "text"]` (default) returns both audio and text.
`voice`	`string`	Overrides the TTS voice for this inference.

Example:

{
  "type": "action",
  "command": "function_call",
  "context": {
    "item": {
      "call_id": "call_xxxx",
      "output": "{\"city\":\"Hangzhou\",\"condition\":\"sunny\",\"temperature\":18}",
      "type": "function_call_output"
    },
    "type": "conversation.item.create"
  }
}

{
  "type": "action",
  "command": "function_call",
  "context": {
    "response": {
      "modalities": ["text", "audio"]
    },
    "type": "response.create"
  }
}

Return value

An error code. See Error code reference.

updateAudio

When audio_update_manually is set to "true", call this method to actively push recording data instead of supplying the data through onNuiNeedAudioData.

Method signature

public synchronized int updateAudio(byte[] data, int len,
                                    boolean first_pack);

Parameters

Parameter	Type	Description
`data`	`byte[]`	The audio data to push.
`len`	`int`	The number of bytes of audio data to push.
`first_pack`	`boolean`	Ignore this parameter.

Return value

An error code. See Error code reference.

updateRefAudio

When audio_update_manually is set to "true" and on-device AEC is enabled, call this method to push the audio played by the player as the reference signal.

Method signature

public synchronized int updateRefAudio(byte[] data, int len,
                                       boolean first_pack);

Parameters

Parameter	Type	Description
`data`	`byte[]`	The audio data to push.
`len`	`int`	The number of bytes of audio data to push.
`first_pack`	`boolean`	Ignore this parameter.

Return value

An error code. See Error code reference.

release

Releases all internal resources of the SDK. After you call this method, the SDK instance becomes unavailable. To use it again, you must reinitialize it by calling initialize.

Method signature

public synchronized int release();

Return value

An error code. See Error code reference.

GetVersion

Gets the current SDK version information.

Method signature

public synchronized String GetVersion();

Return value

The current SDK version information.

INativeNuiCallback: listener callbacks

onNuiEventCallback: listen for event information

Method signature

void onNuiEventCallback(NuiEvent event, final int resultCode, final int arg2, KwsResult kwsResult, AsrResult asrResult);

Parameters

Parameter	Type	Description
`event`	NuiEvent	The callback event.
`resultCode`	`int`	The error code. Valid when the EVENT_ASR_ERROR event occurs.
`arg2`	`int`	A reserved parameter.
`asrResult`	`AsrResult`	The speech recognition result.
`kwsResult`	`KwsResult`	The voice wake-up feature. You do not need to use this parameter.

onNuiAudioStateChanged: listen for the audio state

The SDK uses this callback to notify you when to start or stop recording.

Method signature

void onNuiAudioStateChanged(AudioState state);

AudioState states

State	Description
`STATE_OPEN`	The interaction has started. You can open the recording device and start recording.
`STATE_PAUSE`	The interaction has stopped. You can stop recording.
`STATE_CLOSE`	The SDK instance has been released. You can fully close the recording device.

onNuiNeedAudioData: supply audio data

After the conversation starts, this callback is triggered continuously. Supply the audio data in this callback. You do not need to use this callback when audio_update_manually is set to "true".

Method signature

int onNuiNeedAudioData(byte[] buffer, int len);

Parameters

Parameter	Type	Description
`buffer`	`byte[]`	The audio data to fill.
`len`	`int`	The number of bytes of audio data to fill.

Return value

The number of bytes actually filled.

onNuiAssistEventCallback: receive auxiliary events and data

This callback receives auxiliary events and related data from the SDK.

Method signature

void onNuiAssistEventCallback_(int event, byte[] info, int info_len,
                               byte[] data);

Parameters

Parameter	Type	Description
`event`	`int`	A `NuiEvent` event.
`info`	`String`	Ignore this parameter.
`info_len`	`int`	Ignore this parameter.
`data`	`byte[]`	Auxiliary data, such as TTS audio returned by the model.

onNuiLogTrackCallback: listen for trace logs

This callback receives detailed internal logs from the SDK to help with troubleshooting and debugging.

default void onNuiLogTrackCallback(Constants.LogLevel level, String log)

`NuiEvent`: event types

Event	Description
`EVENT_TRANSCRIBER_STARTED`	The task started successfully.
`EVENT_VAD_START`	Triggered immediately after the task starts. This does not mean that the start of speech has been detected.
`EVENT_VAD_END`	The end of speech was detected.
`EVENT_ASR_PARTIAL_RESULT`	An intermediate speech recognition result.
`EVENT_ASR_ERROR`	An error occurred during the voice conversation.
`EVENT_MIC_ERROR`	Triggered when no audio data is received for two consecutive seconds.
`EVENT_SENTENCE_END`	The end of a sentence was detected and a complete recognition result was returned.
`EVENT_TRANSCRIBER_COMPLETE`	The voice conversation ended.
`EVENT_AUDIO_TRANSCRIPTION`	An incremental text transcript event for audio output. Transcript segments are returned in streaming mode.
`EVENT_AUDIO_TRANSCRIPTION_COMPLETED`	The transcript for audio output is complete.
`EVENT_OTHER_RESULT`	Other event information, such as a Function Calling result.
`EVENT_ASR_TTS_START`	The model started returning TTS audio.
`EVENT_ASR_TTS_DATA`	TTS audio returned by the model.
`EVENT_ASR_TTS_COMPLETE`	The model finished returning TTS audio.
`EVENT_AEC_DATA`	Audio data processed by AEC.

Quick start

Call procedure

Request parameters

Connection and control parameters

Voice conversation parameters

Key APIs

NativeNui

initialize

setParams

startDialog

stopDialog

cancelDialog

updateAction

updateAudio

updateRefAudio

release

GetVersion

INativeNuiCallback: listener callbacks

onNuiEventCallback: listen for event information

onNuiAudioStateChanged: listen for the audio state

onNuiNeedAudioData: supply audio data

onNuiAssistEventCallback: receive auxiliary events and data

onNuiLogTrackCallback: listen for trace logs

NuiEvent: event types

`NuiEvent`: event types