Alibaba Cloud Model Studio:Python SDK

model

str

The Qwen-Omni model to use. See Model list.

ws_url

str

The full WebSocket endpoint. If not specified, the endpoint is automatically selected based on the workspace and region of the Client:

• Singapore region: wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime.

• Beijing region: wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime

Replace {WorkspaceId} with your actual workspace ID.

output_modalities

list[str]

The model output modality. Set to ['text'] for text only, or ['text', 'audio'] for both audio and text.

voice

str

The voice for audio output. For supported voices, see Voice list.

Default voices:

• Qwen3.5-Omniseries model : "Tina"

• qwen3-omni-flash-realtime-2025-09-15series model: "Cherry"

input_audio_config

dict

Input audio format configuration. Use a dict (with type and sample_rate keys) to set type (pcm or wav) and sample_rate (8000, 16000, 24000, or 48000; default: 16000 Hz) together. For usage, see Audio format and sample rate configuration.

Applies to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime.

output_audio_config

dict

Output audio format configuration. Use a dict (with type and sample_rate keys) to set type (pcm or wav) and sample_rate (8000, 16000, 24000, or 48000) together. If no sample rate is specified, 16000 Hz is used by default; explicitly set the desired output rate, such as 24000 Hz.

Applies to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime.

input_audio_format

str

Legacy compatibility field. For new integrations, we recommend that you use input_audio_config to set the format and sample rate together.

output_audio_format

str

Legacy compatibility field. For new integrations, we recommend that you use output_audio_config to set the format and sample rate together.

smooth_output

bool

This parameter is supported only by qwen3-omni-flash-realtime-2025-09-15.

• True: Get a conversational response.

• False: Get a more formal, written-style response.

  However, this may result in poor quality if the content is difficult to read aloud.

• None: The model automatically selects a conversational or formal response style.

instructions

str

A system message that sets the model's objective or role.

For example: You are an AI agent for a five-star hotel. Answer customer questions about room types, facilities, prices, and booking policies. Be accurate and friendly. Always respond with a professional and helpful attitude. Do not provide unverified information or information outside the hotel's scope of services.

enable_input_audio_transcription

bool

Enables speech recognition for input audio.

input_audio_transcription_model

str

Speech recognition model for input audio transcription. The value is always qwen3-asr-flash-realtime. This parameter is not configurable.

turn_detection_type

str

The Voice Activity Detection (VAD) type. Valid values:

• server_vad (default): Detects the end of user speech based on acoustic features.

• semantic_vad: Detects the end of user speech based on semantic validity. This mode can filter out meaningless speech, such as filler words and background noise. This mode is supported by Qwen3.8-Omni-Flash-Realtime and the Qwen3.5-Omni-Realtime series.

turn_detection_threshold

float

VAD detection threshold. Increase in noisy environments and decrease in quiet environments.

• The closer the value is to -1, the more likely noise is to be detected as speech.

• The closer the value is to 1, the less likely noise is to be detected as speech.

Default: 0.2. Valid values: [-1.0, 1.0].

turn_detection_silence_duration_ms

int

Silence duration that indicates end of speech. If exceeded, the model triggers a response. Default: 800. Valid values: [200, 6000].

turn_detection_param

dict

Additional turn_detection configuration parameters. Currently supports idle_timeout_ms (int): the idle timeout in milliseconds. Applies only toqwen3.5-omni-plus-realtimeandqwen3.5-omni-flash-realtimemodels inserver_vadmode. After the server finishes audio playback and the user remains silent beyond this duration (no speech.started triggered), the model proactively generates a response to prompt the user to continue the conversation. Valid range: [5000, 30000].

Example: turn_detection_param={'idle_timeout_ms': 5000}

enable_search

bool

Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models.

Enables web search. Default: false. When enabled, the model can search the web to answer real-time questions.

Tool calling (tools) and web search (enable_search) are incompatible. You cannot enable both at the same time.

search_options

object

Web search options. Takes effect only when enable_search is enabled.

Currently, you can only set enable_source (Boolean), which controls whether to return search result sources. Set to true to enable. Example: search_options={'enable_source': True}.

tools

list[dict]

Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models.

Tool definitions. When provided, the model can call external tools to respond to user questions. For Function Calling, the model does not generate audio and only returns the tool calling parameters.

Qwen3.8-Omni-Flash-Realtime supports Function Calling and MCP tools in the same session. See client events for MCP configuration and MCP call limits for quotas. tools and enable_search are mutually exclusive, including MCP tools.

The following fields describe Function Calling tools.

Each tool is a dictionary that contains the following fields:

• type (string, required): Must be set to "function".

• function (dict, required): The definition of the tool function. It contains the following fields:

  • name (string, required): The custom name of the tool function. We recommend that you use a name that is the same as the function name, such as get_current_weather or get_current_time.

  • description (string, optional): A description of the tool function. The model uses this description to decide whether to call the function.

  • parameters (dict, optional): Input parameter descriptions. The model uses them to extract parameters. If the tool function does not require input parameters, you do not need to specify this field. It contains the following fields:

    • type (string, required): Must be set to "object".

    • properties (dict, optional): Describes the name, data type, and description of each input parameter. The key is the parameter name, and the value is a dictionary that contains the data type (type) and description (description).

    • required (list, optional): Specifies which input parameters are required.

temperature

float

Sampling temperature that controls content diversity.

Higher values produce more diverse content; lower values produce more deterministic content.

Valid values: [0, 2).

Because both temperature and top_p control content diversity, we recommend that you set only one of them.

• qwen3.8-omni-flash-realtime: 0.6

• Qwen3.5-Omni-Realtime series model:0.7

• qwen3-omni-flash-realtime-2025-09-15:0.9

top_p

float

Probability threshold for nucleus sampling that controls content diversity.

Higher values produce more diverse content; lower values produce more deterministic content.

Valid values: (0, 1.0].

Because both temperature and top_p control content diversity, we recommend that you set only one of them.

Default top_p values:

• qwen3.8-omni-flash-realtime: 0.95

• Qwen3.5-Omni-Realtime series model:0.8

• qwen3-omni-flash-realtime-2025-09-15:1.0

top_k

integer

Candidate set size for sampling. For example, 50 means only the 50 highest-scoring tokens form the candidate set. Larger values increase randomness; smaller values increase determinism. Set to None or a value greater than 100 to disable top_k and use only top_p.

The value must be greater than or equal to 0.

Default top_k values:

• qwen3.8-omni-flash-realtime: 20

• Qwen3.5-Omni-Realtime series model:20

• qwen3-omni-flash-realtime-2025-09-15:50

max_tokens

integer

Maximum number of tokens to return.

max_tokens setting does not affect the generation process of the LLM. If the number of tokens generated by the model exceeds max_tokens, the returned content is truncated.

For qwen3.8-omni-flash-realtime, the valid range is [1, 65536]. For other models, default and maximum values equal the model's maximum output length. See the Model Studio console for details.

Use max_tokens to limit output length for generating summaries, keywords, controlling costs, or reducing response time.

repetition_penalty

float

Controls repetition in generated sequences. Higher values reduce repetition. 1.0 means no penalty. qwen3.8-omni-flash-realtime also accepts 0. For other models, the value must be greater than 0.

Default repetition_penalty values:

• qwen3.8-omni-flash-realtime: 0

• Qwen3.5-Omni-Realtime series model:1.0

• Other models: 1.05

presence_penalty

float

Controls content repetition in model output.

Valid values: [-2.0, 2.0]. Positive values reduce repetition; negative values increase it.

Default presence_penalty values:

• qwen3.8-omni-flash-realtime: 1

• Qwen3.5-Omni-Realtime series model:1.5

• Other models: 0.0

Scenarios:

Higher values suit scenarios requiring diversity or creativity, such as creative writing or brainstorming.

Lower values suit scenarios requiring consistency or technical precision, such as technical documents.

seed

integer

Makes generation more deterministic, ensuring consistent results across runs.

Passing the same seed with identical parameters produces the same result as much as possible.

Valid values: 0 to 2³¹−1. Default value: -1.

def connect(self) -> RealtimeSession

Server-side event

Session created

session.updated

Session configuration updated

Creates a connection with the server. This method is automatically called when the with context is entered. Do not call connect() first and then enter the with context of the same session.

def update_session(self,
                   output_modalities: List[str] = ...,
                   *,
                   voice: str = ...,
                   input_audio_format: str = ...,
                   output_audio_format: str = ...,
                   enable_input_audio_transcription: bool = ...,
                   input_audio_transcription_model: str = ...,
                   enable_turn_detection: bool = ...,
                   turn_detection_type: str = ...,
                   prefix_padding_ms: int = ...,
                   turn_detection_threshold: float = ...,
                   turn_detection_silence_duration_ms: int = ...,
                   turn_detection_param: dict = ...,
                   input_audio_config: dict = ...,
                   output_audio_config: dict = ...,
                   **kwargs) -> None

session.updated

Session configuration updated

Updates session configuration. For parameter details, see Request parameters.

After connecting, the server returns default session configurations. Call this method immediately after connecting to update settings.

When the server receives the session.update event, it validates the parameters. Invalid parameters return an error; otherwise the server updates session configuration.

def append_audio(self, audio_b64: Union[str, bytes]) -> None

None

Appends audio data to the cloud input buffer. str is treated as Base64-encoded data; bytes is automatically Base64-encoded. The buffer is temporary storage that can be written to and committed later.

• If "turn_detection" is enabled, the audio buffer is used for voice detection, and the server decides when to commit.
• If "turn_detection" is disabled, the client can send up to 15 MiB of audio per event. Streaming smaller blocks makes VAD more responsive.

def append_video(self, video_b64: Union[str, bytes]) -> None

None

Adds image data to the cloud video buffer. str is treated as Base64-encoded data; bytes is automatically Base64-encoded. Accepts local images or real-time video stream captures.

Image input limits:

• The image format must be JPG or JPEG. The recommended image resolution is 480p or 720p, with a maximum of 1080p.
• A single image after Base64 encoding must not exceed 256 KB. We recommend keeping the raw image size below 190 KB before encoding.
• The image data must be Base64-encoded.
• We recommend that you send images to the server at a frequency of 1 image per second.

def clear_appended_audio(self, ) -> None

input_audio_buffer.cleared

Deletes the audio received by the server

Deletes the audio from the current cloud buffer.

def commit(self, ) -> None

input_audio_buffer.committed

Server received the committed audio

Commits audio and video from the cloud buffer. Returns an error if the buffer is empty.

• If "turn_detection" is enabled, the client does not need to send this event. The server automatically commits the audio buffer.
• If "turn_detection" is disabled, the client must commit the audio buffer to create a user message item.

Note:

1. If audio transcription is configured for the session using input_audio_transcription, the system transcribes the audio.
2. Committing the input audio buffer does not create a response from the model.

def create_response(self,
        modalities: list[str] = ...,
        instructions: str = ...,
        **kwargs) -> None

Server-side event

Server starts generating a response

response.output_item.added

New output content is available in the response

Server-side event

Conversation item created

response.content_part.added

New output content added to the assistant message item

response.audio_transcript.delta

Incrementally generated transcribed text

response.audio.delta

Incrementally generated audio from the model

response.audio_transcript.done

Text transcription completed

response.audio.done

Audio generation completed

response.content_part.done

Streaming of text or audio content for the assistant message is complete

response.output_item.done

Streaming of the entire output item for the assistant message is complete

response.done

Response completed

Instructs the server to create a model response.

When you configure a session in "turn_detection" mode, the server automatically creates a model response.

def cancel_response(self, ) -> None

None

Cancels the in-progress response. If there is no response to cancel, the server responds with an error.

def create_item(self, item: dict) -> None

None

Sends a conversation.item.create event to the server.

For a Function Calling tool result, item is a dictionary with these fields:

• type: Must be function_call_output.
• call_id: The call_id from the response.function_call_arguments.done event.
• output: A string that contains the tool execution result.

For MCP approval, the same event uses item.type="mcp_approval_response". See MCP approval response for its fields.

def close(self, ) -> None

None

Terminates the task and closes the connection (idempotent; does not send session.finish).

session.session_id
session.last_response_id

None

Gets the session_id of the current task and the response_id of the last response (session attributes; additional first-packet delay metrics such as last_first_text_delay/last_first_audio_delay are also available).

def events(self) -> Iterator[Dict]

event: Server-side event

Continuously reads and yields server-side events (dict) one by one. The iteration ends when the connection is closed or session.finished is received. Use a single receiver thread to consume events continuously; do not create multiple concurrent readers.

def recv_event(self) -> Optional[Dict]

None

Blocking read of a single server-side event, without automatic looping. Returns None when there is no event (connection closed). Error events are raised as DashScopeError exceptions.