Alibaba Cloud Model Studio:Client events

session.update

After you establish a WebSocket connection, send this event to update the default session configuration.

After the service receives the session.update event, it validates the parameters. If the parameters are invalid, the service returns a fault. If the parameters are valid, the service updates and returns the complete configuration.

{
  "event_id": "event_ToPZqeobitzUJnt3QqtWg",
  "type": "session.update",
  "session": {
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Cherry",
    "input_audio_format": "pcm16",
    "output_audio_format": "pcm24",
    "input_audio_transcription": {
      "model": "qwen3-asr-flash-realtime",
      "language": "zh"
    },
    "translation": {
      "language": "en"
    }
  }
}

input_audio_buffer.append

This event appends audio bytes to the input audio buffer. The service uses this buffer to detect speech and determine when to submit it.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.append",
    "audio": "xxx"
}

input_image_buffer.append

This event adds image data to the image buffer. The image can be from a local file or captured in real-time from a video stream.

The following limits apply to image inputs:

• The image format must be JPG or JPEG. For optimal performance, a resolution of 480p or 720p is recommended. The maximum resolution is 1080p.

• The size of a single image cannot exceed 500 KB before Base64 encoding.

• The image data must be Base64-encoded.

• The frequency for adding images to the buffer must not exceed 2 images per second.

• You must send at least one input_audio_buffer.append event before you send an input_image_buffer.append event.

{
    "event_id": "event_xxx",
    "type": "input_image_buffer.append",
    "image": "xxx"
}

session.finish

Send this event to end the current session. After you send this event, the server responds as:

• If speech is detected: The server completes speech recognition and sends a conversation.item.input_audio_transcription.completed event with the recognition result. Then, it sends a session.finished event to indicate that the session has ended.

• No speech detected: The server-side sends the session.finished event directly.

The client must disconnect after receiving the session.finished event.

{
    "event_id": "event_xxx",
    "type": "session.finish"
}