全部產品
Search
文件中心

Alibaba Cloud Model Studio:Qwen-ASR即時語音辨識WebSocket API

更新時間:Nov 19, 2026

即時語音辨識-千問服務通過 WebSocket 通訊協定,接收即時音頻流並即時轉寫。支援 VAD 模式 和 Manual 模式 互動流程。

使用者指南:關於模型介紹和選型建議請參見語音辨識,範例程式碼請參見即時語音辨識。

介面地址

WebSocket URL 固定如下,通過查詢參數 model 指定要調用的模型名稱(將 <model_name> 替換為實際的模型):

新加坡

wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>

調用時請將{WorkspaceId}替換為真實的Workspace ID。

華北2(北京)

wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>

調用時請將{WorkspaceId}替換為真實的Workspace ID。

重要阿里雲百鍊為華北2(北京)、新加坡地區推出了業務空間專屬網域名稱,能夠為推理請求提供卓越的效能和更高的穩定性,建議遷移至新網域名稱:

  • 華北2(北京)地區:從 dashscope.aliyuncs.com 遷移至 {WorkspaceId}.cn-beijing.maas.aliyuncs.com
  • 新加坡地區:從 dashscope-intl.aliyuncs.com 遷移至 {WorkspaceId}.ap-southeast-1.maas.aliyuncs.com

{WorkspaceId}需要替換為真實的Workspace ID。現有網域名稱仍可正常使用。

重要URL 必須使用 wss:// 協議。Authorization 在要求標頭中設定(參見要求標頭),模型通過 URL 查詢參數 model 指定。

要求標頭

要求標頭中需添加如下資訊:

參數

類型

是否必選

說明

Authorization

string

是

鑒權令牌,格式為 Bearer <your_api_key>,將 <your_api_key> 替換為實際的 API Key。

user-agent

string

否

用戶端標識,便於服務端追蹤來源。

X-DashScope-WorkSpace

string

否

阿里雲百鍊業務空間ID。

X-DashScope-DataInspection

string

否

是否啟用資料合規檢測功能。預設不傳或設為enable。如非必要,請勿啟用該參數。

重要Authorization 鑒權在 WebSocket 握手階段驗證。如果 API Key 無效或缺失,握手將失敗並返回 HTTP 401/403 錯誤。

互動流程

用戶端事件和服務端事件的詳細說明,請參見用戶端事件和服務端事件。

支援兩種互動模式:

  • VAD 模式(預設):服務端自動檢測語音的起點和終點(斷句),適用於即時對話、會議記錄等情境。
  • Manual 模式:由用戶端控制斷句,適用於用戶端能明確判斷語句邊界的情境,如聊天軟體中的發送語音。

此模型不傳回轉寫文字的句級、字詞級時間戳記。VAD 事件中的語音起止時間用於描述語音活動邊界,欄位說明請參見對應事件。

VAD 模式(預設)

silence_duration_ms 的伺服器預設值為 800,對話、聊天等需要快速斷句的情境可參考 400;threshold 的伺服器預設值為 0.2。完整設定請參見用戶端事件。

服務端自動檢測語音的起點和終點(斷句)。開發人員只需持續發送音頻流,服務端會在檢測到一句話結束時自動返回最終識別結果。此模式適用於即時對話、會議記錄等情境。

啟用方式:配置用戶端session.update事件的session.turn_detection參數。

  • 用戶端通過發送input_audio_buffer.append事件將音頻追加到緩衝區。

  • 服務端在檢測到語音時返回input_audio_buffer.speech_started事件。

    注意:如果用戶端尚未收到該事件,就直接發送session.finish結束會話,服務端會直接返回session.finished事件,隨後用戶端需主動中斷連線。

  • 用戶端繼續發送input_audio_buffer.append事件提交音頻。

  • 用戶端在音頻提交完後,發送session.finish事件通知服務端結束當前會話。

    警告在 VAD 模式下,推完音頻後必須先發送session.finish事件再關閉串連。如果用戶端直接關閉 WebSocket 串連而未發送該事件,服務端將丟棄當前 in_progress item,conversation.item.input_audio_transcription.completed 等事件將不會到達。建議在調用 ws.Close() 之前,先發送 {"type":"session.finish"},並等待收到session.finished事件後再關閉串連。

  • 服務端在檢測到語音結束時返回input_audio_buffer.speech_stopped事件。

  • 服務端返回input_audio_buffer.committed事件。

  • 服務端返回conversation.item.created事件。

  • 服務端返回conversation.item.input_audio_transcription.text事件,其中包含語音辨識即時結果。

  • 服務端返回conversation.item.input_audio_transcription.completed事件,其中包含語音辨識最終結果。

  • 服務端返回session.finished事件,通知用戶端識別結束,此時用戶端需要主動中斷連線。

Manual 模式

由用戶端控制斷句。用戶端需要發送完一整句話的音頻後,再發送一個input_audio_buffer.commit事件來通知服務端。此模式適用於用戶端能明確判斷語句邊界的情境,如聊天軟體中的發送語音。

啟用方式:將用戶端session.update事件的session.turn_detection設為null。

  • 用戶端通過發送input_audio_buffer.append事件將音頻追加到緩衝區。
  • 用戶端通過發送input_audio_buffer.commit事件來提交輸入音頻緩衝區。 該提交會在對話中建立一個新的使用者訊息項。
  • 用戶端發送session.finish事件通知服務端結束當前會話。
  • 服務端返回input_audio_buffer.committed事件進行響應。
  • 服務端返回conversation.item.input_audio_transcription.text事件,其中包含語音辨識即時結果。
  • 服務端返回conversation.item.input_audio_transcription.completed事件,其中包含語音辨識最終結果。
  • 服務端返回session.finished事件,通知用戶端識別結束,此時用戶端需要主動中斷連線。

呼叫範例

以下範例使用 qwen3-asr-flash-realtime。在控制台取得 API Key 和 Base URL,分別設定環境變數 DASHSCOPE_API_KEY 和 DASHSCOPE_BASE_URL。

下載範例音訊並儲存為 asr_example.wav。音訊為 16 kHz、單聲道、16-bit PCM 編碼的 WAV 檔案。

說明範例程式碼讀取 asr_example.pcm(PCM16、16 kHz、單聲道)。如僅有 MP3/WAV 等格式,可使用 ffmpeg 轉換:

ffmpeg -i asr_example.wav -ar 16000 -ac 1 -f s16le asr_example.pcm

說明使用非 VAD(Manual)模式時,建議單次會話持續發送的音頻時間長度累加不超過 60 秒。

Java

在運行樣本前,請確保已安裝Java-WebSocket依賴:

<dependency>
    <groupId>org.java-websocket</groupId>
    <artifactId>Java-WebSocket</artifactId>
    <version>1.5.6</version>
</dependency>
<dependency>
    <groupId>org.json</groupId>
    <artifactId>json</artifactId>
    <version>20240303</version>
</dependency>
implementation 'org.java-websocket:Java-WebSocket:1.5.6'
implementation 'org.json:json:20240303'
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.handshake.ServerHandshake;
import org.json.JSONObject;

import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.*;

public class QwenASRRealtimeClient {

    private static final Logger logger = Logger.getLogger(QwenASRRealtimeClient.class.getName());
    private static final String API_KEY = System.getenv("DASHSCOPE_API_KEY");
    private static final String MODEL = "qwen3-asr-flash-realtime";

    // 控制是否使用 VAD 模式
    private static final boolean enableServerVad = true;

    private static final AtomicBoolean isRunning = new AtomicBoolean(true);
    private static WebSocketClient client;

    public static void main(String[] args) throws Exception {
        initLogger();

        String baseUrl = "wss://" + URI.create(System.getenv("DASHSCOPE_BASE_URL")).getHost() + "/api-ws/v1/realtime";
        String url = baseUrl + "?model=" + MODEL;
        logger.info("Connecting to server: " + url);

        client = new WebSocketClient(new URI(url)) {
            @Override
            public void onOpen(ServerHandshake handshake) {
                logger.info("Connected to server.");
                sendSessionUpdate();
            }

            @Override
            public void onMessage(String message) {
                try {
                    JSONObject data = new JSONObject(message);
                    String eventType = data.optString("type");

                    logger.info("Received event: " + data.toString(2));

                    // 最终识别结果在 transcription.completed 事件中
                    if ("conversation.item.input_audio_transcription.completed".equals(eventType)) {
                        logger.info("Final transcript: " + data.optString("transcript"));
                    }

                    // 收到结束事件 → 停止发送线程并关闭连接
                    if ("session.finished".equals(eventType)) {
                        logger.info("Closing WebSocket connection after session finished...");

                        isRunning.set(false); // 停止发送音频线程
                        if (this.isOpen()) {
                            this.close(1000, "ASR finished");
                        }
                    }
                } catch (Exception e) {
                    logger.severe("Failed to parse message: " + message);
                }
            }

            @Override
            public void onClose(int code, String reason, boolean remote) {
                logger.info("Connection closed: " + code + " - " + reason);
            }

            @Override
            public void onError(Exception ex) {
                logger.severe("Error: " + ex.getMessage());
            }
        };

        // 添加请求头
        client.addHeader("Authorization", "Bearer " + API_KEY);
        client.addHeader("OpenAI-Beta", "realtime=v1");

        client.connectBlocking(); // 阻塞直到连接建立

        // 替换为待识别的音频文件路径
        String localAudioPath = "asr_example.pcm";
        Thread audioThread = new Thread(() -> {
            try {
                sendAudio(localAudioPath);
            } catch (Exception e) {
                logger.severe("Audio sending thread error: " + e.getMessage());
            }
        });
        audioThread.start();
    }

    /** 会话更新事件(开启/关闭 VAD) */
    private static void sendSessionUpdate() {
        JSONObject eventNoVad = new JSONObject()
                .put("event_id", "event_123")
                .put("type", "session.update")
                .put("session", new JSONObject()
                        .put("modalities", new String[]{"text"})
                        .put("input_audio_format", "pcm")
                        .put("sample_rate", 16000)
                        // .put("input_audio_transcription", new JSONObject()
                        //         .put("language", "zh"))
                        .put("turn_detection", JSONObject.NULL) // 手动模式
                );

        JSONObject eventVad = new JSONObject()
                .put("event_id", "event_123")
                .put("type", "session.update")
                .put("session", new JSONObject()
                        .put("modalities", new String[]{"text"})
                        .put("input_audio_format", "pcm")
                        .put("sample_rate", 16000)
                        // .put("input_audio_transcription", new JSONObject()
                        //         .put("language", "zh"))
                        .put("turn_detection", new JSONObject()
                                .put("type", "server_vad")
                                .put("threshold", 0.2)
                                .put("silence_duration_ms", 400))
                );

        if (enableServerVad) {
            logger.info("Sending event (VAD):\n" + eventVad.toString(2));
            client.send(eventVad.toString());
        } else {
            logger.info("Sending event (Manual):\n" + eventNoVad.toString(2));
            client.send(eventNoVad.toString());
        }
    }

    /** 发送音频文件流 */
    private static void sendAudio(String localAudioPath) throws Exception {
        Thread.sleep(3000); // 等会话准备
        byte[] allBytes = Files.readAllBytes(Paths.get(localAudioPath));
        logger.info("文件读取开始");

        int offset = 0;
        while (isRunning.get() && offset < allBytes.length) {
            int chunkSize = Math.min(3200, allBytes.length - offset);
            byte[] chunk = new byte[chunkSize];
            System.arraycopy(allBytes, offset, chunk, 0, chunkSize);
            offset += chunkSize;

            if (client != null && client.isOpen()) {
                String encoded = Base64.getEncoder().encodeToString(chunk);
                JSONObject eventd = new JSONObject()
                        .put("event_id", "event_" + System.currentTimeMillis())
                        .put("type", "input_audio_buffer.append")
                        .put("audio", encoded);

                client.send(eventd.toString());
                logger.info("Sending audio event: " + eventd.getString("event_id"));
            } else {
                break; // 避免在断开后继续发送
            }

            Thread.sleep(100); // 模拟实时发送
        }

        logger.info("文件读取完毕");

        if (client != null && client.isOpen()) {
            // 非 VAD 模式下需要 commit
            if (!enableServerVad) {
                JSONObject commitEvent = new JSONObject()
                        .put("event_id", "event_789")
                        .put("type", "input_audio_buffer.commit");
                client.send(commitEvent.toString());
                logger.info("Sent commit event for manual mode.");
            }

            JSONObject finishEvent = new JSONObject()
                    .put("event_id", "event_987")
                    .put("type", "session.finish");
            client.send(finishEvent.toString());
            logger.info("Sent finish event.");
        }
    }

    /** 初始化日志 */
    private static void initLogger() {
        logger.setLevel(Level.ALL);
        Logger rootLogger = Logger.getLogger("");
        for (Handler h : rootLogger.getHandlers()) {
            rootLogger.removeHandler(h);
        }

        Handler consoleHandler = new ConsoleHandler();
        consoleHandler.setLevel(Level.ALL);
        consoleHandler.setFormatter(new SimpleFormatter());
        logger.addHandler(consoleHandler);
    }
}

Python

在運行樣本前,請確保已使用以下命令安裝依賴:

python -m pip install websocket-client

說明請勿將範例檔案命名為 websocket.py,以免與 websocket 模組同名,造成匯入衝突。

# pip install websocket-client
import os
import time
import json
import threading
import base64
from urllib.parse import urlsplit
import websocket
import logging
import logging.handlers
from datetime import datetime

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

API_KEY = os.environ["DASHSCOPE_API_KEY"]
QWEN_MODEL = "qwen3-asr-flash-realtime"
baseUrl = f"wss://{urlsplit(os.environ['DASHSCOPE_BASE_URL']).netloc}/api-ws/v1/realtime"
url = f"{baseUrl}?model={QWEN_MODEL}"
print(f"Connecting to server: {url}")

# 注意: 如果是非vad模式,建议持续发送的音频时长累加不超过60s
enableServerVad = True
is_running = True  # 增加运行标志位

headers = [
    "Authorization: Bearer " + API_KEY,
    "OpenAI-Beta: realtime=v1"
]

def init_logger():
    formatter = logging.Formatter('%(asctime)s|%(levelname)s|%(message)s')
    f_handler = logging.handlers.RotatingFileHandler(
        "omni_tester.log", maxBytes=100 * 1024 * 1024, backupCount=3
    )
    f_handler.setLevel(logging.DEBUG)
    f_handler.setFormatter(formatter)

    console = logging.StreamHandler()
    console.setLevel(logging.DEBUG)
    console.setFormatter(formatter)

    logger.addHandler(f_handler)
    logger.addHandler(console)

def on_open(ws):
    logger.info("Connected to server.")

    # 会话更新事件
    event_manual = {
        "event_id": "event_123",
        "type": "session.update",
        "session": {
            "modalities": ["text"],
            "input_audio_format": "pcm",
            "sample_rate": 16000,
            # "input_audio_transcription": {
            #     # 语种标识,可选,如果有明确的语种信息,建议设置
            #     "language": "zh"
            # },
            "turn_detection": None
        }
    }
    event_vad = {
        "event_id": "event_123",
        "type": "session.update",
        "session": {
            "modalities": ["text"],
            "input_audio_format": "pcm",
            "sample_rate": 16000,
            # "input_audio_transcription": {
            #     "language": "zh"
            # },
            "turn_detection": {
                "type": "server_vad",
                "threshold": 0.2,
                "silence_duration_ms": 400
            }
        }
    }
    if enableServerVad:
        logger.info(f"Sending event: {json.dumps(event_vad, indent=2)}")
        ws.send(json.dumps(event_vad))
    else:
        logger.info(f"Sending event: {json.dumps(event_manual, indent=2)}")
        ws.send(json.dumps(event_manual))

def on_message(ws, message):
    global is_running
    try:
        data = json.loads(message)
        logger.info(f"Received event: {json.dumps(data, ensure_ascii=False, indent=2)}")
        if data.get("type") == "conversation.item.input_audio_transcription.completed":
            logger.info(f"Final transcript: {data.get('transcript')}")
        elif data.get("type") == "session.finished":
            logger.info("Closing WebSocket connection after session finished...")
            is_running = False  # 停止音频发送线程
            ws.close()
    except json.JSONDecodeError:
        logger.error(f"Failed to parse message: {message}")

def on_error(ws, error):
    logger.error(f"Error: {error}")

def on_close(ws, close_status_code, close_msg):
    logger.info(f"Connection closed: {close_status_code} - {close_msg}")

def send_audio(ws, local_audio_path):
    time.sleep(3)  # 等待会话更新完成
    global is_running

    with open(local_audio_path, 'rb') as audio_file:
        logger.info(f"文件读取开始: {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")
        while is_running:
            audio_data = audio_file.read(3200)  # ~0.1s PCM16/16kHz
            if not audio_data:
                logger.info(f"文件读取完毕: {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")
                if ws.sock and ws.sock.connected:
                    if not enableServerVad:
                        commit_event = {
                            "event_id": "event_789",
                            "type": "input_audio_buffer.commit"
                        }
                        ws.send(json.dumps(commit_event))
                    finish_event = {
                        "event_id": "event_987",
                        "type": "session.finish"
                    }
                    ws.send(json.dumps(finish_event))
                break

            if not ws.sock or not ws.sock.connected:
                logger.info("WebSocket已关闭,停止发送音频。")
                break

            encoded_data = base64.b64encode(audio_data).decode('utf-8')
            eventd = {
                "event_id": f"event_{int(time.time() * 1000)}",
                "type": "input_audio_buffer.append",
                "audio": encoded_data
            }
            ws.send(json.dumps(eventd))
            logger.info(f"Sending audio event: {eventd['event_id']}")
            time.sleep(0.1)  # 模拟实时采集

# 初始化日志
init_logger()
logger.info(f"Connecting to WebSocket server at {url}...")

local_audio_path = "asr_example.pcm"
ws = websocket.WebSocketApp(
    url,
    header=headers,
    on_open=on_open,
    on_message=on_message,
    on_error=on_error,
    on_close=on_close
)

thread = threading.Thread(target=send_audio, args=(ws, local_audio_path))
thread.start()
ws.run_forever()

Go

建立 Go 範例目錄後初始化模組並安裝相依套件:

go mod init asr-example
go get github.com/gorilla/websocket@v1.5.3

在運行樣本前,請確保已安裝相關依賴:

package main

import (
	"encoding/base64"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
	neturl "net/url"
	"os"
	"time"

	"github.com/gorilla/websocket"
)

const (
	model           = "qwen3-asr-flash-realtime"
	audioFile       = "asr_example.pcm" // 替换为您的PCM音频文件路径
	enableServerVad = true                  // 控制是否使用 VAD 模式
)

// 服务端事件结构
type ServerEvent struct {
	Type       string `json:"type"`
	Transcript string `json:"transcript,omitempty"`
}

func main() {
	base, err := neturl.Parse(os.Getenv("DASHSCOPE_BASE_URL"))
	if err != nil {
		log.Fatal(err)
	}
	baseURL := "wss://" + base.Host + "/api-ws/v1/realtime"
	apiKey := os.Getenv("DASHSCOPE_API_KEY")

	url := baseURL + "?model=" + model
	log.Printf("Connecting to server: %s", url)

	conn, err := connect(url, apiKey)
	if err != nil {
		log.Fatal("连接WebSocket失败:", err)
	}
	defer conn.Close()

	// 启动接收消息的 goroutine
	sessionFinished := make(chan bool, 1)
	go receiveMessages(conn, sessionFinished)

	// 发送 session.update
	if err := sendSessionUpdate(conn); err != nil {
		log.Fatal("发送 session.update 失败:", err)
	}

	// 等待会话配置完成
	time.Sleep(3 * time.Second)

	// 发送音频流
	if err := sendAudioStream(conn); err != nil {
		log.Fatal("发送音频失败:", err)
	}

	// 等待 session.finished
	<-sessionFinished
}

// 建立 WebSocket 连接
func connect(url, apiKey string) (*websocket.Conn, error) {
	headers := http.Header{}
	headers.Set("Authorization", "Bearer "+apiKey)
	headers.Set("OpenAI-Beta", "realtime=v1")
	conn, _, err := websocket.DefaultDialer.Dial(url, headers)
	return conn, err
}

// 发送 session.update 事件
func sendSessionUpdate(conn *websocket.Conn) error {
	session := map[string]interface{}{
		"modalities":         []string{"text"},
		"input_audio_format": "pcm",
		"sample_rate":        16000,
		// "input_audio_transcription": map[string]interface{}{
		// 	"language": "zh",
		// },
	}
	if enableServerVad {
		session["turn_detection"] = map[string]interface{}{
			"type":                "server_vad",
			"threshold":           0.2,
			"silence_duration_ms": 400,
		}
	} else {
		session["turn_detection"] = nil
	}
	event := map[string]interface{}{
		"event_id": "event_123",
		"type":     "session.update",
		"session":  session,
	}
	payload, _ := json.Marshal(event)
	log.Printf("Sending session.update: %s", string(payload))
	return conn.WriteMessage(websocket.TextMessage, payload)
}

// 发送音频流(每100ms发送一个PCM chunk)
func sendAudioStream(conn *websocket.Conn) error {
	f, err := os.Open(audioFile)
	if err != nil {
		return err
	}
	defer f.Close()

	chunk := make([]byte, 3200) // 100ms @ 16kHz 16bit 单声道
	for {
		n, err := f.Read(chunk)
		if n > 0 {
			event := map[string]interface{}{
				"event_id": fmt.Sprintf("event_%d", time.Now().UnixMilli()),
				"type":     "input_audio_buffer.append",
				"audio":    base64.StdEncoding.EncodeToString(chunk[:n]),
			}
			payload, _ := json.Marshal(event)
			if err := conn.WriteMessage(websocket.TextMessage, payload); err != nil {
				return err
			}
			time.Sleep(100 * time.Millisecond)
		}
		if err == io.EOF {
			break
		}
		if err != nil {
			return err
		}
	}
	log.Println("音频流结束")
	if !enableServerVad {
		commitEvt := map[string]interface{}{
			"event_id": "event_789",
			"type":     "input_audio_buffer.commit",
		}
		payload, _ := json.Marshal(commitEvt)
		if err := conn.WriteMessage(websocket.TextMessage, payload); err != nil {
			return err
		}
	}
	finishEvt := map[string]interface{}{
		"event_id": "event_987",
		"type":     "session.finish",
	}
	payload, _ := json.Marshal(finishEvt)
	return conn.WriteMessage(websocket.TextMessage, payload)
}

// 接收并处理服务端事件
func receiveMessages(conn *websocket.Conn, sessionFinished chan<- bool) {
	for {
		_, msg, err := conn.ReadMessage()
		if err != nil {
			log.Println("读取消息错误:", err)
			sessionFinished <- true
			return
		}
		var evt ServerEvent
		if err := json.Unmarshal(msg, &evt); err != nil {
			log.Println("解析消息错误:", err)
			continue
		}
		log.Printf("Received event: %s", evt.Type)
		switch evt.Type {
		case "conversation.item.input_audio_transcription.completed":
			log.Printf("Final transcript: %s", evt.Transcript)
		case "session.finished":
			log.Println("Session finished")
			sessionFinished <- true
			return
		}
	}
}

Node.js

在運行樣本前,請確保已使用以下命令安裝依賴:

npm install ws
/**
 * Qwen-ASR Realtime WebSocket 客户端(Node.js版)
 * 功能:
 * - 支持 VAD 模式和 Manual 模式
 * - 发送 session.update 启动会话
 * - 持续发送音频块 input_audio_buffer.append
 * - 如果是Manual模式,需要发送 input_audio_buffer.commit
 * - 发送session.finish事件
 * - 收到 session.finished 事件后关闭连接
 */

import WebSocket from 'ws';
import fs from 'fs';

// ===== 配置 =====
const API_KEY = process.env.DASHSCOPE_API_KEY;
const MODEL = 'qwen3-asr-flash-realtime';
const enableServerVad = true; // true为VAD模式,false为Manual模式
const localAudioPath = 'asr_example.pcm'; // PCM16、16kHz音频文件路径

const baseUrl = `wss://${new URL(process.env.DASHSCOPE_BASE_URL).host}/api-ws/v1/realtime`;
const url = `${baseUrl}?model=${MODEL}`;

console.log(`Connecting to server: ${url}`);

// ===== 状态控制 =====
let isRunning = true;

// ===== 建立连接 =====
const ws = new WebSocket(url, {
    headers: {
        'Authorization': `Bearer ${API_KEY}`,
        'OpenAI-Beta': 'realtime=v1'
    }
});

// ===== 事件绑定 =====
ws.on('open', () => {
    console.log('[WebSocket] Connected to server.');
    sendSessionUpdate();
    // 启动音频发送线程
    sendAudio(localAudioPath);
});

ws.on('message', (message) => {
    try {
        const data = JSON.parse(message);
        console.log('[Received Event]:', JSON.stringify(data, null, 2));

        // 最终识别结果在 transcription.completed 事件中
        if (data.type === 'conversation.item.input_audio_transcription.completed') {
            console.log(`[Final Transcript] ${data.transcript}`);
        }

        // 收到结束事件
        if (data.type === 'session.finished') {
            console.log('[Action] Closing WebSocket connection after session finished...');

            if (ws.readyState === WebSocket.OPEN) {
                ws.close(1000, 'ASR finished');
            }
        }
    } catch (e) {
        console.error('[Error] Failed to parse message:', message);
    }
});

ws.on('close', (code, reason) => {
    console.log(`[WebSocket] Connection closed: ${code} - ${reason}`);
});

ws.on('error', (err) => {
    console.error('[WebSocket Error]', err);
});

// ===== 会话更新 =====
function sendSessionUpdate() {
    const eventNoVad = {
        event_id: 'event_123',
        type: 'session.update',
        session: {
            modalities: ['text'],
            input_audio_format: 'pcm',
            sample_rate: 16000,
            // input_audio_transcription: {
            //     language: 'zh'
            // },
            turn_detection: null
        }
    };

    const eventVad = {
        event_id: 'event_123',
        type: 'session.update',
        session: {
            modalities: ['text'],
            input_audio_format: 'pcm',
            sample_rate: 16000,
            // input_audio_transcription: {
            //     language: 'zh'
            // },
            turn_detection: {
                type: 'server_vad',
                threshold: 0.2,
                silence_duration_ms: 400
            }
        }
    };

    if (enableServerVad) {
        console.log('[Send Event] VAD Mode:\n', JSON.stringify(eventVad, null, 2));
        ws.send(JSON.stringify(eventVad));
    } else {
        console.log('[Send Event] Manual Mode:\n', JSON.stringify(eventNoVad, null, 2));
        ws.send(JSON.stringify(eventNoVad));
    }
}

// ===== 发送音频文件流 =====
function sendAudio(audioPath) {
    setTimeout(() => {
        console.log(`[File Read Start] ${audioPath}`);
        const buffer = fs.readFileSync(audioPath);

        let offset = 0;
        const chunkSize = 3200; // 约0.1s的PCM16音频

        function sendChunk() {
            if (!isRunning) return;
            if (offset >= buffer.length) {
                isRunning = false; // 停止发送音频
                console.log('[File Read End]');
                if (ws.readyState === WebSocket.OPEN) {
                    if (!enableServerVad) {
                        const commitEvent = {
                            event_id: 'event_789',
                            type: 'input_audio_buffer.commit'
                        };
                        ws.send(JSON.stringify(commitEvent));
                        console.log('[Send Commit Event]');
                    }

                    const finishEvent = {
                        event_id: 'event_987',
                        type: 'session.finish'
                    };
                    ws.send(JSON.stringify(finishEvent));
                    console.log('[Send Finish Event]');
                }

                return;
            }

            if (ws.readyState !== WebSocket.OPEN) {
                console.log('[Stop] WebSocket is not open.');
                return;
            }

            const chunk = buffer.slice(offset, offset + chunkSize);
            offset += chunkSize;

            const encoded = chunk.toString('base64');
            const appendEvent = {
                event_id: `event_${Date.now()}`,
                type: 'input_audio_buffer.append',
                audio: encoded
            };

            ws.send(JSON.stringify(appendEvent));
            console.log(`[Send Audio Event] ${appendEvent.event_id}`);

            setTimeout(sendChunk, 100); // 模拟实时发送
        }

        sendChunk();
    }, 3000); // 等待会话配置完成
}

將程式碼儲存為 asr_example.mjs,執行 node asr_example.mjs。

C#

將程式碼儲存為 Program.cs,使用 .NET 主控台專案執行。

範例程式碼如下:

using System.Net.WebSockets;
using System.Text;
using System.Text.Json.Nodes;

class Program {
    private static ClientWebSocket _webSocket = new ClientWebSocket();
    private static CancellationTokenSource _cts = new CancellationTokenSource();
    private static bool _sessionFinished = false;
    private static bool _isRunning = true;

    // 控制是否使用 VAD 模式
    private const bool EnableServerVad = true;

    private static readonly string ApiKey = Environment.GetEnvironmentVariable("DASHSCOPE_API_KEY") ?? throw new InvalidOperationException("DASHSCOPE_API_KEY environment variable is not set.");
    private const string Model = "qwen3-asr-flash-realtime";
    private static readonly string BaseUrl = "wss://" + new Uri(Environment.GetEnvironmentVariable("DASHSCOPE_BASE_URL")!).Authority + "/api-ws/v1/realtime";
    private const string AudioFilePath = "asr_example.pcm"; // 替换为您的PCM音频文件路径

    static async Task Main(string[] args) {
        var url = $"{BaseUrl}?model={Model}";
        Console.WriteLine($"Connecting to server: {url}");

        // 设置鉴权 headers
        _webSocket.Options.SetRequestHeader("Authorization", $"Bearer {ApiKey}");
        _webSocket.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");

        await _webSocket.ConnectAsync(new Uri(url), _cts.Token);
        Console.WriteLine("Connected to server.");

        // 启动消息接收任务
        var receiveTask = ReceiveMessagesAsync();

        // 发送 session.update 配置
        await SendSessionUpdateAsync();

        // 发送音频流
        await SendAudioStreamAsync();

        // 等待 session.finished 事件
        while (!_sessionFinished && !_cts.IsCancellationRequested) {
            await Task.Delay(100);
        }

        if (_webSocket.State == WebSocketState.Open) {
            await _webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "ASR finished", _cts.Token);
        }
    }

    private static async Task SendAsync(string text) {
        var bytes = Encoding.UTF8.GetBytes(text);
        await _webSocket.SendAsync(new ArraySegment<byte>(bytes), WebSocketMessageType.Text, true, _cts.Token);
    }

    // 发送 session.update 事件
    private static async Task SendSessionUpdateAsync() {
        var session = new JsonObject {
            ["modalities"] = new JsonArray { "text" },
            ["input_audio_format"] = "pcm",
            ["sample_rate"] = 16000,
            // ["input_audio_transcription"] = new JsonObject { ["language"] = "zh" }
        };
        if (EnableServerVad) {
            session["turn_detection"] = new JsonObject {
                ["type"] = "server_vad",
                ["threshold"] = 0.2,
                ["silence_duration_ms"] = 400
            };
        } else {
            session["turn_detection"] = null;
        }
        var payload = new JsonObject {
            ["event_id"] = "event_123",
            ["type"] = "session.update",
            ["session"] = session
        };
        Console.WriteLine($"Sending session.update: {payload.ToJsonString()}");
        await SendAsync(payload.ToJsonString());
    }

    // 发送音频流(每100ms发送一个PCM chunk)
    private static async Task SendAudioStreamAsync() {
        await Task.Delay(3000); // 等待会话配置完成
        const int chunkSize = 3200; // 100ms @ 16kHz 16bit 单声道
        using var fs = new FileStream(AudioFilePath, FileMode.Open, FileAccess.Read);
        var buffer = new byte[chunkSize];
        int read;
        while (_isRunning && (read = await fs.ReadAsync(buffer, 0, chunkSize)) > 0) {
            if (_webSocket.State != WebSocketState.Open) break;
            string b64 = Convert.ToBase64String(buffer, 0, read);
            var append = new JsonObject {
                ["event_id"] = $"event_{DateTimeOffset.Now.ToUnixTimeMilliseconds()}",
                ["type"] = "input_audio_buffer.append",
                ["audio"] = b64
            };
            await SendAsync(append.ToJsonString());
            await Task.Delay(100);
        }
        Console.WriteLine("File read end.");
        if (_webSocket.State == WebSocketState.Open) {
            if (!EnableServerVad) {
                var commit = new JsonObject {
                    ["event_id"] = "event_789",
                    ["type"] = "input_audio_buffer.commit"
                };
                await SendAsync(commit.ToJsonString());
            }
            var finish = new JsonObject {
                ["event_id"] = "event_987",
                ["type"] = "session.finish"
            };
            await SendAsync(finish.ToJsonString());
        }
    }

    // 接收并处理服务端事件
    private static async Task ReceiveMessagesAsync() {
        var buffer = new byte[16384];
        var sb = new StringBuilder();
        while (_webSocket.State == WebSocketState.Open && !_cts.IsCancellationRequested) {
            try {
                var result = await _webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), _cts.Token);
                if (result.MessageType == WebSocketMessageType.Close) {
                    await _webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing", _cts.Token);
                    break;
                }
                sb.Append(Encoding.UTF8.GetString(buffer, 0, result.Count));
                if (!result.EndOfMessage) continue;
                string text = sb.ToString();
                sb.Clear();
                var data = JsonNode.Parse(text);
                string? type = data?["type"]?.GetValue<string>();
                Console.WriteLine($"Received event: {type}");
                if (type == "conversation.item.input_audio_transcription.completed") {
                    Console.WriteLine($"Final transcript: {data!["transcript"]}");
                } else if (type == "session.finished") {
                    Console.WriteLine("Session finished, closing...");
                    _sessionFinished = true;
                    _isRunning = false;
                    break;
                }
            } catch (Exception ex) {
                Console.WriteLine($"Receive error: {ex.Message}");
                break;
            }
        }
    }
}

PHP

將相依套件設定儲存為 composer.json,執行 composer install 後執行 PHP 範例。

範例程式碼目錄結構為:

my-php-project/

├── composer.json

├── vendor/

└── index.php

composer.json內容如下,相關依賴的版本號碼請根據實際情況自行決定:

{
    "require": {
        "react/event-loop": "^1.3",
        "react/socket": "^1.11",
        "ratchet/pawl": "^0.4"
    }
}

index.php內容如下:

<?php

require __DIR__ . '/vendor/autoload.php';

use Ratchet\Client\Connector;
use React\EventLoop\Loop;
use React\Socket\Connector as SocketConnector;

$api_key = getenv("DASHSCOPE_API_KEY");
$model = 'qwen3-asr-flash-realtime';
$base_url = 'wss://' . parse_url(getenv('DASHSCOPE_BASE_URL'), PHP_URL_HOST) . '/api-ws/v1/realtime';
$websocket_url = $base_url . '?model=' . $model;
$audio_file_path = 'asr_example.pcm'; // 替换为您的PCM音频文件路径

// 控制是否使用 VAD 模式
$enable_server_vad = true;

$loop = Loop::get();
$socketConnector = new SocketConnector($loop, [
    'tls' => ['verify_peer' => true, 'verify_peer_name' => true],
]);
$connector = new Connector($loop, $socketConnector);

$headers = [
    'Authorization' => 'Bearer ' . $api_key,
    'OpenAI-Beta' => 'realtime=v1',
];

$is_running = true;

$connector($websocket_url, [], $headers)->then(function ($conn) use ($loop, $audio_file_path, $enable_server_vad, &$is_running) {
    echo "连接到WebSocket服务器\n";

    // 监听服务端事件
    $conn->on('message', function($msg) use ($conn, &$is_running) {
        $event = json_decode($msg, true);
        if (!isset($event['type'])) {
            return;
        }
        echo "Received event: {$event['type']}\n";
        if ($event['type'] === 'conversation.item.input_audio_transcription.completed') {
            echo "Final transcript: {$event['transcript']}\n";
        } elseif ($event['type'] === 'session.finished') {
            echo "Session finished, closing...\n";
            $is_running = false;
            $conn->close();
        }
    });
    $conn->on('close', function() {
        echo "连接已关闭\n";
    });

    // 发送 session.update 事件
    sendSessionUpdate($conn, $enable_server_vad);

    // 等待会话配置完成后开始发送音频
    $loop->addTimer(3, function () use ($conn, $audio_file_path, $enable_server_vad, $loop, &$is_running) {
        sendAudioStream($conn, $audio_file_path, $enable_server_vad, $loop, $is_running);
    });
}, function ($e) {
    echo "无法连接:{$e->getMessage()}\n";
});

$loop->run();

// 发送 session.update 事件
function sendSessionUpdate($conn, $enable_server_vad) {
    $session = [
        'modalities' => ['text'],
        'input_audio_format' => 'pcm',
        'sample_rate' => 16000,
        // 'input_audio_transcription' => ['language' => 'zh'],
        'turn_detection' => $enable_server_vad ? [
            'type' => 'server_vad',
            'threshold' => 0.2,
            'silence_duration_ms' => 400,
        ] : null,
    ];
    $event = [
        'event_id' => 'event_123',
        'type' => 'session.update',
        'session' => $session,
    ];
    $conn->send(json_encode($event));
    echo "Sent session.update\n";
}

// 发送音频流(每100ms发送一个PCM chunk)
function sendAudioStream($conn, $audio_file_path, $enable_server_vad, $loop, &$is_running) {
    $fp = fopen($audio_file_path, 'rb');
    if (!$fp) {
        echo "无法打开音频文件\n";
        return;
    }
    $send_chunk = function() use ($conn, $fp, $enable_server_vad, $loop, &$send_chunk, &$is_running) {
        if (!$is_running) {
            fclose($fp);
            return;
        }
        $chunk = fread($fp, 3200); // 100ms @ 16kHz 16bit 单声道
        if ($chunk === false || strlen($chunk) === 0) {
            fclose($fp);
            echo "音频流结束\n";
            if (!$enable_server_vad) {
                $conn->send(json_encode([
                    'event_id' => 'event_789',
                    'type' => 'input_audio_buffer.commit',
                ]));
            }
            $conn->send(json_encode([
                'event_id' => 'event_987',
                'type' => 'session.finish',
            ]));
            return;
        }
        $append = [
            'event_id' => 'event_' . round(microtime(true) * 1000),
            'type' => 'input_audio_buffer.append',
            'audio' => base64_encode($chunk),
        ];
        $conn->send(json_encode($append));
        $loop->addTimer(0.1, $send_chunk);
    };
    $send_chunk();
}

可用的 SDK