Dans les applications de chat en temps réel ou de génération de texte long, des temps d'attente prolongés dégradent l'expérience utilisateur et peuvent provoquer des délais d'expiration côté serveur, entraînant l'échec des tâches. Le streaming output résout ces problèmes en renvoyant continûment des fragments de texte au fur et à mesure que le modèle les génère.
Fonctionnement
Le streaming output utilise le protocole Server-Sent Events (SSE). Lorsqu'une requête en streaming démarre, le serveur établit une connexion HTTP persistante avec le client. Chaque fois que le modèle génère un bloc de texte (appelé chunk), il le transmet immédiatement via cette connexion. Une fois tout le contenu généré, le serveur envoie un signal de fin.
Le client écoute le flux d'événements et reçoit les chunks de texte en temps réel pour les traiter, par exemple en affichant les caractères un par un dans l'interface. Cette approche diffère des appels non streamés, qui renvoient l'intégralité du contenu en une seule fois.
Les composants ci-dessus sont fournis à titre indicatif uniquement et n'envoient pas de requêtes réelles.
Facturation
Le streaming output suit les mêmes règles de facturation que les appels non streamés, avec une tarification basée sur le nombre de tokens d'entrée et de tokens de sortie dans la requête.
Si une requête est interrompue, seuls les tokens de sortie générés avant que le serveur ne reçoive la demande d'arrêt sont comptabilisés.
Utilisation
ImportantLa version open source de Qwen3, les versions commerciale et open source de QwQ, ainsi que QVQ et Qwen-Omni, prennent uniquement en charge le streaming output.
Étape 1 : Configurez votre clé API et sélectionnez une région
Vous devez avoir obtenu une clé API et l'avoir configurée en tant que variable d'environnement.
La configuration de votre clé API en tant que variable d'environnement (
DASHSCOPE_API_KEY) est plus sécurisée que son codage en dur dans votre code.
Étape 2 : Effectuez une requête en streaming
OpenAI compatible
-
Activation
Définissez
streamsurtrue. -
Affichage de l'utilisation des tokens
Le protocole OpenAI ne renvoie pas l'utilisation des tokens par défaut. Définissez
stream_options={"include_usage": true}afin que le dernier chunk de données renvoyé inclue les informations d'utilisation des tokens.
Python
import os
from openai import OpenAI
# 1. Prepare: Initialize the client
client = OpenAI(
# Configure the API key using an environment variable to avoid hard coding.
api_key=os.environ["DASHSCOPE_API_KEY"],
# The API key is tightly bound to a region. Ensure base_url matches the region of your API key.
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
# 2. Make a streaming request
completion = client.chat.completions.create(
model="qwen-plus",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Please introduce yourself"}
],
stream=True,
stream_options={"include_usage": True}
)
# 3. Handle the streaming response
# Store response fragments in a list. Joining them at the end is more efficient than repeated string concatenation.
content_parts = []
print("AI: ", end="", flush=True)
for chunk in completion:
if chunk.choices:
content = chunk.choices[0].delta.content or ""
print(content, end="", flush=True)
content_parts.append(content)
elif chunk.usage:
print("\n--- Request usage ---")
print(f"Input Tokens: {chunk.usage.prompt_tokens}")
print(f"Output Tokens: {chunk.usage.completion_tokens}")
print(f"Total Tokens: {chunk.usage.total_tokens}")
full_response = "".join(content_parts)
# print(f"\n--- Full response ---\n{full_response}")
Réponse
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 87
Total Tokens: 113
Node.js
import OpenAI from "openai";
async function main() {
// 1. Prepare: Initialize the client
// Configure the API key using an environment variable to avoid hard coding.
if (!process.env.DASHSCOPE_API_KEY) {
throw new Error("Set the DASHSCOPE_API_KEY environment variable");
}
// The API key is tightly bound to a region. Ensure baseURL matches the region of your API key.
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
const client = new OpenAI({
apiKey: process.env.DASHSCOPE_API_KEY,
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
});
try {
// 2. Make a streaming request
const stream = await client.chat.completions.create({
model: "qwen-plus",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "Please introduce yourself" },
],
stream: true,
// Purpose: Get token usage in the last chunk.
stream_options: { include_usage: true },
});
// 3. Handle the streaming response
const contentParts = [];
process.stdout.write("AI: ");
for await (const chunk of stream) {
// The last chunk contains no choices but includes usage information.
if (chunk.choices && chunk.choices.length > 0) {
const content = chunk.choices[0]?.delta?.content || "";
process.stdout.write(content);
contentParts.push(content);
} else if (chunk.usage) {
// Request complete. Print token usage.
console.log("\n--- Request usage ---");
console.log(`Input Tokens: ${chunk.usage.prompt_tokens}`);
console.log(`Output Tokens: ${chunk.usage.completion_tokens}`);
console.log(`Total Tokens: ${chunk.usage.total_tokens}`);
}
}
const fullResponse = contentParts.join("");
// console.log(`\n--- Full response ---\n${fullResponse}`);
} catch (error) {
console.error("Request failed:", error);
}
}
main();
Réponse
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 89
Total Tokens: 115
curl
Requête
# ======= Important notes =======
# Ensure the DASHSCOPE_API_KEY environment variable is set
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
--no-buffer \
-d '{
"model": "qwen-plus",
"messages": [
{"role": "user", "content": "Who are you?"}
],
"stream": true,
"stream_options": {"include_usage": true}
}'
Réponse
La réponse suit le protocole SSE. Chaque ligne commençant par data: représente un fragment de données.
data: {"choices":[{"delta":{"content":"","role":"assistant"},"index":0,"logprobs":null,"finish_reason":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"finish_reason":null,"delta":{"content":"I am"},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" from"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" Alibaba"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":"'s large-scale language"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":" model, my name is Qwen"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"delta":{"content":"."},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[{"finish_reason":"stop","delta":{"content":""},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: {"choices":[],"object":"chat.completion.chunk","usage":{"prompt_tokens":22,"completion_tokens":17,"total_tokens":39},"created":1726132850,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-428b414f-fdd4-94c6-b179-8f576ad653a8"}
data: [DONE]
data:: la charge utile du message, généralement une chaîne JSON.[DONE]: indique la fin de l'ensemble de la réponse en streaming.
DashScope
-
Comment activer
La méthode d'activation de la sortie en streaming varie selon le SDK ou l'outil utilisé :
- SDK Python : définissez le paramètre
streamsurTrue. - SDK Java : utilisez l'interface
streamCall. - curl : définissez l'en-tête
X-DashScope-SSEsurenable.
- SDK Python : définissez le paramètre
-
Activer la sortie incrémentielle
Le protocole DashScope prend en charge les sorties en streaming incrémentielles et non incrémentielles :
-
Incrémentiel (recommandé) : chaque fragment de données contient uniquement le contenu nouvellement généré. Définissez
incremental_outputsurtruepour activer la sortie en streaming incrémentielle.Exemple : ["J'aime","manger","des pommes"]
-
Non incrémentiel : chaque fragment de données contient tout le contenu précédemment généré, ce qui gaspille la bande passante réseau et augmente la charge de traitement côté client. Définissez
incremental_outputsurfalsepour activer la sortie en streaming non incrémentielle.Exemple : ["J'aime","J'aime manger","J'aime manger des pommes"]
-
-
Consulter l'utilisation des tokens
Chaque fragment de données inclut des informations en temps réel sur l'utilisation des tokens.
Python
import os
from http import HTTPStatus
import dashscope
from dashscope import Generation
# 1. Prepare: Configure the API key and region
# Configure the API key using an environment variable to avoid hard coding.
try:
dashscope.api_key = os.environ["DASHSCOPE_API_KEY"]
except KeyError:
raise ValueError("Set the DASHSCOPE_API_KEY environment variable")
# The API key is tightly bound to a region. Ensure base_url matches the region of your API key.
dashscope.base_http_api_url = "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1"
# 2. Make a streaming request
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Please introduce yourself"},
]
try:
responses = Generation.call(
model="qwen-plus",
messages=messages,
result_format="message",
stream=True,
# Key: Set to True for incremental output, which offers better performance.
incremental_output=True,
)
# 3. Handle the streaming response
content_parts = []
print("AI: ", end="", flush=True)
for resp in responses:
if resp.status_code == HTTPStatus.OK:
content = resp.output.choices[0].message.content
print(content, end="", flush=True)
content_parts.append(content)
# Check if this is the last packet
if resp.output.choices[0].finish_reason == "stop":
usage = resp.usage
print("\n--- Request usage ---")
print(f"Input Tokens: {usage.input_tokens}")
print(f"Output Tokens: {usage.output_tokens}")
print(f"Total Tokens: {usage.total_tokens}")
else:
# Handle errors
print(
f"\nRequest failed: request_id={resp.request_id}, code={resp.code}, message={resp.message}"
)
break
full_response = "".join(content_parts)
# print(f"\n--- Full response ---\n{full_response}")
except Exception as e:
print(f"An unknown error occurred: {e}")
RéponseAI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can help you answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 91
Total Tokens: 117
Java
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import io.reactivex.Flowable;
import io.reactivex.schedulers.Schedulers;
import java.util.Arrays;
import java.util.concurrent.CountDownLatch;
import com.alibaba.dashscope.protocol.Protocol;
public class Main {
public static void main(String[] args) {
// 1. Get the API key
String apiKey = System.getenv("DASHSCOPE_API_KEY");
if (apiKey == null || apiKey.isEmpty()) {
System.err.println("Set the DASHSCOPE_API_KEY environment variable");
return;
}
// 2. Initialize the Generation instance
// The API key is tightly bound to a region. Ensure baseUrl matches the region of your API key.
Generation gen = new Generation(Protocol.HTTP.getValue(), "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1");
CountDownLatch latch = new CountDownLatch(1);
// 3. Build request parameters
GenerationParam param = GenerationParam.builder()
.apiKey(apiKey)
.model("qwen-plus")
.messages(Arrays.asList(
Message.builder()
.role(Role.USER.getValue())
.content("Introduce yourself")
.build()
))
.resultFormat(GenerationParam.ResultFormat.MESSAGE)
.incrementalOutput(true) // Enable incremental output for streaming
.build();
// 4. Make a streaming call and handle the response
try {
Flowable<GenerationResult> result = gen.streamCall(param);
StringBuilder fullContent = new StringBuilder();
System.out.print("AI: ");
result
.subscribeOn(Schedulers.io()) // Execute request on IO thread
.observeOn(Schedulers.computation()) // Process response on computation thread
.subscribe(
// onNext: Handle each response fragment
message -> {
String content = message.getOutput().getChoices().get(0).getMessage().getContent();
String finishReason = message.getOutput().getChoices().get(0).getFinishReason();
// Output content
System.out.print(content);
fullContent.append(content);
// When finishReason is not null, it indicates the last chunk. Output usage info.
if (finishReason != null && !"null".equals(finishReason)) {
System.out.println("\n--- Request usage ---");
System.out.println("Input Tokens: " + message.getUsage().getInputTokens());
System.out.println("Output Tokens: " + message.getUsage().getOutputTokens());
System.out.println("Total Tokens: " + message.getUsage().getTotalTokens());
}
System.out.flush(); // Flush output immediately
},
// onError: Handle errors
error -> {
System.err.println("\nRequest failed: " + error.getMessage());
latch.countDown();
},
// onComplete: Completion callback
() -> {
System.out.println(); // New line
// System.out.println("Full response: " + fullContent.toString());
latch.countDown();
}
);
// Main thread waits for the async task to complete
latch.await();
System.out.println("Program execution complete");
} catch (Exception e) {
System.err.println("Request exception: " + e.getMessage());
e.printStackTrace();
}
}
}
Réponse
AI: Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can help you answer questions, create content such as stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I support multiple languages, including but not limited to Chinese, English, German, French, and Spanish. If you have any questions or need help, feel free to ask me anytime!
--- Request usage ---
Input Tokens: 26
Output Tokens: 91
Total Tokens: 117
curl
Requête
# ======= Important notes =======
# Ensure the DASHSCOPE_API_KEY environment variable is set
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-H "X-DashScope-SSE: enable" \
-d '{
"model": "qwen-plus",
"input":{
"messages":[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Who are you?"
}
]
},
"parameters": {
"result_format": "message",
"incremental_output":true
}
}'
Réponse
La réponse suit le format Server-Sent Events (SSE). Chaque message comprend :
- id : numéro du fragment de données.
- event : type d'événement, toujours « result ».
- Informations sur le code d'état HTTP.
- data : données au format JSON.
id:1
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"I am","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":27,"output_tokens":1,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:2
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"Qwen","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":30,"output_tokens":4,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:3
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":" from Alibaba","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":33,"output_tokens":7,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
...
id:13
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"or need help, feel free to","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":90,"output_tokens":64,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:14
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"ask me!","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":92,"output_tokens":66,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
id:15
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","role":"assistant"},"finish_reason":"stop"}]},"usage":{"total_tokens":92,"output_tokens":66,"input_tokens":26,"prompt_tokens_details":{"cached_tokens":0}},"request_id":"d30a9914-ac97-9102-b746-ce0cb35e3fa2"}
Sortie en streaming pour les modèles multimodaux
Les modèles multimodaux permettent d'ajouter des images, de l'audio et d'autres contenus aux conversations. Leur implémentation de la sortie en streaming diffère de celle des modèles textuels purs sur les points suivants :
- Construction du message utilisateur : les entrées des modèles multimodaux incluent non seulement du texte, mais aussi des images, de l'audio et d'autres informations multimodales.
- Interface du SDK DashScope : utilisez l'interface MultiModalConversation dans le SDK Python DashScope. Utilisez la classe MultiModalConversation dans le SDK Java DashScope.
Pour les modèles multimodaux, consultez Compréhension d'images et de vidéos , Extraction de texte , Compréhension audio — Qwen3-Omni-Captioner , Kimi , etc. Le modèle Qwen-Omni prend uniquement en charge la sortie en streaming car sa sortie peut inclure du texte ou de l' audio et d'autres contenus multimodaux. L'analyse de ses résultats diffère de celle des autres modèles. Pour plus de détails, consultez Omni-modal .
OpenAI compatible
Python
from openai import OpenAI
import os
client = OpenAI(
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx"
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-vl-plus", # Replace with other multimodal models as needed and adjust messages accordingly
messages=[
{"role": "user",
"content": [{"type": "image_url",
"image_url": {"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream=True,
# stream_options={"include_usage": True}
)
full_content = ""
print("Streaming output content:")
for chunk in completion:
# If stream_options.include_usage is True, the last chunk's choices field is an empty list and should be skipped (token usage can be obtained via chunk.usage)
if chunk.choices and chunk.choices[0].delta.content != "":
full_content += chunk.choices[0].delta.content
print(chunk.choices[0].delta.content)
print(f"Full content: {full_content}")
from openai import OpenAI
import os
client = OpenAI(
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx"
api_key=os.getenv("DASHSCOPE_API_KEY"),
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen3-vl-plus", # Replace with other multimodal models as needed and adjust messages accordingly
messages=[
{"role": "user",
"content": [{"type": "image_url",
"image_url": {"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream=True,
# stream_options={"include_usage": True}
)
full_content = ""
print("Streaming output content:")
for chunk in completion:
# If stream_options.include_usage is True, the last chunk's choices field is an empty list and should be skipped (token usage can be obtained via chunk.usage)
if chunk.choices and chunk.choices[0].delta.content != "":
full_content += chunk.choices[0].delta.content
print(chunk.choices[0].delta.content)
print(f"Full content: {full_content}")
Node.js
import OpenAI from "openai";
const openai = new OpenAI(
{
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: apiKey: "sk-xxx"
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
apiKey: process.env.DASHSCOPE_API_KEY,
// China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
baseURL: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"
}
);
const completion = await openai.chat.completions.create({
model: "qwen3-vl-plus", // Replace with other multimodal models as needed and adjust messages accordingly
messages: [
{role: "user",
content: [{"type": "image_url",
"image_url": {"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream: true,
// stream_options: { include_usage: true },
});
let fullContent = ""
console.log("Streaming output content:")
for await (const chunk of completion) {
// If stream_options.include_usage is true, the last chunk's choices field is an empty array and should be skipped (token usage can be obtained via chunk.usage)
if (chunk.choices[0] && chunk.choices[0].delta.content != null) {
fullContent += chunk.choices[0].delta.content;
console.log(chunk.choices[0].delta.content);
}
}
console.log(`Full output content: ${fullContent}`)
import OpenAI from "openai";
const openai = new OpenAI(
{
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: apiKey: "sk-xxx"
apiKey: process.env.DASHSCOPE_API_KEY,
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
const completion = await openai.chat.completions.create({
model: "qwen3-vl-plus", // Replace with other multimodal models as needed and adjust messages accordingly
messages: [
{role: "user",
content: [{"type": "image_url",
"image_url": {"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},},
{"type": "text", "text": "What scene is depicted in the image?"}]}],
stream: true,
// stream_options: { include_usage: true },
});
let fullContent = ""
console.log("Streaming output content:")
for await (const chunk of completion) {
// If stream_options.include_usage is true, the last chunk's choices field is an empty array and should be skipped (token usage can be obtained via chunk.usage)
if (chunk.choices[0] && chunk.choices[0].delta.content != null) {
fullContent += chunk.choices[0].delta.content;
console.log(chunk.choices[0].delta.content);
}
}
console.log(`Full output content: ${fullContent}`)
curl
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "qwen3-vl-plus",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"
}
},
{
"type": "text",
"text": "What scene is depicted in the image?"
}
]
}
],
"stream":true,
"stream_options":{"include_usage":true}
}'
# ======= Important notes =======
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# === Delete this comment before running ===
curl --location 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
"model": "qwen3-vl-plus",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"
}
},
{
"type": "text",
"text": "What scene is depicted in the image?"
}
]
}
],
"stream":true,
"stream_options":{"include_usage":true}
}'
DashScope
Python
import os
from dashscope import MultiModalConversation
import dashscope
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
dashscope.base_http_api_url = "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1"
messages = [
{
"role": "user",
"content": [
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
responses = MultiModalConversation.call(
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx"
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
model='qwen3-vl-plus', # Replace with other multimodal models as needed and adjust messages accordingly
messages=messages,
stream=True,
incremental_output=True
)
full_content = ""
print("Streaming output content:")
for response in responses:
if response.output.choices[0].message.content:
print(response.output.choices[0].message.content[0]['text'])
full_content += response.output.choices[0].message.content[0]['text']
print(f"Full content: {full_content}")
import os
from dashscope import MultiModalConversation
import dashscope
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'
messages = [
{
"role": "user",
"content": [
{"image": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
responses = MultiModalConversation.call(
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# If you haven't configured an environment variable, replace the next line with your Model Studio API key: api_key="sk-xxx",
api_key=os.getenv("DASHSCOPE_API_KEY"),
model='qwen3-vl-plus', # Replace with other multimodal models as needed and adjust messages accordingly
messages=messages,
stream=True,
incremental_output=True)
full_content = ""
print("Streaming output content:")
for response in responses:
if response["output"]["choices"][0]["message"].content:
print(response.output.choices[0].message.content[0]['text'])
full_content += response.output.choices[0].message.content[0]['text']
print(f"Full content: {full_content}")
Java
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import io.reactivex.Flowable;
import com.alibaba.dashscope.utils.Constants;
public class Main {
// China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
static {Constants.baseHttpApiUrl="https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1";}
public static void streamCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("image", "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"),
Collections.singletonMap("text", "What scene is depicted in the image?"))).build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: .apiKey("sk-xxx")
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3-vl-plus") // Replace with other multimodal models as needed and adjust messages accordingly
.messages(Arrays.asList(userMessage))
.incrementalOutput(true)
.build();
Flowable<MultiModalConversationResult> result = conv.streamCall(param);
result.blockingForEach(item -> {
try {
List<Map<String, Object>> content = item.getOutput().getChoices().get(0).getMessage().getContent();
// Check if content exists and is not empty
if (content != null && !content.isEmpty()) {
System.out.println(content.get(0).get("text"));
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
});
}
public static void main(String[] args) {
try {
streamCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
import io.reactivex.Flowable;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
// Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
public static void streamCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
// must create mutable map.
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("image", "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"),
Collections.singletonMap("text", "What scene is depicted in the image?"))).build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
// API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
// If you haven't configured an environment variable, replace the next line with your Model Studio API key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3-vl-plus") // Replace with other multimodal models as needed and adjust messages accordingly
.messages(Arrays.asList(userMessage))
.incrementalOutput(true)
.build();
Flowable<MultiModalConversationResult> result = conv.streamCall(param);
result.blockingForEach(item -> {
try {
List<Map<String, Object>> content = item.getOutput().getChoices().get(0).getMessage().getContent();
// Check if content exists and is not empty
if (content != null && !content.isEmpty()) {
System.out.println(content.get(0).get("text"));
}
} catch (Exception e){
System.exit(0);
}
});
}
public static void main(String[] args) {
try {
streamCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
curl
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# China (Beijing) region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-H 'X-DashScope-SSE: enable' \
-d '{
"model": "qwen3-vl-plus",
"input":{
"messages":[
{
"role": "user",
"content": [
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
},
"parameters": {
"incremental_output": true
}
}'
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H 'Content-Type: application/json' \
-H 'X-DashScope-SSE: enable' \
-d '{
"model": "qwen3-vl-plus",
"input":{
"messages":[
{
"role": "user",
"content": [
{"image": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"},
{"text": "What scene is depicted in the image?"}
]
}
]
},
"parameters": {
"incremental_output": true
}
}'
Sortie en continu pour les modèles de raisonnement
Les modèles de raisonnement renvoient d'abord le champ reasoning_content (le processus de réflexion), puis le champ content (la réponse). Déterminez si l'étape actuelle correspond à la phase de réflexion ou de réponse en fonction du statut du paquet de données.
Pour plus d'informations sur les modèles de raisonnement, consultez les rubriques Réflexion approfondie , Compréhension des images et des vidéos et Raisonnement visuel .
Pour la mise en œuvre de la sortie en continu de Qwen3-Omni-Flash (mode réflexion), reportez-vous à la section Omni-modal .
OpenAI compatible
Voici le format de réponse lors de l'appel au mode de réflexion du modèle qwen-plus à l'aide du SDK Python OpenAI en mode streaming :
# Thinking stage
...
ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content='Cover all key points while')
ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content='remaining natural and fluent.')
# Response stage
ChoiceDelta(content='Hello! I am **Qwen', function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content=None)
ChoiceDelta(content='** (', function_call=None, refusal=None, role=None, tool_calls=None, reasoning_content=None)
...
- Si
reasoning_contentn'est pas None et quecontentestNone, l'étape actuelle correspond à la phase de réflexion. - Si
reasoning_contentest None et quecontentn'est pasNone, l'étape actuelle correspond à la phase de réponse. - Si les deux champs sont
None, l'étape reste identique à celle du paquet précédent.
Python
Exemple de code
from openai import OpenAI
import os
# Initialize the OpenAI client
client = OpenAI(
# If you haven't configured an environment variable, replace with your Alibaba Cloud Model Studio API key: api_key="sk-xxx"
api_key=os.getenv("DASHSCOPE_API_KEY"),
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
messages = [{"role": "user", "content": "Who are you"}]
completion = client.chat.completions.create(
model="qwen-plus", # Replace with other deep-thinking models as needed
messages=messages,
# The enable_thinking parameter enables the thinking process. This parameter has no effect on models qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, and QwQ.
extra_body={"enable_thinking": True},
stream=True,
# stream_options={
# "include_usage": True
# },
)
reasoning_content = "" # Full thought process
answer_content = "" # Full response
is_answering = False # Whether in the response stage
print("\n" + "=" * 20 + "Thought process" + "=" * 20 + "\n")
for chunk in completion:
if not chunk.choices:
print("\nUsage:")
print(chunk.usage)
continue
delta = chunk.choices[0].delta
# Collect only thinking content
if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None:
if not is_answering:
print(delta.reasoning_content, end="", flush=True)
reasoning_content += delta.reasoning_content
# Received content, start responding
if hasattr(delta, "content") and delta.content:
if not is_answering:
print("\n" + "=" * 20 + "Full response" + "=" * 20 + "\n")
is_answering = True
print(delta.content, end="", flush=True)
answer_content += delta.content
Réponse
====================Thought process====================
Okay, the user asked "Who are you," so I need to give an accurate and friendly answer. First, I should confirm my identity as Qwen, developed by Tongyi Lab under Alibaba Group. Next, explain my main functions, like answering questions, creating text, logical reasoning, etc. Keep the tone approachable and avoid overly technical terms so the user feels comfortable. Also, avoid complex jargon and ensure the answer is concise. Additionally, include some interactive elements to encourage further questions. Finally, check for any missing key information, such as my Chinese name "Tongyi Qianwen" and English name "Qwen," along with my company and lab. Make sure the response is comprehensive and meets user expectations.
====================Full response====================
Hello! I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can answer questions, create text, perform logical reasoning, programming, and more, aiming to provide high-quality information and services. You can call me Qwen or simply Tongyi Qianwen. How can I help you?
Node.js
Exemple de code
import OpenAI from "openai";
import process from 'process';
// Initialize the openai client
const openai = new OpenAI({
apiKey: process.env.DASHSCOPE_API_KEY, // Read from environment variable
baseURL: 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1'
});
let reasoningContent = '';
let answerContent = '';
let isAnswering = false;
async function main() {
try {
const messages = [{ role: 'user', content: 'Who are you' }];
const stream = await openai.chat.completions.create({
// Replace with other Qwen3 models or QwQ models as needed
model: 'qwen-plus',
messages,
stream: true,
// The enable_thinking parameter enables the thinking process. This parameter has no effect on models qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, and QwQ.
enable_thinking: true
});
console.log('\n' + '='.repeat(20) + 'Thought process' + '='.repeat(20) + '\n');
for await (const chunk of stream) {
if (!chunk.choices?.length) {
console.log('\nUsage:');
console.log(chunk.usage);
continue;
}
const delta = chunk.choices[0].delta;
// Collect only thinking content
if (delta.reasoning_content !== undefined && delta.reasoning_content !== null) {
if (!isAnswering) {
process.stdout.write(delta.reasoning_content);
}
reasoningContent += delta.reasoning_content;
}
// Received content, start responding
if (delta.content !== undefined && delta.content) {
if (!isAnswering) {
console.log('\n' + '='.repeat(20) + 'Full response' + '='.repeat(20) + '\n');
isAnswering = true;
}
process.stdout.write(delta.content);
answerContent += delta.content;
}
}
} catch (error) {
console.error('Error:', error);
}
}
main();
Réponse
====================Thought process====================
Okay, the user asked "Who are you," so I need to state my identity. First, I should clearly say I am Qwen, a large-scale language model developed by Alibaba Cloud. Next, mention my main functions, like answering questions, creating text, logical reasoning, etc. Also emphasize my multilingual support, including Chinese and English, so users know I can handle requests in different languages. Additionally, explain my application scenarios, such as helping with learning, work, and daily life. However, since the user's question is direct, detailed information might not be necessary—keep it concise. Also, ensure a friendly tone and invite further questions. Check for any missing key information, like my version or latest updates, but the user probably doesn't need that much detail. Finally, confirm the response is accurate and error-free.
====================Full response====================
I am Qwen, a large-scale language model independently developed by Tongyi Lab under Alibaba Group. I can handle various tasks like answering questions, creating text, logical reasoning, and programming, supporting multiple languages including Chinese and English. If you have any questions or need help, feel free to tell me anytime!
HTTP
Exemple de code
curl
Pour les modèles open source Qwen3, définissez enable_thinking sur true pour activer le mode de réflexion. Le paramètre enable_thinking n'a aucun effet sur les modèles qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, QwQ .
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-plus",
"messages": [
{
"role": "user",
"content": "Who are you"
}
],
"stream": true,
"stream_options": {
"include_usage": true
},
"enable_thinking": true
}'
Réponse
data: {"choices":[{"delta":{"content":null,"role":"assistant","reasoning_content":""},"index":0,"logprobs":null,"finish_reason":null}],"object":"chat.completion.chunk","usage":null,"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
.....
data: {"choices":[{"finish_reason":"stop","delta":{"content":"","reasoning_content":null},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
data: {"choices":[],"object":"chat.completion.chunk","usage":{"prompt_tokens":10,"completion_tokens":360,"total_tokens":370},"created":1745485391,"system_fingerprint":null,"model":"qwen-plus","id":"chatcmpl-e2edaf2c-8aaf-9e54-90e2-b21dd5045503"}
data: [DONE]
DashScope
Voici le format de réponse en streaming lors de l'appel au mode de réflexion du modèle qwen-plus à l'aide du SDK Python DashScope :
# Thinking stage
...
{"role": "assistant", "content": "", "reasoning_content": "High information density,"}
{"role": "assistant", "content": "", "reasoning_content": "making users feel helped."}
# Response stage
{"role": "assistant", "content": "I am Qwen", "reasoning_content": ""}
{"role": "assistant", "content": ", developed by Tongyi Lab", "reasoning_content": ""}
...
- Si
reasoning_contentn'est pas vide et quecontentest vide, la phase actuelle est la réflexion. - Si
reasoning_contentest vide et quecontentn'est pas vide, la phase actuelle est la réponse. - Si les deux sont vides, la phase reste identique à celle du paquet précédent.
Python
Exemple de code
import os
from dashscope import Generation
import dashscope
dashscope.base_http_api_url = "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/"
messages = [{"role": "user", "content": "Who are you?"}]
completion = Generation.call(
# If you haven't configured an environment variable, replace the next line with your Alibaba Cloud Model Studio API key: api_key = "sk-xxx",
api_key=os.getenv("DASHSCOPE_API_KEY"),
# Replace with other deep-thinking models as needed
model="qwen-plus",
messages=messages,
result_format="message", # Qwen3 open-source models only support "message"; for better experience, we recommend setting this to "message" for other models too.
# Enable deep thinking. This parameter has no effect on models qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, and QwQ.
enable_thinking=True,
stream=True,
incremental_output=True, # Qwen3 open-source models only support true; for better experience, we recommend setting this to true for other models too.
)
# Define full thought process
reasoning_content = ""
# Define full response
answer_content = ""
# Determine if finished thinking and started responding
is_answering = False
print("=" * 20 + "Thought process" + "=" * 20)
for chunk in completion:
# If both thought process and response are empty, skip
if (
chunk.output.choices[0].message.content == ""
and chunk.output.choices[0].message.reasoning_content == ""
):
pass
else:
# If currently in thinking stage
if (
chunk.output.choices[0].message.reasoning_content != ""
and chunk.output.choices[0].message.content == ""
):
print(chunk.output.choices[0].message.reasoning_content, end="", flush=True)
reasoning_content += chunk.output.choices[0].message.reasoning_content
# If currently in response stage
elif chunk.output.choices[0].message.content != "":
if not is_answering:
print("\n" + "=" * 20 + "Full response" + "=" * 20)
is_answering = True
print(chunk.output.choices[0].message.content, end="", flush=True)
answer_content += chunk.output.choices[0].message.content
# To print the full thought process and full response, uncomment the following lines and run
# print("=" * 20 + "Full thought process" + "=" * 20 + "\n")
# print(f"{reasoning_content}")
# print("=" * 20 + "Full response" + "=" * 20 + "\n")
# print(f"{answer_content}")
Réponse
====================Thought process====================
Okay, the user asked: "Who are you?" I need to answer this question. First, clarify my identity as Qwen, a large-scale language model developed by Alibaba Cloud. Next, explain my functions and purposes, like answering questions, creating text, logical reasoning, etc. Also, emphasize my goal of being a helpful assistant.
Keep the expression conversational, avoiding professional jargon or complex sentence structures. Add friendly phrases like "Hello there~" to make the conversation natural. Also, ensure accuracy and don't omit key points like my developer, main functions, and usage scenarios.
Consider possible follow-up questions from the user, such as specific application examples or technical details, so subtly hint at further inquiries in the response. For example, mention "Whether it's everyday questions or professional issues, I'll do my best to help," which is both comprehensive and open-ended.
Finally, check for fluency, repetition, or redundant information to keep the response concise. Maintain a balance between friendliness and professionalism so users feel both approachable and reliable.
====================Full response====================
Hello there~ I'm Qwen, a large-scale language model developed by Alibaba Cloud. I can answer questions, create text, perform logical reasoning, programming, and more, aiming to provide help and support. Whether it's everyday questions or professional issues, I'll do my best to help. How can I assist you?
Java
Exemple de code
// dashscope SDK version >= 2.19.4
import java.util.Arrays;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import io.reactivex.Flowable;
import java.lang.System;
import com.alibaba.dashscope.utils.Constants;
public class Main {
static {
Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";
}
private static final Logger logger = LoggerFactory.getLogger(Main.class);
private static StringBuilder reasoningContent = new StringBuilder();
private static StringBuilder finalContent = new StringBuilder();
private static boolean isFirstPrint = true;
private static void handleGenerationResult(GenerationResult message) {
String reasoning = message.getOutput().getChoices().get(0).getMessage().getReasoningContent();
String content = message.getOutput().getChoices().get(0).getMessage().getContent();
if (!reasoning.isEmpty()) {
reasoningContent.append(reasoning);
if (isFirstPrint) {
System.out.println("====================Thought process====================");
isFirstPrint = false;
}
System.out.print(reasoning);
}
if (!content.isEmpty()) {
finalContent.append(content);
if (!isFirstPrint) {
System.out.println("\n====================Full response====================");
isFirstPrint = true;
}
System.out.print(content);
}
}
private static GenerationParam buildGenerationParam(Message userMsg) {
return GenerationParam.builder()
// If you haven't configured an environment variable, replace the next line with your Alibaba Cloud Model Studio API key: .apiKey("sk-xxx")
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen-plus")
.enableThinking(true)
.incrementalOutput(true)
.resultFormat("message")
.messages(Arrays.asList(userMsg))
.build();
}
public static void streamCallWithMessage(Generation gen, Message userMsg)
throws NoApiKeyException, ApiException, InputRequiredException {
GenerationParam param = buildGenerationParam(userMsg);
Flowable<GenerationResult> result = gen.streamCall(param);
result.blockingForEach(message -> handleGenerationResult(message));
}
public static void main(String[] args) {
try {
Generation gen = new Generation();
Message userMsg = Message.builder().role(Role.USER.getValue()).content("Who are you?").build();
streamCallWithMessage(gen, userMsg);
// Print final result
// if (reasoningContent.length() > 0) {
// System.out.println("\n====================Full response====================");
// System.out.println(finalContent.toString());
// }
} catch (ApiException | NoApiKeyException | InputRequiredException e) {
logger.error("An exception occurred: {}", e.getMessage());
}
System.exit(0);
}
}
Réponse
====================Thought process====================
Okay, the user asked "Who are you?", so I need to answer based on previous settings. First, my role is Qwen, a large-scale language model under Alibaba Group. Keep it conversational and simple.
The user might be new to me or confirming my identity. Start by directly stating who I am, then briefly explain my functions and purposes, like answering questions, creating text, programming, etc. Also mention multilingual support so users know I handle different languages.
Also, per guidelines, maintain human-like qualities, so use a friendly tone, maybe add emojis for warmth. Guide users to ask further questions or use my features, like asking what they need help with.
Avoid complex terms and keep it concise. Check for missing key points like multilingual support and specific capabilities. Ensure the response meets all requirements, including conversational style and simplicity.
====================Full response====================
Hello! I'm Qwen, a large-scale language model under Alibaba Group. I can answer questions, create text like stories, official documents, emails, scripts, perform logical reasoning, programming, express opinions, play games, and more. I'm proficient in multiple languages, including but not limited to Chinese, English, German, French, and Spanish. How can I help you?
HTTP
Exemple de code
curl
Pour les modèles de réflexion hybride, définissez enable_thinking sur true pour activer le mode de réflexion. Le paramètre enable_thinking n'a aucun effet sur les modèles qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, QwQ .
# ======= Important notes =======
# API keys differ by region. Get your API key: https://www.alibabacloud.com/help/zh/model-studio/get-api-key
# Singapore region URL. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation" \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-H "X-DashScope-SSE: enable" \
-d '{
"model": "qwen-plus",
"input":{
"messages":[
{
"role": "user",
"content": "Who are you?"
}
]
},
"parameters":{
"enable_thinking": true,
"incremental_output": true,
"result_format": "message"
}
}'
Réponse
id:1
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"Hmm","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":14,"input_tokens":11,"output_tokens":3},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:2
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":",","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":15,"input_tokens":11,"output_tokens":4},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:3
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"user","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":16,"input_tokens":11,"output_tokens":5},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:4
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"asked","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":17,"input_tokens":11,"output_tokens":6},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:5
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"\"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":18,"input_tokens":11,"output_tokens":7},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
......
id:358
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"help","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":373,"input_tokens":11,"output_tokens":362},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:359
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":",","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":374,"input_tokens":11,"output_tokens":363},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:360
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"welcome","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":375,"input_tokens":11,"output_tokens":364},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:361
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"anytime","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":376,"input_tokens":11,"output_tokens":365},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:362
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"tell","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":377,"input_tokens":11,"output_tokens":366},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:363
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"me","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:364
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"!","reasoning_content":"","role":"assistant"},"finish_reason":"null"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
id:365
event:result
:HTTP_STATUS/200
data:{"output":{"choices":[{"message":{"content":"","reasoning_content":"","role":"assistant"},"finish_reason":"stop"}]},"usage":{"total_tokens":378,"input_tokens":11,"output_tokens":367},"request_id":"25d58c29-c47b-9e8d-a0f1-d6c309ec58b1"}
Mise en production
-
Gestion des performances et des ressources : Dans les services backend, le maintien d'une connexion HTTP persistante pour chaque requête en streaming consomme des ressources. Configurez votre service avec une taille de pool de connexions et des valeurs de délai d'attente appropriées. Dans les scénarios de forte concurrence, surveillez l'utilisation des descripteurs de fichiers pour éviter leur épuisement.
-
Rendu côté client : Sur les interfaces web frontales, utilisez les API
ReadableStreametTextDecoderStreampour gérer et restituer fluidement les flux d'événements SSE, offrant ainsi la meilleure expérience utilisateur. -
- Métriques clés : Surveillez le Time to First Token (TTFT), la métrique centrale de l'expérience en streaming. Surveillez également le taux d'erreur de l'API et le temps de réponse moyen.
- Alertes : Définissez des alertes pour les taux d'erreur d'API anormaux, en particulier les erreurs 4xx et 5xx.
-
Configuration du proxy Nginx : Si vous utilisez Nginx comme proxy inverse, sa mise en tampon de sortie par défaut (proxy_buffering) rompt la nature en temps réel des réponses en streaming. Pour garantir que les données soient poussées immédiatement vers les clients, désactivez cette fonctionnalité en définissant
proxy_buffering offdans votre fichier de configuration Nginx.
Codes d'erreur
Si l'appel au modèle échoue et renvoie un message d'erreur, consultez les Codes d'erreur pour résoudre le problème.
FAQ
Q : Pourquoi n'y a-t-il aucune information d'utilisation dans la réponse ?
R : Le protocole OpenAI ne renvoie pas d'informations d'utilisation par défaut. Définissez le paramètre stream_options pour inclure les informations d'utilisation dans le dernier paquet renvoyé.
Q : L'activation de la sortie en streaming affecte-t-elle la qualité de la réponse du modèle ?
R : Non. Cependant, certains modèles ne prennent en charge que la sortie en streaming, et les appels non streamés peuvent provoquer des erreurs de délai d'attente. Nous recommandons d'utiliser la sortie en streaming.
Q : Quelle est la différence entre les appels non streamés et les appels en streaming ?
R : Différences principales :
- Limite de délai d'attente : Pour les appels non streamés, le délai d'attente maximal est d'au moins 300 secondes et varie selon la région et le modèle. Si le traitement n'est pas terminé à temps, la requête est interrompue.
- Structure de sortie : Les appels non streamés renvoient la réponse complète (un seul objet JSON) en une fois. Les appels en streaming renvoient progressivement des fragments de données via le protocole SSE, chaque fragment contenant une partie du contenu généré. Le client doit assembler ces fragments.
- Compatibilité des fonctionnalités : Les deux méthodes prennent en charge des fonctionnalités telles que le mode JSON et Function Call, sans différences fonctionnelles.
Nous recommandons d'utiliser la sortie en streaming pour éviter les délais d'attente et améliorer l'expérience utilisateur.
Q : La sortie en streaming prend-elle en charge le mode JSON (sortie structurée) ?
R : Oui. Définissez stream sur true et response_format sur {"type": "json_object"} dans la requête. Le modèle renverra progressivement des fragments de contenu au format JSON. La sortie finale assemblée sera un JSON valide.