Large Language Models (LLMs) cannot access real-time data or external systems. Function Calling enables models to call external tools, such as APIs, databases, and user-defined functions. This allows a model to retrieve information or perform actions beyond its built-in capabilities.
How it works
Function Calling works through a multi-step interaction between your application and the LLM:
Make the first model call
The application sends the user's question and a list of available tools to the LLM.
Receive tool calling instructions from the model
If the model decides to call an external tool, it returns a JSON instruction that specifies the function name and input parameters.
If the model decides not to call a tool, it returns a natural language response.
Run the tool in the application
The application runs the specified tool and obtains the output.
Make the second model call
Add the tool's output to the messages array and call the model again.
Receive the final response from the model
The model combines the tool's output with the user's question to generate a natural language response.
The following figure shows the workflow.
Supported models
Qwen
Text generation models
Qwen-Max: Qwen3.7-Max series, Qwen3.6-Max series, Qwen3-Max series, and Qwen-Max series
Qwen-Plus: The Qwen3.7-Plus, Qwen3.6-Plus, Qwen3.5-Plus, and Qwen-Plus series
Qwen-Flash: Qwen3.6-Flash series, Qwen3.5-Flash series, and Qwen-Flash series
Qwen-Coder: Qwen3-Coder series, Qwen2.5-Coder series, and Qwen-Coder series
Qwen-Turbo: Qwen-Turbo series
Qwen3.6 open source series
Qwen3.5 open source series
Qwen3 open source series
Qwen2.5 open source series
Multimodal models
Qwen-VL: Qwen3-VL-Plus series and Qwen3-VL-Flash series
Qwen-Omni: Qwen3.5-Omni-Plus series, Qwen3.5-Omni-Flash series, and Qwen3-Omni-Flash series
Qwen-Omni-Realtime: Qwen3.5-Omni-Plus-Realtime series and Qwen3.5-Omni-Flash-Realtime series
Qwen3-VL open source series
DeepSeek
deepseek-v4-pro
deepseek-v4-flash
deepseek-v3.2
deepseek-v3.2-exp (non-thinking mode)
deepseek-v3.1 (non-thinking mode)
deepseek-r1
deepseek-r1-0528
deepseek-v3
GLM
When you use Function Calling with GLM series models, you must include `extra_body={"tool_stream": True}` in your request. Otherwise, the model will not return `tool_calls`, and the tool calling will not work.
glm-5.2
glm-5.1
glm-5
glm-4.7
glm-4.6
Kimi
kimi-k2.6
kimi-k2.5
kimi-k2-thinking
Moonshot-Kimi-K2-Instruct
MiniMax
MiniMax-M2.5
Getting started
Before you begin, obtain an API key and configure it as an environment variable. If you use the OpenAI SDK or DashScope SDK, you must also install the SDK.
The following example shows the complete Function Calling flow for a weather query scenario.
OpenAI compatible
from openai import OpenAI
from datetime import datetime
import json
import os
import random
client = OpenAI(
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
# If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
api_key=os.getenv("DASHSCOPE_API_KEY"),
# If you use a model in the China (Beijing) region, replace the base_url with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
# Simulate a user question
USER_QUESTION = "What's the weather like in Singapore?"
# Define the tool list
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Singapore or New York.",
}
},
"required": ["location"],
},
},
},
]
# Simulate a weather query tool
def get_current_weather(arguments):
weather_conditions = ["Sunny", "Cloudy", "Rainy"]
random_weather = random.choice(weather_conditions)
location = arguments["location"]
return f"The weather in {location} today is {random_weather}."
# Encapsulate the model response function
def get_response(messages):
completion = client.chat.completions.create(
model="qwen3.6-plus",
extra_body={"enable_thinking": False},
messages=messages,
tools=tools,
)
return completion
messages = [{"role": "user", "content": USER_QUESTION}]
response = get_response(messages)
assistant_output = response.choices[0].message
if assistant_output.content is None:
assistant_output.content = ""
messages.append(assistant_output)
# If no tool call is needed, print the content directly
if assistant_output.tool_calls is None:
print(f"No tool call needed. Direct response: {assistant_output.content}")
else:
# Enter the tool calling loop
while assistant_output.tool_calls is not None:
tool_call = assistant_output.tool_calls[0]
tool_call_id = tool_call.id
func_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
print(f"Calling tool [{func_name}], arguments: {arguments}")
# Run the tool
tool_result = get_current_weather(arguments)
# Construct the tool return message
tool_message = {
"role": "tool",
"tool_call_id": tool_call_id,
"content": tool_result, # Keep the original tool output
}
print(f"Tool returns: {tool_message['content']}")
messages.append(tool_message)
# Call the model again to get a summarized natural language response
response = get_response(messages)
assistant_output = response.choices[0].message
if assistant_output.content is None:
assistant_output.content = ""
messages.append(assistant_output)
print(f"Assistant's final response: {assistant_output.content}")import OpenAI from 'openai';
// Initialize the client
const openai = new OpenAI({
apiKey: process.env.DASHSCOPE_API_KEY,
// If you use a model in the China (Beijing) region, replace the baseURL with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
});
// Define the tool list
const tools = [
{
type: "function",
function: {
name: "get_current_weather",
description: "Useful for when you want to query the weather in a specific city.",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "A city or district, such as Singapore or New York.",
},
},
required: ["location"],
},
},
},
];
// Simulate a weather query tool
const getCurrentWeather = (args) => {
const weatherConditions = ["Sunny", "Cloudy", "Rainy"];
const randomWeather = weatherConditions[Math.floor(Math.random() * weatherConditions.length)];
const location = args.location;
return `The weather in ${location} today is ${randomWeather}.`;
};
// Encapsulate the model response function
const getResponse = async (messages) => {
const response = await openai.chat.completions.create({
model: "qwen3.6-plus",
enable_thinking: false,
messages: messages,
tools: tools,
});
return response;
};
const main = async () => {
const input = "What's the weather like in Singapore?";
let messages = [
{
role: "user",
content: input,
}
];
let response = await getResponse(messages);
let assistantOutput = response.choices[0].message;
// Make sure content is not null
if (!assistantOutput.content) assistantOutput.content = "";
messages.push(assistantOutput);
// Determine if a tool call is needed
if (!assistantOutput.tool_calls) {
console.log(`No tool call needed. Direct response: ${assistantOutput.content}`);
} else {
// Enter the tool calling loop
while (assistantOutput.tool_calls) {
const toolCall = assistantOutput.tool_calls[0];
const toolCallId = toolCall.id;
const funcName = toolCall.function.name;
const funcArgs = JSON.parse(toolCall.function.arguments);
console.log(`Calling tool [${funcName}], arguments:`, funcArgs);
// Run the tool
const toolResult = getCurrentWeather(funcArgs);
// Construct the tool return message
const toolMessage = {
role: "tool",
tool_call_id: toolCallId,
content: toolResult,
};
console.log(`Tool returns: ${toolMessage.content}`);
messages.push(toolMessage);
// Call the model again to get a natural language summary
response = await getResponse(messages);
assistantOutput = response.choices[0].message;
if (!assistantOutput.content) assistantOutput.content = "";
messages.push(assistantOutput);
}
console.log(`Assistant's final response: ${assistantOutput.content}`);
}
};
// Start the program
main().catch(console.error);DashScope
import os
from dashscope import MultiModalConversation
import dashscope
import json
import random
# The following URL is for the China (Beijing) region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
dashscope.base_http_api_url = 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1'
# 1. Define the tool list
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Singapore or New York.",
}
},
"required": ["location"],
},
},
}
]
# 2. Simulate a weather query tool
def get_current_weather(arguments):
weather_conditions = ["Sunny", "Cloudy", "Rainy"]
random_weather = random.choice(weather_conditions)
location = arguments["location"]
return f"The weather in {location} today is {random_weather}."
# 3. Encapsulate the model response function
def get_response(messages):
response = MultiModalConversation.call(
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
# If you have not configured the environment variable, replace the following line with api_key="sk-xxx"
api_key=os.getenv("DASHSCOPE_API_KEY"),
# This example uses the multimodal model qwen3.6-plus. To call a text-only model such as qwen3.6-max-preview or qwen-plus, use the text-only model API. For more information, see https://www.alibabacloud.com/help/model-studio/qwen-api-via-dashscope
model="qwen3.6-plus",
enable_thinking=False,
messages=messages,
tools=tools,
result_format="message",
)
return response
# 4. Initialize the conversation history
messages = [
{
"role": "user",
"content": [{"text": "What's the weather like in Singapore?"}]
}
]
# 5. Call the model for the first time
response = get_response(messages)
assistant_output = response.output.choices[0].message
messages.append(assistant_output)
# 6. Determine if a tool call is needed
if "tool_calls" not in assistant_output or not assistant_output["tool_calls"]:
print(f"No tool call needed. Direct response: {assistant_output['content']}")
else:
# 7. Enter the tool calling loop
# Loop condition: as long as the latest model response contains a tool call request
while "tool_calls" in assistant_output and assistant_output["tool_calls"]:
tool_call = assistant_output["tool_calls"][0]
# Parse the tool call information
func_name = tool_call["function"]["name"]
arguments = json.loads(tool_call["function"]["arguments"])
tool_call_id = tool_call.get("id") # Get the tool_call_id
print(f"Calling tool [{func_name}], arguments: {arguments}")
# Run the corresponding tool function
tool_result = get_current_weather(arguments)
# Construct the tool return message
tool_message = {
"role": "tool",
"content": tool_result,
"tool_call_id": tool_call_id
}
print(f"Tool returns: {tool_message['content']}")
messages.append(tool_message)
# Call the model again to get a response based on the tool result
response = get_response(messages)
assistant_output = response.output.choices[0].message
messages.append(assistant_output)
# 8. Print the final natural language response
content = assistant_output["content"]
if isinstance(content, list) and content:
content = content[0].get("text", "") if isinstance(content[0], dict) else str(content[0])
print(f"Assistant's final response: {content}")import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.exception.UploadFileException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.tools.FunctionDefinition;
import com.alibaba.dashscope.tools.ToolCallBase;
import com.alibaba.dashscope.tools.ToolCallFunction;
import com.alibaba.dashscope.tools.ToolFunction;
import com.alibaba.dashscope.utils.JsonUtils;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Random;
public class Main {
/**
* Extracts plain text from the content of a MultiModalMessage.
* The content format is List<Map<String, String>>, for example: [{text=The weather is sunny}]
*/
@SuppressWarnings("unchecked")
public static String getTextContent(Object content) {
if (content instanceof List) {
for (Object item : (List<?>) content) {
if (item instanceof Map) {
Object text = ((Map<String, Object>) item).get("text");
if (text != null) return text.toString();
}
}
}
return content != null ? content.toString() : "";
}
/**
* Defines the local implementation of the tool.
* @param arguments A JSON string from the model containing the required parameters for the tool.
* @return A string with the result of the tool's execution.
*/
public static String getCurrentWeather(String arguments) {
try {
// The parameters provided by the model are in JSON format and need to be parsed manually.
ObjectMapper objectMapper = new ObjectMapper();
JsonNode argsNode = objectMapper.readTree(arguments);
String location = argsNode.get("location").asText();
// Simulates a real API call or business logic with a random result.
List<String> weatherConditions = Arrays.asList("Sunny", "Cloudy", "Rainy");
String randomWeather = weatherConditions.get(new Random().nextInt(weatherConditions.size()));
return "The weather in " + location + " today is " + randomWeather + ".";
} catch (Exception e) {
// Exception handling to ensure program robustness.
return "Failed to parse location parameter.";
}
}
public static void main(String[] args) {
try {
// Describe (register) our tools to the model.
String weatherParamsSchema =
"{\"type\":\"object\",\"properties\":{\"location\":{\"type\":\"string\",\"description\":\"A city or district, such as Singapore or New York.\"}},\"required\":[\"location\"]}";
FunctionDefinition weatherFunction = FunctionDefinition.builder()
.name("get_current_weather") // The unique name of the tool, which must correspond to the local implementation.
.description("Useful for when you want to query the weather in a specific city.") // A clear description helps the model better decide when to use the tool.
.parameters(JsonUtils.parseString(weatherParamsSchema).getAsJsonObject())
.build();
// The following URL is for the China (Beijing) region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
MultiModalConversation conv = new MultiModalConversation(Protocol.HTTP.getValue(), "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1");
String userInput = "What's the weather like in Singapore?";
List<MultiModalMessage> messages = new ArrayList<>();
messages.add(MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("text", userInput))).build());
// First call to the model. Send the user's request and the defined tool list to the model.
MultiModalConversationParam param = MultiModalConversationParam.builder()
.model("qwen3.6-plus") //This example uses the multimodal model qwen3.6-plus. To call a text-only model such as qwen3.6-max-preview or qwen-plus, use the text-only model API. For more information, see https://www.alibabacloud.com/help/model-studio/qwen-api-via-dashscope
.enableThinking(false)
.apiKey(System.getenv("DASHSCOPE_API_KEY")) // Get the API key from the environment variable. API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
.messages(messages) // Pass the current conversation history.
.tools(Arrays.asList(ToolFunction.builder().function(weatherFunction).build())) // Pass the list of available tools.
.build();
MultiModalConversationResult result = conv.call(param);
MultiModalMessage assistantOutput = result.getOutput().getChoices().get(0).getMessage();
messages.add(assistantOutput); // Add the model's first response to the conversation history.
// Check the model's response to determine if it requests a tool call.
if (assistantOutput.getToolCalls() == null || assistantOutput.getToolCalls().isEmpty()) {
// Case A: The model does not call a tool and provides a direct answer.
System.out.println("No tool call needed. Direct response: " + getTextContent(assistantOutput.getContent()));
} else {
// Case B: The model decides to call a tool.
// Use a while loop to handle scenarios where the model calls tools multiple times in a row.
while (assistantOutput.getToolCalls() != null && !assistantOutput.getToolCalls().isEmpty()) {
ToolCallBase toolCall = assistantOutput.getToolCalls().get(0);
// Parse the specific information of the tool call (function name, parameters) from the model's response.
ToolCallFunction functionCall = (ToolCallFunction) toolCall;
String funcName = functionCall.getFunction().getName();
String arguments = functionCall.getFunction().getArguments();
System.out.println("Calling tool [" + funcName + "], arguments: " + arguments);
// Run the corresponding Java method locally based on the tool name.
String toolResult = getCurrentWeather(arguments);
// Construct a message with the role "tool" that contains the tool's execution result.
MultiModalMessage toolMessage = MultiModalMessage.builder()
.role("tool")
.toolCallId(toolCall.getId())
.content(Arrays.asList(Collections.singletonMap("text", toolResult)))
.build();
System.out.println("Tool returns: " + toolResult);
messages.add(toolMessage); // Add the tool's return result to the conversation history.
// Call the model again.
param.setMessages((List) messages);
result = conv.call(param);
assistantOutput = result.getOutput().getChoices().get(0).getMessage();
messages.add(assistantOutput);
}
// Print the final response generated by the model after summarization.
System.out.println("Assistant's final response: " + getTextContent(assistantOutput.getContent()));
}
} catch (NoApiKeyException | UploadFileException e) {
System.err.println("Error: " + e.getMessage());
} catch (Exception e) {
e.printStackTrace();
}
}
}After running the code, the following output is displayed:
Calling tool [get_current_weather], arguments: {'location': 'Singapore'}
Tool returns: The weather in Singapore today is Cloudy.
Assistant's final response: The weather in Singapore today is cloudy.How to use
Function Calling supports two ways to pass tool information:
Method 1: Pass information through the tools parameter (recommended)
For more information, see How to use. Follow the steps to define tools, create a messages array, make a Function Calling, run the tool function, and have the LLM summarize the tool function output.
Method 2: Pass information through a System Message
Passing information through the `tools` parameter provides the best results because the server automatically adapts to the optimal prompt template. If you are using a Qwen model and do not want to use the `tools` parameter, see Pass tool information through a System Message.
The following sections use the OpenAI compatible API as an example to describe the detailed usage of Function Calling with the `tools` parameter.
Assume a business scenario that receives two types of questions: weather queries and time queries.
1. Define tools
Tools connect LLMs to external services. You must first define the tools.
1.1. Create tool functions
Create two tool functions: a weather query tool and a time query tool.
Weather query tool
This tool receives the
argumentsparameter. The format ofargumentsis{"location": "queried location"}. The tool's output is a string in the format:"{location} today is {weather}".For demonstration purposes, the weather query tool defined here does not actually query the weather. It randomly selects from sunny, cloudy, or rainy. In a real business scenario, you can replace this with a tool such as Amap Weather.
Time query tool
The time query tool does not require any input parameters. The tool's output is a string in the format:
"Current time: {queried time}.".If you use Node.js, run
npm install date-fnsto install the date-fns package for obtaining the time.
## Step 1: Define tool functions
# Add the import for the random module
import random
from datetime import datetime
# Simulate a weather query tool. Example output: "The weather in Beijing today is rainy."
def get_current_weather(arguments):
# Define a list of alternative weather conditions
weather_conditions = ["Sunny", "Cloudy", "Rainy"]
# Randomly select a weather condition
random_weather = random.choice(weather_conditions)
# Extract location information from JSON
location = arguments["location"]
# Return the formatted weather information
return f"The weather in {location} today is {random_weather}."
# A tool to query the current time. Example output: "Current time: 2024-04-15 17:15:18."
def get_current_time():
# Get the current date and time
current_datetime = datetime.now()
# Format the current date and time
formatted_time = current_datetime.strftime('%Y-%m-%d %H:%M:%S')
# Return the formatted current time
return f"Current time: {formatted_time}."
# Test the tool functions and print the results. You can remove the following four lines of test code when running the subsequent steps.
print("Testing tool output:")
print(get_current_weather({"location": "Shanghai"}))
print(get_current_time())
print("\n")// Step 1: Define tool functions
// Import the time query tool
import { format } from 'date-fns';
function getCurrentWeather(args) {
// Define a list of alternative weather conditions
const weatherConditions = ["Sunny", "Cloudy", "Rainy"];
// Randomly select a weather condition
const randomWeather = weatherConditions[Math.floor(Math.random() * weatherConditions.length)];
// Extract location information from JSON
const location = args.location;
// Return the formatted weather information
return `The weather in ${location} today is ${randomWeather}.`;
}
function getCurrentTime() {
// Get the current date and time
const currentDatetime = new Date();
// Format the current date and time
const formattedTime = format(currentDatetime, 'yyyy-MM-dd HH:mm:ss');
// Return the formatted current time
return `Current time: ${formattedTime}.`;
}
// Test the tool functions and print the results. You can remove the following four lines of test code when running the subsequent steps.
console.log("Testing tool output:")
console.log(getCurrentWeather({location:"Shanghai"}));
console.log(getCurrentTime());
console.log("\n")After running the tool, the following output is displayed:
Testing tool output:
The weather in Shanghai today is Cloudy.
Current time: 2025-01-08 20:21:45.1.2. Create the tools array
Before humans can choose a tool, they need to understand its function, usage scenarios, and input parameters. The same applies to LLMs. The model selects the appropriate tool based on this information. Provide the tool information in the following JSON format.
| For the weather query tool, the format of the tool description information is as follows: |
Before making a Function Calling, define a tool information array (`tools`) in your code. This array includes the function name, description, and parameter definition for each tool. The array is passed as a parameter in subsequent requests.
# Paste the following code after the Step 1 code
## Step 2: Create the tools array
tools = [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Useful for when you want to know the current time.",
"parameters": {}
}
},
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
"required": ["location"]
}
}
}
]
tool_name = [tool["function"]["name"] for tool in tools]
print(f"Created {len(tools)} tools: {tool_name}\n")// Paste the following code after the Step 1 code
// Step 2: Create the tools array
const tools = [
{
type: "function",
function: {
name: "get_current_time",
description: "Useful for when you want to know the current time.",
parameters: {}
}
},
{
type: "function",
function: {
name: "get_current_weather",
description: "Useful for when you want to query the weather in a specific city.",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
required: ["location"]
}
}
}
];
const toolNames = tools.map(tool => tool.function.name);
console.log(`Created ${tools.length} tools: ${toolNames.join(', ')}\n`);2. Create the messages array
Function Calling passes instructions and context to the LLM through the `messages` array. Before making a call, the `messages` array must contain a System Message and a User Message.
System Message
Although the function and usage scenarios of the tools have been described when you created the tools array, further emphasizing when to call the tools in the System Message usually improves the accuracy of tool calling. For the current scenario, you can set the System Prompt to:
You are a helpful assistant. If the user asks about the weather, call the 'get_current_weather' function;
if the user asks about the time, call the 'get_current_time' function.
Please answer the questions in a friendly tone.User Message
The User Message is used to pass the user's question. Assuming the user asks "Weather in Shanghai", the `messages` array at this point is:
# Step 3: Create the messages array
# Paste the following code after the Step 2 code
# Example User Message for a text generation model
messages = [
{
"role": "system",
"content": """You are a helpful assistant. If the user asks about the weather, call the 'get_current_weather' function;
if the user asks about the time, call the 'get_current_time' function.
Please answer the questions in a friendly tone.""",
},
{
"role": "user",
"content": "Weather in Shanghai"
}
]
# Example User Message for a multimodal model
# messages=[
# {
# "role": "system",
# "content": """You are a helpful assistant. If the user asks about the weather, call the 'get_current_weather' function;
# if the user asks about the time, call the 'get_current_time' function.
# Please answer the questions in a friendly tone.""",
# },
# {"role": "user",
# "content": [{"type": "image_url","image_url": {"url": "https://img.alicdn.com/imgextra/i2/O1CN01FbTJon1ErXVGMRdsN_!!6000000000405-0-tps-1024-683.jpg"}},
# {"type": "text", "text": "Query the current weather for the location in the image"}]},
# ]
print("messages array created\n") // Step 3: Create the messages array
// Paste the following code after the Step 2 code
const messages = [
{
role: "system",
content: "You are a helpful assistant. If the user asks about the weather, call the 'get_current_weather' function; if the user asks about the time, call the 'get_current_time' function. Please answer the questions in a friendly tone.",
},
{
role: "user",
content: "Weather in Shanghai"
}
];
// Example User Message for a multimodal model,
// const messages: [{
// role: "user",
// content: [{type: "image_url", image_url: {"url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"}},
// {type: "text", text: "What is depicted in the image?"}]
// }];
console.log("messages array created\n");Because the available tools include weather and time queries, you can also ask about the current time.
3. Make a Function Calling
Pass the created `tools` and `messages` to the LLM to make a Function Calling. The LLM determines whether to call a tool. If it does, it returns the tool's function name and parameters.
For supported models, see Supported models.
# Step 4: Make a function calling
# Paste the following code after the Step 3 code
from openai import OpenAI
import os
client = OpenAI(
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
# If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
api_key=os.getenv("DASHSCOPE_API_KEY"),
# If you use a model in the China (Beijing) region, replace the base_url with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
def function_calling():
completion = client.chat.completions.create(
# This example uses qwen3.6-plus. You can change the model name as needed. For a list of models, see https://www.alibabacloud.com/help/en/model-studio/getting-started/models
model="qwen3.6-plus",
extra_body={"enable_thinking": False},
messages=messages,
tools=tools
)
print("Returned object:")
print(completion.choices[0].message.model_dump_json())
print("\n")
return completion
print("Making a function calling...")
completion = function_calling()// Step 4: Make a function calling
// Paste the following code after the Step 3 code
import OpenAI from "openai";
const openai = new OpenAI(
{
// API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
// If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
apiKey: process.env.DASHSCOPE_API_KEY,
// If you use a model in the China (Beijing) region, replace the baseURL with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
async function functionCalling() {
const completion = await openai.chat.completions.create({
model: "qwen3.6-plus", // This example uses qwen3.6-plus. You can change the model name as needed. For a list of models, see https://www.alibabacloud.com/help/en/model-studio/getting-started/models
enable_thinking: false,
messages: messages,
tools: tools
});
console.log("Returned object:");
console.log(JSON.stringify(completion.choices[0].message));
console.log("\n");
return completion;
}
const completion = await functionCalling();Because the user asked about the weather in Shanghai, the LLM specifies the tool function name to use as "get_current_weather" and the function's input parameter as "{\"location\": \"Shanghai\"}".
{
"content": "",
"refusal": null,
"role": "assistant",
"audio": null,
"function_call": null,
"tool_calls": [
{
"id": "call_6596dafa2a6a46f7a217da",
"function": {
"arguments": "{\"location\": \"Shanghai\"}",
"name": "get_current_weather"
},
"type": "function",
"index": 0
}
]
}Note that if the LLM determines that no tool is needed for the question, it will respond directly through the content parameter. When you input "Hello", the tool_calls parameter is empty, and the returned object format is:
{
"content": "Hello! How can I help you? I'm particularly good at answering questions about the weather or time.",
"refusal": null,
"role": "assistant",
"audio": null,
"function_call": null,
"tool_calls": null
}If thetool_callsparameter is empty, your program can directly return thecontentwithout running the following steps.
If you want the LLM to select a specific tool every time you make a Function Calling, see Forced tool calling.
4. Run the tool function
Running the tool function translates the model's decision into an actual operation.
The process of running the tool function is completed by your computing environment, not the LLM.
The LLM only outputs a string. Before running the tool function, you need to parse the tool function name and its input parameters separately.
Tool function
Create a mapping
function_mapperfrom the tool function name to the tool function entity to map the returned tool function string to the tool function entity.Input parameters
The input parameters returned by Function Calling are a JSON string. Use a tool to parse it into a JSON object to extract the input parameter information.
After parsing, pass the parameters to the tool function and run it to obtain the output result.
# Step 5: Run the tool function
# Paste the following code after the Step 4 code
import json
print("Running the tool function...")
# Get the function name and input parameters from the returned result
function_name = completion.choices[0].message.tool_calls[0].function.name
arguments_string = completion.choices[0].message.tool_calls[0].function.arguments
# Use the json module to parse the parameter string
arguments = json.loads(arguments_string)
# Create a function mapping table
function_mapper = {
"get_current_weather": get_current_weather,
"get_current_time": get_current_time
}
# Get the function entity
function = function_mapper[function_name]
# If the input parameter is empty, call the function directly
if arguments == {}:
function_output = function()
# Otherwise, pass the parameters and then call the function
else:
function_output = function(arguments)
# Print the tool's output
print(f"Tool function output: {function_output}\n")// Step 5: Run the tool function
// Paste the following code after the Step 4 code
console.log("Running the tool function...");
const function_name = completion.choices[0].message.tool_calls[0].function.name;
const arguments_string = completion.choices[0].message.tool_calls[0].function.arguments;
// Use the JSON module to parse the parameter string
const args = JSON.parse(arguments_string);
// Create a function mapping table
const functionMapper = {
"get_current_weather": getCurrentWeather,
"get_current_time": getCurrentTime
};
// Get the function entity
const func = functionMapper[function_name];
// If the input parameter is empty, call the function directly
let functionOutput;
if (Object.keys(args).length === 0) {
functionOutput = func();
} else {
// Otherwise, pass the parameters and then call the function
functionOutput = func(args);
}
// Print the tool's output
console.log(`Tool function output: ${functionOutput}\n`);After running the code, the following output is displayed:
The weather in Shanghai today is Cloudy.In real business scenarios, many tools perform specific actions (such as sending emails or uploading files) rather than querying data, and do not output a string. We recommend adding status description information (such as "Email sent successfully" or "Operation failed") for such tools to help the LLM understand the execution status.
5. Let the LLM summarize the tool function output
The output format of the tool function is relatively fixed. Directly returning it to the user might sound robotic. Submit the tool output to the model context and call the model again to generate a natural language style response.
Add an Assistant Message
After you make a Function Calling, you obtain an Assistant Message through
completion.choices[0].message. First, add it to the `messages` array.Add a Tool Message
Add the tool's output to the `messages` array in the format
{"role": "tool", "content": "tool output", "tool_call_id": completion.choices[0].message.tool_calls[0].id}.NoteMake sure the tool's output is in string format.
tool_call_idis a unique identifier generated by the system for each tool call request. The model may request to call multiple tools at once. When returning multiple tool results to the model,tool_call_idensures that the tool's output result can be matched with its calling intent.
# Step 6: Submit the tool output to the LLM
# Paste the following code after the Step 5 code
messages.append(completion.choices[0].message)
print("Assistant message added")
messages.append({"role": "tool", "content": function_output, "tool_call_id": completion.choices[0].message.tool_calls[0].id})
print("Tool message added\n")// Step 6: Submit the tool output to the LLM
// Paste the following code after the Step 5 code
messages.push(completion.choices[0].message);
console.log("Assistant message added")
messages.push({
"role": "tool",
"content": functionOutput,
"tool_call_id": completion.choices[0].message.tool_calls[0].id
});
console.log("Tool message added\n");At this point, the `messages` array is:
[
System Message -- Guides the model's tool calling strategy
User Message -- The user's question
Assistant Message -- The tool calling information returned by the model
Tool Message -- The tool's output information (there may be multiple Tool Messages if parallel tool calling is used, as described below)
]After updating the `messages` array, run the following code.
# Step 7: Let the LLM summarize the tool output
# Paste the following code after the Step 6 code
print("Summarizing the tool output...")
completion = function_calling()// Step 7: Let the LLM summarize the tool output
// Paste the following code after the Step 6 code
console.log("Summarizing the tool output...");
const completion_1 = await functionCalling();You can retrieve the response content from content: "The weather in Shanghai today is cloudy. If you have any other questions, feel free to ask."
{
"content": "The weather in Shanghai today is cloudy. If you have any other questions, feel free to ask.",
"refusal": null,
"role": "assistant",
"audio": null,
"function_call": null,
"tool_calls": null
}You have now completed a full Function Calling flow.
Advanced usage
Specify the tool calling method
Parallel tool calling
A single city weather query requires only one tool call. If a question requires multiple tool calls, such as "What's the weather like in Beijing and Shanghai?" or "What's the weather in Hangzhou and what time is it now?", after you make a Function Calling, only one piece of tool call information will be returned. For example, if you ask "What's the weather like in Beijing and Shanghai?":
{
"content": "",
"refusal": null,
"role": "assistant",
"audio": null,
"function_call": null,
"tool_calls": [
{
"id": "call_61a2bbd82a8042289f1ff2",
"function": {
"arguments": "{\"location\": \"Beijing\"}",
"name": "get_current_weather"
},
"type": "function",
"index": 0
}
]
}The returned result contains only the input parameters for Beijing. To ensure the result includes all tool functions and input parameters, you can set the parallel_tool_calls request parameter to true when you make a Function Calling.
Parallel tool calling is suitable for tasks that have no dependencies. If there are dependencies between tasks (the input of tool A is related to the output of tool B), see Getting started to implement serial tool calling (calling one tool at a time) through a `while` loop.
def function_calling():
completion = client.chat.completions.create(
model="qwen3.6-plus", # This example uses qwen3.6-plus. You can change the model name as needed.
extra_body={"enable_thinking": False},
messages=messages,
tools=tools,
# New parameter
parallel_tool_calls=True
)
print("Returned object:")
print(completion.choices[0].message.model_dump_json())
print("\n")
return completion
print("Making a function calling...")
completion = function_calling()async function functionCalling() {
const completion = await openai.chat.completions.create({
model: "qwen3.6-plus", // This example uses qwen3.6-plus. You can change the model name as needed.
enable_thinking: false,
messages: messages,
tools: tools,
parallel_tool_calls: true
});
console.log("Returned object:");
console.log(JSON.stringify(completion.choices[0].message));
console.log("\n");
return completion;
}
const completion = await functionCalling();The tool_calls array in the returned object contains the input parameter information for both Beijing and Shanghai:
{
"content": "",
"role": "assistant",
"tool_calls": [
{
"function": {
"name": "get_current_weather",
"arguments": "{\"location\": \"Beijing\"}"
},
"index": 0,
"id": "call_c2d8a3a24c4d4929b26ae2",
"type": "function"
},
{
"function": {
"name": "get_current_weather",
"arguments": "{\"location\": \"Shanghai\"}"
},
"index": 1,
"id": "call_dc7f2f678f1944da9194cd",
"type": "function"
}
]
}Forced tool calling
LLMs generate content with a degree of uncertainty and may choose the wrong tool. To force the use or disabling of a specific tool for a certain type of question, you can modify the tool_choice parameter. The default value of the tool_choice parameter is "auto", which means the LLM autonomously decides how to make a tool call.
When the LLM summarizes the tool function output, remove the tool_choice parameter. Otherwise, the API will still return tool call information.Force the use of a specific tool
If you want Function Calling to forcibly call a specific tool for a certain type of question, you can set the
tool_choiceparameter to{"type": "function", "function": {"name": "the_function_to_call"}}. The LLM will not participate in the tool selection and will only output the input parameter information.Assuming the current scenario only involves weather query questions, you can modify the `function_calling` code to:
def function_calling(): completion = client.chat.completions.create( model="qwen3.6-plus", extra_body={"enable_thinking": False}, messages=messages, tools=tools, tool_choice={"type": "function", "function": {"name": "get_current_weather"}} ) print(completion.model_dump_json()) function_calling()async function functionCalling() { const response = await openai.chat.completions.create({ model: "qwen3.6-plus", enable_thinking: false, messages: messages, tools: tools, tool_choice: {"type": "function", "function": {"name": "get_current_weather"}} }); console.log("Returned object:"); console.log(JSON.stringify(response.choices[0].message)); console.log("\n"); return response; } const response = await functionCalling();No matter what question is input, the tool function in the returned object will be
get_current_weather.Before using this strategy, make sure the question is related to the selected tool. Otherwise, it may return unexpected results.
Force the use of at least one tool
For some questions that require a tool, the LLM may decide that no call is needed. To force Function Calling to always make a tool call (the
tool_callsparameter in the returned object is not empty), you can set thetool_choiceparameter to"required". Function Calling will then always return tool and input parameter information.Assuming that all questions in the current scenario require a tool call, you can modify the `function_calling` code to:
def function_calling(): completion = client.chat.completions.create( model="qwen3.6-plus", extra_body={"enable_thinking": False}, messages=messages, tools=tools, tool_choice="required" ) print(completion.model_dump_json()) function_calling()async function functionCalling() { const completion = await openai.chat.completions.create({ model: "qwen3.6-plus", enable_thinking: false, messages: messages, tools: tools, tool_choice: "required" }); console.log("Returned object:"); console.log(JSON.stringify(completion.choices[0].message)); console.log("\n"); return completion; } const completion = await functionCalling();No matter what question is input, the
tool_callsparameter in the returned object will never be empty.Before using this strategy, make sure the question is related to the tools. Otherwise, it may return unexpected results.
Force no tool usage
If you need Function Calling to never make a tool call (the returned object contains response content in
contentand thetool_callsparameter is empty), you can set thetool_choiceparameter to"none", or do not pass thetoolsparameter. Thetool_callsparameter returned by Function Calling will always be empty.Assuming that no questions in the current scenario require a tool call, you can modify the `function_calling` code to:
def function_calling(): completion = client.chat.completions.create( model="qwen3.6-plus", extra_body={"enable_thinking": False}, messages=messages, tools=tools, tool_choice="none" ) print(completion.model_dump_json()) function_calling()async function functionCalling() { const completion = await openai.chat.completions.create({ model: "qwen3.6-plus", enable_thinking: false, messages: messages, tools: tools, tool_choice: "none" }); console.log("Returned object:"); console.log(JSON.stringify(completion.choices[0].message)); console.log("\n"); return completion; } const completion = await functionCalling();
Multi-turn conversation
A user might ask "Weather in Beijing" in the first turn, and then "What about Shanghai?" in the second. If the model context lacks the information from the first turn, the model cannot determine which tool to call. In a multi-turn conversation scenario, keep the `messages` array complete after each turn. Add the new User Message to this array and then make a Function Calling and subsequent steps. The `messages` structure is as follows:
[
System Message -- Guides the model's tool calling strategy
User Message -- The user's question
Assistant Message -- The tool calling information returned by the model
Tool Message -- The tool's output information
Assistant Message -- The model's summary of the tool call information
User Message -- The user's second-turn question
]Streaming output
Using streaming output lets you obtain the tool function name and input parameter information in real time, which improves the user experience. In this case:
The parameter information for the tool call is returned in chunks as a data stream.
The tool function name is returned in the first data chunk of the stream response.
from openai import OpenAI
import os
client = OpenAI(
api_key=os.getenv("DASHSCOPE_API_KEY"),
# If you use a model in the China (Beijing) region, replace it with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
"required": ["location"],
},
},
},
]
stream = client.chat.completions.create(
model="qwen3.6-plus",
extra_body={"enable_thinking": False},
messages=[{"role": "user", "content": "Weather in Hangzhou?"}],
tools=tools,
stream=True
)
for chunk in stream:
delta = chunk.choices[0].delta
print(delta.tool_calls)import { OpenAI } from "openai";
const openai = new OpenAI(
{
// API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
// If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
apiKey: process.env.DASHSCOPE_API_KEY,
// If you use a model in the China (Beijing) region, replace the baseURL with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
baseURL: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"
}
);
const tools = [
{
"type": "function",
"function": {
"name": "getCurrentWeather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"]
}
}
}
];
const stream = await openai.chat.completions.create({
model: "qwen3.6-plus",
enable_thinking: false,
messages: [{ role: "user", content: "Weather in Beijing" }],
tools: tools,
stream: true,
});
for await (const chunk of stream) {
const delta = chunk.choices[0].delta;
console.log(delta.tool_calls);
}After running the code, the following output is displayed:
[ChoiceDeltaToolCall(index=0, id='call_8f08d2b0fc0c4d8fab7123', function=ChoiceDeltaToolCallFunction(arguments='{"location":', name='get_current_weather'), type='function')]
[ChoiceDeltaToolCall(index=0, id='', function=ChoiceDeltaToolCallFunction(arguments=' "Hangzhou"}', name=None), type='function')]
NoneRun the following code to assemble the input parameter information (arguments):
tool_calls = {}
for response_chunk in stream:
delta_tool_calls = response_chunk.choices[0].delta.tool_calls
if delta_tool_calls:
for tool_call_chunk in delta_tool_calls:
call_index = tool_call_chunk.index
tool_call_chunk.function.arguments = tool_call_chunk.function.arguments or ""
if call_index not in tool_calls:
tool_calls[call_index] = tool_call_chunk
else:
tool_calls[call_index].function.arguments += tool_call_chunk.function.arguments
print(tool_calls[0].model_dump_json())const toolCalls = {};
for await (const responseChunk of stream) {
const deltaToolCalls = responseChunk.choices[0]?.delta?.tool_calls;
if (deltaToolCalls) {
for (const toolCallChunk of deltaToolCalls) {
const index = toolCallChunk.index;
toolCallChunk.function.arguments = toolCallChunk.function.arguments || "";
if (!toolCalls[index]) {
toolCalls[index] = { ...toolCallChunk };
if (!toolCalls[index].function) {
toolCalls[index].function = { name: '', arguments: '' };
}
}
else if (toolCallChunk.function?.arguments) {
toolCalls[index].function.arguments += toolCallChunk.function.arguments;
}
}
}
}
console.log(JSON.stringify(toolCalls[0]));The following output is displayed:
{"index":0,"id":"call_16c72bef988a4c6c8cc662","function":{"arguments":"{\"location\": \"Hangzhou\"}","name":"get_current_weather"},"type":"function"}In the step where the LLM summarizes the tool function output, the added Assistant Message needs to conform to the format below. Simply replace the elements in tool_calls below with the content above.
{
"content": "",
"refusal": None,
"role": "assistant",
"audio": None,
"function_call": None,
"tool_calls": [
{
"id": "call_xxx",
"function": {
"arguments": '{"location": "xx"}',
"name": "get_current_weather",
},
"type": "function",
"index": 0,
}
],
}Tool calling with the Responses API
The preceding examples are based on the OpenAI Chat Completions and DashScope APIs. If you use the OpenAI Responses API, the overall process is the same, but the API format has the following differences:
Dimension | Chat Completions | Responses API |
Tool definition format | | |
Tool call output | response.choices[0].message.tool_calls | Items in `response.output` where `type` is `function_call` |
Tool result passback | | |
Final response | response.choices[0].message.content | response.output_text |
from openai import OpenAI
import json
import os
import random
# Initialize the client
client = OpenAI(
# If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
# Simulate a user question
USER_QUESTION = "What's the weather like in Singapore?"
# Define the tool list
tools = [
{
"type": "function",
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Singapore or London.",
}
},
"required": ["location"],
},
}
]
# Simulate a weather query tool
def get_current_weather(arguments):
weather_conditions = ["Sunny", "Cloudy", "Rainy"]
random_weather = random.choice(weather_conditions)
location = arguments["location"]
return f"The weather in {location} today is {random_weather}."
# Encapsulate the model response function
def get_response(input_data):
response = client.responses.create(
model="qwen3.6-plus",
extra_body={"enable_thinking": False},
input=input_data,
tools=tools,
)
return response
# Maintain the conversation context
conversation = [{"role": "user", "content": USER_QUESTION}]
response = get_response(conversation)
function_calls = [item for item in response.output if item.type == "function_call"]
# If no tool call is needed, print the content directly
if not function_calls:
print(f"Assistant's final response: {response.output_text}")
else:
# Enter the tool calling loop
while function_calls:
for fc in function_calls:
func_name = fc.name
arguments = json.loads(fc.arguments)
print(f"Calling tool [{func_name}], arguments: {arguments}")
# Run the tool
tool_result = get_current_weather(arguments)
print(f"Tool returns: {tool_result}")
# Append the tool call and result as a pair to the context
conversation.append(
{
"type": "function_call",
"name": fc.name,
"arguments": fc.arguments,
"call_id": fc.call_id,
}
)
conversation.append(
{
"type": "function_call_output",
"call_id": fc.call_id,
"output": tool_result,
}
)
# Call the model again with the full context
response = get_response(conversation)
function_calls = [
item for item in response.output if item.type == "function_call"
]
print(f"Assistant's final response: {response.output_text}")
import OpenAI from "openai";
// Initialize the client
const openai = new OpenAI({
// API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
// If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
apiKey: process.env.DASHSCOPE_API_KEY,
baseURL:
"https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
});
// Define the tool list
const tools = [
{
type: "function",
name: "get_current_weather",
description: "Useful for when you want to query the weather in a specific city.",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "A city or district, such as Singapore or London.",
},
},
required: ["location"],
},
},
];
// Simulate a weather query tool
const getCurrentWeather = (args) => {
const weatherConditions = ["Sunny", "Cloudy", "Rainy"];
const randomWeather =
weatherConditions[Math.floor(Math.random() * weatherConditions.length)];
const location = args.location;
return `The weather in ${location} today is ${randomWeather}.`;
};
// Encapsulate the model response function
const getResponse = async (inputData) => {
const response = await openai.responses.create({
model: "qwen3.6-plus",
enable_thinking: false,
input: inputData,
tools: tools,
});
return response;
};
const main = async () => {
const userQuestion = "Weather in Singapore";
// Maintain the conversation context
const conversation = [{ role: "user", content: userQuestion }];
let response = await getResponse(conversation);
let functionCalls = response.output.filter(
(item) => item.type === "function_call"
);
// If no tool call is needed, print the content directly
if (functionCalls.length === 0) {
console.log(`Assistant's final response: ${response.output_text}`);
} else {
// Enter the tool calling loop
while (functionCalls.length > 0) {
for (const fc of functionCalls) {
const funcName = fc.name;
const args = JSON.parse(fc.arguments);
console.log(`Calling tool [${funcName}], arguments:`, args);
// Run the tool
const toolResult = getCurrentWeather(args);
console.log(`Tool returns: ${toolResult}`);
// Append the tool call and result as a pair to the context
conversation.push({
type: "function_call",
name: fc.name,
arguments: fc.arguments,
call_id: fc.call_id,
});
conversation.push({
type: "function_call_output",
call_id: fc.call_id,
output: toolResult,
});
}
// Call the model again with the full context
response = await getResponse(conversation);
functionCalls = response.output.filter(
(item) => item.type === "function_call"
);
}
console.log(`Assistant's final response: ${response.output_text}`);
}
};
// Start the program
main().catch(console.error);
Tool calling for omni-modal models
Omni-modal models support tool calling. The calling methods for the Qwen-Omni series and Qwen-Omni-Realtime series are different.
Qwen-Omni series
The Qwen3.5-Omni-Plus, Qwen3.5-Omni-Flash, and Qwen3-Omni-Flash series support tool calling through the OpenAI compatible API. The stage of obtaining tool information differs from other models in the following ways:
Streaming output is mandatory: Qwen-Omni only supports streaming output. When obtaining tool information, you must also set `
stream=True`.Text-only output is recommended: The model only needs text information when obtaining tool information (function name and parameters). To avoid generating unnecessary audio, we recommend setting `
modalities=["text"]`. When the output includes both text and audio modalities, you need to skip the audio data chunks when obtaining tool information.
For more information about Qwen-Omni, see Non-real-time (Qwen-Omni).
from openai import OpenAI
import os
client = OpenAI(
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
api_key=os.getenv("DASHSCOPE_API_KEY"),
# If you use a model in the China (Beijing) region, replace it with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
"required": ["location"],
},
},
},
]
completion = client.chat.completions.create(
model="qwen3.5-omni-plus",
messages=[{"role": "user", "content": "Weather in Hangzhou?"}],
# Set the modality of the output data. Valid values: ["text"], ["text","audio"]. We recommend setting it to ["text"].
modalities=["text"],
# stream must be set to True, otherwise an error will occur.
stream=True,
tools=tools
)
for chunk in completion:
# If the output includes the audio modality, change the following condition to: if chunk.choices and not hasattr(chunk.choices[0].delta, "audio"):
if chunk.choices:
delta = chunk.choices[0].delta
print(delta.tool_calls)import { OpenAI } from "openai";
const openai = new OpenAI(
{
// API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
// If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
apiKey: process.env.DASHSCOPE_API_KEY,
// If you use a model in the China (Beijing) region, replace the baseURL with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
}
);
const tools = [
{
"type": "function",
"function": {
"name": "getCurrentWeather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"]
}
}
}
];
const stream = await openai.chat.completions.create({
model: "qwen3-omni-flash",
messages: [
{
"role": "user",
"content": "Weather in Hangzhou"
}],
stream: true,
// Set the modality of the output data. Valid values: ["text"], ["text","audio"]. We recommend setting it to ["text"].
modalities: ["text"],
tools:tools
});
for await (const chunk of stream) {
// If the output includes audio, replace the conditional statement with: if (chunk.choices?.length && chunk.choices[0].delta && !('audio' in chunk.choices[0].delta))
if (chunk.choices?.length){
const delta = chunk.choices[0].delta;
console.log(delta.tool_calls);
}}After running the code, the following output is displayed:
[ChoiceDeltaToolCall(index=0, id='call_391c8e5787bc4972a388aa', function=ChoiceDeltaToolCallFunction(arguments=None, name='get_current_weather'), type='function')]
[ChoiceDeltaToolCall(index=0, id='call_391c8e5787bc4972a388aa', function=ChoiceDeltaToolCallFunction(arguments=' {"location": "Hangzhou"}', name=None), type='function')]
NoneFor the code to assemble the input parameter information (arguments), see Streaming output.
Qwen-Omni-Realtime series
The Qwen3.5-Omni-Plus-Realtime and Qwen3.5-Omni-Flash-Realtime series support tool calling and are suitable for voice conversation scenarios. You can call them through the DashScope SDK or the native WebSocket protocol.
Workflow:
After establishing a WebSocket connection, pass the tool definition through session.update to enter the following interaction flow:
Phase 1: Speech input and tool calling
The user asks a question by voice. The client collects the audio and sends it to the server (corresponding to the
append_audio()method). After the server's VAD detects the end of speech, it performs model inference and determines that a tool needs to be called.The server returns the tool call information to the client (corresponding to the
response.function_call_arguments.doneevent), including the function name (name), function input parameters (arguments), and call identifier (call_id). An example is as follows:{ "type": "response.function_call_arguments.done", "response_id": "resp_JnTOsWXlFhKcFohZbtfz6", "item_id": "item_Rhcms7CauTNsQprV5S4Hr", "output_index": 0, "name": "get_current_weather", "call_id": "call_2be200f4cafe419b9530dd", "arguments": "{\"location\": \"Hangzhou\"}" }The client runs the corresponding tool function locally based on the function name and input parameters to obtain the execution result.
Phase 2: Client sends back tool results and triggers the final response
The client sends the tool execution result back to the server (corresponding to the
conversation.item.createevent), including the call identifier (call_id) and execution result (output). An example is as follows:{ "type": "conversation.item.create", "item": { "type": "function_call_output", "call_id": "call_2be200f4cafe419b9530dd", "output": "The weather in Hangzhou today is sunny, with a temperature of 25°C and a light breeze." } }The client continues to send a
response.createevent to trigger the server to generate the final voice answer based on the tool execution result.The client receives the voice and text returned by the server (corresponding to the
response.audio.deltaandresponse.audio_transcript.deltaevents) and plays the voice response to the user.
The Qwen-Omni-Realtime series does not support thetool_choiceandparallel_tool_callsparameters.
For more information about Qwen-Omni-Realtime, see Real-time (Qwen-Omni-Realtime), Client events, and Server-side events.
DashScope Python SDK
import os
import uuid
import threading
import traceback
import json
import base64
import signal
import sys
import time
from typing import Dict, Any, Optional, List
import pyaudio
import queue
import contextlib
import dashscope
from dashscope.audio.qwen_omni import *
# ==================== Constant Definitions ====================
VOICE = 'Tina'
MODEL = "qwen3.5-omni-plus-realtime"
# To access the Beijing region, replace WS_URL with: wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime
WS_URL = "wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime"
# Configure the API key. If you have not set the environment variable, replace the following line with your API key: dashscope.api_key = "sk-xxx"
dashscope.api_key = os.getenv('DASHSCOPE_API_KEY')
AUDIO_SAMPLE_RATE = 16000
AUDIO_CHUNK_SIZE = 3200
OUTPUT_AUDIO_SAMPLE_RATE = 24000
# ==================== Tool Definitions ====================
def get_train_price(src: str, dst: str) -> str:
"""Query train ticket prices"""
return f"The train ticket price from {src} to {dst} is 100-200 CNY."
def get_flight_price(src: str, dst: str) -> str:
"""Query flight ticket prices"""
return f"The flight ticket price from {src} to {dst} is 200-300 USD."
def get_current_weather(location: str) -> str:
"""Query the weather in a specific city"""
return f"The weather in {location} today is changing from haze to sunny, with a temperature of 4/-4°C and a light breeze."
# Unified OpenAI format tool definitions
TOOLS = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
"required": ["location"],
},
},
},
{
"type": "function",
"function": {
"name": "get_flight_price",
"description": "Useful for when you want to query flight ticket prices.",
"parameters": {
"type": "object",
"properties": {
"src": {
"type": "string",
"description": "The departure city of the flight, such as Beijing or Hangzhou.",
},
"dst": {
"type": "string",
"description": "The arrival city of the flight, such as Beijing or Hangzhou.",
},
},
"required": ["src", "dst"],
},
},
},
{
"type": "function",
"function": {
"name": "get_train_price",
"description": "Useful for when you want to query train ticket prices.",
"parameters": {
"type": "object",
"properties": {
"src": {
"type": "string",
"description": "The departure city of the train, such as Beijing or Hangzhou.",
},
"dst": {
"type": "string",
"description": "The arrival city of the train, such as Beijing or Hangzhou.",
},
},
"required": ["src", "dst"],
},
},
},
]
# Mapping of tool names to functions
TOOL_FUNCTIONS = {
"get_current_weather": get_current_weather,
"get_flight_price": get_flight_price,
"get_train_price": get_train_price,
}
# ==================== Tool Call Handling ====================
def handle_tool_call(tool_call_response: Dict[str, Any]) -> Dict[str, Any]:
"""
Handles a tool call request
Args:
tool_call_response: Tool call information containing name, arguments, and call_id
Returns:
Updated tool call response containing the output field
"""
try:
function_name = tool_call_response['name']
tool_call_arguments = json.loads(tool_call_response['arguments'])
print(f'[Tool Call] Start processing: name={function_name}, args={tool_call_arguments}')
# Find the corresponding function
if function_name not in TOOL_FUNCTIONS:
tool_call_response['output'] = f"Client did not find the tool: {function_name}"
print(f'[Tool Call] Error: Tool not found {function_name}')
return tool_call_response
# Call the function
func = TOOL_FUNCTIONS[function_name]
result = func(**tool_call_arguments)
tool_call_response['output'] = result
print(f'[Tool Call] Completed: {result}')
return tool_call_response
except Exception as e:
error_msg = f"Tool call failed: {str(e)}"
tool_call_response['output'] = error_msg
print(f'[Tool Call] Exception: {error_msg}')
traceback.print_exc()
return tool_call_response
def send_tool_call_response(conversation: OmniRealtimeConversation, response: Dict[str, Any]) -> None:
"""Sends the tool call result to the server"""
conversation.create_item({
"id": 'item_' + uuid.uuid4().hex,
"type": "function_call_output",
"call_id": response['call_id'],
"output": response["output"],
})
# ==================== PCM Audio Player ====================
class PCMPlayer:
"""
PCM Audio Player
Uses a dual-thread architecture for real-time audio playback:
- Decoding thread: Decodes base64-encoded audio data into raw PCM data
- Playback thread: Writes PCM data to the audio output device
Supports dynamically adding audio data, canceling playback, saving audio files, etc.
"""
def __init__(self, pya: pyaudio.PyAudio, sample_rate=24000, chunk_size_ms=100, save_file=False):
"""
Initializes the PCM player
Args:
pya: pyaudio.PyAudio instance
sample_rate: Audio sampling rate (Hz), default 24000
chunk_size_ms: Audio chunk size (milliseconds), affects playback cancellation latency, default 100ms
save_file: Whether to save the played audio to a file (result.pcm), default False
"""
self.pya = pya
self.sample_rate = sample_rate
self.chunk_size_bytes = chunk_size_ms * sample_rate * 2 // 1000
self.player_stream = pya.open(format=pyaudio.paInt16,
channels=1,
rate=sample_rate,
output=True)
self.raw_audio_buffer: queue.Queue = queue.Queue()
self.b64_audio_buffer: queue.Queue = queue.Queue()
self.status_lock = threading.Lock()
self.status = 'playing'
self.decoder_thread = threading.Thread(target=self.decoder_loop)
self.player_thread = threading.Thread(target=self.player_loop)
self.decoder_thread.start()
self.player_thread.start()
self.complete_event: threading.Event = None
self.save_file = save_file
if self.save_file:
self.out_file = open('result.pcm', 'wb')
def decoder_loop(self):
"""Decoding thread: Decodes base64 audio data into raw PCM data"""
while self.status != 'stop':
recv_audio_b64 = None
with contextlib.suppress(queue.Empty):
recv_audio_b64 = self.b64_audio_buffer.get(timeout=0.1)
if recv_audio_b64 is None:
continue
recv_audio_raw = base64.b64decode(recv_audio_b64)
# push raw audio data into queue by chunk
for i in range(0, len(recv_audio_raw), self.chunk_size_bytes):
chunk = recv_audio_raw[i:i + self.chunk_size_bytes]
self.raw_audio_buffer.put(chunk)
if self.save_file:
self.out_file.write(chunk)
def player_loop(self):
"""Playback thread: Writes PCM data to the audio output device"""
while self.status != 'stop':
recv_audio_raw = None
with contextlib.suppress(queue.Empty):
recv_audio_raw = self.raw_audio_buffer.get(timeout=0.1)
if recv_audio_raw is None:
if self.complete_event:
self.complete_event.set()
continue
# write chunk to pyaudio audio player, wait until finish playing this chunk.
self.player_stream.write(recv_audio_raw)
def cancel_playing(self):
"""Cancel playback: Clear all buffer queues"""
self.b64_audio_buffer.queue.clear()
self.raw_audio_buffer.queue.clear()
def add_data(self, data):
"""Add base64-encoded audio data to the playback queue"""
self.b64_audio_buffer.put(data)
def wait_for_complete(self):
"""Wait for playback to complete"""
self.complete_event = threading.Event()
self.complete_event.wait()
self.complete_event = None
def shutdown(self):
"""Shut down the player and release resources"""
self.status = 'stop'
self.decoder_thread.join()
self.player_thread.join()
self.player_stream.close()
if self.save_file:
self.out_file.close()
# ==================== Audio Manager ====================
class AudioManager:
"""Manages audio input and output resources"""
def __init__(self):
self.pya: Optional[pyaudio.PyAudio] = None
self.mic_stream: Optional[pyaudio.Stream] = None
self.player: Optional[PCMPlayer] = None
def initialize(self) -> None:
"""Initialize audio devices"""
print('Initializing audio devices...')
self.pya = pyaudio.PyAudio()
self.mic_stream = self.pya.open(
format=pyaudio.paInt16,
channels=1,
rate=AUDIO_SAMPLE_RATE,
input=True
)
self.player = PCMPlayer(self.pya, sample_rate=OUTPUT_AUDIO_SAMPLE_RATE)
print('Audio devices initialized')
def read_audio_chunk(self) -> Optional[bytes]:
"""Read an audio data chunk"""
if not self.mic_stream:
return None
try:
return self.mic_stream.read(AUDIO_CHUNK_SIZE, exception_on_overflow=False)
except Exception as e:
print(f'[Error] Failed to read audio data: {e}')
return None
def cleanup(self) -> None:
"""Clean up audio resources"""
print('Cleaning up audio resources...')
if self.player:
self.player.shutdown()
if self.mic_stream:
self.mic_stream.close()
if self.pya:
self.pya.terminate()
print('Audio resources cleaned up')
# ==================== Callback Handler ====================
class OmniCallback(OmniRealtimeCallback):
"""Omni real-time conversation callback handler"""
def __init__(self, audio_manager: AudioManager):
self.audio_manager = audio_manager
self.tool_calls: Dict[str, Dict[str, Any]] = {}
self.all_response_text: str = ''
self.last_package_time: float = 0
self.is_first_text: bool = True
self.is_first_audio: bool = True
self.conversation: Optional[OmniRealtimeConversation] = None
def set_conversation(self, conversation: OmniRealtimeConversation) -> None:
"""Set the conversation instance reference"""
self.conversation = conversation
def on_open(self) -> None:
"""Callback on connection establishment"""
print('Connection established')
self.audio_manager.initialize()
self.last_package_time = time.time() * 1000
self.is_first_text = True
self.is_first_audio = True
self.tool_calls = {}
self.all_response_text = ''
def on_close(self, close_status_code: int, close_msg: str) -> None:
"""Callback on connection closure"""
print(f'Connection closed: code={close_status_code}, msg={close_msg}')
self.audio_manager.cleanup()
sys.exit(0)
def on_event(self, response: Dict[str, Any]) -> None:
"""Handle event callbacks"""
try:
event_type = response.get('type', '')
# Session created
if event_type == 'session.created':
print(f'Session started: {response["session"]["id"]}')
# Speech-to-text completed
elif event_type == 'conversation.item.input_audio_transcription.completed':
print(f'User question: {response.get("transcript", "")}')
# Incremental text response
elif event_type in ('response.audio_transcript.delta', 'response.text.delta'):
if self.is_first_text:
self.is_first_text = False
latency = time.time() * 1000 - self.last_package_time
print(f'Time to first token (VAD end): {latency:.0f} ms')
text = response.get('delta', '')
self.all_response_text += text
# Incremental audio response
elif event_type == 'response.audio.delta':
if self.is_first_audio:
self.is_first_audio = False
latency = time.time() * 1000 - self.last_package_time
print(f'Time to first audio (VAD end): {latency:.0f} ms')
audio_interval = time.time() * 1000 - self.last_package_time
print(f'Audio interval: {audio_interval:.0f} ms')
self.last_package_time = time.time() * 1000
recv_audio_b64 = response.get('delta', '')
if self.audio_manager.player:
self.audio_manager.player.add_data(recv_audio_b64)
# VAD detected speech start
elif event_type == 'input_audio_buffer.speech_started':
print('====== VAD detected speech start ======')
if self.audio_manager.player:
self.audio_manager.player.cancel_playing()
# VAD detected speech end
elif event_type == 'input_audio_buffer.speech_stopped':
print('====== VAD detected speech end ======')
self.last_package_time = time.time() * 1000
self.is_first_text = True
self.is_first_audio = True
self.tool_calls = {}
# Function call arguments completed
elif event_type == 'response.function_call_arguments.done':
print('====== Received tool call request ======')
call_id = response.get('call_id', '')
self.tool_calls[call_id] = response.copy()
self.tool_calls[call_id]['processed'] = False
# Response completed
elif event_type == 'response.done':
print('====== Response completed ======')
print(f'Full response: {self.all_response_text}')
if self.conversation:
response_id = self.conversation.get_last_response_id()
text_delay = self.conversation.get_last_first_text_delay()
audio_delay = self.conversation.get_last_first_audio_delay()
# Print detailed metrics only when all are available
if response_id is not None and text_delay is not None and audio_delay is not None:
print(f'[Metric] Response ID: {response_id}, '
f'Time to first token: {text_delay:.0f}ms, '
f'Time to first audio: {audio_delay:.0f}ms')
else:
print('[Metric] Metric information is temporarily unavailable (possibly a response after a tool call)')
self.all_response_text = ''
except Exception as e:
print(f'[Error] Exception handling event: {e}')
traceback.print_exc()
def process_pending_tool_calls(self) -> bool:
"""
Processes pending tool calls
Returns:
Whether there are new tool calls that need a response
"""
has_pending = False
for call_id, tool_call in self.tool_calls.items():
if not tool_call.get('processed', False):
has_pending = True
tool_call['processed'] = True
# Handle the tool call
result = handle_tool_call(tool_call)
# Send the result to the server
if self.conversation:
send_tool_call_response(self.conversation, result)
return has_pending
# ==================== Main Program ====================
def main():
"""Main function"""
print('Initializing Omni real-time conversation...')
# Create an audio manager
audio_manager = AudioManager()
# Create a callback handler
callback = OmniCallback(audio_manager)
# Create a conversation instance
conversation = OmniRealtimeConversation(
api_key=dashscope.api_key,
url=WS_URL,
model=MODEL,
callback=callback,
)
# Set the conversation reference in the callback
callback.set_conversation(conversation)
# Establish the connection
conversation.connect()
# Configure session parameters
omni_output_modalities = [MultiModality.AUDIO, MultiModality.TEXT]
conversation.update_session(
output_modalities=omni_output_modalities,
voice=VOICE,
input_audio_format=AudioFormat.PCM_16000HZ_MONO_16BIT,
output_audio_format=AudioFormat.PCM_24000HZ_MONO_16BIT,
enable_input_audio_transcription=True,
enable_turn_detection=True,
turn_detection_type='server_vad',
tools=TOOLS,
)
# Set up signal handling
def signal_handler(sig, frame):
print('\nReceived Ctrl+C, stopping...')
conversation.close()
audio_manager.cleanup()
print('Omni real-time conversation stopped')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print("Press Ctrl+C to stop the conversation...\n")
# Main loop: continuously send audio and check for tool calls
try:
while True:
# Process pending tool calls
has_tool_calls = callback.process_pending_tool_calls()
if has_tool_calls:
print("*** Tool call completed, creating new response ***")
conversation.create_response(
instructions=None,
output_modalities=omni_output_modalities
)
print('====== Tool call processing completed ======\n')
# Read and send audio data
audio_data = audio_manager.read_audio_chunk()
if audio_data:
audio_b64 = base64.b64encode(audio_data).decode('ascii')
conversation.append_audio(audio_b64)
else:
break
except KeyboardInterrupt:
signal_handler(signal.SIGINT, None)
except Exception as e:
print(f'[Error] Main loop exception: {e}')
traceback.print_exc()
finally:
conversation.close()
audio_manager.cleanup()
if __name__ == '__main__':
main()DashScope Java SDK
import com.alibaba.dashscope.audio.omni.*;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import javax.sound.sampled.*;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Function;
public class Main {
public static void main(String[] args) {
try {
// Initialize components
AudioPlayer audioPlayer = new AudioPlayer();
ToolRegistry toolRegistry = new ToolRegistry();
ConversationHandler handler = new ConversationHandler(audioPlayer, toolRegistry);
// Create and configure the session
OmniRealtimeParam param = OmniRealtimeParam.builder()
.model("qwen3.5-omni-plus-realtime")
.apikey(System.getenv("DASHSCOPE_API_KEY"))
// To access the Beijing region, replace the url with: wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime
.url("wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime")
.build();
OmniRealtimeConversation conversation = new OmniRealtimeConversation(param, handler);
conversation.connect();
// Configure session parameters
configureSession(conversation, toolRegistry);
// Start audio capture
startAudioCapture(conversation, handler);
// Clean up resources
cleanup(conversation, audioPlayer);
} catch (NoApiKeyException e) {
System.err.println("API KEY not found: Please set the DASHSCOPE_API_KEY environment variable");
} catch (Exception e) {
e.printStackTrace();
}
}
private static void configureSession(OmniRealtimeConversation conversation, ToolRegistry toolRegistry) {
HashMap<String, Object> additionalConfig = new HashMap<>();
additionalConfig.put("tools", toolRegistry.buildToolsDefinition());
conversation.updateSession(OmniRealtimeConfig.builder()
.modalities(Arrays.asList(OmniRealtimeModality.AUDIO, OmniRealtimeModality.TEXT))
.voice("Tina")
.enableTurnDetection(true)
.enableInputAudioTranscription(true)
.parameters(additionalConfig)
.build());
System.out.println("Tool calling is enabled. Please start speaking (Press Ctrl+C to exit)...");
}
private static void startAudioCapture(OmniRealtimeConversation conversation, ConversationHandler handler)
throws LineUnavailableException {
AudioFormat format = new AudioFormat(16000, 16, 1, true, false);
TargetDataLine mic = AudioSystem.getTargetDataLine(format);
mic.open(format);
mic.start();
ByteBuffer buffer = ByteBuffer.allocate(3200);
while (!handler.getShouldStop().get()) {
int bytesRead = mic.read(buffer.array(), 0, buffer.capacity());
if (bytesRead > 0) {
conversation.appendAudio(Base64.getEncoder().encodeToString(buffer.array()));
// Check and process pending tool calls
if (handler.hasPendingToolCalls()) {
System.out.println("*** create response after call tools");
handler.processPendingToolCalls(conversation);
conversation.createResponse(null, Arrays.asList(OmniRealtimeModality.AUDIO, OmniRealtimeModality.TEXT));
System.out.println("======TOOL CALL END======");
}
}
try {
Thread.sleep(20);
} catch (InterruptedException ignored) {}
}
mic.close();
}
private static void cleanup(OmniRealtimeConversation conversation, AudioPlayer audioPlayer) {
try {
conversation.close(1000, "Normal exit");
audioPlayer.close();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Audio Player - Responsible for sequential playback of audio data
*/
static class AudioPlayer {
private final SourceDataLine line;
private final Queue<byte[]> audioQueue = new ConcurrentLinkedQueue<>();
private final Thread playerThread;
private final AtomicBoolean shouldStop = new AtomicBoolean(false);
public AudioPlayer() throws LineUnavailableException {
AudioFormat format = new AudioFormat(24000, 16, 1, true, false);
line = AudioSystem.getSourceDataLine(format);
line.open(format);
line.start();
playerThread = new Thread(this::playLoop, "AudioPlayer");
playerThread.start();
}
private void playLoop() {
while (!shouldStop.get()) {
byte[] audio = audioQueue.poll();
if (audio != null) {
line.write(audio, 0, audio.length);
} else {
try {
Thread.sleep(10);
} catch (InterruptedException ignored) {}
}
}
}
public void play(String base64Audio) {
audioQueue.add(Base64.getDecoder().decode(base64Audio));
}
public void close() {
shouldStop.set(true);
try {
playerThread.join(1000);
} catch (InterruptedException ignored) {}
line.drain();
line.close();
}
}
/**
* Tool Registry - Manages available tools and their implementations
*/
static class ToolRegistry {
private final Map<String, Function<JsonObject, String>> tools = new ConcurrentHashMap<>();
private final Map<String, JsonObject> pendingToolCalls = new ConcurrentHashMap<>();
public ToolRegistry() {
registerDefaultTools();
}
private void registerDefaultTools() {
registerTool("get_current_weather", this::getCurrentWeather);
registerTool("get_flight_price", this::getFlightPrice);
registerTool("get_train_price", this::getTrainPrice);
}
public void registerTool(String name, Function<JsonObject, String> handler) {
tools.put(name, handler);
}
/**
* Build tool definitions (OpenAI format)
*/
public List<Map<String, Object>> buildToolsDefinition() {
List<Map<String, Object>> definitions = new ArrayList<>();
definitions.add(createFunctionDefinition(
"get_current_weather",
"Useful for when you want to query the weather in a specific city.",
createParamsSchema(
Collections.singletonMap("location",
createProperty("string", "A city or district, such as Beijing, Hangzhou, or Yuhang.")),
Collections.singletonList("location")
)
));
Map<String, Object> flightProps = new HashMap<>();
flightProps.put("src", createProperty("string", "The departure city of the flight, such as Beijing or Hangzhou."));
flightProps.put("dst", createProperty("string", "The arrival city of the flight, such as Beijing or Hangzhou."));
definitions.add(createFunctionDefinition(
"get_flight_price",
"Useful for when you want to query flight ticket prices.",
createParamsSchema(flightProps, Arrays.asList("src", "dst"))
));
Map<String, Object> trainProps = new HashMap<>();
trainProps.put("src", createProperty("string", "The departure city of the train, such as Beijing or Hangzhou."));
trainProps.put("dst", createProperty("string", "The arrival city of the train, such as Beijing or Hangzhou."));
definitions.add(createFunctionDefinition(
"get_train_price",
"Useful for when you want to query train ticket prices.",
createParamsSchema(trainProps, Arrays.asList("src", "dst"))
));
return definitions;
}
private Map<String, Object> createFunctionDefinition(String name, String description, Map<String, Object> parameters) {
Map<String, Object> function = new HashMap<>();
function.put("name", name);
function.put("description", description);
function.put("parameters", parameters);
Map<String, Object> tool = new HashMap<>();
tool.put("type", "function");
tool.put("function", function);
return tool;
}
private Map<String, Object> createParamsSchema(Map<String, Object> properties, List<String> required) {
Map<String, Object> schema = new HashMap<>();
schema.put("type", "object");
schema.put("properties", properties);
schema.put("required", required);
return schema;
}
private Map<String, Object> createProperty(String type, String description) {
Map<String, Object> prop = new HashMap<>();
prop.put("type", type);
prop.put("description", description);
return prop;
}
/**
* Add a tool call to the pending queue
*/
public void addPendingToolCall(String callId, JsonObject toolCall) {
pendingToolCalls.put(callId, toolCall);
}
/**
* Check if there are pending tool calls
*/
public boolean hasPendingToolCalls() {
return !pendingToolCalls.isEmpty();
}
/**
* Process all pending tool calls
*/
public void processPendingToolCalls(OmniRealtimeConversation conversation) {
if (pendingToolCalls.isEmpty()) {
return;
}
for (Map.Entry<String, JsonObject> entry : pendingToolCalls.entrySet()) {
String callId = entry.getKey();
JsonObject toolCall = entry.getValue();
String result = executeTool(toolCall);
sendToolResult(conversation, callId, result);
}
pendingToolCalls.clear();
}
private String executeTool(JsonObject toolCall) {
String functionName = toolCall.get("name").getAsString();
JsonObject arguments = new Gson().fromJson(
toolCall.get("arguments").getAsString(),
JsonObject.class
);
System.out.println("[Tool Call] start handling: " + functionName + ", args: " + arguments);
Function<JsonObject, String> handler = tools.get(functionName);
if (handler == null) {
return "Client did not find this tool. Call failed.";
}
String result = handler.apply(arguments);
System.out.println("[Tool Call] response: " + result);
return result;
}
private void sendToolResult(OmniRealtimeConversation conversation, String callId, String output) {
JsonObject item = new JsonObject();
item.addProperty("id", "item_" + UUID.randomUUID().toString().replace("-", ""));
item.addProperty("type", "function_call_output");
item.addProperty("call_id", callId);
item.addProperty("output", output);
conversation.createItem(item);
}
// ===== Tool Implementations =====
private String getCurrentWeather(JsonObject args) {
String location = args.get("location").getAsString();
return "The weather in " + location + " today is changing from haze to sunny, with a temperature of 4/-4°C and a light breeze.";
}
private String getFlightPrice(JsonObject args) {
String src = args.get("src").getAsString();
String dst = args.get("dst").getAsString();
return "The flight ticket price from " + src + " to " + dst + " is 200-300 USD.";
}
private String getTrainPrice(JsonObject args) {
String src = args.get("src").getAsString();
String dst = args.get("dst").getAsString();
return "invalid apikey error";
}
}
/**
* Conversation Handler - Handles WebSocket events
*/
static class ConversationHandler extends OmniRealtimeCallback {
private final AudioPlayer audioPlayer;
private final ToolRegistry toolRegistry;
private final AtomicBoolean shouldStop = new AtomicBoolean(false);
private final AtomicReference<StringBuilder> responseTextRef = new AtomicReference<>(new StringBuilder());
private long lastPackageTime = 0;
private boolean isFirstText = true;
private boolean isFirstAudio = true;
public ConversationHandler(AudioPlayer audioPlayer, ToolRegistry toolRegistry) {
this.audioPlayer = audioPlayer;
this.toolRegistry = toolRegistry;
}
public AtomicBoolean getShouldStop() {
return shouldStop;
}
@Override
public void onOpen() {
System.out.println("Connection established");
}
@Override
public void onClose(int code, String reason) {
System.out.println("Connection closed");
shouldStop.set(true);
}
@Override
public void onEvent(JsonObject message) {
String type = message.get("type").getAsString();
switch (type) {
case "session.created":
handleSessionCreated(message);
break;
case "conversation.item.input_audio_transcription.completed":
handleTranscriptionCompleted(message);
break;
case "response.audio_transcript.delta":
case "response.text.delta":
handleTextDelta(message);
break;
case "response.audio.delta":
handleAudioDelta(message);
break;
case "input_audio_buffer.speech_started":
handleSpeechStarted();
break;
case "input_audio_buffer.speech_stopped":
handleSpeechStopped();
break;
case "response.function_call_arguments.done":
handleFunctionCall(message);
break;
case "response.done":
handleResponseDone();
break;
default:
break;
}
}
private void handleSessionCreated(JsonObject message) {
String sessionId = message.get("session").getAsJsonObject().get("id").getAsString();
System.out.println("start session: " + sessionId);
}
private void handleTranscriptionCompleted(JsonObject message) {
System.out.println("question: " + message.get("transcript").getAsString());
}
private void handleTextDelta(JsonObject message) {
if (isFirstText) {
isFirstText = false;
System.out.println("first text latency from vad end: " +
(System.currentTimeMillis() - lastPackageTime) + " ms");
}
String text = message.get("delta").getAsString();
responseTextRef.get().append(text);
}
private void handleAudioDelta(JsonObject message) {
if (isFirstAudio) {
isFirstAudio = false;
System.out.println("first audio latency from vad end: " +
(System.currentTimeMillis() - lastPackageTime) + " ms");
}
System.out.println("audio interval: " + (System.currentTimeMillis() - lastPackageTime) + " ms");
lastPackageTime = System.currentTimeMillis();
audioPlayer.play(message.get("delta").getAsString());
}
private void handleSpeechStarted() {
System.out.println("======VAD Speech Start======");
}
private void handleSpeechStopped() {
System.out.println("======VAD Speech End======");
lastPackageTime = System.currentTimeMillis();
isFirstText = true;
isFirstAudio = true;
}
private void handleFunctionCall(JsonObject message) {
System.out.println("======TOOL CALL======");
String callId = message.get("call_id").getAsString();
toolRegistry.addPendingToolCall(callId, message);
}
private void handleResponseDone() {
System.out.println("======RESPONSE DONE======");
System.out.println("all response text: " + responseTextRef.get());
responseTextRef.set(new StringBuilder());
}
/**
* Check if there are pending tool calls
*/
public boolean hasPendingToolCalls() {
return toolRegistry.hasPendingToolCalls();
}
/**
* Process all pending tool calls
*/
public void processPendingToolCalls(OmniRealtimeConversation conversation) {
toolRegistry.processPendingToolCalls(conversation);
}
}
}WebSocket(Python)
import asyncio
import json
import base64
import os
import pyaudio
import websockets
# ==================== Constant Definitions ====================
API_KEY = os.getenv("DASHSCOPE_API_KEY")
# To access the Beijing region, replace with:
# wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime
URL = "wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime"
MODEL = "qwen3.5-omni-plus-realtime"
VOICE = "Ethan"
# ==================== Tool Definitions ====================
def get_current_weather(location):
"""Query the weather in a specific city"""
return f"The weather in {location} today is changing from haze to sunny, with a temperature of 4/-4°C and a light breeze."
def get_flight_price(src, dst):
"""Query flight ticket prices"""
return f"The flight ticket price from {src} to {dst} is 200-300 USD."
def get_train_price(src, dst):
"""Query train ticket prices"""
return f"The train ticket price from {src} to {dst} is 100-200 CNY."
# Mapping of tool names to functions
TOOL_FUNCTIONS = {
"get_current_weather": get_current_weather,
"get_flight_price": get_flight_price,
"get_train_price": get_train_price,
}
TOOLS = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang.",
}
},
"required": ["location"],
},
},
},
{
"type": "function",
"function": {
"name": "get_flight_price",
"description": "Useful for when you want to query flight ticket prices.",
"parameters": {
"type": "object",
"properties": {
"src": {
"type": "string",
"description": "The departure city of the flight, such as Beijing or Hangzhou.",
},
"dst": {
"type": "string",
"description": "The arrival city of the flight, such as Beijing or Hangzhou.",
},
},
"required": ["src", "dst"],
},
},
},
{
"type": "function",
"function": {
"name": "get_train_price",
"description": "Useful for when you want to query train ticket prices.",
"parameters": {
"type": "object",
"properties": {
"src": {
"type": "string",
"description": "The departure city of the train, such as Beijing or Hangzhou.",
},
"dst": {
"type": "string",
"description": "The arrival city of the train, such as Beijing or Hangzhou.",
},
},
"required": ["src", "dst"],
},
},
},
]
# ==================== Tool Call Handling ====================
def handle_tool_call(name, arguments_str):
"""
Handles a tool call request
Args:
name: Tool function name
arguments_str: JSON formatted input parameter string
Returns:
Tool execution result string
"""
try:
arguments = json.loads(arguments_str)
print(f'[Tool Call] Start processing: name={name}, args={arguments}')
func = TOOL_FUNCTIONS.get(name)
if func is None:
result = f"Client did not find the tool: {name}"
print(f'[Tool Call] Error: {result}')
return result
result = func(**arguments)
print(f'[Tool Call] Completed: {result}')
return result
except Exception as e:
error_msg = f"Tool call failed: {str(e)}"
print(f'[Tool Call] Exception: {error_msg}')
return error_msg
# ==================== Main Program ====================
async def main():
"""Main function: Establishes a WebSocket connection and conducts a voice conversation"""
pya = pyaudio.PyAudio()
speaker = pya.open(format=pyaudio.paInt16, channels=1, rate=24000, output=True)
# Establish WebSocket connection
headers = {
"Authorization": f"bearer {API_KEY}",
"X-DashScope-OmniRealtime": "true",
}
async with websockets.connect(
f"{URL}?model={MODEL}", additional_headers=headers,
) as ws:
await ws.recv()
# Configure session parameters
await ws.send(json.dumps({
"type": "session.update",
"session": {
"modalities": ["text", "audio"],
"voice": VOICE,
"input_audio_format": "pcm16",
"output_audio_format": "pcm16",
"instructions": "You are a personal assistant named Xiaoyun",
"turn_detection": {"type": "server_vad"},
"input_audio_transcription": {"model": "qwen3-asr-flash-realtime"},
"tools": TOOLS,
},
}))
await ws.recv()
# Audio capture coroutine
async def send_audio():
mic = pya.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True)
try:
while True:
data = mic.read(3200, exception_on_overflow=False)
await ws.send(json.dumps({
"type": "input_audio_buffer.append",
"audio": base64.b64encode(data).decode(),
}))
await asyncio.sleep(0.01)
except asyncio.CancelledError:
mic.close()
pending = {}
all_response_text = ""
send_task = asyncio.create_task(send_audio())
print("Tool calling is enabled. Speak into the microphone (Ctrl+C to exit)...")
# Event handling loop
async for raw in ws:
msg = json.loads(raw)
t = msg["type"]
# Session created
if t == "session.created":
print(f"Session started: {msg['session']['id']}")
# Play audio
elif t == "response.audio.delta":
speaker.write(base64.b64decode(msg["delta"]))
# Incremental text response
elif t in ("response.audio_transcript.delta", "response.text.delta"):
all_response_text += msg.get("delta", "")
# User speech-to-text
elif t == "conversation.item.input_audio_transcription.completed":
print(f"[User] {msg['transcript']}")
# VAD detected speech start
elif t == "input_audio_buffer.speech_started":
print("====== VAD detected speech start ======")
# VAD detected speech end
elif t == "input_audio_buffer.speech_stopped":
print("====== VAD detected speech end ======")
# Received tool call request
elif t == "response.function_call_arguments.done":
print("====== Received tool call request ======")
pending[msg["call_id"]] = {
"name": msg["name"],
"arguments": msg["arguments"],
}
# Response completed
elif t == "response.done":
if pending:
# Run pending tool calls
for cid, info in pending.items():
result = handle_tool_call(info["name"], info["arguments"])
# Send tool execution result
await ws.send(json.dumps({
"type": "conversation.item.create",
"item": {
"type": "function_call_output",
"call_id": cid,
"output": result,
},
}))
pending.clear()
# Trigger the server to continue generating a response
await ws.send(json.dumps({
"type": "response.create",
"response": {"modalities": ["text", "audio"]},
}))
print("====== Tool call processing completed ======")
else:
# Normal response completed, print the full response
if all_response_text:
print(f"[Model] {all_response_text}")
all_response_text = ""
send_task.cancel()
speaker.close()
pya.terminate()
asyncio.run(main())
Tool calling for deep thinking models
Deep thinking models perform inference before outputting tool call information, which improves the interpretability and reliability of decisions.
Thinking process
The model analyzes the user's intent, identifies the required tools, verifies the legality of parameters, and plans the calling strategy step by step.
Tool calling
The model outputs one or more function call requests in a structured format.
Parallel tool calling is supported.
The following shows an example of a tool call using a streaming deep thinking model.
For more information about text generation thinking models, see Deep thinking. For more information about multimodal thinking models, see Image and video understanding and Non-real-time (Qwen-Omni).
tool_choiceparameter only supports being set to"auto"(default value, which means the model autonomously selects the tool) or"none"(forces the model not to select a tool).
OpenAI compatible
Python
Example code
import os
from openai import OpenAI
# Initialize the OpenAI client and configure the Alibaba Cloud DashScope service
client = OpenAI(
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
# If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
api_key=os.getenv("DASHSCOPE_API_KEY"), # Read the API key from the environment variable
base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)
# Define the list of available tools
tools = [
# Tool 1: Get the current time
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Useful for when you want to know the current time.",
"parameters": {} # No parameters required
}
},
# Tool 2: Get the weather in a specific city
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"] # Required parameter
}
}
}
]
messages = [{"role": "user", "content": input("Please enter your question:")}]
# Example message for a multimodal model
# messages = [{
# "role": "user",
# "content": [
# {"type": "image_url","image_url": {"url": "https://img.alicdn.com/imgextra/i4/O1CN014CJhzi20NOzo7atOC_!!6000000006837-2-tps-2048-1365.png"}},
# {"type": "text", "text": "Based on the location in the image, what is the current weather there?"}]
# }]
completion = client.chat.completions.create(
# This example uses qwen3.6-plus. You can replace it with other deep thinking models.
model="qwen3.6-plus",
messages=messages,
extra_body={
# Enable deep thinking. This parameter is invalid for qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, and QwQ models.
"enable_thinking": True
},
tools=tools,
parallel_tool_calls=True,
stream=True,
# Uncomment to get token consumption information
# stream_options={
# "include_usage": True
# }
)
reasoning_content = "" # Define the complete thinking process
answer_content = "" # Define the complete response
tool_info = [] # Store tool call information
is_answering = False # Determine if the thinking process has ended and the response has begun
print("="*20+"Thinking Process"+"="*20)
for chunk in completion:
if not chunk.choices:
# Process usage statistics information
print("\n"+"="*20+"Usage"+"="*20)
print(chunk.usage)
else:
delta = chunk.choices[0].delta
# Process the AI's thinking process (chain of thought)
if hasattr(delta, 'reasoning_content') and delta.reasoning_content is not None:
reasoning_content += delta.reasoning_content
print(delta.reasoning_content,end="",flush=True) # Output the thinking process in real time
# Process the final response content
else:
if not is_answering: # Print the title when entering the response phase for the first time
is_answering = True
print("\n"+"="*20+"Response Content"+"="*20)
if delta.content is not None:
answer_content += delta.content
print(delta.content,end="",flush=True) # Stream the response content
# Process tool call information (supports parallel tool calling)
if delta.tool_calls is not None:
for tool_call in delta.tool_calls:
index = tool_call.index # Tool call index, for parallel calling
# Dynamically expand the tool information storage list
while len(tool_info) <= index:
tool_info.append({})
# Collect the tool call ID (for subsequent function calls)
if tool_call.id:
tool_info[index]['id'] = tool_info[index].get('id', '') + tool_call.id
# Collect the function name (for subsequent routing to specific functions)
if tool_call.function and tool_call.function.name:
tool_info[index]['name'] = tool_info[index].get('name', '') + tool_call.function.name
# Collect function parameters (JSON string format, requires subsequent parsing)
if tool_call.function and tool_call.function.arguments:
tool_info[index]['arguments'] = tool_info[index].get('arguments', '') + tool_call.function.arguments
print(f"\n"+"="*19+"Tool Call Information"+"="*19)
if not tool_info:
print("No tool call")
else:
print(tool_info)Return result
Enter "Weather in the four municipalities" to obtain the following result:
====================Thinking Process====================
Okay, the user is asking about the weather in the four municipalities. First, I need to clarify which four municipalities they are. According to China's administrative divisions, the municipalities include Beijing, Shanghai, Tianjin, and Chongqing. So the user wants to know the weather conditions in these four cities.
Next, I need to check the available tools. The provided tools include the get_current_weather function, which takes a location parameter of type string. Each city needs to be queried separately because the function can only query one location at a time. Therefore, I need to call this function once for each municipality.
Then, I need to consider how to generate the correct tool calls. Each call should include the city name as a parameter. For example, the first call is for Beijing, the second for Shanghai, and so on. I need to make sure the parameter name is `location` and the value is the correct city name.
Also, the user probably wants the weather information for each city, so I need to ensure each function call is correct. This might require making four consecutive calls, one for each city. However, based on the tool usage rules, it might need to be handled in multiple steps, or multiple calls might be generated at once. But according to the example, it seems only one function is called at a time, so it might need to be done step by step.
Finally, I need to confirm if there are any other factors to consider, such as whether the parameters are correct, the city names are accurate, and whether I need to handle possible error situations, like a city not existing or the API being unavailable. But for now, the four municipalities are clear, so it should be fine.
====================Response Content====================
===================Tool Call Information===================
[{'id': 'call_767af2834c12488a8fe6e3', 'name': 'get_current_weather', 'arguments': '{"location": "Beijing"}'}, {'id': 'call_2cb05a349c89437a947ada', 'name': 'get_current_weather', 'arguments': '{"location": "Shanghai"}'}, {'id': 'call_988dd180b2ca4b0a864ea7', 'name': 'get_current_weather', 'arguments': '{"location": "Tianjin"}'}, {'id': 'call_4e98c57ea96a40dba26d12', 'name': 'get_current_weather', 'arguments': '{"location": "Chongqing"}'}]Node.js
Example code
import OpenAI from "openai";
import readline from 'node:readline/promises';
import { stdin as input, stdout as output } from 'node:process';
const openai = new OpenAI({
// API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
apiKey: process.env.DASHSCOPE_API_KEY,
baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
});
const tools = [
{
type: "function",
function: {
name: "get_current_time",
description: "Useful for when you want to know the current time.",
parameters: {}
}
},
{
type: "function",
function: {
name: "get_current_weather",
description: "Useful for when you want to query the weather in a specific city.",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
required: ["location"]
}
}
}
];
async function main() {
const rl = readline.createInterface({ input, output });
const question = await rl.question("Please enter your question:");
rl.close();
const messages = [{ role: "user", content: question }];
// Example message for a multimodal model
// const messages= [{
// role: "user",
// content: [{type: "image_url", image_url: {url: "https://img.alicdn.com/imgextra/i2/O1CN01FbTJon1ErXVGMRdsN_!!6000000000405-0-tps-1024-683.jpg"}},
// {type: "text", text: "What's the weather like in the location shown in the image?"}]
// }];
let reasoningContent = "";
let answerContent = "";
const toolInfo = [];
let isAnswering = false;
console.log("=".repeat(20) + "Thinking Process" + "=".repeat(20));
try {
const stream = await openai.chat.completions.create({
// This example uses qwen3.6-plus. You can replace it with other deep thinking models.
model: "qwen3.6-plus",
messages,
// Enable deep thinking. This parameter is invalid for qwen3-30b-a3b-thinking-2507, qwen3-235b-a22b-thinking-2507, and QwQ models.
enable_thinking: true,
tools,
stream: true,
parallel_tool_calls: true
});
for await (const chunk of stream) {
if (!chunk.choices?.length) {
console.log("\n" + "=".repeat(20) + "Usage" + "=".repeat(20));
console.log(chunk.usage);
continue;
}
const delta = chunk.choices[0]?.delta;
if (!delta) continue;
// Process the thinking process
if (delta.reasoning_content) {
reasoningContent += delta.reasoning_content;
process.stdout.write(delta.reasoning_content);
}
// Process the response content
else {
if (!isAnswering) {
isAnswering = true;
console.log("\n" + "=".repeat(20) + "Response Content" + "=".repeat(20));
}
if (delta.content) {
answerContent += delta.content;
process.stdout.write(delta.content);
}
// Process the tool call
if (delta.tool_calls) {
for (const toolCall of delta.tool_calls) {
const index = toolCall.index;
// Make sure the array is long enough
while (toolInfo.length <= index) {
toolInfo.push({});
}
// Update the tool ID
if (toolCall.id) {
toolInfo[index].id = (toolInfo[index].id || "") + toolCall.id;
}
// Update the function name
if (toolCall.function?.name) {
toolInfo[index].name = (toolInfo[index].name || "") + toolCall.function.name;
}
// Update the parameters
if (toolCall.function?.arguments) {
toolInfo[index].arguments = (toolInfo[index].arguments || "") + toolCall.function.arguments;
}
}
}
}
}
console.log("\n" + "=".repeat(19) + "Tool Call Information" + "=".repeat(19));
console.log(toolInfo.length ? toolInfo : "No tool call");
} catch (error) {
console.error("An error occurred:", error);
}
}
main(); Return result
Enter "Weather in the four municipalities" to obtain the following result:
Please enter your question:Weather in the four municipalities
====================Thinking Process====================
Okay, the user is asking about the weather in the four municipalities. First, I need to clarify which are the four municipalities of China. Beijing, Shanghai, Tianjin, and Chongqing, right? Next, I need to call the weather query function for each city.
But the user's question might require me to get the weather conditions for these four cities separately. I need to call the get_current_weather function once for each city, with their respective city names as parameters. I need to make sure the parameters are correct, such as the full names of the municipalities, like "Beijing", "Shanghai", "Tianjin", and "Chongqing".
Then, I need to call the weather API for these four cities in order. Each call requires a separate tool_call. The user probably wants the current weather information for each city, so I need to ensure each call is correct. I might need to pay attention to the correct spelling and names of each city to avoid errors. For example, Chongqing is sometimes abbreviated, so the full name should be used in the parameter.
Now, I need to generate four tool_calls, one for each municipality. I'll check if each parameter is correct and then arrange them in order. This way, the user will get the weather data for all four municipalities.
====================Response Content====================
===================Tool Call Information===================
[
{
id: 'call_21dc802e717f491298d1b2',
name: 'get_current_weather',
arguments: '{"location": "Beijing"}'
},
{
id: 'call_2cd3be1d2f694c4eafd4e5',
name: 'get_current_weather',
arguments: '{"location": "Shanghai"}'
},
{
id: 'call_48cf3f78e02940bd9085e4',
name: 'get_current_weather',
arguments: '{"location": "Tianjin"}'
},
{
id: 'call_e230a2b4c64f4e658d223e',
name: 'get_current_weather',
arguments: '{"location": "Chongqing"}'
}
]HTTP
Example code
curl
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.6-plus",
"messages": [
{
"role": "user",
"content": "What's the weather like in Hangzhou?"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Useful for when you want to know the current time.",
"parameters": {}
}
},
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location":{
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"]
}
}
}
],
"enable_thinking": true,
"stream": true
}'DashScope
Python
Example code
import dashscope
from dashscope import MultiModalConversation
# If you use a model in the China (Beijing) region, replace the base_http_api_url with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/"
tools = [
# Tool 1: Get the current time
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Useful for when you want to know the current time.",
"parameters": {} # Since getting the current time requires no input parameters, parameters is an empty dictionary
}
},
# Tool 2: Get the weather in a specific city
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
# A location is required to query the weather, so the parameter is set to location
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"]
}
}
}
]
# Define the question
messages = [{"role": "user", "content": [{"text": input("Please enter your question:")}]}]
# Example message for a multimodal model
# messages = [
# {
# "role": "user",
# "content": [
# {"image": "https://img.alicdn.com/imgextra/i2/O1CN01FbTJon1ErXVGMRdsN_!!6000000000405-0-tps-1024-683.jpg"},
# {"text": "What's the weather like in the location shown in the image?"}]
# }]
completion = MultiModalConversation.call(
# This example uses qwen3.6-plus. You can replace it with other deep thinking models.
model="qwen3.6-plus",
messages=messages,
enable_thinking=True,
tools=tools,
parallel_tool_calls=True,
stream=True,
incremental_output=True,
result_format="message"
)
reasoning_content = ""
answer_content = ""
tool_info = []
is_answering = False
print("="*20+"Thinking Process"+"="*20)
for chunk in completion:
if chunk.status_code == 200:
msg = chunk.output.choices[0].message
# Process the thinking process
if 'reasoning_content' in msg and msg.reasoning_content:
reasoning_content += msg.reasoning_content
print(msg.reasoning_content, end="", flush=True)
# Process the response content
if 'content' in msg and msg.content:
if not is_answering:
is_answering = True
print("\n"+"="*20+"Response Content"+"="*20)
answer_content += msg.content
print(msg.content, end="", flush=True)
# Process the tool call
if 'tool_calls' in msg and msg.tool_calls:
for tool_call in msg.tool_calls:
index = tool_call['index']
while len(tool_info) <= index:
tool_info.append({'id': '', 'name': '', 'arguments': ''}) # Initialize all fields
# Incrementally update the tool ID
if 'id' in tool_call:
tool_info[index]['id'] += tool_call.get('id', '')
# Incrementally update the function information
if 'function' in tool_call:
func = tool_call['function']
# Incrementally update the function name
if 'name' in func:
tool_info[index]['name'] += func.get('name', '')
# Incrementally update the parameters
if 'arguments' in func:
tool_info[index]['arguments'] += func.get('arguments', '')
print(f"\n"+"="*19+"Tool Call Information"+"="*19)
if not tool_info:
print("No tool call")
else:
print(tool_info)Return result
Enter "Weather in the four municipalities" to obtain the following result:
Please enter your question:Weather in the four municipalities
====================Thinking Process====================
Okay, the user is asking about the weather in the four municipalities. First, I need to confirm which are the four municipalities of China. Beijing, Shanghai, Tianjin, and Chongqing, right? Next, the user needs the weather conditions for each city, so I need to call the weather query function.
However, the problem is that the user did not specify the city names, just "the four municipalities". I might need to explicitly state the name of each municipality and then query them separately. For example, Beijing, Shanghai, Tianjin, and Chongqing. I need to make sure each city is correct.
Then, I need to check the available tools. The user has provided the get_current_weather function, which takes a location parameter. Therefore, I need to call this function for each municipality, passing the corresponding city name as the parameter. For example, the first call's location is Beijing, the second is Shanghai, the third is Tianjin, and the fourth is Chongqing.
However, I might need to be careful. For a municipality like Chongqing, sometimes a more specific district is needed, but the user might only want the city-level weather. So using the municipality name directly should be fine. Next, I need to generate four separate function calls, one for each municipality. This way, the user will get the weather conditions for all four cities.
Finally, I need to make sure the parameters for each call are correct and that none are missed. This will ensure the user's question is fully answered.
===================Tool Call Information===================
[{'id': 'call_2f774ed97b0e4b24ab10ec', 'name': 'get_current_weather', 'arguments': '{"location": "Beijing"}'}, {'id': 'call_dc3b05b88baa48c58bc33a', 'name': 'get_current_weather', 'arguments': '{"location": "Shanghai"}}'}, {'id': 'call_249b2de2f73340cdb46cbc', 'name': 'get_current_weather', 'arguments': '{"location": "Tianjin"}'}, {'id': 'call_833333634fda49d1b39e87', 'name': 'get_current_weather', 'arguments': '{"location": "Chongqing"}}'}]Java
Example code
// dashscope SDK version >= 2.19.4
import java.util.Arrays;
import com.alibaba.dashscope.exception.UploadFileException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.utils.Constants;
import com.alibaba.dashscope.utils.JsonUtils;
import com.alibaba.dashscope.tools.ToolFunction;
import com.alibaba.dashscope.tools.FunctionDefinition;
import io.reactivex.Flowable;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.lang.System;
import com.github.victools.jsonschema.generator.Option;
import com.github.victools.jsonschema.generator.OptionPreset;
import com.github.victools.jsonschema.generator.SchemaGenerator;
import com.github.victools.jsonschema.generator.SchemaGeneratorConfig;
import com.github.victools.jsonschema.generator.SchemaGeneratorConfigBuilder;
import com.github.victools.jsonschema.generator.SchemaVersion;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Collections;
public class Main {
private static final Logger logger = LoggerFactory.getLogger(Main.class);
private static ObjectNode jsonSchemaWeather;
private static ObjectNode jsonSchemaTime;
static {Constants.baseHttpApiUrl="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1";}
static class TimeTool {
public String call() {
LocalDateTime now = LocalDateTime.now();
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
return "Current time: " + now.format(formatter) + ".";
}
}
static class WeatherTool {
private String location;
public WeatherTool(String location) {
this.location = location;
}
public String call() {
return location + " is sunny today";
}
}
static {
SchemaGeneratorConfigBuilder configBuilder = new SchemaGeneratorConfigBuilder(
SchemaVersion.DRAFT_2020_12, OptionPreset.PLAIN_JSON);
SchemaGeneratorConfig config = configBuilder
.with(Option.EXTRA_OPEN_API_FORMAT_VALUES)
.without(Option.FLATTENED_ENUMS_FROM_TOSTRING)
.build();
SchemaGenerator generator = new SchemaGenerator(config);
jsonSchemaWeather = generator.generateSchema(WeatherTool.class);
jsonSchemaTime = generator.generateSchema(TimeTool.class);
}
private static void handleGenerationResult(GenerationResult message) {
System.out.println(JsonUtils.toJson(message));
}
// Create a tool calling method for the text generation model
public static void streamCallWithMessage(Generation gen, Message userMsg)
throws NoApiKeyException, ApiException, InputRequiredException {
GenerationParam param = buildGenerationParam(userMsg);
Flowable<GenerationResult> result = gen.streamCall(param);
result.blockingForEach(message -> handleGenerationResult(message));
}
// Build text generation model parameters that support tool calling
private static GenerationParam buildGenerationParam(Message userMsg) {
FunctionDefinition fdWeather = buildFunctionDefinition(
"get_current_weather", "Get the weather for a specific area", jsonSchemaWeather);
FunctionDefinition fdTime = buildFunctionDefinition(
"get_current_time", "Get the current time", jsonSchemaTime);
return GenerationParam.builder()
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3.6-plus")
.enableThinking(true)
.messages(Arrays.asList(userMsg))
.resultFormat(GenerationParam.ResultFormat.MESSAGE)
.incrementalOutput(true)
.tools(Arrays.asList(
ToolFunction.builder().function(fdWeather).build(),
ToolFunction.builder().function(fdTime).build()))
.build();
}
// Create a tool calling method for the multimodal model
public static void streamCallWithMultiModalMessage(MultiModalConversation conv, MultiModalMessage userMsg)
throws NoApiKeyException, ApiException, UploadFileException {
MultiModalConversationParam param = buildMultiModalConversationParam(userMsg);
Flowable<MultiModalConversationResult> result = conv.streamCall(param);
result.blockingForEach(message -> System.out.println(JsonUtils.toJson(message)));
}
// Build multimodal model parameters that support tool calling
private static MultiModalConversationParam buildMultiModalConversationParam(MultiModalMessage userMsg) {
FunctionDefinition fdWeather = buildFunctionDefinition(
"get_current_weather", "Get the weather for a specific area", jsonSchemaWeather);
FunctionDefinition fdTime = buildFunctionDefinition(
"get_current_time", "Get the current time", jsonSchemaTime);
return MultiModalConversationParam.builder()
.apiKey(System.getenv("DASHSCOPE_API_KEY"))
.model("qwen3-vl-plus") // Use the multimodal model Qwen3-VL
.enableThinking(true)
.messages(Arrays.asList(userMsg))
.tools(Arrays.asList( // Configure the tool list
ToolFunction.builder().function(fdWeather).build(),
ToolFunction.builder().function(fdTime).build()))
.build();
}
private static FunctionDefinition buildFunctionDefinition(
String name, String description, ObjectNode schema) {
return FunctionDefinition.builder()
.name(name)
.description(description)
.parameters(JsonUtils.parseString(schema.toString()).getAsJsonObject())
.build();
}
public static void main(String[] args) {
try {
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMsg = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("text", "Please tell me the weather in Hangzhou"))).build();
try {
streamCallWithMultiModalMessage(conv, userMsg);
} catch (UploadFileException e) {
throw new RuntimeException(e);
}
// Uncomment the following lines when using a text generation model for tool calling
// Generation gen = new Generation();
// Message userMessage = Message.builder()
// .role(Role.USER.getValue())
// .content("Please tell me the weather in Hangzhou")
// .build();
// try {
// streamCallWithMessage(gen, userMessage);
// } catch (InputRequiredException e) {
// throw new RuntimeException(e);
// }
} catch (ApiException | NoApiKeyException e) {
logger.error("An exception occurred: {}", e.getMessage());
}
System.exit(0);
}
}
Return result
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":6,"total_tokens":244},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"Okay, the user asked me"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":12,"total_tokens":250},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"to tell them the weather in Hangzhou. I"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":16,"total_tokens":254},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"need to first determine if there are"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":22,"total_tokens":260},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"any relevant tools available. Looking at the provided"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":28,"total_tokens":266},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"tools, I see there is a get_current"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":34,"total_tokens":272},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"_weather function with a location parameter"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":38,"total_tokens":276},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":". So I should call"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":43,"total_tokens":281},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"this function with the parameter"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":48,"total_tokens":286},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"set to Hangzhou. No other"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":52,"total_tokens":290},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"tools are needed because"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":56,"total_tokens":294},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"the user only"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":60,"total_tokens":298},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"asked about the weather. Next, I will construct"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":64,"total_tokens":302},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"the tool_call, filling"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":68,"total_tokens":306},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"in the name and parameters"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":73,"total_tokens":311},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":". I need to make sure the parameter is a"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":78,"total_tokens":316},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"JSON object and location is a"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":82,"total_tokens":320},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"string. After checking for"}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":88,"total_tokens":326},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"errors, I will return it."}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":106,"total_tokens":344},"output":{"choices":[{"finish_reason":"null","message":{"role":"assistant","content":"","reasoning_content":"","tool_calls":[{"type":"function","id":"call_ecc41296dccc47baa01567","function":{"name":"get_current_weather","arguments":"{\"location\": \"Hangzhou"}}]}}]}}
{"requestId":"4edb81cd-4647-9d5d-88f9-a4f30bc6d8dd","usage":{"input_tokens":238,"output_tokens":108,"total_tokens":346},"output":{"choices":[{"finish_reason":"tool_calls","message":{"role":"assistant","content":"","reasoning_content":"","tool_calls":[{"type":"function","id":"","function":{"arguments":"\"}"}}]}}]}}HTTP
Example code
curl
# ======= Important =======
# If you use a text-only generation model, replace the url with https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation
# API keys vary by region. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
# The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
# === Delete this comment before running ===
curl -X POST "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-H "X-DashScope-SSE: enable" \
-d '{
"model": "qwen3.6-plus",
"input":{
"messages":[
{
"role": "user",
"content": [{"text": "Weather in Hangzhou"}]
}
]
},
"parameters": {
"enable_thinking": true,
"incremental_output": true,
"result_format": "message",
"tools": [{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Useful for when you want to know the current time.",
"parameters": {}
}
},{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for when you want to query the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "A city or district, such as Beijing, Hangzhou, or Yuhang."
}
},
"required": ["location"]
}
}
}]
}
}'Going live
Test tool calling accuracy
Establish an evaluation system:
Build a test dataset that reflects real-world business scenarios and define clear evaluation metrics, such as tool selection accuracy, parameter extraction accuracy, and the end-to-end success rate.
Optimize prompts
Based on problems identified during testing, such as incorrect tool selections or parameters, you can optimize the system prompts, tool descriptions, and parameter descriptions.
Upgrade the model
If prompt tuning fails to improve performance, upgrading to a more powerful model version, such as `
qwen3.6-plus`, is the most direct and effective method.
Dynamically control the number of tools
When an application integrates dozens or even hundreds of tools, providing all of them to the model can cause the following problems:
Performance degradation: The model's difficulty in selecting the correct tool from a large set of tools increases dramatically.
Cost and latency: Many tool descriptions will consume a large amount of input tokens, leading to increased costs and slower responses.
Solution: Add a tool routing/retrieval layer before calling the model. This layer filters the tool library based on the user's query to provide a small, relevant subset of tools to the model.
Mainstream methods for implementing tool routing:
Semantic retrieval
Convert tool descriptions (
description) into vectors using an embedding model and store them in a vector database. When a user submits a query, you can perform a vector similarity search on the query vector to recall the top K most relevant tools.Hybrid retrieval
This method combines the fuzzy match of semantic retrieval with the exact match of traditional keywords or metadata tags. To do this, add `
tags` or `keywords` fields to the tools. During retrieval, performing both vector search and keyword filtering can significantly improve recall accuracy, especially for high-frequency or specific scenarios.Lightweight LLM router
For more complex routing logic, you can use a smaller, faster, and less expensive model, such as Qwen-Flash, as a router model. This model's task is to output a list of relevant tool names based on the user's query.
Practical advice
Keep the candidate set concise: Regardless of the method used, we recommend providing no more than 20 tools to the main model. This provides an optimal balance between the model's cognitive load, cost, latency, and accuracy.
Layered filtering strategy: You can build a funnel-style routing strategy. For example, you can first use low-cost keyword or rule matching to filter out clearly irrelevant tools. Then, you can perform semantic retrieval on the remaining tools to improve efficiency and quality.
Tool security principles
When granting tool execution capabilities to an LLM, security is the primary consideration. The core principles are least privilege and human confirmation.
Principle of least privilege: The toolset provided to the model must strictly adhere to the principle of least privilege. By default, tools should be read-only, such as tools for querying weather or searching documents. Avoid providing any "write" permissions that involve state changes or resource operations.
Isolate dangerous tools: Do not provide dangerous tools directly to the LLM, such as tools for executing arbitrary code (
code interpreter), operating the file system (fs.delete), performing database delete or update operations (db.drop_table), or handling financial transactions (payment.transfer).Human involvement: A manual review and confirmation process is required for all high-privilege or irreversible operations. The model can generate an operation request, but the final "execute" button must be clicked by a human user. For example, the model can prepare an email, but the user must confirm the send operation.
User experience optimization
The function calling process involves multiple steps, and a problem at any step can negatively affect the user experience.
Handle tool run failures
Tool execution failures are common. You can adopt the following strategies:
Maximum retries: Set a reasonable retry limit, such as 3, to avoid long user waits or system resource waste due to continuous failures.
Provide fallback responses: If retries are exhausted or an unresolvable error is encountered, return a clear and friendly prompt to the user, such as: "Sorry, I can't find the relevant information at the moment. The service might be busy. Please try again later."
Cope with processing latency
High latency can reduce user satisfaction. You can implement optimizations on both the frontend and backend.
Set a timeout: Set an independent and reasonable timeout for each step of the function calling process. If a timeout occurs, the operation should be immediately interrupted and feedback should be provided to the user.
Provide instant feedback: When a function call starts, we recommend displaying a prompt on the interface, such as "Querying the weather for you..." or "Searching for relevant information...". This gives the user real-time feedback on the progress.
Billing
In addition to the tokens in the `messages` array, tool descriptions are also billed as input tokens.
Pass tool information through a System Message
Error codes
If a model call fails and returns an error message, see Error codes to resolve the issue.