This article is included in the open-source project Alibaba Cloud AI Agent Handbook. For a better reading experience, visit: https://aliyun.github.io/ai-agent-handbook/run/chapter-09.html
New models have been arriving almost every few weeks lately, priced at a fraction of the cost of flagship models while already coming close on general benchmark scores. Epoch AI's report The plunging price of thought, published this September, estimated that since 2023 the cost of reaching the same capability level has fallen by about 47% per quarter on average. There will only be more inexpensive models that are good enough. The real challenges have become two things: which steps in your own work you can hand over to them, and which Agent should handle a given task.
HiRoute is an open-source, locally running intelligent routing engine for Agents. You keep using familiar Agents such as Codex, Claude Code, Qoder, Pi, OpenCode, Hermes, and DeepSeek Harness, while HiRoute decides which model should handle each step. This update focuses on the two challenges above:
A common approach is to divide the work by scenario: use a powerful model for coding and a cheaper one for organizing information. Different scenarios do require different capabilities, but in actual conversations we have seen that difficulty changes from step to step: complex scenarios contain plenty of simple work, and difficult problems can suddenly arise even in simple scenarios.
We ran a comparative experiment on technology selection: first research six candidate technologies and verify official materials, then write a decision memo on the key solutions. Assigning information organization to a cheaper model and key judgments to the primary model delivered the same quality of key judgments as using the primary model throughout, at only about one-twelfth of the total cost. Conversely, using the cheaper model throughout produced reasonably good information organization, but major errors in two of the three key judgments.
Figure 1: Total task cost comparison under the same acceptance criteria
So HiRoute divides the responsibilities as follows: the scenario determines the candidate models and acceptance criteria; the choice of model for each step is made when that step is reached. Switching occurs only when the user sends a new message or the context is compacted or rebuilt. Tool calls within the same turn always use the same model, and only the first request after a switch misses the cache.
For details, see the real-world case study and the cost analysis of model switching and caching.
Below is a screen recording of a real conversation. On the left is DeepSeek Harness, and on the right is HiRoute. The routing plan uses DeepSeek V4.1 Flash as the new model to try and GPT-6 Astra as the primary model. The user simply makes requests in the Agent as usual, without manually switching models at any point.
Figure 2: The conversation timeline records which model was used in each turn and why it was chosen
Benchmarks and leaderboards are useful for initial screening, but they cannot answer "Will it work well in my work?" SWE-Bench Pro, released by Scale AI last September, is one example: leading models that solve more than 70% of the issues on SWE-bench Verified solve only about 23% of the public problems in SWE-Bench Pro. On startups' private codebases, the best result falls further to 17.8%.
The problems themselves are harder, so the first drop cannot be attributed entirely to the models not having seen them before. But the public and private problems are constructed using the same methodology, making the latter drop more revealing. To a model, your codebase is precisely this kind of unseen private code.
New models arrive too quickly to build a dedicated evaluation set for every release. Our approach is to evaluate them as we use them: start by assigning sufficiently simple steps to the new model, inspect the result after every step, let the primary model take over when problems arise, and gradually give the new model more responsibility once there is enough evidence. This update completes that workflow:
Figure 3: Trust-level settings for a new model
The most direct evidence for assessing "How well did this step go?" is in the user's next message: explicitly rejecting the previous step's result is evidence that it fell short. Simply assigning further work, or saying nothing, does not indicate failure. This update also aligns scoring more closely with real feedback: it recognizes explicit approval, distinguishes partial approval from full acceptance, and credits partially usable progress.
After a period of use, the model performance report summarizes stage scores, the proportion of work handled by the new model, and cost estimates. Individual scores can be inaccurate, so trust-level adjustments require enough evidence:
Figure 4: Each rule's actual value and requirement are shown side by side, and your records are replayed to show the effect of the adjustment. Costs are API-equivalent estimates.
The bar for raising the trust level is higher than for lowering it, intentionally: a few occasional successes are not enough to justify handing over more work; when problems are clear enough, responsibility should be withdrawn more promptly. Recommendations are only recommendations and do not change routing until published. Keeping the current trust level is also a normal choice. After enabling "HiRoute Management" for your everyday Agent under "Agent", you can also ask it to read the model performance report and trust-level adjustment recommendations directly. For periodic reviews, use the Agent's own scheduled tasks.
The judgments above must be made once every turn. The most direct approach is to give the task, history, and routing policy to the decision model and ask, "Which model should be used for this step?" We tried it, but the results were not stable: the model has to weigh task difficulty, past performance, whether problems have been resolved, and what the policy specifies all at once. A misjudgment on any one of these leads to the wrong conclusion.
System One breaks this down into three independent questions, each answered separately within a single request:
Figure 5: System One's three questions per turn
The decision model only answers the questions; thresholds, model-group selection, and execution are all handled by HiRoute. Adjusting the trust level changes the threshold, not the questions themselves. Starting with this update, the third question is part of the default assessment: if the previous step fell short and the issue will persist, the primary model takes over; once the issue is resolved or no longer relevant, the safeguard is lifted.
We documented this mechanism in a paper and released datasets in both Chinese and English, raw model responses, and tools for offline reproduction. In the experiment, both methods read the same task context and complete routing policy: one directly selects a model group, while the other answers the three questions and leaves HiRoute to combine the answers into a decision.
Figure 6: Final model-group selection accuracy on the reviewed subset
One point in these results deserves a closer look: when selecting groups directly, the two decision models made errors in exactly opposite directions. In the mathematics scenario, Jev 1.14 was too bold, correct in only 6 of the 24 records requiring the safeguard; OpenAI Decisions was too conservative, correct in only 2 of the 16 records where the safeguard should have been lifted. After switching to the three questions, Jev 1.14 selected correctly in all 24 safeguard cases and remained correct in all 16 cases requiring the safeguard to be lifted. OpenAI Decisions improved to 14 correct decisions to lift the safeguard, while its correct safeguard decisions fell from 24 to 22.
The same decomposition substantially reduced both biases, despite their opposite directions.
Cost and latency are not an issue either. Based on Jev's actual billing, a thousand three-question decisions cost about USD 0.084–0.086. Excluding connection setup, the measured median decision latency is between half a second and one second, making reassessment on every turn affordable.
It is important to note that these results measure "whether the correct model group was selected according to the stated policy", not task completion rates or actual monetary savings. This comparison uses a subset reviewed after inspecting the outputs. All models and both methods use the same exclusion list, and the size of the improvement varies by decision model and scenario. The OpenAI results come from the experiment's research adapter and do not represent the performance of the built-in OpenAI connection.
For the full data, exclusion list, and limitations, see the paper.
Decision-model connections are now more flexible as well:
Model routing decides "Which model should handle this step?", while task routing decides "Which execution Agent should handle this work?" Each routing plan specifies the types of work it suits and can be bound to an execution Agent: Codex CLI, Claude Code, Qoder CLI, Pi, DeepSeek Harness, or OpenCode. During execution, each turn of this Agent still follows the plan's model routing, choosing between the economy group and the primary group.
In this update:
On October 31, the DeepSeek Harness team published the long-form article "Exploring Super-agent Design: Toward More Autonomous Agent Systems" and open-sourced the research prototype dsh-superagent-prototype.
It divides the system into two layers: a persistent Router handles external communication, global coordination, and task assignment, and need not execute any tasks itself. Workers focus on executing individual tasks in isolated contexts and return their results to the Router, which consolidates the reply. This keeps the Router responsive to new messages without being blocked by a long-running task. The article starts from a practical observation: one person can oversee only about three to five Agent conversations at once, making human attention the bottleneck.
A similar architecture can be built using HiRoute's existing capabilities, without building a separate routing mechanism specifically for a Super Agent:
Figure 7: The Super Agent architecture of Hermes and HiRoute
hiroute worker exec: Pass the task, working directory, and acceptance requirements to hiroute worker exec. Add --no-wait to return as soon as the task is accepted, allowing the Router to resume the conversation immediately. Multiple tasks can run in parallel.notify_on_complete. Once the task finishes, the process exits, waking Hermes to retrieve the result and report it to you in the conversation.# Delegate information gathering to the plan bound to Pi; return immediately after submission
hiroute worker exec --plan <PLAN_ID> --cwd /work/mq-research --no-wait \
-- "Gather the official documentation and release notes for three message queues and organize them into a comparison table"
# Check progress at any time; retrieve the result after the task finishes
hiroute worker status --run <RUN_ID>
hiroute worker result --run <RUN_ID>The Router only understands intent, assigns tasks, and consolidates results. How expensive a model each execution Agent uses is then determined turn by turn by the plan it belongs to. For the complete setup process with Hermes, see "Build a DeepSeek-Style Super Agent with Hermes and HiRoute".
hiroute console open generates a login link.curl -fsSL https://hiroute.ai/install.sh | sh -s -- --console-mode loopback --console-port 5838
hiroute service start --output json
hiroute console openWhen connecting a Codex or Claude subscription, you can also enter the corresponding OpenAI or Anthropic API Key. Select the models once and save once to import the models from both sources at the same time.
This puts the API credits bundled with subscriptions to good use: eligible ChatGPT Pro / Business and Claude Max / Ultra / Team subscriptions can receive API credits each billing cycle, and unused credits do not roll over (see the official explanations from OpenAI and Anthropic).
Enter the Key for the organization receiving the credits here, then create a route for the subscription model in its model details. HiRoute will discover the model with the same name in the API, which can be added as a candidate too. This combines the same-named models from the subscription and API into a single model: the Agent uses just one model name, and HiRoute uses the subscription first, automatically falling back to the API when the subscription is temporarily unavailable.
You can set a one-time budget, replenish it on a calendar cycle, or let multiple Keys share one budget. HiRoute reserves an amount before forwarding a request and settles it after completion. Amounts are estimated from traffic forwarded by HiRoute, not from the provider's account balance. This also works well for managing credits: set a monthly replenishing budget for that Key equal to the credit amount. Once it is exhausted, the Key will be skipped.
Each API connection can use a direct connection, inherit the background service's proxy environment settings, or use a manually entered HTTP(S) proxy and bypass rules. Decision models sharing the connection follow the same settings.
Long conversations no longer repeatedly save full request snapshots and protocol logs. Only the content needed to view history, scores, and which model handled the work is retained. By default, local records use at most 20 GB; this limit can be adjusted or disabled, while the seven-day retention period still applies.
Official model information, such as context window sizes, and new-model recommendations on the home page are released separately through HiRoute's model catalog, without waiting for an application update.
HiRoute runs on your computer or server and is licensed under Apache-2.0.
HiRoute is not perfect, and decision models can make mistakes too. We encourage you to try it in your own scenarios and to tell us where its judgments go wrong.
Using Nacos 3.3.0 ARD to Enable Task-Based Tool and Skill Discovery for Agents
814 posts | 61 followers
FollowAlibaba Cloud Native Community - October 22, 2026
Alibaba Cloud Native Community - October 22, 2026
Alibaba Developer - December 25, 2018
Alibaba Cloud Community - July 29, 2026
Alibaba Clouder - December 26, 2016
Alibaba Cloud Community - April 2, 2026
814 posts | 61 followers
Follow
Token Plan
Build more, spend less. One plan, every modality.
Learn More
QwenWork
QwenWork is dedicated to helping employees strengthen their professional competitiveness in the AI era and to enabling enterprises to improve organizational effectiveness.
Learn More
DeepSeek
DeepSeek-V4.1-Flash is the lightweight flagship of DeepSeek's new architecture family, packing 552B total MoE parameters to deliver flagship-surpassing intelligence across key benchmarks.
Learn More
AgentCore
AgentCore unifies agent construction, governance, and execution capabilities so enterprises can deploy diverse agents with confidence.
Learn MoreMore Posts by Alibaba Cloud Native Community