Tokenmaxxing Dilemma: Are There Immediate Solutions for Improvement?

By WangChen

A little over 4 months ago, Uber began rolling out Claude Code across its engineering team of about 5,000, and the tool quickly sparked a wave of enthusiasm among engineers. However, four months later, usage far exceeded the expectations of the company's financial model, burning through the entire year's AI programming budget. This case triggered a chain of discussions in the community, focus on two areas: leading practices for controlling token consumption, and how to quantify business value. This highlights that while encouraging developers to use AI for efficiency gains and to accelerate product iteration and innovation, establishing transparent cost control mechanisms will become an important subject for major enterprises.

I. Where Are Tokens Being Burned?

This is a complex question that relies on both rigorous statistics and a large number of compliant samples. We attempt to make an initial investigation into the distribution of Token consumption from the following three data sources: the academic paper arXiv:2604.22750,engineering field measurements from Vantage.sh, Jake Nesler, and consumption tracking of a 100 million token sample on Reddit.

arXiv:2604.22750

"How Do AI Agents Spend Your Money?", published in April this year, conducts a highly systematic study on Agent Token consumption. The core finding is that Agentic programming tasks consume about 1,000 times more tokens than ordinary chats, with Input Tokens rather than Output Tokens driving the vast majority of the costs. The paper also reports a counterintuitive conclusion: there is almost no correlation between token consumption and task accuracy (r < 0.15), meaning spending more money does not buy better results.

Vantage.sh

Vantage.sh is a cloud cost management platform company that helps enterprises understand and optimize expenses in multi-cloud environments, providing features such as cost reports, budget alerts, and resource optimization recommendations. In April this year, they published "The Hidden Cost Driver in Agentic Coding: It's Not the Per-Token Price" [3], providing more precise data on Token consumption: the Input/Output Token ratio of Agentic programming sessions is about 25:1 (approx. 1 million Input vs. 40,000 Output), with Input accounting for about 85% of the total cost. Although Agentic sessions only account for 1/10 of the total request volume, they are about 200 times more expensive than non-Agentic usage; they also identified that sessions are divided into three phases (exploration -> implementation -> test iteration), with the highest density of Token consumption occurring during the exploration phase of the first 10 turns.

Reddit 100M Token Tracking

This is individual tracking data from a heavy Claude Code user posted in r/ClaudeAI (1,289 requests, 100M token sample size) [4], where 99.4% of AI expenses came from Input Tokens. As independent, unfiltered, first-hand usage data, it confirms from a practical level the directional judgments of the academic papers and FinOps reports: the cost problem of Agents is fundamentally about reading too much, rather than writing too much.

Synthesizing the above sources, the consensus of the three reports is: the vast majority of Agent Token consumption is Input (comprehension/retrieval), not Output (generation/output). It should be noted that the above three reports do not involve multimodal output; if multimodality is considered, the proportion of output will increase significantly. However, none of them further subcategorized Input Tokens. In actual practice, knowing that the Agent reads a lot does not help us locate optimization directions; we need a more granular classification framework.

The table below is a qualitative attribution we created based on personal experience using Agents combined with the industry consensus on common classifications of Agent Input behaviors, divided into File Retrieval, Relationship Tracking, Context Management, Generation Loop, and Tool Interaction. The high/medium/low rankings reflect the relative contribution of various actions to Token consumption in daily practice, rather than precise measurements from a specific experiment.

C1 (Blind File Reading) and C2 (Dependency Exploration) are the two heaviest sources of consumption, together constituting the bulk of Input Tokens. This corroborates with findings from Jake Nesler's "80% wasted finding things", Vantage.sh's "re-reading files", and arXiv's "input tokens dominate". While C4 (Generation Iteration) and C5 (Tool Trial and Error) also consume Tokens, they are far less exaggerated in daily experience than the first two.

Among these five major cost sources, C2 (Dependency Exploration) is the most structural type and is best suited for intervention through architectural means.

Why say so? Although Blind File Reading (C1) is equally heavy in consumption, it is essentially a search problem that can be alleviated with better indexing and more precise file positioning. Context Reconstruction (C3) can be optimized with larger context caching and memory capabilities. But Dependency Exploration is different; the Agent tracks relationships between entities: A calls B, B is deployed on C, what level is C's SLA, what changed recently on C. If this relationship is not structured in advance, the Agent has to infer it on the spot in plain text every time, and if the inference fails, it starts over. Next, using Dependency Exploration as a keyword, let's talk about our practices.

II. Three Paradigms of Dependency Exploration

If we only look at how Agents acquire dependency/relationship information, we have gone through roughly three stages over the past three years. Each generation resolved the core pain points of the previous one while exposing new bottlenecks. In the evolutionary comparison below, we connect these using examples from operations scenarios.

Scenario Description: An alert for a service named shopping-user goes off, but the root cause is actually downstream in shopping-cart (one user sleep + NPE, another spinning for 500ms). Traditional root cause analysis targets shopping-user itself. Only when the dependency chain "shopping-user calls shopping-cart, shopping-cart calls shopping-item" is explicitly known can the Agent follow the topology down to locate the true point of failure.

This is a classic scenario where dependency exploration determines the success or failure of diagnosis. Using it to show how the three generations of paradigms handle dependency information is the most intuitive approach.

Stuffing is limited by capacity, and RAG opened up capacity using retrieval; RAG in turn got stuck on semantic fragmentation—when we get a piece of relevant text, we still don't know if it has a direct dependency relationship with the entity in the current issue; Ontology, however, elevates entities and entity relationships to first-class citizens, moving the Agent from inferring relationships in text to querying relationships on a graph.

III. How Does Ontology Reduce Token Consumption in Dependency Exploration?

We provide two sets of empirical evidence from code scenarios and operations scenarios to explore the specific gap between "with Ontology" and "without Ontology".

Code Knowledge Graph: 10× Token Compression

In March 2026, Martin Vogel et al. published Codebase-Memory (arXiv:2603.27277) , using Tree-Sitter to parse code into a persistent knowledge graph of functions/classes/call chains stored in SQLite, exposed to LLMs via 14 MCP tools. The experimental results, validated on hub detection and caller ranking tasks across 31 code repositories, showed: with the graph, consumption was about 1,000 tokens, representing a 10x compression in token usage and a 2.1x reduction in tool calls; whereas without the graph, consumption reached up to 10,000 tokens. The experiment covered 31 code repositories and 66 programming languages, using Tree-Sitter to parse code into persistent knowledge graphs stored in SQLite, exposed to the LLM via 14 MCP tools.

UModel: Intelligent Anomaly Detection with a Single Line of Code

An ontology in code scenarios is just the beginning. In enterprise operations scenarios, because domain knowledge is completely outside the scope of model pre-training, the effect of an ontology is even more significant.

The Alibaba Cloud Observability team, in their UModel PaaS API architecture practice, achieved intelligent anomaly detection with just one line of code and provided a straightforward comparison. Below is the path developers must take when manually integrating observability data. If an Agent were to retrace this same path at runtime, each step would incur corresponding Token costs.

However, if an ontology like UModel is provided, the Token overhead for three types of tasks can be avoided.

Multi-round Metadata Queries: No longer repeatedly querying the relationships between EntitySet, MetricSet, and Storage. Previously, each step required returning to the model for a round of inference; now, it is resolved all at once at the ontology layer.
On-the-spot Field Mapping Inference: No need for the Agent to ponder whether service_id and acs_arms_service_id refer to the same thing—the mapping relationship is already built into DataLink.
Query Syntax Correction Loops: No need to retry after misspelling PromQL/SPL. In Object mode, the Agent only needs to express its intent (get_metric()), and the underlying syntax is completely transparent to it.

IV. After Models Become Stronger, Do We Still Need Ontology?

At this point, an unavoidable question surfaces: if the context window continues to grow and inference costs continue to fall, will the value of Ontology be consumed by "brute force"?

Our judgment is: it depends on the domain.

For code scenarios, the answer is indeed somewhat ambiguous. Code is the main battlefield for model pre-training. As model capabilities continue to enhance, its ability to understand a new codebase is indeed growing stronger. In this domain, the value of ontology may gradually be internalized by the model.

But once you switch to enterprise-level domains like operations (Ops), the situation is completely different. We believe this is a classic case where the value of Ontology is clearest and least likely to be consumed by the model. There are three reasons:

Entities and entity relationships within an enterprise are never in the pre-training corpus of general models. An enterprise's instance scale, service topology, CMDB configurations, alerting rules, change windows, and on-call rosters change every day.
The essence of operations is relationship reasoning. An alert is on A, but the root cause is on B, with three layers of dependencies in between. If this dependency chain is not provided to the Agent in a structured way, it has to search through logs and documents chunk by chunk and guess step by step. A stronger model can speed up inference, but it cannot eliminate the problem of "no information".
As a high-stakes scenario, operations has extremely low tolerance for inaccuracies. An autonomous operations misjudgment of a root cause could restart a healthy service, or miss a real failure and lead to a P0 incident. In this kind of scenario, "83% accuracy" is unacceptable; you need ontology to push quality loss close to zero.

Palantir is the most convincing validation case for the business value of "Enterprise Ontology." Their core product AIP (AI Platform) is built on an ontology layer that maps out an enterprise's personnel, assets, processes, and data into a unified graph for the AI Agent to perform reasoning on. Palantir's market value of over $300 billion is essentially the market pricing this "Enterprise Ontology" capability—it's not just about saving tokens, but more importantly, making AI output reliable, auditable, and actionable.

V. Alibaba Cloud's Ontology Practice: STAROps

With that said, we want to anchor the discussion on a specific engineering practice already running online. Alibaba Cloud recently released STAROps, an all-domain intelligent operations platform. It organically combines LLM technology, UModel, RCA, and RCA benchmark, representing a comprehensive implementation of Ontology in the AIOps direction in China. Among them, UModel is the ontology layer, responsible for defining entities and enterprise relationships in the operations world. RCA (Root Cause Analysis) is the practice methodology of the Alibaba Cloud Observability Team for distributed system troubleshooting, and the RCA benchmark provides a standardized root cause analysis evaluation dataset and evaluation protocol system, jointly initiated with the China Academy of Information and Communications Technology (CAICT), Xpeng Motors, and the Institute of Software, Chinese Academy of Sciences.

We can get a sense of the role ontology plays in fault diagnosis for operations scenarios through the following example of a frontend application error.

Entity Awareness: Determine the entity to be diagnosed, frontend.
Automated Multi-Source Query Orchestration: The system automatically generates an investigation topology map showing the call relationships of frontend and its upstream/downstream services, queries ERROR level logs, and extracts the specific exception stack.
Deep Investigation: The system automatically orchestrates multi-step observability data queries, including Pod-level storage/filesystem metrics, K8s Events queries, resource change record queries, downstream service topology acquisition, etc.

Output Fault Entity Symptoms:
Symptom: The frontend service has an extremely high proportion of HTTP 500 status codes within the alert window.
Evidence: Trace difference analysis shows a diff value up to 0.958 for http.status_code=500; the key error stack shows TypeError: Cannot read properties of undefined (reading 'streetAddress'), occurring when rendering the /cart/checkout/[orderId] page, which indicates a data structure the frontend depends on (expected from the ad service) is empty.
Causal Chain Confirmation: Ad service rolling update -> some Pods unreachable (ECONNREFUSED) -> Frontend calls Ad service failed -> Frontend code did not catch this dependency failure or handle the returned empty data -> throws TypeError -> User request returns 500 error.
Data Integrity Self-Assessment: Identified missing data: None. Core link Trace, Pod metrics, K8s events are all covered; some Pod topology queries (such as obs_117, obs_124) were not executed, but do not affect the root cause determination.

Handling Recommendations:
Urgent Fix: Check the status of all Pods in the ad service to ensure they are all Ready. If there are abnormal Pods, immediately restart or roll back the ad service deployment.
Code Optimization: Fix the frontend service code, add null checks for the data returned by the ad service or add try-catch blocks to prevent single-point dependency failures from causing the entire page to crash (achieving graceful degradation).
Configuration Check: Review the ad service's rolling update strategy (maxSurge, maxUnavailable) and health probe (readinessProbe) configurations to ensure service availability during updates.

Community

Tokenmaxxing Dilemma: Are There Immediate Solutions for Improvement?

I. Where Are Tokens Being Burned?

arXiv:2604.22750

Vantage.sh

Reddit 100M Token Tracking

II. Three Paradigms of Dependency Exploration

III. How Does Ontology Reduce Token Consumption in Dependency Exploration?

Code Knowledge Graph: 10× Token Compression

UModel: Intelligent Anomaly Detection with a Single Line of Code

IV. After Models Become Stronger, Do We Still Need Ontology?

V. Alibaba Cloud's Ontology Practice: STAROps

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Qwen

Alibaba Cloud Model Studio

Alibaba Cloud for Generative AI

AI Acceleration Solution