Summary: Across releases 1.30 through 1.32, we continuously improved performance, merged 39 changes, and built a benchmark covering 40 scenarios to evaluate every iteration.
Hardware keeps getting faster, but software does not necessarily feel any smoother.
Andy's and Bill's Law describes this phenomenon: performance gains from faster hardware are quickly consumed by software's growing resource demands. "Andy" and "Bill" refer to Intel's Andy Grove and Microsoft's Bill Gates. The quip captures an experience familiar to many software users.
In the AI era, code is written faster and teams iterate on features more frequently. At the same time, users are running multiple Agent tasks in parallel, asking AI to operate browsers and computers, and working through longer conversations. Software is being asked to do more, raising the bar for performance.
How much can Agents help? We recently completed a round of performance optimization for Qoder IDE, merging 39 related merge requests (MRs). Many of the performance issues were even identified automatically by Qoder Agent, which also implemented and validated the fixes.
We want to share what we learned from this effort, along with the performance benchmark we built afterward.
In practice, we found that although Agents occasionally introduce performance issues, they are also highly effective at identifying those issues by analyzing code, logs, and tests together.
The 39 MRs merged in this effort fall into four categories:
The first three categories account for 35 MRs. Many of these optimizations simply focused on making the software do less work that could have been avoided in the first place.
One issue involving long conversations is a typical example.
When you scroll upward through message history, the interface loads earlier messages and inserts them at the beginning of the list. Previously, we generated each message's key—the identifier React uses to recognize individual messages—from its position in the “loaded list.”
When new messages were inserted at the beginning, the positions of the existing messages that followed also changed. As a result, the same message's key would change, potentially triggering unnecessary component remounts and causing lag while browsing message history.
To fix this, we replaced that position with the message's fixed sequence number within the current request. Combined with the conversation request identifier, this gives the same message a stable key:
// Before the fix: use the message's position in the loaded list
key = `${requestId}-message-${i}`
// After the fix: use the message's fixed sequence number within the current request
key = `${requestId}-message-${indexInRequest}`This allows React to continue recognizing and reusing existing components when earlier messages are loaded, reducing the overhead of unnecessary remounts. A stable key helps React handle insertions and moves within a list correctly.
The change itself affected only one line of code, but the problem could only be fully understood in the context of loading message history. For this kind of issue, having an Agent investigate the code alongside runtime logs and then verify the effect of the change is an effective approach.
After fixing a performance issue, another question remains: could a later release reintroduce it?
We see benchmark scores for large language models every day. If model capabilities can be evaluated using a set of tasks, IDE performance can likewise be continuously evaluated using a fixed set of scenarios.
We therefore built an internal performance benchmark for Qoder IDE: a repeatable suite of performance tests.
It took less than one day to go from proposing the idea to completing the first version and obtaining results. Expanding the scenarios, adding support for macOS and Windows, and completing preliminary validation took half a month in total. During that period, the work required approximately 15% of one engineer's time.
The benchmark currently covers 40 scenarios, including terminal commands, streaming output in tool-intensive conversations, loading long conversations, and editing large files. Each scenario is scored on a 100-point scale according to its own criteria. The scenarios are then weighted by frequency of use and importance to calculate a composite score.
From 1.28.0 to 1.32.0, the internal composite scores for the two platforms changed as follows:
Versions 1.28.0 and 1.29.0 predated this optimization effort and had similar composite scores. After performance optimizations were introduced in the following three releases, the scores increased in each successive release.
These figures are intended to track changes between versions within the same internal benchmark. They cannot be directly translated into a claim about how much faster Qoder IDE became, nor can they be used to rank the performance of the two operating systems.
Looking at individual scenarios makes it easier to see where the optimizations had an effect. On macOS, for example, from 1.28.0 to 1.32.0:
These scores also remind us that more work remains. Although streaming output in tool-intensive conversations has continued to improve, there is still room for improvement under the current scoring criteria. The full-text search score, meanwhile, changed only slightly, from 62.9 to 63.5. An increase in the composite score is no substitute for examining every individual scenario.
The following two charts show scores for all 40 scenarios across five recent releases. Darker colors indicate higher scores. First are the 13 highest-weighted scenarios, which together account for 60% of the composite score:
The remaining 27 scenarios account for a combined 40%:
By running this benchmark continuously, we have already identified two performance regressions introduced during iterative development and fixed them before release. A performance regression occurs when subsequent changes cause previously achieved performance to deteriorate.
For us, this is a tangible benefit of building the benchmark: it allows the team to spot changes that warrant further investigation before a release goes out.
Of course, 40 scenarios cannot fully cover a product as complex as Qoder IDE. We plan to expand the suite to 100 or more scenarios and adjust the weights to reduce the disproportionate influence of individual scenarios on the overall score. The performance of important scenarios also needs to be monitored individually.
More comprehensive test coverage increases the chance of identifying problems early. New features, new usage patterns, and cases not covered by the existing benchmark will still need to be added continuously.
This brings us back to Andy's and Bill's Law.
If Agents are used only to accelerate feature development, the amount of code in the software will grow, while existing resource waste may continue to accumulate. We can also ask them to inspect existing code, identify unnecessary computation and memory usage, and fix the issues affecting responsiveness one by one.
In this effort, building the benchmark itself also became work that an Agent could help with. The first version was up and running quickly; scenarios were then added incrementally so that the benefits of each optimization could be verified repeatedly in subsequent releases.
This effort alone is not enough to tell us whether Agents can break Andy's and Bill's Law. It does, however, point to a direction worth continued investment: involving Agents in performance investigation and continuous validation alongside the development of new features.
Beyond writing more code, it is equally worthwhile to invest time in making existing software run better.
Update Qoder IDE and try it out. If you still encounter specific cases of lag, you can submit feedback with one click from within the product to help us continue investigating and improving performance.
How TNG Digital's Data Chief Got 400+ People to Trust the Same Number
1,556 posts | 524 followers
FollowAlibaba Cloud Community - June 16, 2026
Alibaba Cloud Native Community - August 5, 2026
Alibaba Cloud Community - July 31, 2026
Alibaba Cloud Native Community - July 23, 2026
Alibaba Cloud Indonesia - March 11, 2026
CloudSecurity - July 24, 2026
1,556 posts | 524 followers
Follow
QwenWork
QwenWork is dedicated to helping employees strengthen their professional competitiveness in the AI era and to enabling enterprises to improve organizational effectiveness.
Learn More
Token Plan
Build more, spend less. One plan, every modality.
Learn More
Qwen
Full-range, open-source, multimodal, and multi-functional
Learn More
Wan3.0
Unleash Your Creativity Try Wan 3.0 now — longer narratives, greater control, endless creativity.
Learn MoreMore Posts by Alibaba Cloud Community