This article introduces the open-source release of Better Harness, a toolkit for evaluating and optimizing AI coding agent workflows based on actual task execution.

Learn More about Qoder

Explore Qoder – The All in One AI Coder

Recently, we built Better Harness into Qoder Desktop. Since its launch, many users have asked: will this feature be open source?

The answer is yes!

Today, we’re officially open-sourcing Better Harness. Find the project at: https://github.com/QoderAI/better-harness

Better Harness is an open-source toolkit for analyzing and continuously improving coding agent workflows. It connects the engineering practices, evaluation models, and runtime capabilities of Harness Engineering and Loop Engineering, with support for Claude Code, Codex, Qoder, and Cursor.

All four platforms share the same evaluation model, but they do not yet offer full parity in session analysis, evidence coverage, or output capabilities. The Qoder integration has been repeatedly validated in real development workflows and is currently the most complete reference implementation.

Host	Installation or Loading Method	Default Output
Claude Code	Add the repository as a marketplace, then install the plugin	HTML + Markdown
Codex	Install through Git Marketplace	HTML + Markdown
Qoder Desktop	Built in; no installation required	Canvas
Cursor Agent	Currently loaded from source	HTML / Markdown

Once you have installed or loaded it on your platform, run the following command in the project you want to analyze:

/better-harness

You can start it now and review the report as you continue reading. If a type of evidence is unavailable, Better Harness explicitly preserves that limitation in its results rather than substituting configuration counts or data from another platform.

Better Harness Focuses on What the Agent Actually Did in a Task

Suppose an agent changes a module, runs a test command, and declares the task complete. The real question is not whether the repository “has tests,” but whether that test is relevant to the change, covers the main risks, and produces results that justify handing over the work.

That is why Better Harness does not treat the presence of AGENTS.md, Rules, Skills, MCP, Hooks, Memory, tests, or CI as proof that they actually contributed to the task. Every proposed improvement—a Finding—must include traceable evidence, a concrete impact on the user, the smallest necessary fix scope, and a way to verify the fix.

Here is a snapshot of Better Harness analyzing its own project in Codex:

The report does not equate “missing Codex host tests” with “Codex has failed.” Instead, it preserves the distinction between tests that have not been run and evidence of failure. Scores only help locate issues; what matters is the conclusion, its impact, the smallest necessary fix scope, and how to verify the fix.

This is what separates Better Harness from a configuration checklist. A checklist tells you what a project has; Better Harness examines whether those capabilities actually helped the agent complete a task with trustworthy results.

And if the report’s judgments are to be inspected, revised, and verified again, open source has to mean more than a runnable entry point.

The Three Layers of Better Harness’s Open-Source Architecture

Simply publishing the prompt behind /better-harness on GitHub would certainly qualify as open source.

But for a coding agent, the outcome is never determined by a single line in a prompt. It depends on the entire way of working behind that prompt: what is worth checking, what counts as evidence, how judgments are formed, and how those judgments are continuously applied and refined in real projects.

That is why this release opens up three interconnected layers of Better Harness.

Layer One: Harness Engineering Best Practices

This layer answers a fundamental question: when examining sessions, CLI tools, observability, Rules, Skills, MCP, Memory, Hooks, and automation, what should we check—and which conclusions cannot be drawn merely from the presence of configuration?

This knowledge is organized by problem domain in references:

Domain	Question It Answers	Key Areas Examined
Session Evidence	How did the agent actually complete the task?	Sessions, task episodes, tool calls, retries after failures, usage, and evidence of outcomes
Project Harness	Does the project provide reliable paths for execution, verification, and delivery?	CLI tools, observability, design contracts, tests, Git Hooks, sensitive code, and recovery mechanisms
Agent Customize	Are agent assets discoverable, applicable, and actually effective?	Rules, Skills, MCP, Memory, Hooks, Custom Agents, and platform configuration
Loop Engineering	Which mechanism should handle proven recurring work over the long term?	Skill, Hook, Script, Automation, Rule, Custom Agent, MCP, and other mechanisms

Rather than loading an ever-growing master prompt on every run, Better Harness consults the relevant domain guidance for the issues it finds. A diagnostic problem leads to observability practices; a Skill issue leads to Skill Review. When a recurring workflow is identified, it determines whether a Skill, Hook, Script, or Automation should handle that work on an ongoing basis.

Layer Two: The Agent Work Loop Evaluation Model

This layer turns engineering practices into questions that can be checked individually and governs the relationship between evidence, scores, and conclusions. The current model and definitions of evidence states are published in Agent Work Loop.

Because the standards are still evolving, we avoided letting a single model subjectively define a “good harness.” For the first round of internal evaluation, we selected 30 real GitHub projects. Four types of models evaluated them independently using OpenAI’s Harness Engineering article, producing 120 standardized reports. We then compared results across models and applied human calibration to clarify evidence requirements, the limits of valid conclusions, and differences between project types, before reassessing every project under the updated criteria.

This loop—automated evaluation, automated synthesis, human calibration, and automated reruns—gave us our first reproducible Harness Engineering evaluation model, one that could be continuously refined.

The first version was still closer to a traditional software engineering maturity assessment, checking whether projects had documentation, tests, CI, and security mechanisms. But we soon realized that static assets cannot prove an agent has actually completed a task.

To track changes in both the model and project capabilities over time, we adopted a spec-driven approach to building Better Harness. As we accumulated more than 200 specs, the model gradually shifted from asking “What is in the repository?” to judging “What actually happened during the task?”

The unit of evaluation narrowed from a repository or session to a specific task. A session was no longer the object being evaluated, but simply a container for evidence. The model also converged on five dimensions: task understanding, controlled execution, change verification, reliable delivery, and retained learning. The existence of a file, a configuration count, events occurring close together in time, or even a successful command could no longer be treated as direct proof that a capability had actually been effective.

Ultimately, Agent Work Loop became more than a static scorecard: it became a reproducible system for making judgments about real tasks, with continuous calibration.

Layer Three: A Runnable Engineering Implementation

This layer brings the first two beyond documents and models, making them repeatable in real projects. Better Harness starts an analysis through a plugin or CLI—the JavaScript code in the project’s scripts directory—first fixing the task scope, then collecting three types of evidence separately:

Session Evidence reconstructs the agent’s behavior in real tasks;
Project Harness checks whether the project can be started, diagnosed, verified, and recovered;
Agent Customize examines configuration, routing, and evidence of use for Rules, Skills, MCP, Memory, and Hooks.

These three evidence streams remain independent during collection and analysis. Only at the end does the Lead combine them with the guidance in References and the Agent Work Loop evaluation model to reach an overall judgment. “The project has a capability” and “the agent actually used it in the task” are, and always will be, two different facts.

The final output is not just a score, but a set of Findings that specify the limits of the evidence, user impact, fix scope, and verification method. Once the report has been rendered and validated, the process can move on to remediation. If stable, recurring work is identified, Loop Engineering determines whether a Skill, Hook, Script, Automation, or another mechanism should handle it over the long term.

Completing a fix does not, by itself, mean the workflow has improved. The improvement loop is only complete when better results are observed again in subsequent tasks of the same kind.

Start with One Issue You Can Verify

For Qoder Users

Better Harness is built into Qoder Desktop. Update to the latest version, open the Quest window, and select Better Harness (Beta), or simply run /better-harness. It is also available directly in Qoder CLI and the JetBrains plugin on machines where Qoder Desktop has been installed.

The Open-Source Repository

Visit the Better Harness GitHub repository, follow the README to install it, and run/better-harnessfor the first time. For example, Claude Code users can run these commands in order:

/plugin marketplace add QoderAI/better-harness
/plugin install better-harness@better-harness

Then run:

/better-harness Analyze my project’s harness and generate an HTML report

On your first run, you do not need to build a complete engineering framework for agents or aim for a perfect score. A more practical starting point is one issue with clear evidence, a concrete impact, and a fix you can verify quickly.

It might be a check command the agent cannot discover, an error log that offers no next step for diagnosis, or an existing Skill that has never been included in task routing.

Fix one issue, rerun the checks, and observe whether subsequent tasks of a similar kind change. Harness Engineering is not a one-off configuration exercise. It is the ongoing work of making a project easier for agents to understand, execute, and verify.

We Know There Is More to Do

Better Harness has been run and calibrated repeatedly in Qoder’s real development workflows. Even so, the current model is primarily shaped by the project types and task scenarios we know best.

It may still have blind spots across different technology stacks, project scales, team constraints, and coding agents. More real-world evidence is needed to keep refining it.

If you would like to get involved, here are a few ways to contribute:

Contribute engineering practices: add guidance for evaluating a language, framework, or common workflow to references/. No coding required.
Broaden the evaluation: add an evidence-backed evaluation dimension or detector to models/ or scripts/, complete with fixtures and tests.
Extend host support: complete evidence collection and validation for another coding agent. The repository’s Roadmap already has a to-do list.
Share real-world examples: contribute a sanitized example of your team’s practices to case-studies/.

If you disagree with a Finding in a report, we warmly welcome an issue. To us, a counterexample from a real project is far more useful than a star—though a star is always appreciated, too.

Built at Qoder. Shared with the community. Helping every coding agent build on proven engineering practices.

Community

The Better Harness Evaluation Model Used by 100,000 Users Is Now Open Source

Learn More about Qoder

Explore Qoder – The All in One AI Coder

Better Harness Focuses on What the Agent Actually Did in a Task

The Three Layers of Better Harness’s Open-Source Architecture

Layer One: Harness Engineering Best Practices

Layer Two: The Agent Work Loop Evaluation Model

Layer Three: A Runnable Engineering Implementation

Start with One Issue You Can Verify

For Qoder Users

The Open-Source Repository

We Know There Is More to Do

Read previous post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Qoder

AgentCore

Managed Service for Prometheus