By Caichen
This article is included in the Alibaba Cloud AI Agent Handbook. Project URL: https://github.com/aliyun/ai-agent-handbook. Scan the QR code at the end of this article to join the Handbook reading group and learn more about the technical details.
Collaboration in software R&D is one of the most important areas for AI adoption today. Alibaba Cloud's Cloud Communications core product engineering team began using AI to improve efficiency early on. In 2024, we rolled out code completion, conversational programming, and generative IDE tools for everyday development. In 2025, we standardized our workflows, codifying standards and domain knowledge into constitutions and Skill resources to give AI clear rules to follow. In 2026, we have fully transitioned to an Agent-based operating model and are moving toward end-to-end delivery.
When discussing how AI can improve software R&D efficiency, the industry's first thought is often AI Coding, which has indeed made programming significantly more efficient. In practice, however, greater coding efficiency does not necessarily translate into greater overall delivery efficiency.
Looking back at the traditional requirement delivery process, from the emergence of a requirement to its release into production, coding is only one step. An AI Native delivery process also involves considerable work both before and after AI Coding: beforehand, requirements must be translated into language that AI can understand; afterward, AI-generated code must undergo quality checks and functional testing. If people still handle these tasks, their efficiency cannot match that of AI, and the manual stages become bottlenecks constraining delivery efficiency across the entire process. The upper limit of system throughput is determined by the stage with the greatest concentration of human intervention.
Therefore, the best way to improve end-to-end delivery efficiency is to have AI perform the work throughout the entire process once a requirement has been submitted: an engineering model with Agent-driven operations from end to end. Implementing this model requires answers to three questions: first, how should multiple Agents collaborate? Second, how should Agents' operating boundaries and permissions be defined? Third, how can this new production model be safely implemented within existing systems?
On the first question, our team's early attempts at multi-Agent collaboration encountered a challenge common across the industry: missing context.
Consider the handoff of a technical design. When an engineering Agent writes the design, it focuses on the overall technical architecture and middleware choices. The downstream testing Agent, however, is more interested in the boundaries of the system being worked on and how to diagnose problems. As a result, each Agent sees its own output as rich and detailed, yet upstream-downstream collaboration frequently suffers from the same issue: the upstream Agent has not produced the information the downstream Agent needs. The root cause is that early Agent collaboration still followed an artisanal production model, relying primarily on human experience and informal conventions. This cannot meet enterprise production requirements for stability and consistency.
Our solution draws on factory production lines: define standards and procedures for the engineering workflow, specify what is required of Agents, and clearly define the inputs and outputs of every stage. Organizing Agent collaboration as a pipeline brings it into standardized production, with unified collaboration contracts and explicit collaboration paths. This aligns semantics, transfers context in full, and makes handoffs plug-and-play.
The second question, how to define Agents' operating boundaries and permissions, is not just technical. It is also organizational: how should AI be managed when it is introduced into the system as a new factor of production?
There are broadly two approaches to using Agents in the industry today. One treats AI as an independent entity: a digital employee or digital human. The other treats AI as an extension of a person in the digital world: a digital twin. We chose digital twins. Treating AI as an independent entity requires a separate set of operating procedures and behavioral rules, making it the more complex engineering choice. A digital twin, by contrast, can draw on established organizational management practices and operate under the permissions of the person it represents.
First, people within an organization already have different permissions. A digital twin has the same permissions as its human counterpart, naturally defining what it can and cannot do. Second, when a digital twin commits code or generates artifacts using that person's permissions, the system treats the actions as having been performed by that person, so any problems can be traced back to them. This accountability obliges the owner to take responsibility for their digital twin's behavior and continuously iterate on and improve it. Only by clearly defining Agents' operating boundaries and permissions through digital twins can Agents be incorporated into the engineering system as a factor of production that the organization can trust.
For the third question, how to implement this organizational system, our guiding principles are safe production and an evolutionary rollout.
Stability comes first. Whether a requirement is implemented by a person or by AI, it must not cause production issues or disrupt customers' use of the service. To ensure this, we defined two gates.
• The first is the Harness engineering gate. Before a digital twin can begin work, the application's Harness engineering must be in place so that AI operations have clear rules to follow. The digital twin's capability level (L) is defined based on the maturity of that Harness engineering.
• The second is the requirement risk-level gate. Requirements are assigned risk levels (R): simple requirements, such as operations tools and data dashboards, carry lower risk, while complex requirements, such as cross-application coordination and changes to core processing paths, carry higher risk.
Before starting work, a digital twin assesses the requirement's risk level and accepts only requirements within its capabilities. If a requirement exceeds its capabilities (R > L), it is handed over to the appropriate engineering lead for manual execution. When the system was first launched, Harness engineering and the knowledge base were still in their early stages, so only simple requirements could be accepted. As Harness engineering and the knowledge base are continuously refined and enriched through ongoing work, the digital twin's capability boundaries gradually expand, enabling it to take on increasingly complex requirements.
This is our approach to a gradual rollout: prioritize stability and progress from simple requirements to complex ones.
Building on these ideas, we developed an engineering model that combines cloud-based digital twins with automated delivery through requirement pipelines, enabling continuous 7×24-hour operation.
In this new engineering model, a digital twin serves as both the entry point for requirements and their router. People in product, business, and technical roles can submit requirements to an engineering digital twin for execution. On receiving a requirement, the digital twin first assesses its risk and determines whether it has the capability to take it on. If it does, it completes the subsequent engineering work in a self-contained closed loop. If the requirement is beyond its capabilities, it routes the work to the appropriate engineering lead.
This operating system has now been rolled out across four key business lines within Cloud Communications core products: domestic SMS, intelligent voice, intelligent scheduling, and international SMS. With current model capabilities, a single digital twin can handle three to four requirements in parallel, greatly increasing engineering concurrency. Results from actual deployments show that, among the requirements digital twins assessed as within their capabilities, approximately two-thirds achieved genuine end-to-end delivery. The end-to-end delivery rate for simple requirements was 66.7%, and the overall deliverables met expectations.
Returning to the first question: how should multiple Agents collaborate?
Our answer is to define multi-Agent collaboration through a pipeline. We design the pipeline by dividing all engineering work in the AI Native delivery model into nine main stages: PRD production, gating, readiness, design, coding, self-checks, testing and quality checks, release and acceptance, and Loop feedback. These nine stages form the backbone of the pipeline.
Three principles govern this backbone. The first is the contract-chain principle: the inputs and outputs of every stage, along with the content and format of its artifacts, are defined in advance. Each stage's deliverables constitute its contract. The second is the minimum-standard principle: if a preceding stage's deliverables contain problems, they must be sent back for rework. Nonconforming outputs must not proceed, and problems must never be passed downstream to compromise the overall quality of the work. The third is pluggable Skill resources: an explicitly cataloged set of capabilities supports the execution of each stage and connects the stages into a complete process. Skill resources can be replaced, supplemented, and continuously upgraded, enabling greater adaptability to different business scenarios.
Of these three principles, the first two ensure that overall execution quality does not slip, while the third enables the operating model to extend to different business scenarios.
Once the pipeline has been defined, how do digital twins collaborate across stages to achieve end-to-end delivery?
Consider the quality checks and testing that follow AI Coding. First, we establish separate workspaces for a digital twin's sub-Agents so that they can work independently. When sub-Agents share a workspace, they can develop context-induced hallucinations, and one sub-Agent's conclusions can influence another Agent's judgments. Separate workspaces isolate these hallucinations. Each role-specific digital twin reviews only the aspects relevant to its role within its own workspace: the product digital twin verifies that the business requirements have been implemented in full; the engineering digital twin checks for code defects; the security digital twin reviews deliverables for security compliance; and the testing digital twin automatically writes test cases and executes them in the test environment.
Once all digital twins have completed their checks, a consolidated report is produced and passed to the technical lead Agent. This primary Agent checks each declarative checkpoint and issues a pass or fail decision. Any problems trigger automatic reruns and fixes; exceptions and failed checks are always returned to the same stage for rework. This combination of multi-role collaboration and automated testing directly removes the human throughput bottleneck in testing and quality checks.
The final question is: how can this system be rolled out gradually?
We rely primarily on post-delivery quality checks and the Loop mechanism. Every delivery completed by a digital twin is followed by an audit; this post-delivery audit is a standard production stage. The audit focuses on whether the process was followed correctly, whether the deliverables met expectations, and whether AI execution metrics across the overall workflow, including cache hit rates and Token consumption, were within normal ranges.
If an audit identifies good practices or key information, these are automatically captured in the knowledge base for direct reuse on similar requirements in the future. If it identifies opportunities for improvement, the corresponding recommendations are sent directly to the digital twin's owner. A person decides whether to implement each proposed improvement and optimizes the full set of engineering assets accordingly.
Loop follows a cycle of detection → root-cause attribution → correction → verification. Under this mechanism, each requirement delivered is accompanied by an audit, a round of corrections and iteration on engineering assets, and an update to the knowledge base. The digital twin's capabilities gradually improve, progressing from L1 toward L2 and L3. Its operating boundaries expand step by step, naturally increasing the range of requirements it can deliver. Post-delivery audits and the Loop mechanism enable the gradual rollout of the entire end-to-end Agent collaboration model for software R&D, making the system stronger with use.
A real requirement, with sensitive information removed, illustrates the complete end-to-end delivery journey.
A business stakeholder sends a requirement ticket through DingTalk to the engineering lead's digital twin. On receiving it, the digital twin first evaluates whether it can accept the requirement. If it can, Agents representing multiple roles hold a roundtable discussion to clarify and align on the requirement, then return the agreed outcome to the requester. Once the requester confirms it, the corresponding requirement document is generated. The Harness engineering assets are then checked, and subsequent work begins only after these assets pass the checks.
The first step in the subsequent workflow is technical design and internal review. Designs that fail review are automatically returned for correction. Once a design passes, it is submitted to the technical lead for approval. After approval, the process automatically proceeds to AI Coding and deployment to the test environment. Next, the different roles perform quality checks on the artifacts, with automatic fixes applied if problems are found. Once the checks are complete, automated functional testing begins. Only after every test case has passed are the deliverables submitted to the platform for quality checks. When all engineering stages are complete, a post-delivery audit is performed: knowledge gained during the audit is stored in the knowledge base, and the corresponding improvement proposals are submitted to the relevant owners for decisions.
After passing the task acceptance gate, the entire process runs unattended, with all nine stages self-driven. Every step leaves a complete audit trail of who performed it, the basis for the action, and the outcome, allowing retrospective inspection.
This engineering system is already in use within the delivery framework of the Cloud Communications core product engineering team. Our future plans focus on three areas.
First, continue expanding the operating boundaries. As engineering assets and experience accumulate, we will gradually extend from simple requirements to complex ones, allowing the range of deliverable requirements to grow naturally. Second, control costs. The baseline requirement is that this delivery model must not cost more than the existing manual model. We reduce delivery costs through tier-based model routing, context governance, and batch execution during off-peak nighttime hours. Third, adapt to new organizational models. We have already experimented with operating units made up of one engineer and several digital twins, directly responsible for a domain of work, so that team throughput is no longer constrained by headcount. The current digital-twin engineering model remains compatible with organizations centered on people. If organizational structures evolve further in the AI era, we will actively adapt to them as well.
Looking back on our journey to improve efficiency with AI, the team has taken its share of wrong turns. Our experience is that neither of two extremes is advisable.
The first is abandoning AI altogether out of fear that hallucinations might pose risks to production stability, and continuing to rely on traditional programming methods. The second is rushing for results: skipping the organization of knowledge and the establishment of Harness engineering, and expecting a sufficiently powerful model alone to guarantee delivery quality.
In practice, we have found that even the most advanced models struggle to understand our domain knowledge all at once. That knowledge is often nonstandard and internal, with many terms and pieces of jargon used only within the team. Some of it is unstructured, held in people's heads and passed along verbally to guide their work. Much of it is counterintuitive: one defect may coexist with another bug in such a way that the two negatives cancel out, allowing the system to run stably. Fixing just one of them, however, may cause a failure. AI cannot understand all of this in a single pass.
We therefore chose to define processes and standards for AI. We first constrain its operating scope to a relatively small area, then have people guide AI's learning and iterative improvement within that engineering framework. This gradual approach improves software R&D efficiency and enables AI-driven engineering workflows to evolve. Our experience also led us to a conclusion: the long-term differentiator in AI-enabled software R&D efficiency is not how capable the model is, but the depth of the assets built around it across the entire engineering delivery system.
AgentTeams Dashboard: Unified Management of Teams, Projects, and Assets
Agent as a Service: The Service Architecture Powering Personal Agents and Enterprise Agents
793 posts | 60 followers
FollowAlibaba Cloud Community - January 11, 2023
Alibaba Clouder - September 30, 2019
ApsaraDB - September 21, 2022
Alibaba Clouder - December 25, 2019
yanmin - June 25, 2019
Alibaba Cloud Native Community - November 6, 2025
793 posts | 60 followers
Follow
Qoder
From Assisted to Autonomous — Adapting to Every Programming Need
Learn More
DevOps Solution
Accelerate software development and delivery by integrating DevOps with the cloud
Learn More
AgentCore
AgentCore unifies agent construction, governance, and execution capabilities so enterprises can deploy diverse agents with confidence.
Learn More
Digital Marketing Solution
Transform your business into a customer-centric brand while keeping marketing campaigns cost effective.
Learn MoreMore Posts by Alibaba Cloud Native Community
