Enterprises are moving AI capabilities from lab-based proofs of concept to large-scale deployments in the production environment. This process is far more than "plugging in an API"—it demands a comprehensive overhaul of architectural philosophy, engineering practices, and organizational governance. This chapter first dissects the unique challenges that distinguish AI workloads from traditional systems, explores the core, recurring tensions in architectural design, and distills a set of design principles to guide AI architecture decisions.
Challenges of AI workloads
Non-deterministic outputs
Traditional software systems follow deterministic logic: call the same interface with the same parameters, and you will always get the same result. An AI workload fundamentally breaks this premise. Even with identical input, a large language model can produce different outputs on each inference run—varying in wording, structure, or even conclusions.
Consider an enterprise customer service scenario. When a user asks, "When will my order arrive?", a traditional system queries a database for the logistics status and returns a definite date. In contrast, an AI-powered customer service agent, after retrieving the same logistics information, might respond once with "Your order is expected to arrive tomorrow afternoon," and another time with "Based on the shipping information, your package will be delivered to your address before 14:00 on June 21." The two responses are semantically equivalent, but the phrasing is entirely different. This non-determinism renders traditional, exact-match-based automated testing methods ineffective and requires architects to introduce a semantic-level quality assessment framework.
When building conversational applications with a large language model, you can reduce randomness by setting the temperature parameter close to zero. However, this does not completely eliminate non-determinism. Factors like floating-point arithmetic precision within the model or different computation orders on various hardware can still cause subtle output drift. Therefore, the testing strategy for AI systems must shift from "verifying whether the output is correct" to "verifying whether the output is within an acceptable range."
To address this, architects should establish a semantic-level evaluation system. This system moves beyond traditional "does the output equal the expected value?" checks and instead builds a quality baseline using multi-dimensional scoring (relevance, accuracy, completeness, and safety) and continuously monitors for quality drift.
Subjective and contextual quality
The definition of "good" becomes unusually complex in AI systems. A summary of the same document needs to extract strategic insights for executives, preserve technical details for engineers, and highlight risk exposure for legal teams. There is no single "absolutely correct" summary—quality judgment depends entirely on the use case and audience expectations.
This subjectivity has profound implications at the product level. Defects in traditional software can be found before deployment through unit and integration testing. In contrast, "quality defects" in AI systems often surface only in specific business contexts. An answer from a medical consultation assistant might be medically accurate, but if it uses too much professional jargon for a patient to understand, it is "low-quality" in that scenario.
In engineering practice, this means quality assessment must be layered: a foundational layer checks for factuality, safety, and format compliance (automatable); a business layer evaluates information completeness and logical consistency (semi-automatable); and an experience layer judges tone, level of detail, and audience fit (requires manual sampling).
Cost scales with usage
Traditional cloud architecture costs are primarily tied to infrastructure resources like server count, storage capacity, and network bandwidth. An AI workload introduces a new cost dimension: token consumption. Each model call is billed based on input and output tokens, making the cost directly proportional to business volume and request complexity.
Consider an enterprise Q&A system. Assume an inference model costs A dollars per 1,000 input tokens and B dollars per 1,000 output tokens, where B is typically 2 to 4 times A. A single 8,000-token interaction (7,000 input, 1,000 output) would cost approximately 7A + B. Multiplied by an average of one million requests per day, the monthly inference cost can escalate significantly. If you choose a model with more parameters, A and B could increase by 5 to 10 times. More critically, as retrieval-augmented generation (RAG) systems retrieve more context and agents perform more rounds of tool calls, the token consumption per request will continue to swell, driving costs far beyond expectations.
Token economics requires architects to treat cost as a first-class architectural constraint. This means not only choosing more cost-effective models (for example, using Qwen-Plus instead of Qwen-Max for simple queries) but also designing cost control mechanisms at the architectural level, such as context window compression, cache hit optimization, and complexity-based request routing. On the Model Studio platform, you can use the model routing capability to dynamically select models of different specifications based on request characteristics, minimizing token expenditure while meeting quality requirements.
Extended supply chain
In traditional software, supply chain risks are mainly concentrated in code dependencies and infrastructure—for example, whether an open-source library has a vulnerability or a container image is trustworthy. An AI workload extends the supply chain to an unprecedented scope. Model weights, training data, fine-tuning datasets, prompt templates, evaluation benchmarks, and even knowledge base documents in a RAG system all become components of the supply chain, where each link can become an attack surface.
There have been multiple model supply chain attacks. Attackers have published carefully poisoned model weight files on open-source model repositories. On the surface, the model functions normally, but under specific trigger conditions, it can leak user input data or generate harmful content. If an enterprise deploys such a model into a production environment without rigorous validation, the consequences could be disastrous.
In addition, the prompt itself poses a supply chain risk. Enterprises often store core prompt templates in version control systems or configuration centers but frequently lack audit and approval processes for prompt changes. A single unreviewed modification to a prompt can cause a fundamental shift in model behavior, such as bypassing content safety filters or leaking sensitive business logic contained in system instructions.
Compliance is not optional
China is at the forefront of global regulation for generative AI. The Interim Measures for the Management of Generative Artificial Intelligence Services, implemented in 2023, clearly state that enterprises providing generative AI services to the public must complete an Algorithm Filing; generated content must reflect core socialist values; service providers are responsible for the legality of their training data sources; and users have the right to know they are interacting with an AI, not a human.
Algorithm Filing is not a one-time compliance action but a continuous obligation. A new filing may be required whenever a model undergoes a major update, such as switching the base model version, modifying the core prompt strategy, or introducing new training data. This places clear demands on architectural design: the system must have complete capabilities for model version tracing, training data provenance, and generated content auditing.
On Alibaba Cloud, the content moderation capabilities of Model Studio and Alibaba Cloud Content Security provide the technical foundation for compliance. However, architects must understand that compliance is not simply a matter of adding a filter before launch. It means weaving compliance requirements into every layer of the architecture—from source tagging during data ingestion and data cleansing during model training, to content moderation during inference and server-side log retention and traceability.
Rapid model evolution
The pace of technological iteration in the AI field far exceeds that of the traditional software ecosystem. The Qwen series of models has undergone multiple major version upgrades in two years—from Qwen to Qwen2, Qwen2.5, and Qwen3. Each upgrade brings significant capability improvements and minor interface changes. A prompt strategy that is carefully tuned today may perform poorly with the next model version, and the resources invested in fine-tuning a model today may become unnecessary due to the capability improvements of a new base model.
The key architectural takeaway from this rapid evolution is that AI systems must be designed for model replaceability. Abstracting model calls into a unified interface, versioning prompts, and persisting evaluation benchmarks for quick regression testing on new models are all necessary architectural investments. On the Model Studio platform, separating application configuration from model configuration allows you to switch the underlying model without modifying application code, significantly reducing the engineering burden of model iteration.
At the same time, architects need to be wary of the "over-optimization trap": avoid investing excessive custom engineering in a model that might be obsolete in six months. During a phase of rapid model capability improvement, lightweight prompt engineering and retrieval-augmented generation (RAG) often provide a better return on investment than heavyweight model fine-tuning. This is because optimizations from the former are more easily migrated to new models.
Five tensions in AI architecture
AI architecture design is not about choosing between "good" and "bad," but about finding a balance between multiple, equally valid but conflicting goals. The following five tensions are present throughout the lifecycle of an AI system. Architects must make explicit trade-off decisions in each specific scenario.
Quality vs. cost
A model with a larger parameter scale usually delivers better understanding, more accurate reasoning, and more fluent expression—but it also means a multiplicative increase in inference cost. Take the Qwen series for example: Qwen-Max performs significantly better than Qwen-Plus on complex reasoning tasks, but its cost per call is also several times higher.
In an intelligent customer service system that handles 500,000 user queries per day, routing all requests to Qwen-Max could push monthly inference costs over a million yuan. But if all requests use Qwen-Plus, the quality of answers to complex questions will drop significantly, leading to lower user satisfaction and a higher rate of escalation to human agents.
The architectural pattern to resolve this tension is intelligent routing: use a lightweight intent classifier to determine request complexity. Simple queries (like "What are your business hours?") are routed to Qwen-Plus, while complex queries that require multi-step reasoning (like "Compare the long-term costs of plan A and plan B") are routed to Qwen-Max. In large model application scenarios, you can use workflow orchestration to implement this tiered routing, finding a Pareto-optimal solution between achieving quality goals and controlling costs. However, the routing strategy itself has a cost. Classifier misjudgments can lead to quality loss or wasted spend, so you need to continuously monitor routing accuracy and iterate on optimizations.
For daily request volumes below 100,000, the engineering complexity of routing and classifier maintenance may outweigh the saved inference costs; in this case, using a single medium-sized model is more economical. When the classifier's misclassification rate for complex queries exceeds 15% (routing them to the simpler model), using the large model for all requests might actually reduce overall costs by minimizing follow-up requests and manual interventions caused by poor-quality answers.
Speed vs. safety
Users expect AI systems to respond as instantly as a human conversation. Streaming technology allows the model's generated tokens to be sent to the user one by one, creating a "typewriter" effect and reducing the time-to-first-token (TTFT) to a few hundred milliseconds. However, streaming means content is displayed to the user as it is generated. If the model suddenly produces inappropriate content in the third paragraph, the first two paragraphs have already been sent and cannot be retracted.
The traditional solution is "review then release": wait for the model to generate the complete response, run it through a content security check, and then return it all at once. This increases the first-token response time from the sub-second range to several or even tens of seconds, severely harming the user experience. In conversational scenarios, a wait time over three seconds can lead to significant user churn.
Practical compromises include segmented review (running a quick check after each sentence is generated), real-time streaming with asynchronous review and post-hoc retraction, and differentiated strategies based on user trust levels (internal users get streamed output, while public users require pre-moderated output). Alibaba Cloud Content Security supports streaming text detection, which allows for real-time content filtering without significantly increasing latency. However, architects must evaluate the acceptable trade-off between the risk of missed detections and the degradation of user experience for their specific scenario.
For applications aimed at internal users with low content risk, you can adopt a 'stream first, review later' model, only performing a delayed review on flagged content. For public-facing applications or those in regulated domains like financial advice or medical information, 'review first, release later' is the baseline requirement, even if it adds 200–500 ms of latency.
Autonomy vs. control
The value of an AI agent lies in its ability to autonomously complete complex tasks—automatically retrieving information, calling tools, making decisions, and executing actions. But the higher the autonomy, the greater the risk of losing control. An agent authorized to perform database operations could cause a disaster if it executes an unintended DELETE statement due to a prompt injection attack or a model hallucination.
Consider an internal procurement agent designed to automatically place restocking orders based on inventory levels. If given full autonomy, it might over-purchase due to a misunderstanding of market signals, tying up significant capital. But if every purchase requires manual approval, the efficiency advantage of automation is lost.
A viable architectural strategy is "tiered autonomy": set clear action boundaries, such as a single-transaction limit of 50,000 yuan and a daily cumulative limit of 200,000 yuan. Within these boundaries, the agent can execute automatically. When a boundary is exceeded, it pauses and requests manual approval. When building an agent on the Model Studio platform, you can implement this tiered control using conditional and human review nodes in a workflow. The key is to make the agent escalate when it encounters uncertainty, rather than silently making a potentially wrong decision.
When the impact of a single operation is below a certain monetary threshold (e.g., 5,000 yuan) and is reversible, the agent can be authorized to execute autonomously. If the threshold is exceeded or the operation is irreversible (such as data deletion or sending external notifications), a human confirmation step is mandatory.
Innovation vs. governance
Business teams want to experiment with AI capabilities quickly—discover a new use case this week and launch a chatbot next week. This tension is particularly acute in large organizations, where different business lines might incubate dozens of AI applications simultaneously. If each application must go through the full compliance process independently, the governance team will be overwhelmed, and business teams will lose market opportunities due to long waits.
The solution is platform-based governance: build a unified AI service platform and integrate compliance capabilities (such as content moderation, log auditing, and user notification) as platform-level services. Business applications built on this platform inherit its compliance baseline and only require an incremental assessment for application-specific risks. This approach maintains a compliance floor while enabling rapid business innovation. AI Governance can be configured with an AI governance baseline to automatically detect application deployments that violate security policies, providing governance guardrails without blocking the development process.
For AI applications open to the public, Algorithm Filing must be completed before they go live. This is a legal requirement, not a recommendation.
Customization vs. maintainability
Through model fine-tuning, enterprises can make a base model excel in specific domains. Vertical scenarios such as medical consultation, legal advice, and financial analysis can all benefit significantly from fine-tuning. But each fine-tuned model is a technical asset that requires ongoing maintenance: it needs to be re-tuned when the base model is upgraded, retrained when the training data is updated, and diagnosed when performance degrades.
An enterprise that maintains customized fine-tuned models for 12 different business lines is effectively maintaining 12 independent model training pipelines, 12 sets of evaluation benchmarks, and 12 separate operational responsibilities. When Qwen upgrades from Qwen2.5 to Qwen3, all 12 models need to be evaluated for compatibility and may need to be re-tuned. The engineering burden grows linearly.
An alternative strategy is to prioritize achieving customization with prompt engineering and RAG. By using carefully designed system prompts and context injection through retrieval augmentation, you can make a general-purpose model perform well in a specific domain. The results of this approach are not tied to a specific model version; when the model is upgraded, you only need to verify that the prompt effects remain stable, which significantly lowers maintenance costs. You should only escalate to a fine-tuning solution when prompt engineering and RAG cannot meet quality requirements.
If the accuracy of a domain-specific task reaches an acceptable business level (e.g., >90% satisfaction) through prompt engineering, there is no need to introduce the complexity of fine-tuning. Consider a fine-tuning investment only when the limits of prompt engineering are reached, business requirements are still not met, and you have at least several thousand high-quality labeled data points.
Design principles
The five tensions reveal the inherent contradictions in AI architecture design. The following five design principles provide a systematic approach to resolving them. These principles are not rigid rules but a guiding framework to help architects make better decisions in specific scenarios.
Principle 1: Controlled autonomy
The value of an AI system lies in its ability to complete complex tasks autonomously, but this autonomy must be exercised within explicit boundaries. The principle of "controlled autonomy" requires that every AI component has clearly defined capability boundaries and behavioral norms. When a situation exceeds these boundaries, the system should actively escalate to human decision-makers rather than failing silently or making a potentially wrong choice.
Implementing this principle requires a three-layer mechanism. The first layer is guardrails, which ensure that the AI's behavior does not cross boundaries by filtering inputs and checking outputs. This includes content safety filtering, sensitive information masking, and format validation. On the Model Studio platform, application-level guardrails can be configured through system prompts and post-processing rules. The second layer is the human-in-the-loop (HITL) model, which defines when human intervention is needed and designs the trigger conditions and escalation channels for manual review. The third layer is the agent permission boundary, which clearly defines which tools the agent can call, what data it can access, and what actions it can perform, using the principle of least privilege to tighten its authorization scope.
In practice, controlled autonomy means designing a "red line list" for an agent. This list specifies actions that are absolutely forbidden (e.g., deleting production data without approval) and gray-area actions that require human confirmation (e.g., payments exceeding a certain threshold). The system should provide a clear audit trail, recording the context, rationale, and outcome of every autonomous decision.
Principle 2: End-to-end measurability
"You can't improve what you can't measure"—this engineering truth is even more critical in the AI domain. The output quality, cost efficiency, and risk level of an AI system all need to be tracked and optimized through quantitative metrics. The "end-to-end measurability" principle requires that every AI decision generates observable metric data across three dimensions: quality, cost, and latency.
An observability architecture is the foundation for implementing this principle. In addition to traditional application performance monitoring (APM), an AI system needs: token consumption tracking to record the input and output token count for each request and associate it with business scenarios and cost centers; quality scoring to perform automated, multi-dimensional evaluation of model outputs—relevance, accuracy, completeness, and safety; and end-to-end tracing in RAG and agent scenarios to record the complete call chain from user request to final response, including retrieved results, raw model output, and post-processing modifications.
A token accounting mechanism should allocate AI inference costs to business departments and specific applications, making cost responsibility traceable. On Alibaba Cloud, you can use the usage statistics in Model Studio combined with the log storage and query capabilities of Log Service to build a cross-application, cross-model token consumption dashboard, enabling fine-grained management of AI costs.
Principle 3: Progressive authorization
A newly deployed AI system should not be immediately granted full production permissions and user access. The "progressive authorization" principle requires that the permissions and scope of an AI system be expanded gradually based on its demonstrated reliability—first validated in a controlled environment, then incrementally exposed to production traffic.
A phased rollout is the core mechanism for practicing this principle. Shadow mode is the first stage: the AI system runs in parallel with the existing process, but its output is logged and not acted upon. Humans compare the AI's decisions with actual decisions to gather quality data. Grayscale release is the second stage: a small amount of real traffic (e.g., 5%) is routed to the AI system, with close monitoring of quality metrics and anomalies. Full rollout is the third stage: after the grayscale validation is successful, the traffic percentage is gradually increased until the old process is completely replaced.
Graduated permission expansion is suitable for agent scenarios. A newly launched agent is granted only read permissions. After two consecutive weeks of meeting quality targets, it gains write permissions. After another month of stable operation, it is granted permission to perform high-risk operations. Each permission expansion is based on quantifiable reliability evidence, not subjective judgment.
Principle 4: Cost awareness
A token budget is not an afterthought for the finance team to calculate at month-end. It is a first-class constraint that must be incorporated into the architectural design from day one. The "cost awareness" principle requires setting a token budget for each request type during the system design phase and implementing dynamic cost control at runtime.
A per-request cost budget means setting a token consumption cap for different types of business requests. For example, a simple FAQ query might have a token budget of 2,000 (input and output combined), while a complex report generation request might have a budget of 30,000. When a request's context is about to exceed its budget, the system should trigger a degradation strategy, such as compressing the context, truncating low-relevance retrieval results, or rejecting an overly broad request and guiding the user to narrow its scope.
Complexity-based model routing is a core component of a cost-aware architecture. By performing a quick complexity assessment of a request (using keyword matching, historical query classification, or intent recognition), requests of different complexities can be routed to models of different cost levels. On an AI gateway, you can configure Qwen-Plus as the default processing model and only escalate requests assessed as high-complexity to Qwen-Max. This achieves cost optimality under quality constraints.
Principle 5: Built-in compliance
Transform compliance from a pre-launch checklist into an intrinsic attribute of the architecture. The "built-in compliance" principle requires that regulatory requirements are incorporated into the architectural design from day one, rather than being added as an extra layer after the system is developed.
Integrating content filtering from the start of the architectural design means that the system's data flow naturally includes content moderation nodes, instead of relying on a developer remembering to add them manually before deployment. When creating an AI application on the Model Studio platform, the content security policy should be a required configuration item, not an optional one.
Built-in audit logging means that the input, output, model version, and configuration parameters of every model call are persistently recorded and associated with a user identity. This satisfies regulatory requirements for service providers to retain logs (for example, for at least six months). These logs are not optional, switchable debugging information but an indispensable component of the system's normal operation.
Algorithm Filing integration means that the architecture is designed with capabilities for model version management, training data provenance, and security assessment report generation. When an audit requires relevant materials, the system can automatically generate the necessary technical documentation, rather than relying on engineers to compile it manually.