The core challenge in AI architecture design is not a lack of options, but an abundance of them without a systematic decision-making methodology. When a company decides to build an AI application, it faces not a single decision, but a series of interconnected ones: Which model to use? How to deploy it? How to organize data? How to orchestrate agents? Build or buy? Each choice impacts subsequent decisions, and many are difficult to reverse.
These decision trees are interdependent and should not be used in isolation. This chapter provides five systematic decision trees to help architects make evidence-based choices at critical junctures in AI architecture. The model strategy determines the selection space for the inference architecture, the data architecture affects the feasibility of the model strategy, the agent architecture builds on the combination of the first three, and the build-versus-buy decision constrains all technical choices from both economic and organizational perspectives.
Decision tree 1: Model strategy
Model strategy is the most fundamental decision in AI architecture, as it determines the system's capability ceiling, cost baseline, and operational complexity. Architects must answer three sequential questions: Whose model to use? How to customize the model? How to combine models?
API vs. self-hosted models
The first decision is whether to call a model hosted in the cloud via an API or to download and run the model on your own infrastructure.
Choosing an API model from the Model Studio (Bailian) platform, such as the Tongyi Qianwen series, offers zero operational overhead, pay-as-you-go billing, and automatic model upgrades. This approach is suitable when: data sensitivity is moderate (encrypted transmission is sufficient for compliance), latency on the order of seconds is acceptable, deep modifications to model behavior are not required, and the business workload has not yet reached the tipping point where fixed GPU costs are more economical.
Choosing to deploy an open-source model, such as Qwen3-72B, on PAI-EAS provides complete data control, customizable inference optimizations, and potential cost advantages at scale. Self-hosting is preferable if:
The model must process data classified as sensitive personal information under the Personal Information Protection Law (PIPL), and cannot leave the corporate network.
Strict latency requirements necessitate low-level optimization of the inference engine.
Your extensive model customization needs, such as custom vocabularies or special decoding strategies, exceed the flexibility of API models.
Your daily call volume exceeds a threshold where the fixed cost of GPUs becomes lower than pay-as-you-go billing.
Fine-tuning, RAG, and prompt engineering
After determining the model source, the next decision is how to adapt a general-purpose model to a specific business scenario. The three main customization methods each have their ideal conditions:
Prompt engineering has the lowest startup cost and fastest iteration speed. By carefully designing system prompts, constructing few-shot examples, and using techniques like chain of thought, you can significantly improve model performance on target tasks. This is suitable when the task can be clearly described in natural language, public knowledge is sufficient to ensure response quality, and business logic changes frequently, requiring rapid adjustments. On the Model Studio (Bailian) platform, you can rapidly iterate and A/B test prompts through the application configuration interface.
Retrieval-augmented generation (RAG) builds on prompt engineering by introducing external knowledge retrieval. RAG is suitable when: the model must access private enterprise knowledge (such as product documentation, internal policies, or customer data); the knowledge is frequently updated (such as a weekly product price list); or answers must be traceable to their sources (citing specific documents and paragraphs). The knowledge base feature on Model Studio (Bailian) provides an end-to-end RAG pipeline—from document upload and automatic chunking to vectorization, storage, and retrieval—lowering the barrier to entry for building RAG systems.
Use model fine-tuning when prompt engineering and RAG cannot meet your requirements. It is ideal when: the model needs to learn a new stylistic voice or specialized terminology (such as a specific legal document format); the model must complete a task in a few tokens that would otherwise require long prompts (reducing runtime costs); or evaluation criteria are highly subjective and must be learned from large volumes of human preference data. The PAI platform supports both full-parameter fine-tuning and parameter-efficient methods like LoRA, providing a complete toolchain for training data management and model evaluation.
In practice, these three methods are often used in combination. A typical progression is to first use prompt engineering to quickly validate feasibility, then add RAG to incorporate enterprise knowledge, and finally use fine-tuning for core scenarios to improve consistency. DataWorks can be used to manage data preprocessing workflows in the RAG pipeline, ensuring the quality and freshness of the knowledge base.
Single model, model routing, and model ensemble
The final dimension of model strategy is the combination approach:
The single-model approach is the simplest: select one highly capable model to handle all requests. This is suitable for single-purpose business scenarios, uniform request complexity, and situations where cost is not the primary constraint. For example, a strategic analysis assistant for internal executives may have low request volume but require extremely high quality, making a single Qwen-Max model sufficient.
Model routing is valuable when request complexity is uneven. A typical pattern involves a lightweight classifier (which can be a rules engine, a small model, or a heuristic algorithm) that routes requests to different models based on their features. For example, you can route simple chats to Qwen-Flash, standard questions requiring RAG to Qwen-Plus, and complex reasoning requests to Qwen-Max. Model Studio (Bailian) workflows can orchestrate this routing logic and log routing decisions for future optimization.
A model ensemble is used when decision accuracy is paramount. Multiple models independently generate responses to the same request, and the final output is determined by voting, weighted fusion, or a consistency check. This is suitable for scenarios with extremely low fault tolerance, such as financial risk control or medical diagnostic assistance. The trade-off is a multiplicative increase in cost and latency. In practice, a consensus from two smaller models can sometimes replace a single massive model, achieving a better cost-to-quality ratio in certain scenarios.
Trade-off matrix
The core trade-offs in model strategy can be analyzed along four dimensions:
Latency: API models typically have network overhead (50-200ms). Self-hosted models can be optimized for extremely low latency but require operational investment. Larger models have slower inference, but architectures like mixture of experts (MoE) can maintain large-model capabilities while controlling inference overhead.
Cost: APIs are billed per token, so costs scale linearly with usage, which is ideal for fluctuating workloads. Self-hosting is billed by GPU time, which involves a fixed cost but low marginal costs, making it suitable for stable, high-volume workloads. Fine-tuning adds a one-time training cost but can reduce runtime token consumption by enabling tasks to be completed with shorter prompts.
Quality: Models with more parameters generally offer better quality, but not for every task. A fine-tuned smaller model can outperform a general-purpose large model on a specific vertical task. RAG significantly improves factual accuracy but increases system complexity.
Data Privacy: With API models, data leaves the enterprise boundary (even if encrypted in transit). With self-hosted models, data never leaves your network. Fine-tuning requires uploading training data to a training platform (PAI supports training within a VPC to reduce risk). For RAG, you must evaluate the compliance of the storage location for your knowledge base data.
Re-evaluate your model strategy when: model API costs exceed the budget by more than 20% for three consecutive months; a new generation of models offers a price-performance improvement of over 50%; data privacy requirements increase due to regulatory updates; or the self-hosting operations team cannot keep up with security patches.
Decision tree 2: Inference architecture
Once a model is chosen, the next key decision is how to deploy and run the model's inference service. The inference architecture directly determines the system's response speed, scalability, availability, and operational cost.
Serverless, dedicated GPU, and hybrid deployment
Serverless inference means fully entrusting model calls to a cloud platform without provisioning or managing GPU instances. By triggering API calls through Function Compute (FC) or using the managed inference services on Model Studio (Bailian), you avoid managing underlying GPU supply, scheduling, and operations. The advantages are pay-per-use billing with zero idle cost, automatic scaling to handle traffic peaks, and zero operational burden for service availability. This approach is suitable for workloads with high traffic volatility (peaks are 10x or more than troughs), tolerance for potential cold-start latency (the first request may take an extra 1-3 seconds), and no need for low-level inference engine customization.
Dedicated GPU deployment (via PAI-EAS) means reserving a set of GPU instances exclusively for your model inference service. You gain full control over the inference engine configuration, allowing fine-tuned optimizations like batch size, quantization strategies, KV cache management, and speculative decoding. This approach is suitable for stable, large-scale traffic where GPU utilization can be maintained above 60%; extreme latency requirements that demand millisecond-level optimization; use of custom or fine-tuned models not in the standard DashScope model list; or security requirements that mandate model execution within a dedicated VPC.
A hybrid deployment strategy combines the advantages of both. Baseline traffic is handled by dedicated GPU instances (maintaining 70-80% utilization), while sudden traffic spikes overflow to Serverless inference as an API-based fallback. This architecture is especially useful for scenarios with predictable traffic patterns but extremely high peaks, such as e-commerce promotions. PAI-EAS supports elastic scaling policies that automatically scale GPU instances based on queue depth, but this scaling speed (minutes) is slower than Serverless (seconds). A Serverless fallback is therefore necessary for handling sudden traffic spikes.
Real-time streaming, batch processing, and asynchronous queues
The inference execution model depends on the business scenario's timeliness requirements:
Real-time streaming is for interactive scenarios where the user is waiting for an AI response and expects a "typewriter" effect of word-by-word generation. Model Studio (Bailian) natively supports Server-Sent Events (SSE) for streaming responses. Key architectural considerations include implementing streaming rendering logic on the client, using a load balancer that supports long-lived connections, and setting a timeout policy that accounts for the total time to generate long text (which may exceed 30 seconds). In streaming mode, time to first token (TTFT) is a key user experience metric and should generally be kept under one second.
Batch processing is for offline tasks that are not time-sensitive, such as large-scale document summarization, historical data labeling, or periodic report generation. Model Studio (Bailian) provides a batch inference API that supports submitting large request lists and retrieving results asynchronously. The advantages include reduced costs from using low-priority compute resources (like preemptible instances), improved throughput from sharing the KV cache between requests, and global scheduling optimizations by the platform. DataWorks can orchestrate batch inference jobs for scheduled triggers and dependency management.
An asynchronous queue is for workflow-driven AI tasks where the user submits a request but does not need an immediate result, instead receiving a notification when the result is ready. For example, a user uploads a 100-page contract for AI review; the system accepts the request, returns a task ID, and notifies the user upon completion. The architectural pattern typically involves a frontend submitting requests to a message queue (like RocketMQ), a consumer retrieving requests to call the model, writing the results to storage, and sending a final notification to the user. This pattern decouples request reception from processing, allowing the system to consume the queue at its own pace and naturally smooth out traffic peaks.
Single-region, multi-region, and on-device inference
Geographic deployment strategy involves balancing data compliance, user experience, and system robustness:
Single-region deployment is the simplest solution. If your users and data are located in Chinese mainland, deploying the AI inference service in one core region is sufficient. The Model Studio (Bailian) services in Chinese mainland regions cover most domestic scenarios. The advantages are a simple architecture, data residency within the region, and minimal operational overhead.
Multi-region deployment is suitable for two cases: global operations that require serving users from a nearby region to reduce network latency (for example, using the China (Beijing) region for domestic users and the Singapore region for Southeast Asian users), and high-availability requirements that necessitate region-level disaster recovery. The challenges of multi-region deployment include cross-region synchronization of models and knowledge bases, user request routing strategies, and differing regulatory requirements (such as the EU's GDPR vs. China's PIPL). On PAI-EAS, you can deploy model services independently in multiple regions and use Global Accelerator or DNS-based routing to provide nearest-region access.
On-device inference (or edge inference) deploys small models to end-user devices or edge nodes. This is suitable for offline environments with unreliable networks (such as mines or ships), real-time interactions with extremely low latency requirements (like voice assistants in AR/VR), and high-sensitivity scenarios where data must never leave the device (such as local diagnostic assistance on medical equipment). On-device models are typically smaller versions (such as Qwen3-0.6B or Qwen3-4B) with limited capabilities but extremely fast responses. A hybrid architecture can allow an on-device model to handle simple requests and forward complex ones to the cloud when a network is available.
Capacity planning
The choice of inference architecture can be initially guided by request volume:
Fewer than 10,000 requests per day: Use Serverless mode (direct API calls to Model Studio (Bailian)). No dedicated resources are needed, and costs scale elastically with usage. This is suitable for early-stage AI applications and internal tools.
10,000 to 1 million requests per day: Evaluate the cost break-even point between Serverless and dedicated GPU. If the request distribution is relatively uniform, the per-request cost of dedicated GPUs (PAI-EAS) may be lower. If there are significant peaks and troughs, Serverless is often the more economical choice. At this stage, it is recommended to build a performance monitoring system to track P50, P95, and P99 latency distributions.
Over 1 million requests per day: A dedicated GPU deployment is almost always necessary to control costs. The architectural focus shifts to inference engine optimization (quantization, KV cache optimization, speculative decoding), multi-instance load balancing, auto-scaling policies, and multi-region deployment for high availability. Use a hybrid deployment strategy to handle traffic spikes.
Over 10 million requests per day: This scale requires deep inference optimization and customized deployment strategies. It may involve advanced techniques like model distillation (compressing a large model's knowledge into a smaller one to reduce per-request cost), using specialized inference chips, and merging requests for batch processing.
Re-evaluate your inference architecture when: GPU utilization remains below 20% for over a quarter (a signal of over-provisioning); cold-start latency leads to a significant increase in user complaints; traffic growth makes Serverless costs exceed dedicated instance costs; or new inference acceleration technologies (such as new hardware or optimization frameworks) cause a generational shift in price-performance.
Decision tree 3: AI data architecture
If a model is an AI system's "brain," then the data architecture is its "memory." The quality ceiling for RAG is determined by the quality of the data pipeline—if the pipeline cannot retrieve the right knowledge, even the most powerful model cannot provide an accurate answer. Data architecture design must balance retrieval precision, response latency, and storage cost.
Pure vector retrieval, hybrid retrieval, and graph-enhanced RAG
Pure vector retrieval is the most basic form of a RAG system. Documents are split into chunks, converted into embedding vectors, and stored in a vector database. During retrieval, the user's query is vectorized to perform an approximate nearest neighbor search, returning the most semantically similar document chunks as context. This method is simple to implement and works well for unstructured text. It is suitable for FAQ-style Q&A, where questions and answers are semantically related, and for simple document retrieval (like product manual lookups). The default mode of the Model Studio (Bailian) knowledge base is vector retrieval.
However, pure vector retrieval has clear limitations. It only captures semantic similarity and struggles with exact-match requirements (such as querying a specific product ID like SKU-20240315), cannot use structured metadata for filtering (for example, "show only documents published in 2026"), and performs poorly on complex questions that require understanding relationships between entities.
Hybrid retrieval combines vector semantic search with traditional keyword search (like BM25) and adds metadata filtering. It executes both search types simultaneously and then uses a reranker model to merge and re-rank the results. This approach delivers both semantic understanding and exact-match capabilities, making it the mainstream choice for enterprise RAG systems. It is suitable for corporate document libraries with clear metadata, technical knowledge bases with both concepts and exact parameters, and customer service knowledge bases with both semantic FAQs and exact-match IDs. The Model Studio (Bailian) knowledge base supports a hybrid retrieval mode, and AnalyticDB for PostgreSQL provides the underlying fused vector and full-text indexing capabilities.
Graph-enhanced RAG builds on hybrid retrieval by introducing a knowledge graph to explicitly model relationships between entities. When a question involves multi-hop reasoning (for example, "Which components from Supplier A are used in projects managed by John Doe?"), semantic retrieval struggles to connect information across documents. A knowledge graph, however, can quickly find answers by traversing entity relationship paths. This approach is suitable for domains with complex entity relationships (like supply chain management or academic citation networks) and for multi-hop Q&A. The complexity and maintenance cost of graph-enhanced RAG are significantly higher, so it should only be considered when hybrid retrieval cannot meet precision requirements.
Managed knowledge base vs. self-built pipeline
The knowledge base feature on Model Studio (Bailian) provides an out-of-the-box RAG pipeline: upload documents, and the platform handles automatic parsing, intelligent chunking, vectorization, index construction, and retrieval APIs. This requires almost no engineering effort, making it ideal for quickly validating RAG feasibility and for small- to medium-sized knowledge bases. Its advantages are high development efficiency (setup in minutes), zero operational overhead, and seamless integration with Model Studio (Bailian) applications.
However, a managed knowledge base offers limited customization. You cannot finely control chunking strategies (for example, using different logic for code versus tables), insert custom cleaning steps into the preprocessing pipeline (like data masking or terminology normalization), or extensively tune retrieval strategies. When your RAG quality requirements are exceptionally high, you may need to build your own data pipeline.
A self-built pipeline means creating the entire chain from data ingestion to retrieval service yourself. This includes: document parsing (PDF, Word, web pages); preprocessing (cleaning, masking, metadata extraction); chunking (by semantic paragraph, fixed size, or hierarchy); embedding (calling DashScope models or a self-trained model); index storage (using AnalyticDB for PostgreSQL, Lindorm, or Elasticsearch); and a retrieval service with custom recall and reranking logic. DataWorks can orchestrate the scheduled execution and incremental updates of this pipeline.
The key consideration is your team's focus. If your team has fewer than five people and AI is not a core business, start with a managed knowledge base. If you have dedicated data engineers and RAG quality is a core differentiator, build your own pipeline for complete control. If you are in between, start with a managed knowledge base and self-build specific components only after identifying bottlenecks.
Data freshness and architectural implications
The required timeliness of data in your knowledge base directly determines the complexity of your data pipeline architecture:
Real-time updates (seconds to minutes): Suitable for rapidly changing data, such as order status or inventory information. The architecture is a streaming pipeline that uses a message queue (like Kafka or RocketMQ) to listen for data change events and trigger incremental vectorization and index updates. This is the most complex option, requiring you to handle consistency, out-of-order events, and duplicate messages.
Near-real-time updates (hourly): Suitable for content with a moderate update frequency, such as news, announcements, or product documentation. The architecture is a scheduled job. Use DataWorks to configure hourly or daily data synchronization tasks that process incremental data in batches and update the index. This has moderate complexity and is a pragmatic choice for most enterprise knowledge bases.
Static or infrequent updates (weekly to monthly): Suitable for content that rarely changes, such as regulations, training materials, or historical archives. The architecture is a manual or periodic full rebuild. You process all documents at once and manually trigger a rebuild only when major content changes occur. This is the least complex option and is well-supported by the default mode of the Model Studio (Bailian) knowledge base.
The trade-off: Recall precision, latency, and storage cost
The data architecture design must always navigate this three-way trade-off:
Methods to improve recall precision (such as smaller chunk sizes, higher-dimensional embedding vectors, adding a reranker, or increasing retrieval candidates) usually increase storage costs and retrieval latency. An index with 100 million 1536-dimensional vectors requires about 600 GB of storage, and each search may take 50-100 ms.
Methods to reduce latency (such as using an in-memory vector index, reducing retrieval candidates, or simplifying reranking logic) may sacrifice recall precision. Loading an index fully into memory can reduce search latency to single-digit milliseconds, but memory is far more expensive than disk storage.
Methods to control storage cost (such as reducing vector dimensionality, using quantization compression, or increasing chunk size to reduce the total number of vectors) may negatively affect recall precision. Product Quantization (PQ) can compress vector storage to a quarter of its original size but introduces some precision loss.
Architects must set priorities based on the specific scenario. User-facing real-time Q&A should prioritize low latency (P95 < 200ms) and baseline precision. An in-depth research assistant for analysts should prioritize recall precision, even if latency is several seconds. Archival retrieval of massive document stores should prioritize low storage cost.
Re-evaluate your data architecture when: the RAG system's accuracy consistently falls below business requirements (you may need to upgrade from pure vector to hybrid or graph-enhanced retrieval); knowledge base growth causes retrieval latency to exceed SLAs; or data freshness requirements shift from daily to minute-level.
Decision tree 4: Agent architecture
An agent is a key evolution for AI systems, moving them from passive question-answering to proactive action. When an AI must not only answer questions but also call tools, perform operations, and coordinate multi-step tasks, agent architecture becomes a central design consideration.
Single-agent vs. multi-agent orchestration
A single-agent architecture is the most direct implementation. One agent has all the capabilities to understand intent, call tools, and generate responses. This is suitable when the task scope is clear and limited, the number of required tools is under ten, and no complex role-playing or process coordination is needed. For example, a calendar management agent can understand "Schedule a meeting with Mr. Zhang for 3 PM tomorrow," then call the calendar API, find an open slot, create the invitation, and send notifications—all handled by one agent.
When task complexity exceeds a single agent's capacity, multi-agent orchestration becomes necessary. Signs of this complexity include: the need for different specialized knowledge and toolsets (one agent for code analysis, another for document comprehension); the need to execute sub-tasks in parallel and aggregate results; the need for checks and balances between roles (one agent generates a plan, another reviews it for compliance); or a single agent's context window being too small to hold all task information.
Common multi-agent orchestration patterns include: the Orchestrator Pattern, where a central agent decomposes and dispatches tasks to multiple specialist agents; the Pipeline Pattern, where the output of one agent becomes the input for the next; and the Debate Pattern, where multiple agents analyze a problem from different angles and "debate" to converge on the best answer.
Tool-calling, workflow-driven, and autonomous agents
An agent's internal architecture can be classified into three types based on its degree of autonomy:
A tool-calling agent is centered on the model's ability to call tools. The model decides when to call which tool and how to parse the result. The execution path is dynamic, with the model determining the next action at each step based on the context. This is suitable for open-ended problem spaces where not all execution paths can be predefined and flexibility is required. For example, a general data analysis assistant must dynamically decide which data source to query, what analysis to perform, and how to visualize results for any given question.
A workflow-driven agent executes a predefined flowchart. Each node is a deterministic step, such as a model call, a tool call, a conditional branch, or a manual approval. The execution path, including any branches, is defined at design time. This is suitable for business scenarios with fixed processes and strict compliance requirements, such as loan approval (credit check → income verification → risk assessment → final decision) or medical triage (symptom collection → urgency assessment → department recommendation).
An autonomous agent operates with the greatest degree of autonomy. It can set its own sub-goals, plan execution steps, and reflect on and adjust its strategy during execution. This is suitable for complex, long-running research tasks (such as "Analyze the competitive landscape of this industry and generate a report") and exploratory tasks with high uncertainty. Autonomous agents also carry the highest risk. An uncontrolled agent could fall into an infinite loop, incur high API costs, or perform unintended actions. You must implement clear termination conditions (max steps, token budget, timeout) and checkpoints (pausing for manual confirmation before critical decisions).
Synchronous vs. asynchronous execution
The agent's execution mode depends on the task duration and user interaction model:
Synchronous execution is for tasks that complete quickly (within seconds). The user submits a request, the agent processes it, and the result is returned directly. This is the default mode for most conversational scenarios. Key considerations include controlling the total execution time with timeouts and placing latency constraints on tool calls so that an unresponsive API does not block the entire request.
Asynchronous execution is for tasks that take a long time to complete. The user submits the task and can leave, receiving a notification upon completion. For example, "Analyze the sales data for the past three months and generate a weekly report" might take several minutes. An asynchronous architecture must include task status tracking (pending, running, completed, failed), progress reporting to keep the user informed, persistence of intermediate results to prevent data loss on failure, and result notifications (via message, email, or UI push).
A hybrid model combines both. The agent first returns a quick, preliminary response synchronously (for example, "I am analyzing your data and expect to finish in about 3 minutes"). It then performs the detailed analysis asynchronously and pushes the full result upon completion.
Decision tree 5: Build, buy, or customize
Ultimately, all technical decisions are constrained by a fundamental strategic question: what is the role of AI in your company's technology stack? Is it a core competency or a general-purpose utility? This judgment determines your investment model.
SaaS vs. PaaS vs. IaaS
A SaaS AI solution is the most lightweight option. You directly purchase off-the-shelf software with integrated AI capabilities, such as a customer service SaaS with an AI summarization feature or an IDE with AI-assisted coding. You do not need to know the underlying model or manage prompts and knowledge bases; you consume AI as a product feature. This is suitable when AI is not a core differentiator, standard features meet your needs, you have no data residency concerns, and you want the fastest time-to-market.
The PaaS model, represented by Model Studio (Bailian), provides foundational capabilities like model invocation, knowledge bases, agent orchestration, and content security, upon which you build your own AI applications. You control the application logic, prompt strategies, knowledge base content, and business processes, but you do not manage the underlying GPU resources or model serving. This is the recommended starting point for most enterprise AI applications, especially when you need some customization, have application developers but lack deep MLOps expertise, and require platform-level security and compliance.
The IaaS model means you manage the entire AI technology stack yourself. You deploy open-source models on ECS or ACK, build your own inference services and training pipelines, and maintain the full model lifecycle. You gain maximum flexibility and control but also assume all engineering complexity and operational responsibility. This is suitable when AI is a core competency that requires extreme customization, your team has deep ML engineering capabilities, you have strict data control requirements (for example, core financial system data), or your scale is so large that building is significantly cheaper than using platform services.
Total cost of ownership (TCO)
The total cost of ownership (TCO) of an AI system extends far beyond cloud resource fees. A complete TCO assessment should include:
Development Cost: The engineering effort to build from scratch. This is nearly zero for SaaS, requires application-layer development for PaaS (person-weeks), and full-stack development for IaaS (person-months to person-years).
Operations Cost: The ongoing human effort for maintenance, including monitoring, troubleshooting, and updates. This is handled by the vendor for SaaS, shared for PaaS (Alibaba Cloud manages the platform, you manage the application), and is your full responsibility for IaaS.
Model Update Cost: The investment to keep up with evolving foundation models. SaaS updates are typically automatic. PaaS allows you to switch to new models via configuration. IaaS requires you to manage the entire replacement, evaluation, and redeployment process, including retraining fine-tuned models.
Compliance Cost: The effort to meet regulatory requirements. SaaS vendors often handle this, but you must confirm it covers your use case. PaaS provides platform-level compliance, but you are still responsible for your application. With IaaS, all compliance work is your responsibility.
Opportunity Cost: The potential benefits lost by choosing one path over another. Building an AI system over several months means delaying its launch. In a fast-moving market, a six-month delay can mean losing your first-mover advantage. This hidden cost is often underestimated.
Exit Cost: The future expense of migrating to a new solution. Deep integration with a SaaS vendor can make data migration difficult. A highly customized self-built system may be hard to migrate due to architectural lock-in. Design with reasonable abstraction layers to reduce exit costs.
Team capability and recommended path
Your team's AI engineering capability is a key constraint. A technically optimal solution that exceeds your team's abilities becomes the worst choice in practice.
No dedicated AI/ML team: Business developers handle your AI needs, with no dedicated algorithm engineers or MLOps roles. Recommended path: First, evaluate if a SaaS solution meets your needs. If not, use the low-code capabilities of Model Studio (Bailian) to build your application. Leverage the platform's agent orchestration, knowledge base management, and model evaluation features to lower the expertise barrier. Focus on business scenarios and prompt optimization, not underlying engineering.
Small AI team (3-10 people): You have a few algorithm engineers and AI application developers with some model evaluation and data processing capabilities. Recommended path: Use Model Studio (Bailian) as your core foundation. Deliver quickly where standard features suffice, and invest in deep customization only for core differentiators. Use the PAI platform for selective model fine-tuning and DataWorks to build custom data pipelines. Focus on identifying which parts are worth building yourself and which should remain on the platform.
Large ML engineering team (10+ people): You have a complete AI R&D organization, including research, training, inference optimization, data engineering, and MLOps. Recommended path: A hybrid IaaS+PaaS approach. Self-build core model capabilities on IaaS (deploying custom inference services on ACK, running large-scale training on PAI) for maximum control and cost advantage. Use PaaS capabilities (like content safety from Model Studio (Bailian)) for peripheral needs to improve efficiency. Focus your engineering power on what builds your competitive moat.
Regardless of team size, a common recommendation is to start with a proof of concept (PoC) on the Model Studio (Bailian) platform to validate business value with minimal investment. After the PoC confirms that an AI solution can solve a real business problem, decide whether to stay on the platform or evolve toward a self-built solution based on your team's capabilities and long-term strategy. Premature investment in self-building is a common waste of resources in AI projects; many companies spend months building a custom service only to find that the business case was not viable or that a platform solution already met their needs.
Architects should document the rationale and assumptions for each decision. This allows for quick re-evaluation and adjustment when external conditions change, such as team expansion, business growth, new model releases, or regulatory updates. AI architecture is not a one-time design but a dynamic process of continuous optimization that evolves with the business.
Re-evaluate your technical path when: the PaaS platform's limitations repeatedly block your requirements (signaling a need to move toward IaaS); the customization cost of a SaaS solution begins to approach the cost of building it yourself; or your team's AI engineering capability grows significantly over a 6-12 month period.