Well-Architected Framework:Cost pillar

Token economics

Model Studio prices services per million tokens. For mainstream text generation models: qwen3-max in the under-32K tier costs CNY 2.5 for input and CNY 10 for output; qwen-plus in the under-128K tier costs CNY 0.8 for input and CNY 2 for output (unit: CNY per million tokens). The price gap between lightweight and flagship tiers can range from several times to more than an order of magnitude. This is why model selection is such a high-leverage decision.

Note two pricing characteristics. First, output costs more than input. For mainstream text models, the output-to-input price ratio is typically between 2.5x and 4x. Your estimates must separate input tokens and output tokens. Using a single average token count obscures the higher cost of output tokens. Second, some models use tiered pricing: the unit price depends on the total input tokens in a single request, and all tokens in that request are billed at the same tier. For qwen3-max, input costs CNY 2.5 for up to 32K tokens, CNY 4 from 32K to 128K, and CNY 7 from 128K to 256K. This means the cost impact of context length is not smooth or linear. Once you cross a tier boundary, the pricing baseline for the entire request increases. This mechanism has a more direct impact on your bill than theoretical factors like attention complexity.

Example estimate for customer service: Process 1 million conversations per month, where each consumes 2,000 input tokens and 500 output tokens. With qwen3-max, the input cost is 1,000,000 × 2,000 ÷ 1,000,000 × 2.5 = CNY 5,000. The output cost is 1,000,000 × 500 ÷ 1,000,000 × 10 = CNY 5,000. The total is about CNY 10,000 per month.

Compute cost

For enterprises self-hosting inference or performing their own fine-tuning, GPU computing power is the most expensive resource. Alibaba Cloud offers GPU compute instance families such as gn8is, gn7i, gn6v, gn6e, and gn6i, corresponding to L20, A10, V100, T4, and other GPU types. Large-scale training and high-concurrency inference typically run on Lingjun Intelligent Computing Service rather than publicly sold GPU Elastic Compute Service (ECS) instance families. Choose the smallest option that meets your memory and throughput requirements. Use real-time pricing in the ECS console and the Pricing Calculator. As a cost anchor, public resources in Platform for AI-Elastic Algorithm Service (PAI-EAS) are billed at CNY 0.194 per vCPU core-hour and CNY 0.024 per GB-hour for memory.

Fine-tuning cost is roughly linear with model size. For two model tiers whose parameter counts differ by one order of magnitude, the required GPU-hours and parallel GPU count typically differ by a similar order of magnitude. Before you start fine-tuning, validate whether a smaller tier can meet the target after thorough tuning. You can estimate inference compute roughly as follows: for dense models, the forward compute per generated token is about twice the parameter count. That is, a 7-billion-parameter model is about 14 GFLOPs per token, and a 72-billion-parameter model is about 144 GFLOPs per token. This is a per-token scale, not per-request. For a single request, multiply this value by the output length and add the input-side prefill overhead.

GPU utilization in self-hosted inference clusters is usually low. Common causes include uneven traffic distribution, cold start waits, inefficient batching, and fragmentation from teams deploying separate stacks. Use your monitoring data to evaluate utilization and to quantify the benefits of pooling resources.

Storage cost

Take text-embedding-v4 as an example. The default output is 1,024 dimensions. If you store each vector as 4-byte floats, one vector uses 4,096 bytes (about 4 KB). With metadata and index overhead, each text chunk typically consumes about 5 KB to 6 KB. For 10 million text chunks, the raw vectors require about 40 GB to 60 GB of storage. After you add an HNSW graph index, the total usually increases by another 10% to 30%. This model supports multiple output sizes from 2,048 to 64 dimensions. Reducing dimensions directly reduces storage and retrieval cost. Dropping from 1,024 to 512 halves storage, an often-overlooked cost lever.

Model files also matter. A 7-billion-parameter model stored in FP16 is about 14 GB. A 72-billion-parameter model is about 144 GB. If you keep multiple fine-tuned versions for A/B testing and rollback, costs grow fast. Training sets, evaluation sets, and intermediate checkpoints also add significant costs.

Hidden costs

Evaluation compute. Every model change requires rerunning hundreds to thousands of test cases. If your evaluation set has 1,000 samples and each sample uses 800 input tokens and 200 output tokens, a single full evaluation with qwen-plus costs about CNY 1. In fast iteration, one team may run it more than ten times a day. Add multi-model comparisons and regression checks, and the monthly total becomes significant. Evaluation is not latency-sensitive, so run it with batch inference at a lower unit price.

Cost amplification from chain-of-thought output. For reasoning models, chain-of-thought is billed as output tokens. When the model emits its thinking process, all output tokens are billed at the thinking-mode output price. In an official Alibaba Cloud example, one call generates about 360 output tokens, of which 286 are thinking tokens—close to 80%. If you enable deep thinking by default for simple tasks, output cost can multiply. This is the easiest hidden cost to lose control over today. Enable thinking mode only by task type, and set an upper bound on thinking length.

Sunk cost from failed experiments. A significant portion of fine-tuning experiments never reach production, but they consume computing power. Make experimental loss an explicit line item in the budget. Overprovisioned development environments are also common. Teams often request the same compute tier as production for debugging, but utilization is low. Prefer elastic resources that can scale to zero. In addition, Model Studio provides free call quotas for many models for a limited time after activation, which helps reduce POC trial-and-error cost.

Energy use and green computing

The carbon footprint of large language model (LLM) workloads is now a topic in ESG reporting. Training energy use correlates with parameter count, data volume, and duration. Inference energy per call is small, but it accumulates at scale. If you need quantified emissions, estimate them based on actual computing power time and the grid emissions factor in your region, rather than using generalized analogies. The core principle of green computing is right-sizing the model. If a 7-billion-parameter model can handle the job, using a 72-billion-parameter model consumes roughly an order of magnitude more computing power and produces a similar multiple of emissions.

BP-COST-01 Match model tier to task complexity

Best practice: Build a task-to-model mapping matrix. Choose the most cost-effective tier based on cognitive complexity. Avoid using the largest model for every scenario.

Key points: Divide tasks into four complexity levels. Level 1 covers structured tasks (classification, entity extraction, format conversion, keyword extraction). Logic is simple and patterns are stable, so use lightweight tiers (such as the Qwen Flash series) or a specialized small model. Level 2 covers standard conversational tasks (simple Q&A, information retrieval, templated replies) and fits the Qwen Plus tier. Level 3 covers complex reasoning tasks (multi-step reasoning, creative generation, complex summarization, cross-document synthesis) and needs the Qwen Max tier. Level 4 covers expert tasks (code generation, mathematical proofs, specialized domain analysis) and may require a domain fine-tuned model or the largest general-purpose tier. In the matrix, record the model family and capability level instead of hard-coding version numbers, and retest when a new version is released. Anti-pattern: Use the strongest model in POC to maximize quality, then skip downgrade evaluation when you scale.

Alibaba Cloud implementation: Use AI Gateway to centrally manage multi-tier model calls and API credentials. Configure scenario-based model selection policies and use model evaluation capabilities to verify that lower tiers meet your quality bar.

BP-COST-02 Use model routing to balance cost and quality

Best practice: Deploy a request classifier to score each inference request and route it to the most cost-effective model instance.

Key points: The architecture has three components: a request classifier that evaluates inputs in real time and outputs a complexity score; a routing decision engine that dispatches to the right endpoint based on the score and thresholds; and a quality monitoring module that continuously collects quality metrics and adjusts thresholds. You can measure savings directly. Using the customer service example in 6.1.1, the baseline of calling qwen3-max for all requests costs about CNY 10,000 per month. If you route 70% of requests to qwen-plus and keep 30% on qwen3-max, the total input cost is CNY 2,620 (CNY 1,500 for qwen3-max and CNY 1,120 for qwen-plus), total output cost is CNY 2,200 (CNY 1,500 for qwen3-max and CNY 700 for qwen-plus), and the total is about CNY 4,820 per month, saving about 52%. The savings depend on the down-routing ratio and the price ratio between tiers. Use real unit prices for your model pair and do not rely on rule-of-thumb multipliers. You can train a lightweight classifier (a small pretrained encoder or logistic regression). Its inference latency can stay within a few milliseconds. You can also use heuristics based on input length, intent type, and whether structured output is required. This needs no training but is less accurate. Anti-pattern: Launch routing without quality monitoring. Misclassification pushes complex requests to a weaker model, and you often only discover the quality loss after user complaints.

Alibaba Cloud implementation: Manage tiered model endpoints in Model Studio. Deploy a lightweight classifier on Function Compute (FC) 3.0 as a routing pre-gateway. Use AI Gateway for request dispatch and load balancing.

BP-COST-03 Reduce cost with distillation and quantization

Best practice: For internal or non-customer-facing and non-critical AI functions, use knowledge distillation and model quantization to reduce inference resource usage. Keep quality loss within an acceptable range while reducing deployment cost.

Key points: Knowledge distillation trains a smaller student model using outputs from a large teacher model as the training signal. By learning the teacher’s probability distribution rather than only hard labels, the student inherits part of the teacher’s capability. A typical pattern is to use a large-tier teacher and distill a student with one order of magnitude fewer parameters, while retaining most of the teacher’s quality on a specific task. Model quantization compresses weights from high-precision floats (FP16/FP32) to low-precision integers. INT8 typically halves model size and inference memory usage. INT4 can achieve about 4x compression, with larger quality loss as bit-width decreases. Use these techniques for internal tools and ops backends, non-customer-facing data pipelines (batch classification, label generation), and dev/test model services. Avoid aggressive quantization on core user-facing experience paths. Anti-pattern: Ship a quantized model without testing on your business evaluation set. Sensitivity to accuracy loss varies widely by task, and generic benchmarks cannot replace business validation.

Alibaba Cloud implementation: Use the model compression capability in PAI-Model Gallery to run quantization. It is based on Weight-only Quantization technology and provides MinMax-8Bit and MinMax-4Bit strategies. Run quantization as a separate pre-deployment job, then deploy the compressed artifact as an online service through PAI-EAS. PAI-EAS supports inference engines such as BladeLLM and vLLM to further improve throughput.

BP-COST-04 Share a GPU pool for inference

Best practice: Consolidate resource silos created by separate team deployments. Build a unified GPU inference resource pool. Use multi-model sharing and intelligent scheduling to increase utilization.

Key points: A typical anti-pattern is five teams each deploying a 2-GPU inference service for a total of 10 GPUs, but with different peak hours. Average utilization stays at only about 30%. Measure pooling benefits by effective load: 10 GPUs at 30% utilization equals about 3 GPUs at full load. After pooling, you can configure 4 GPUs to handle peak overlap while raising average utilization to about 75%. Compared with the original 10 GPUs, this saves about 60% of investment. The actual reduction depends on how peaks overlap. Model this based on historical traffic curves rather than copying ratios. Implement the solution with Kubernetes GPU sharing scheduling: deploy GPU sharing and isolation components to support multiple containers per GPU; deploy a unified model service gateway for multi-model endpoint routing; configure horizontal scaling based on queue depth and GPU utilization. Fewer GPUs also means lower energy use and emissions. Anti-pattern: Reduce GPU count aggressively based on average utilization rather than overlapping peaks. When multiple services spike together, performance degrades across the board.

Alibaba Cloud implementation: Use Shared GPU Scheduling in Container Service for Kubernetes (ACK) to share one GPU across multiple services. The cGPU component isolates both memory and compute. Note that shared GPU scheduling and topology-aware scheduling for distributed training acceleration are separate capabilities. On the PAI side, use a dedicated resource group in PAI-EAS with GPU slicing to build a unified inference pool, and use a Virtual Resource Group for mixed-resource scheduling and priority management.

BP-COST-05 Engineer prompts and output budgets

Best practice: Treat prompts and output length as measurable cost drivers. Use systematic trimming, structured output constraints, and thinking-budget controls to minimize per-request token usage without sacrificing quality.

Key points: The system prompt is a fixed overhead in every request. Remove redundant descriptions and repeated instructions. Replace long natural-language sentences with concise, structured formats. Move low-frequency rules into conditional branches.

Important: Evaluate the benefit of trimming the system prompt together with context caching. Model Studio enables implicit caching by default for supported models. A cached input prefix is billed at a lower percentage of the input unit price (see BP-COST-06). The system prompt sits at the prefix and is the easiest part to cache. When the cache hits steadily, the cost saved by trimming the system prompt is only a small fraction of what you would save at full price. Use the right order: first ensure a long, stable prefix hits the cache, then trim the parts not covered by the cache. Doing this in reverse leads to overstated savings.

Output-side optimization often saves more because output is several times more expensive than input and is not affected by input caching. Require structured output (JSON, lists) to reduce verbose explanations. For reasoning models, control thinking mode by task type. Disable deep thinking for simple tasks. When you must enable it, use thinking-budget parameters to cap chain-of-thought length. Because thinking tokens can account for most output, this is the most effective single control on the output side. Other strategies include summarizing long documents before sending them to the model, compressing chat history with a sliding window, and maintaining a library of scenario-specific compact templates. These also help you avoid crossing tiered pricing boundaries. Anti-pattern: Report savings by multiplying reduced tokens by the full unit price without validating cache hits. This can significantly overstate savings. Base calculations on billed tokens from usage reports.

Alibaba Cloud implementation: Use Prompt Management in Model Studio to build a versioned template library. Use deep-thinking parameters to control chain-of-thought spending. Use the Model Usage report to monitor billed performance for each template by workspace, API key, and model.

BP-COST-06 Reuse with context and semantic caches

Best practice: Build a two-layer cache system. Use the platform-native context cache to reuse computation for request prefixes, and deploy an application-layer semantic cache to intercept semantically equivalent repeated requests.

Key points: The two caches work differently and offer different levels of potential savings. For maximum effect, use them in combination.

Layer 1 is the platform context cache. It reuses the shared request prefix and reduces input-side cost. Implicit caching is enabled by default for supported models and requires no changes. The cached portions are billed at a fraction of the input unit price (20% for most models, and lower for some). Explicit caching requires you to mark content for caching in the request. The billing rate for cache hits is lower (typically 10% of the input unit price), but cache creation is billed at 125% of the input unit price, and there are limits on TTL and rewind range. Context caching is prefix-based. Place stable content such as the system prompt and knowledge snippets at the start of the message sequence, and keep tool definitions serialized in exactly the same structure across requests. For critical paths that need predictable cost and latency, use explicit caching. For most use cases, implicit caching is sufficient.

Layer 2 is the application-layer semantic cache. It returns a historical result directly and avoids the entire call’s input and output cost, offering greater potential savings. Exact-match caching has low hit rates for natural language because users phrase the same intent in many ways. A semantic cache encodes the request text into a vector and matches by similarity (a cosine similarity threshold is often set to 0.95 or higher). Your invalidation strategy must cover three dimensions: time, with a TTL to prevent stale answers, and shorter TTLs for time-sensitive data such as markets and weather; knowledge updates, by actively clearing related entries when the underlying knowledge base changes; and quality feedback, by removing low-quality entries based on negative user signals. Hit rates vary widely by scenario: FAQ queries are the most concentrated and have the highest hit rates; general customer service sits in the middle; open-ended creative writing needs unique output and is not a good fit. Before launching, evaluate the hit rate offline using historical request logs. Architecturally, place the cache in front of the model service as an interceptor. Use the request vector as the key, and store the output and metadata (timestamp, model version, confidence) as the value. Continuously monitor hit rate and user satisfaction. Anti-pattern: Use a semantic cache for scenarios that require output diversity or include private user context. This can cause irrelevant answers or cross-user data leakage. Include tenant and permission dimensions in the cache key.

Alibaba Cloud implementation: Context caching is provided natively by Model Studio. Implicit caching works without changes. Enable explicit caching through request parameters. For an application-layer semantic cache, use the TairVector vector data structure in Tair (Redis-compatible). It supports an HNSW index and key-level TTL. Combine it with Model Studio text-embedding models for request vectorization.

BP-COST-07 Use batch and asynchronous inference

Best practice: For non-real-time scenarios, use batch inference and asynchronous inference. Prefer platform batch inference discounts, and combine them with off-peak scheduling and spot instance options to reduce cost further.

Key points: Many inference workloads do not need real-time responses. Typical scenarios include batch document processing (contract analysis, resume screening, content moderation), offline data pipelines (log analysis, profile updates), scheduled report generation, and model evaluation and data labeling. For Model Studio API calls, batch inference provides the most direct savings. Submit a set of requests as a JSONL file and process it asynchronously. Billing is 50% of real-time pricing. You can set the maximum wait time from 1 to 14 days. If the job does not finish by the deadline, it is terminated, and only successfully executed requests are billed. Move evaluation, labeling, and document batch processing to this API to halve the cost of those tasks without changing models or prompts. Note that if you enable deep thinking, chain-of-thought tokens are still billed at the output price.

For self-hosted inference, the benefits come from three areas: spot instance pricing is much lower than pay-as-you-go, and while reclamation is possible, batch jobs usually support checkpointing and resuming; scheduling non-urgent jobs in low-demand, off-peak windows for better resource availability; batching merges many small requests into one batch to increase GPU throughput and reduce communication and scheduling overhead. The core asynchronous design is simple: the frontend submits requests to a message queue, inference workers consume and process the queue, and on completion, they notify via callback or write results to storage for downstream pull. This design supports traffic smoothing, retry and fault tolerance, and horizontal scaling. Anti-pattern: Convert end-user interactive paths to batch inference just to get discounts. Its wait window is measured in days and fits only latency-insensitive workloads.

Alibaba Cloud implementation: Call the Model Studio batch inference API for offline tasks and get discounted pricing. For self-hosted inference, use the LLM batch inference capability of PAI-EAS (which provides an OpenAI-compatible Batch API through LLM intelligent routing). For AIGC and long-running near-real-time scenarios such as audio and video processing, use the asynchronous inference service and queue service in PAI-EAS. Function Compute (FC) 3.0 supports asynchronous invocation and delayed invocation. You can deliver asynchronous results to Message Service, EventBridge, or ApsaraMQ for RocketMQ as the decoupling layer.

BP-COST-08 Improve cost predictability with commitments

Best practice: Based on historical usage and growth forecasts, lock in discounts through Savings Plans or a resource plan. Establish usage monitoring and capacity planning to ensure your commitment aligns with actual consumption.

Key points: Model Studio provides an AI general-purpose Savings Plans, a discount option designed for pay-as-you-go usage of large language models (LLMs). You commit to a monthly spending amount for a term (3, 6, 12, or 24 months). You keep pay-as-you-go flexibility while receiving tiered discounts, which lowers model usage costs.

Note that Model Studio also offers a Token Plan subscription for AI coding and agent tools. It uses credits with fixed-period allowances and pauses calls when the allowance is exhausted. It is suitable for developer tools and is not the same cost-optimization model as enterprise API usage. Do not confuse these options when making a selection.

Alibaba Cloud implementation: Purchase Savings Plans or a resource plan in the Model Studio console. Use the Model Usage report to monitor spending progress and remaining quota by workspace, API key, and model. Use Budget Management in Billing Management to set threshold alerts (based on actual or forecast values).

BP-COST-09 Use autoscaling to pay as you go

Best practice: Configure multi-layer elasticity based on inference traffic patterns. Minimize idle capacity while meeting your SLOs. For development environments, scale to zero when there is no traffic to eliminate spending on idle resources.

Key points: AI inference traffic is often periodic and bursty. Fixed sizing either overprovisions to peak and stays idle most of the time, or sizes to average and degrades at peak. Use three layers of elasticity. First, scale to zero: for dev/test and low-frequency services, scale replicas down to zero after a period with no requests. The trade-off is a cold start. The first request can take tens of seconds to minutes; mitigate it with pre-warming. Second, predictive scaling: predict load based on historical patterns and scale out before peak. Use it for production services with regular traffic to avoid lag from reactive scaling. Third, reactive scaling with a cooldown period: trigger scaling based on real-time metrics (GPU utilization, queue depth, response latency) and set a reasonable cooldown period to avoid thrashing. Anti-pattern: Enable scale to zero for customer-facing real-time paths. When traffic returns, the initial requests will experience cold start latency. Keep a minimum always-on replica for customer-facing services.

Alibaba Cloud implementation: PAI-EAS provides horizontal autoscaling and scheduled autoscaling. It supports setting the minimum instance count to 0 to scale to zero (services with a dedicated gateway do not support this). Trigger metrics include QPS, CPU, GPU utilization, queue backlog (for asynchronous services), and custom metrics. Function Compute (FC) is billed per request and scales automatically. Its GPU instances can handle near-real-time inference and provide shallow-sleep instances to reduce reservation cost. ACK and ACK Serverless can deploy Advanced Horizontal Pod Autoscaler (AHPA), which uses historical Prometheus metrics for predictive scaling. Keep at least 7 days of history.

BP-COST-10 Manage the training data lifecycle

Best practice: Classify training-related data into four tiers—hot, warm, cold, and delete—based on access frequency and business value. Automate storage tiering and cleanup. Minimize storage cost while meeting compliance requirements.

Key points: The hot tier includes active fine-tuning training sets, active evaluation benchmarks, and in-progress training checkpoints. This tier requires high-throughput, low-latency access, so use performance or advanced NAS, Extreme NAS, or local SSD. For large-scale intelligent computing training, use Cloud Paralleled File System (CPFS). The warm tier includes recently completed experiment data, historical evaluation results, and the original training data of online models (for reproducibility and audit). Use Object Storage Service (OSS) standard storage. The cold tier includes long-unused historical data, checkpoints of retired models, and early experimental datasets that you must retain for compliance or reproducibility. Use OSS infrequent access, archive, cold archive, or deep cold archive storage. The delete tier includes data that must be permanently removed based on retention policies, such as personal data beyond the retention period (Personal Information Protection Law (PIPL) compliance), intermediate artifacts from failed experiments with no reuse value, and old datasets fully replaced by new versions.

When you choose a cold storage type, include hidden costs in your estimate. Comparing only storage unit price understates total spend. Using pay-as-you-go locally redundant storage in the Chinese mainland as a reference: standard storage is about CNY 0.12 per GB-month, infrequent access about CNY 0.08 (about two-thirds of standard), archive about CNY 0.033, cold archive about CNY 0.015, and deep cold archive about CNY 0.0075. Also consider minimum storage durations (30 days for infrequent access, 60 days for archive, and 180 days for cold archive and deep cold archive). Early deletion requires paying the difference. Data retrieval incurs retrieval fees. Cold archive and deep cold archive require unfreezing before reads. Use infrequent access for warm/cold data that is still read occasionally. Use archive and lower-cost classes only for data you are confident you will not access for a long time but must retain. Anti-pattern: Move data that you still need in the near term into archive or cold archive based only on unit price. Retrieval fees and minimum-duration charges can increase total cost.

Alibaba Cloud implementation: Use Lifecycle Rules in OSS to automate storage class transitions and expiration deletion. Rules support matching by prefix or tags. You can transition to infrequent access, archive, cold archive, and deep cold archive. You can also delete expired objects, incomplete multipart uploads, historical versions, and delete markers. In data lake scenarios, use Data Lake Formation (DLF) for centralized metadata and access policy management, and combine it with DataWorks to implement data quality and lifecycle processes.

BP-COST-11 Use data tiering for vector storage

Best practice: Implement data tiering based on query frequency. Keep hot vectors in memory indexes for performance, move low-frequency vectors to disk indexes or compressed storage, and reduce overall scale by lowering embedding dimensions.

Key points: vector database cost includes storage cost and compute cost (index build and query). In-memory indexes (such as HNSW) offer the best query latency, but memory is much more expensive than disk. For knowledge bases with tens of millions to hundreds of millions of vectors, an all-in-memory index is unsustainable. Before tiering, shrink the vectors themselves. text-embedding-v4 supports outputs from 2,048 to 64 dimensions. Dimension reduction cuts storage and retrieval cost proportionally. Validate recall quality across dimensions on your business retrieval evaluation set and choose the lowest dimension that meets your quality threshold. This step often saves more than later tiering work. For tiering strategy, keep the hot tier in an in-memory index for frequently accessed knowledge chunks and core business knowledge (product manuals, policy documents). Use a disk index for the warm tier to store low-frequency but still-needed historical embeddings. Latency increases, but unit cost drops significantly. For the cold tier, archive offline in a compressed format. Load on demand when needed. Access latency is in seconds, but storage cost is lowest. Track access frequency by recording it for each vector ID at the retrieval gateway layer, and migrate on a schedule. Anti-pattern: Assume the storage engine handles tiering automatically. Most vector stores do not currently provide automatic, frequency-based tiering. You must implement heat statistics and migration orchestration in the application layer.

Alibaba Cloud implementation: DashVector provides a fully managed vector search service with serverless autoscaling. AnalyticDB for PostgreSQL provides vector analytics and supports data tiering at the product level, but tiering requires an explicit call to storage transition functions (you can run scheduled jobs by partition). It does not migrate automatically based on access heat. Note that after you move a hot table to a cold table, primary keys, indexes, and sequences are removed. Include an indexing strategy in your migration design. Alibaba Cloud Elasticsearch can use Index Lifecycle Management (ILM) for hot/cold separation, or use the OpenStore engine for storage-compute separation to store massive historical data at lower cost while keeping it searchable.

Use a five-layer governance architecture

Layer 1: Usage governance at the call layer. This delivers the most direct savings and requires no infrastructure changes. Use model routing by task complexity, rely on context caching to reuse stable prefixes, move evaluation and document batch processing to batch inference for a 50% discount, control thinking mode and set thinking budgets by task, and cap context length to avoid crossing tiered pricing thresholds.

Layer 2: Discount with commitment. After your usage baseline stabilizes, use Savings Plans or a resource plan to lock in discounts. Put only the stable baseline into the commitment, and keep the variable portion on pay-as-you-go, because unused quota resets each cycle. The core value is to convert uncertain consumption spending into budgetable planned spending.

Layer 3: Elastic compute and spot instance usage. For self-hosted online inference, configure the three-layer elasticity strategy. For offline batch jobs, prefer spot GPU instances.

Layer 4: GPU pooling. Use shared GPU scheduling in ACK and cGPU isolation to run multiple services on one GPU, or use a dedicated resource group in PAI-EAS with GPU slicing. Combine it with vLLM or BladeLLM to increase per-GPU throughput.

Layer 5: Storage tiering and observability and governance. Use OSS Lifecycle Rules to automate storage class transitions and expiration deletion. Reduce vector dimensions and implement hot/warm/cold tiering for vector data. Use Model Usage and Application Monitoring in Model Studio for model-side usage, and use Cost Management in Billing Management for cost analysis, cost allocation, and budget alerts. Enable cost anomaly detection to catch abnormal growth and idle resources.

Estimate costs for a typical mid-size enterprise

Consider a mid-size enterprise with 1 million calls per month as a baseline scenario. It mainly calls the Model Studio API and does not self-host an inference cluster. Assumptions: three applications—intelligent customer service, document processing, and internal knowledge base Q&A—with a call volume distribution of 60%, 25%, and 15%. Each call uses an average of 2,000 input tokens and 500 output tokens. The unoptimized baseline uses qwen3-max (in the under-32K tier) for all calls.

Baseline cost: model inference input is CNY 5,000, output is CNY 5,000, total about CNY 10,000 per month. Vector search is about CNY 4,000 per month. Object Storage Service (OSS) is about CNY 300 per month. Evaluation and experimental compute is about CNY 1,500 per month. Baseline total is about CNY 15,800 per month, where model inference accounts for about 63%, making it the largest cost item.

Optimize inference cost in this order. Step 1, model routing: route 70% of requests to qwen-plus and keep 30% on qwen3-max. Input is CNY 2,620, output is CNY 2,200, subtotal about CNY 4,820. Step 2, context caching: assume about 60% of input is a stable prefix, and about 80% of that prefix hits the implicit cache, with cached portions billed at 20% of the input unit price. The effective input billing factor is about 0.62, dropping input to about CNY 1,614 and subtotal to about CNY 3,814. Step 3, batch inference: move document processing (about 25%) to batch inference with a 50% discount, reducing the subtotal to about CNY 3,337. Step 4, committed use: apply Savings Plans to the stable baseline. Using a conservative 10% discount estimate, inference cost drops to about CNY 3,000 per month, about 70% lower than the baseline. Check actual discounts in the console based on your commitment tier.

Other cost items: reduce vector search to about CNY 2,000 per month through dimension reduction and tiering; reduce OSS to about CNY 150 per month through lifecycle rules; reduce evaluation compute to about CNY 750 per month through batch inference. The optimized total is about CNY 5,900 per month, a total savings rate of about 63%.

These numbers are only an estimation framework. Actual cost varies significantly by use case, traffic patterns, quality targets, and technical choices. If you self-host an inference cluster, always-on GPU cost becomes the dominant cost, and the cost structure and optimization focus change. The value of this framework lies in how it defines cost analysis, accounting scope, and the optimization sequence.

Establish organizational mechanisms for cost governance

Cost optimization is not only a technical problem. It is also an organizational governance problem. Establish four mechanisms: form a virtual AI FinOps team where architects, finance, and business stakeholders review spending trends and optimization opportunities on a regular basis; implement cost allocation and attribute inference cost to each business line by call volume to provide visibility to business owners; define unit economics by setting a per-interaction cost metric for each application and an optimization target; and run a monthly cost audit to identify abnormal growth, idle resources, and optimization opportunities.

The goal of the cost pillar is not to minimize spending. It is to maximize the business return on every dollar spent. A strong AI architecture strikes a dynamic balance across cost, performance, and quality. Do not sacrifice user experience for savings, and do not let costs run away in pursuit of marginal quality gains.