AI Center in EMR Serverless Spark lets you integrate Large Language Model (LLM) capabilities into large-scale data processing using SQL. Its two core features—AI Function and model service—embed AI inference directly into your data workflows without custom code.
AI Center has been generally available since April 27, 2026 and is now a paid service. For details, see the EMR Serverless Spark AI Center general availability.
Activate AI Center
Prerequisites
An Alibaba Cloud account, or a RAM user with the AliyunRAMFullAccess permission.
Procedure
-
Log on to the E-MapReduce console.
-
In the left-side navigation pane, choose EMR Serverless > Spark.
-
On the Spark page, click the target workspace name.
-
In the left-side navigation pane, choose AI > Serving.
-
In the upper-right corner, click Enable AI Center.
Billing
Built-in model invocations in AI Center use pay-as-you-go billing. Charges are based on actual token consumption of built-in models in your workspace. For detailed billing rules, see Model calls (pay-as-you-go).
Key benefits
-
SQL-based AI, no code required
Built-in functions such as
ai_query, sentiment analysis, and vectorization let developers call LLMs directly from SQL. Complex AI inference embeds into existing ETL flows with no Python, Java, or SDK code required. -
Unified model registration
By default, AI Function calls Alibaba Cloud Qwen3.5-Plus for out-of-the-box inference. You can also register models from Model Studio, PAI-EAS, or your own private endpoints in one click. A unified access layer abstracts protocol and authentication differences—register once, query from any SQL job.
-
In-place processing, end-to-end workflow
Vector generation and batch inference run in place—no data movement required. The full loop of data cleaning, AI feature engineering, and result write-back stays within one system, reducing cross-system transfer costs and simplifying compliance.
Use cases
Content understanding
-
Sentiment and feedback analysis: Classify sentiment in comments and categorize user feedback at scale.
-
Intelligent ticket routing: Categorize tickets as complaints, inquiries, or suggestions based on content and route them to the right department.
-
Cross-border business support: Translate content end-to-end across languages to generate multilingual reports or adapt marketing copy for overseas markets.
-
General summary generation: Generate batch summaries of long documents and extract key information with Qwen.
Structured data extraction
-
Key information extraction: Extract predefined fields—such as party A's name, amount, and date—from contracts, logs, or comments and output them as JSON.
-
Automatic copy polishing: Correct syntax errors and improper wording before data export or report generation.
-
Unstructured to structured conversion: Convert natural-language business rules into structured data records for SQL analysis.
Semantic retrieval and RAG
-
Building RAG vector databases: Chunk document sets and convert them to semantic embeddings for enterprise knowledge bases.
-
Intelligent Q&A matching: Match user questions to knowledge-base entries by semantic similarity for accurate auto-replies.
-
Intelligent data deduplication: Identify semantically duplicate records—such as similar news articles or customer service entries—to cleanse datasets.
Data security and compliance
-
Data masking: Scan text to identify sensitive personal information—names, ID numbers, phone numbers, bank card numbers—and mask or replace it.
-
Privacy compliance auditing: Audit historical data assets in batches to find unmasked sensitive fields.
SQL development efficiency
-
Smooth job migration: Convert HiveQL to Spark SQL, resolving UDF adaptation and window function differences.
-
Query performance tuning: Analyze SQL logic and suggest optimizations for predicate pushdown, bucketing, and join strategies.
-
Code commenting and suggestions: Generate comments for complex SQL scripts to help developers understand legacy code.