Alibaba Cloud ApsaraDB Data Agent Achieves Dual Breakthroughs in Performance and Experience

This article introduces how Alibaba Cloud ApsaraDB Data Agent achieved dual breakthroughs in performance and user experience for AI-powered data analytics.

In many enterprises, data is not scarce. What is truly scarce is—data insights that can be quickly understood, trusted, and truly used in decision-making. We have been thinking: What would it be like if obtaining data value no longer required complex switching between tools, high professional thresholds, and long waits?

This is exactly the starting point for us to build the Data Agent.

1. Performance Validation: No. 1 on the BIRD-CRITIC Leaderboard

In the globally authoritative SQL evaluation benchmark BIRD-CRITIC, Alibaba Cloud ApsaraDB Data Agent topped the list with its excellent performance. The BIRD-CRITIC leaderboard aims to validate "whether LLMs can solve database problems in real-world application scenarios." The overall difficulty of this benchmark is significantly higher than that of traditional natural language to SQL (NL2SQL) testing. The top ranking of ApsaraDB Data Agent marks that its generalization capabilities in complex scenarios have reached a world-leading level!

Why choose BIRD-CRITIC?

BIRD-CRITIC is currently one of the most challenging benchmarks in the global SQL Diagnosis realm. It breaks the limitation of traditional natural language to SQL focusing only on "statement generation," diving deep into real O&M core scenarios such as query repair, DDL Change security, and Performance Optimization. This benchmark not only spans four major mainstream dialects such as MySQL and Oracle, but also implements nearly harsh column-level matching standards in the evaluation. This high-difficulty setting truly maps the business challenges that Data Agent must face—adhering to strict execution logic in complex scenarios.

Core Engine: Technical Accumulation of DMS

The ability of Data Agent to achieve excellent results is inseparable from the technical accumulation of Alibaba Cloud DMS in the database realm. We transformed the practical experience of DMS in multi-dialect syntax rules, Performance Optimization patterns, and data governance into a knowledge base available to the agent through engineering means. This enables the agent to be more accurate and standardized when detailed issues such as special paging in Oracle or implicit transformation in MySQL are handled.

Real Architecture: Multi-Agent Collaboration

In terms of architecture design, we followed the Multi-Agent collaboration mechanism in the Data Agent production environment:

• Intent Planning Agent (Coordinator): It is mainly responsible for parsing vague requirements, utilizing metadata capabilities to detect data distribution, and assisting in resolving business ambiguities.

• Execution Validation Agent (Critic): It generates SQL based on the planning, performs determinism verification (Determinism Check) and security evaluation, and ensures the reliability of the execution procedure.

This closed-loop flow of "planning-execution-validation" not only verified its Validity in Testing but also serves as the foundational paradigm for us to handle complex Data Jobs.

For Users, the meaning behind this is only one: It is not a Demo, but a data agent that can be safely entrusted with real business!

It can simultaneously cover traditional BI Analysis (descriptive, diagnostic) and Advanced Analysis (predictive, prescriptive). It can serve as a conversion tool from natural language to SQL query (NL2SQL), and also generate chat-style BI (including ChatBI) for predefined reports. It is an autonomous intelligent System capable of understanding analysis intents, planning analysis paths, executing complex Jobs, and generating deep Insights, and can stably complete complex, multi-step data analytics Jobs.

2. Experience Authentication: Won National Innovation Awards and Selected for CCF Class A International Conference

Besides Performance, the product experience of Data Agent has also obtained authoritative industry recognition. Data Agent won the China Design Intelligence Award (DIA). DIA is one of the most influential international innovation design awards in China, with Strict review and covering multi-dimensional values. It is regarded as an important indicator of measuring product innovation strength and international competitiveness.

Meanwhile, centering on the practical achievements of Data Agent in the transparency of the Analysis procedure and the human-computer collaboration experience, it was successfully accepted by the CCF Class A international conference CSCW 2025. This further demonstrates the engineering depth and industry attention regarding Data Agent in terms of complex interactions and experience trustworthiness.

https://dl.acm.org/doi/10.1145/3715070.3749256

These authoritative endorsements validate one point:

When AI undertakes more complex data analytics Jobs, the experience itself has become an important component of product competitiveness.

3. Data Security: "Three-in-one" End-to-end Security Assurance

DMS Data Agent constructs a "identity-environment-control" three-in-one security system:

• Resource Access Management: Through "security hosting," the Account and Password are not exposed. The System Supports fine-grained permissions, automatic masking, and full-process auditing.
• Environment fencing: The System adopts kernel-level sandboxes and VPC closed-loop fencing to ensure that Data interaction is completed within the closed loop, blocking external network threats.
• Control security: Tenant-level session fencing is implemented. When the Job Ends, the environment is destroyed and the Data is purged to eliminate Data residue.

This solution realizes the end-to-end security assurance of "Account non-exposure, full environment fencing, auditable operations, and no Data residue."

4. Data-for-All: Data Analytics Accessible to Everyone, Starting from the Job

We always adhere to a judgment: Not all Analysis Jobs require the same procedure rendering method.

Therefore, Data Agent adopts a Job-driven transparency policy. It automatically selects the most suitable experience method based on the Job complexity.

Simple job: Low transparency, efficiency first

In daily data querying and rapid judgment scenarios, Data Agent directly provides the Result and Generates interactive charts.

The User does not need to be interrupted by the execution procedure. They can use the shortest path to obtain usable answers.

Complex Job: High transparency, boosting cognition

In complex scenarios such as business reviews and policy analysis, Data Agent first Generates an Analysis plan, executes the Analysis in steps, and finally produces a structured web Insight report.

The User can not only see the conclusion but also understand "how this conclusion is derived." The entire Analysis procedure is clearly visible, controllable, and credible. Key steps can be backtracked to ensure the Result is both reliable and transparent.

5. Human in the Loop: Human-Intelligence Collaboration, Accurate Analysis

During the AI agent-based Analysis procedure, the User is always in the decision-making loop.

The Analysis procedure is clear and traceable. When critical or high-risk operations are involved, the System actively confirms them. The User can adjust or supplement the Analysis direction at any time.

AI does not override judgment. Instead, it becomes a collaborative and correctable Analysis partner.

6. Conclusion

We firmly believe that Data Intelligence that truly delves into business is by no means a simple "intelligence stacking." Instead, it represents extreme efficiency in daily operations and transparent credibility in key links. From topping the BIRD-CRITIC list to gaining dual recognition from DIA and CCF Class A top conferences, Data Agent has achieved dual breakthroughs in Performance and experience. This is also an important step for us towards "data analytics accessible to everyone."

Learn more about product details:

🔗 https://www.alibabacloud.com/help/dms/data-agent-for-analytics

Community

Alibaba Cloud ApsaraDB Data Agent Achieves Dual Breakthroughs in Performance and Experience

1. Performance Validation: No. 1 on the BIRD-CRITIC Leaderboard

Why choose BIRD-CRITIC?

Core Engine: Technical Accumulation of DMS

Real Architecture: Multi-Agent Collaboration

2. Experience Authentication: Won National Innovation Awards and Selected for CCF Class A International Conference

3. Data Security: "Three-in-one" End-to-end Security Assurance

4. Data-for-All: Data Analytics Accessible to Everyone, Starting from the Job

5. Human in the Loop: Human-Intelligence Collaboration, Accurate Analysis

6. Conclusion

Read previous post:

ApsaraDB

You may also like

Comments

ApsaraDB

Related Products

Container Compute Service (ACS)

Container Service for Kubernetes

ApsaraDB RDS for SQL Server

ApsaraDB for HBase