New Release: Alibaba Cloud DAS – Reduce Database Management Costs by 90%

This article discusses the capabilities of Alibaba Cloud DAS and how it automates database administration and optimization and helps to reduce database management costs by 90%.

By ApsaraDB

Introduction

On 22 April, Alibaba Cloud released a new database product Database Autonomy Service (DAS). DAS is jointly developed by Alibaba Cloud and the Alibaba DAMO Academy (an Alibaba division dedicated to exploring the unknown through scientific and technological research and innovation). It provides comprehensive database management capabilities including self-awareness, auto-recovery, auto-optimization, and auto-safeguard. DAS enables enterprises to use databases without human interaction. It provides autopilot-like experience and reduces database management costs by 90%.

Migrating databases to the cloud has become an industry consensus. Gartner predicted that by 2023, 3/4th of the world's databases will be running on the cloud. However, managing cloud databases using conventional approaches requires significant manpower, and exceptions cannot be detected or handled in a timely manner. Therefore, enterprises are in dire need of management solutions tailored for cloud databases.

Alibaba Cloud Database Autonomy Service (DAS) is developed based on Alibaba's extensive experience in operating and maintaining (O&M) large-scale databases for over a decade. DAS integrates AI technologies developed by the DAMO Academy, enabling enterprises to manage databases in an autopilot mode. It provides comprehensive database management capabilities such as 24x7 exception detection, root cause analysis, fault recovery, fault tracking, and fault evaluation, to ensure continuous, stable, and efficient operation of databases. The research achievements in DAS have been recognized for two consecutive years by the Very Large Data Bases (VLDB) Conference, a premier forum in the database field.

Compared with conventional database management models, DAS has six unique features.

Six Core Autonomy Features

24x7 real-time exception detection: DAS detects exceptions based on database workloads in real-time by using machine learning algorithms. Compared with fault-driven threshold-based alerts, this approach detects database exceptions almost immediately after their occurrence.
Auto-recovery from exceptions: After detecting an exception, DAS automatically performs root cause analysis and implements any required damage control, repair, or optimization operations to facilitate database auto-recovery, thereby reducing the impact on enterprise services.
Auto-optimization: DAS constantly performs SQL review and optimization on the database based on global workload and real service scenarios instead of individual SQL statements to continuously protect your database like an always-on database administrator (DBA).
Intelligent parameter tuning: A database has hundreds of parameters and various user service scenarios, rendering it impossible to manually tune the parameters to optimal configurations. The DAS team developed the intelligent parameter tuning feature in cooperation with the DAMO Academy. This feature combines artificial intelligence (AI) technologies with intelligent stress testing to automatically recommend an optimal parameter template for each database instance.
AutoScale: DAS automatically calculates and predicts the service model and capacity level of databases based on machine learning to achieve proactive auto-scaling.
Intelligent stress testing: DAS provides you with custom stress testing services. It is able to automatically learn the service model of a database, generate workloads by simulating real services as required, and provide various and tailored testing scenarios. DAS helps you solve database management problems such as challenges with major promotions and database model selection.

Four Core Innovations and Breakthroughs

World's first comprehensive database autonomy engine: DAS implements comprehensive database autonomy in various scenarios through centralized decision-making, conflict settlement, as well as decision-making and decision distribution in special autonomy scenarios, based on root cause analysis and aggregated instance information.
World's first external cost-based SQL diagnosis engine: Driven by a cost-based diagnosis engine, DAS uses a set of database-independent optimizers to implement precise SQL diagnosis and provide optimization suggestions through execution plan-based evaluation under an adaptive statistical information collection mechanism.
SQL optimization technology based on global workload: DAS performs overall optimization based on the global workload of the database by comprehensively considering database workload metrics that may affect the overall database performance (such as SQL execution resource usage and read/write ratio) to minimize storage space consumption and maximize the global database performance.
Workload exception detection and prediction based on machine learning: The AI-based workload exception detection feature of DAS automatically detects database exceptions and triggers global optimization upon the occurrence of these exceptions. It optimizes databases proactively and globally in real-time.

Alibaba Cloud DAS has been widely used in Alibaba's internal business scenarios. It has optimized more than 42 million SQL statements, reclaimed more than 4 PB of space, and served a large number of enterprises in industries such as e-commerce, finance, and gaming.

Take Douyu, a leading game live-streaming platform in China, as an example. Douyu has to process about one hundred billion service calls daily. Its database cluster has more than 1,000 databases. Conventional database management solutions cannot cope with such a large scale of database traffic.

"When migrating Douyu's services to the cloud, DAS helped us achieve unified database management in on-premises and cloud environments," said Ding Tieqiu, Director of Douyu's Infrastructure Department. "The intelligent diagnosis and autonomy services provided by DAS have improved the cost-effectiveness of activities such as routine operations, major promotions, and important events; lowered the thresholds for database protection; and given us a clear picture of the global operation status during activities. It also reduces the O&M costs of our database cluster. This allows us to invest more in business innovation and to provide better gaming and entertainment experience for our users."

"DAS shall usher in an 'autopilot' era of databases to make them more intelligent, stable, secure, and cost-efficient. It helps users focus on business innovation, and 'push' enterprises to the fast lane of business development," said Li Guangsheng, a senior expert from Alibaba Cloud, "Within the next three years, 'autopilot' will be enabled for 80% of databases in the cloud."

Multiple research achievements made by the DAS team in cooperation with the DAMO Academy have been recognized by international academia:

Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Application, WWW, 2018
iBTune: Individualized Buffer Tuning for Large-scale Cloud Databases, VLDB, 2019
Diagnosing Root Causes of Intermittent Slow Queries in Large-Scale Cloud Databases, VLDB, 2020

As the world's third and Asia's largest cloud service provider, Alibaba Cloud has a comprehensive database product portfolio. Currently, over 400,000 database instances are running on Alibaba Cloud, including those of leading enterprises in various industries such as retail, finance, e-government, telecommunications, manufacturing, and logistics.

Ready to give it a try? Visit Database Autonomy Service (DAS) to learn more!

Community

New Release: Alibaba Cloud DAS – Reduce Database Management Costs by 90%

Introduction

Six Core Autonomy Features

Four Core Innovations and Breakthroughs

Read previous post:

Read next post:

Alibaba Cloud New Products

You may also like

Comments

Alibaba Cloud New Products

Related Products

Bastionhost

Database Overview

Managed Service for Grafana

AIRec