Databases are the foundations of all the enterprise business. Employees such as R&D engineers, O&M staff, and testers need to query or manage database data every day. However, employees have different levels of knowledge on databases. Therefore, enterprises face a large number of challenges in database stability.

Challenges in database O&M and management

  • Frequent database failures during fast business iteration
    • During the rollout of business, a large number of slow SQL queries are generated.
    • During promotions, the capacity of databases does not meet the business requirements.
    • Schemas or indexes that are improperly designed hinder databases from performing well.
    • SQL statements or tables are published to an online environment without being reviewed.
  • Difficulties in troubleshooting and performance optimization

    Database troubleshooting and performance optimization are long-standing challenges in the database field. Skilled database administrators (DBAs) may still take a long time to troubleshoot a fault and then fail to identify root causes. These challenges are due to the following reasons:

    • Useful data is difficult to obtain. You can diagnose issues and optimize performance only based on large amounts of system data or historical data. DBAs can deliver effective solutions only when complete data is available.
    • Effective solutions are difficult to find. Only DBAs who have years of experience can find effective solutions. DBAs must also be familiar with various troubleshooting scenarios. However, due to the complexity and fast development of database technologies, DBAs cannot gain enough experience. DBAs must proactively obtain more experience and become familiar with more troubleshooting scenarios on their own. However, experience cannot be obtained from teaching and the experience that applies to others may not apply to your scenarios.
    • Databases are difficult to be optimized. After DBAs work out a solution to an issue, they may find that the solution is difficult to implement. For example, if DBAs need to tune a database engine, they may take a long time to optimize code.
  • High O&M costs
    Cloud technologies allow enterprises to use different types of databases based on business types or deploy databases in different environments. This brings the following challenges to database O&M:
    • DBAs who are proficient in various types of databases are scarce.
    • DBAs must develop advanced skills to manage various types of databases across environments.
    • Experience in database O&M is difficult to obtain and teach.
  • High-security risks
    When data value increases, enterprises are faced with internal and external security risks. The following security issues are prone to occur:
    • Unauthorized access to databases and unpredictable or inappropriate use of databases
    • Data breaches
    • Data corruption
    • Hacker attacks
    • Data exceptions that are caused by software or hardware bugs
    • Data loss that is caused by operational faults

Solution

Database Autonomy Service (DAS) is a cloud service that uses machine learning and expert experience to automate perception, healing, optimization, O&M, and security assurance for databases. DAS makes database management simple and avoids service failures that are caused by manual operations. DAS enables a stable, secure, and efficient database service.

DAS has been used to manage all the databases of Alibaba Group for more than two years. By April 2020, DAS has improved database performance in the following aspects:

  • More than 40 million SQL statements are automatically optimized.
  • Four petabytes of storage are automatically reclaimed.
  • Memory is optimized and 20% of storage is saved.