Databases are the foundations of all the enterprise business. Employees such as R&D engineers, operations and maintenance (O&M) staff, and testers need to query or manage database data every day. However, employees have different levels of knowledge on databases. Therefore, enterprises face many challenges in ensuring database stability.

Challenges in database O&M and management

  • Frequent database faults during fast business iteration
    • While business data is being published, a large number of slow SQL queries are generated.
    • During promotions, the capacity of databases does not meet the business demand.
    • Schemas or indexes that are poorly designed hinder databases from performing well.
    • SQL statements or tables are published to an online environment without being reviewed.
  • Difficulties in troubleshooting and performance optimization

    Database troubleshooting and performance optimization are long-standing challenges in the database field. Even the most skilled database administrators (DBAs) may take a long time to troubleshoot a fault and still fail to identify root causes. These challenges are due to the following reasons:

    • Useful data is difficult to obtain. Both cause identification and performance optimization depend on a large amount of system data or historical data. DBAs can deliver effective solutions only when complete data is available.
    • Effective solutions are difficult to find. Generally, only DBAs who have many years of experience can find effective solutions. DBAs must also be familiar with various troubleshooting scenarios. However, due to the complexity and fast development of database technologies, DBAs cannot gain enough experience. They must also proactively obtain more experience and become familiar with more troubleshooting scenarios on their own. This is because experience cannot be obtained from teaching and the experience shared by others cannot be directly copied.
    • Optimization is difficult to carry out. After DBAs work out a solution to an issue, they may find the solution difficult to implement. For example, it takes a long time for DBAs to optimize code in a database engine so that an issue can be solved.
  • High O&M costs
    Cloud technologies allow enterprises to choose different types of databases based on business types or deploy databases in different environments. This brings the following challenges to database O&M:
    • DBAs who are proficient in various types of databases are scarce.
    • DBAs require advanced skills to manage various types of databases across environments.
    • Experience in database O&M is difficult to obtain and teach.
  • High security risks
    As data value increases, enterprises are faced with both internal and external security risks. The following security issues are prone to occur:
    • Unauthorized access to databases, and unpredictable or inappropriate use of databases
    • Data breaches
    • Data corruption
    • Hacker attacks
    • Data exceptions that are caused by software or hardware bugs
    • Data loss that is caused by operational faults

Solutions

Database Autonomy Service (DAS) is a cloud service that uses machine learning and expert experience to automate perception, healing, optimization, O&M, and security assurance for databases. DAS makes database management simpler and avoids service failures that may be caused by manual operations. DAS makes database services stable, secure, and efficient.

DAS has been used to manage all the databases of Alibaba Group for more than two years. As of April 2020, DAS has greatly improved database performance in the following aspects:

  • More than 40 million SQL statements are automatically optimized.
  • Four petabytes of data space are automatically reclaimed.
  • 20% of memory is automatically optimized.