The PolarDB for AI feature provides the one-stop service for database-based data intelligence and uses MLOps and built-in models to integrate data, features, and models. This topic describes the PolarDB for AI feature.

Background information

With the accumulation of data, data-driven intelligent applications such as search, recommendation, and Q&A) are used in more and more scenarios. In the past few years, data-driven intelligent applications have gradually transformed from simple data analysis and statistics to features and model applications based on machine learning and deep learning. The speed of transformation is faster than expected, partly due to technological breakthroughs in machine learning and artificial intelligence. Deep neural networks have made unprecedented progress in disciplines such as image analysis and natural language processing. Reinforcement learning becomes a powerful paradigm that complements traditional supervised learning.

However, in data-driven intelligent applications, data, features, and models are still not associated. First, data engineers perform data cleansing and data integration by developing their own processes. Then, algorithm engineers perform periodic production features and models by customizing feature engineering processes, model training scripts, and scheduled task scripts. Finally, development engineers are responsible for model launch, stability assurance, and monitoring and O&M. This means that data migration is required between different systems. Data may be redundant between different sources and therefore inconsistent. Difficulties arise in managing features and upgrading models. In addition, the high labor costs of data engineers, algorithm engineers, and development engineers also hamper large-scale growth of current data-driven intelligent decision-making applications.

Based on expertise in data discovery, data management, version control, data cleansing, and data integration, PolarDB for AI provides one-stop data services to integrate data, features, and models, reduces labor costs for developing data-driven intelligent applications, and overcomes existing difficulties.

Introduction

PolarDB for AI is an in-database distributed machine learning component based on PolarDB for MySQL. PolarDB for AI adopts the cloud-native architecture and provides a series of MLOps that support machine learning by using SQL statements: creating models, training models, viewing model status, viewing model lists, model evaluation, and model reasoning. It also offers multiple built-in algorithms for machine learning and artificial intelligence: classification algorithms, regression algorithms, and clustering algorithms. MLOps and built-in models allow PolarDB for AI to provide efficient, reliable, and convenient data intelligence, break the isolation between databases and business applications, and deliver one-stop data intelligence services based on databases.

Pricing

The PolarDB for AI feature is in the public preview. You can try it free of charge.

Technical architecture

PolarDB for AI SQL statements are routing by PolarProxy. The SQL statements that contain /*polar4ai*/ are sent to clusters that support AI for computing. The returned results follow the same protocols as those of ordinary SQL statements. Therefore, you can connect to a cluster as set out in Connect to a cluster and execute SQL statements that contain /*polar4ai*/ to use this feature. The following figure shows the architecture. Technical architecture
Note PolarDB PolarProxy must be 2.7.5 or later. For more information about how to view and upgrade the PolarProxy version, see Upgrade the cluster version.
In addition to the storage and computing features of ordinary clusters, PolarDB for AI also provides three layers: access layer, feature layer, and model layer.
  • Access layer: processes SQL statements (including SQL parsing, SQL verification, cost estimation, and execution plans) and optimizes the node tree of SQL statements.
  • Feature layer: converts data to features, including data access, feature generation, data synchronization, and feature update.
  • Model layer: handles MLOps capabilities related to models, including creating models, model training, model evaluation, model inference, and model management.

Scenarios

In the following figure, PolarDB for AI transforms traditional data intelligence applications of databases and AI into one-stop data intelligence applications. In traditional data intelligence applications, only databases can be used by traditional data engineers and algorithm engineers. In one-stop data intelligence applications, both databases and AI are used by business engineers and connected to business systems.

System architecture

PolarDB for AI can be used in the following scenarios:

  • ID-Mapping

    ID-Mapping is usually suitable for platform customers, such as in gaming and e-commerce. In game platforms, one user may have multiple game accounts. These accounts are stored in the databases of different games or platforms, and are not related to each other. This brings various problems. For example, the same user cannot be accurately profiled in precision marketing. Traffic usage across channels is inefficient. The PolarDB for AI models associate accounts to optimize basic data and provide high-quality data for upstream services.

  • Q&A chatbots

    Q&A chatbots use data in databases. A Q&A service can be provided based on business scenarios and in combination with AI capabilities (such as dialogue control, machine learning, and natural language understanding). Q&A chatbots can provide 24/7 service to support more customers, improve customer satisfaction, enhance efficiency and reduce costs. Q&A chatbots can help enterprises deliver online consultation, online marketing and online services.

  • Search recommendation

    In traditional databases, search capabilities are usually based on the full-text retrieval capabilities of databases. No natural language retrieval such as semantic retrieval and synonym match are supported The mature search solutions in PolarDB for AI can greatly improve search accuracy.

    The combination of artificial intelligence recommendation algorithms and knowledge graph technology in PolarDB for AI with Alibaba e-commerce policies provides one-stop services and assists quick transition to the cold start process. Custom solutions for different business scenarios can continuously improve core business capabilities and enhance business revenue growth.

Benefits

PolarDB for AI supports PolarDB built-in MLOps such as model training and model inference, delivers compatibility with MySQL statements, and provides industry intelligence algorithms by Alibaba DAMO Academy to avoid data conversion and migration between two or more systems. This reduces the development costs of data intelligence applications and accelerates monetization of data business.

Usage

After you enable the feature as set out in Enable the PolarDB for AI feature, you can create features, update features, create AI models, evaluate models, infer models, and delete models. For more information, see Feature management and Model management.