With various MLOps and built-in models, PolarDB for AI makes PolarDB a one-stop database service to integrate data, features, and models. This topic provides an overview of PolarDB for AI.
Background information
With the accumulation of data, data-driven intelligent applications such as search, recommendation, and Q&A) are used in more and more scenarios. In the past few years, data-driven intelligent applications have gradually transformed from simple data analysis and statistics to features and model applications based on machine learning and deep learning. The speed of transformation is faster than expected, partly due to technological breakthroughs in machine learning and artificial intelligence. Deep neural networks have made unprecedented progress in disciplines such as image analysis and natural language processing. Reinforcement learning becomes a powerful paradigm that complements traditional supervised learning.
However, in data-driven intelligent applications, data, features, and models are still not associated. First, data engineers perform data cleansing and data integration by developing their own processes. Then, algorithm engineers perform periodic production features and models by customizing feature engineering processes, model training scripts, and scheduled task scripts. Finally, development engineers are responsible for model launch, stability assurance, and monitoring and O&M. This means that data migration is required between different systems. Data may be redundant between different sources and therefore inconsistent. Difficulties arise in managing features and upgrading models. In addition, the high labor costs of data engineers, algorithm engineers, and development engineers also hamper large-scale growth of current data-driven intelligent decision-making applications.
Based on expertise in data discovery, data management, version control, data cleansing, and data integration, Alibaba Cloud provides PolarDB for AI to address the isolation of data, features, and models, thereby reducing the labor costs for developing data intelligence applications.
Introduction
PolarDB for AI is an in-database distributed machine learning component based on PolarDB for MySQL. PolarDB for AI supports various MLOps that can be used through SQL statements, such as creating models, training models, viewing model status, viewing model lists, model evaluation, and performing model inference. PolarDB for AI also offers a wide variety of built-in AI algorithms such as classification algorithms, regression algorithms, and clustering algorithms. The MLOps and built-in models allow PolarDB for AI to provide efficient, reliable, and convenient data intelligence capabilities, break the isolation between databases and business applications, and deliver one-stop data intelligence services based on databases.
Supported versions
To use PolarDB for AI, your PolarDB for MySQL cluster must meet the following requirements:
The cluster is of Enterprise Edition and Cluster Edition.
The cluster runs PolarDB for MySQL 8.0.1 or later.
PolarProxy is 2.7.5 or later.
For more information about how to view or upgrade the database engine and PolarProxy, see Minor version updates.
Pricing
PolarDB for AI charges you only for the compute nodes. Regular AI nodes are billed based on regular compute nodes. For more information about the specifications and prices of regular AI nodes, see Compute node specifications of PolarDB for MySQL Enterprise Edition and Compute node pricing.
Architecture
To implement PolarDB for AI, PolarProxy routes the SQL statements that contain the /*polar4ai*/ hint to clusters that support AI for computing and returns the results in the same way as returning those of ordinary SQL statements. Therefore, you can connect to a cluster and execute SQL statements that contain the /*polar4ai*/ hint to use PolarDB for AI. The following figure shows the architecture.
PolarDB PolarProxy must be of V2.7.5 or later. For more information about how to view and upgrade the PolarProxy version, see Minor version update.
In addition to the storage and computing features of ordinary clusters, PolarDB for AI includes other three layers.
Access layer: processes SQL statements (including SQL parsing, SQL verification, cost estimation, and execution plans) and optimizes the node tree of SQL statements.
Feature layer: converts data to features, including data access, feature generation, data synchronization, and feature update.
Model layer: handles MLOps capabilities related to models, including creating models, model training, model evaluation, model inference, and model management.
Scenarios
As shown in the following figure, PolarDB for AI transforms traditional data intelligence applications from the traditional architecture where databases and AI are separated into an integrated architecture. In traditional data intelligence applications, only databases can be used by traditional data engineers and algorithm engineers. In one-stop data intelligence applications, both database and AI are used by business engineers and connected to business systems.
PolarDB for AI can be used in the following scenarios:
ID-Mapping
ID-Mapping is usually suitable for platform customers, such as in gaming and e-commerce. In game platforms, one user may have multiple game accounts. These accounts are stored in the databases of different games or platforms, and are not related to each other. This brings various problems. For example, the same user cannot be accurately profiled in precision marketing. Traffic usage across channels is inefficient. The PolarDB for AI models associate accounts to optimize basic data and provide high-quality data for upstream services.
Q&A chatbots
Q&A chatbots use data in databases. A Q&A service can be provided based on business scenarios and in combination with AI capabilities (such as dialogue control, machine learning, and natural language understanding). Q&A chatbots can provide
24/7service to support more customers, improve customer satisfaction, enhance efficiency and reduce costs. Q&A chatbots can help enterprises deliver online consultation, online marketing and online services.Search recommendation
In traditional databases, search is usually implemented based on the full-text retrieval capabilities of databases. No natural language retrieval such as semantic retrieval and synonym match are supported. PolarDB for AI provides a mature search solution that can greatly improve search accuracy.
The combination of AI recommendation algorithms and knowledge graph technology in PolarDB for AI with the e-commerce solutions provided by Alibaba Group makes for a one-stop solution for recommendation and helps enterprises smoothly navigate the startup stage Custom solutions for different business scenarios can continuously improve core business capabilities and enhance business revenue growth.
Benefits
PolarDB for AI allows MLOps operations built in PolarDB such as model training and model inference to be performed through SQL statements, and supports industry intelligence algorithms by Alibaba DAMO Academy to avoid data conversion and migration across systems. This reduces the development costs of data intelligence applications and accelerates the growth of data business.
Usage notes
Quick start
Advanced usage
Sample solutions