PAI-Rec is an end-to-end recommendation system development platform built by Alibaba Cloud. It gives enterprise developers full control to build, iterate, and maintain recommendation systems — from offline data processing and model training to online serving and real-time experimentation.
Use cases
PAI-Rec is a good fit if you need to:
Build a personalized recommendation engine from scratch — PAI-Rec provides the full pipeline: recall, coarse ranking, fine ranking, filtering, and re-ranking.
Iterate quickly on recommendation quality — A/B testing, experiment reporting, and online learning are built in, so you can measure the impact of algorithm changes by day or by hour and roll them out progressively.
Maintain transparency over your algorithms — PAI-Rec is a white-box platform. The source code for feature engineering, recall and ranking models (via EasyRec), and the recommendation engine is available for you to inspect and customize.
Accelerate time-to-first-recommendation — if your algorithm or engineering team is still ramping up, start with the industry-specific algorithm models provided by the Alibaba Cloud algorithm team. Switch to fully custom models as your team matures.
How it works
PAI-Rec spans four dimensions of a recommendation system:
| Dimension | What it covers |
|---|---|
| Offline processing | Feature engineering, sample generation, model training, and scheduling via DataWorks, MaxCompute, or Machine Learning Designer of PAI |
| Online services | The PAI-Rec engine (Go) handles real-time request serving, recall, ranking, filtering, and re-ranking |
| Real-time data streaming | Frontend event tracking feeds behavioral signals back into the pipeline for online learning |
| Engineering architecture | Built on the Alibaba Cloud Apsara big data infrastructure; select service types based on your technology stack |
The platform also supports cold start, recommendation control, and online learning for production scenarios.
Key features
| Feature | Description |
|---|---|
| White-box development | Access and customize source code for feature engineering and sample processing, recall and ranking model scripts, EasyRec model source code, and the PAI-Rec engine |
| Streamlined solution setup | Configure user, item, and behavior tables to automatically generate recall and ranking scripts and configuration files |
| Engine and experiment management | Manage recall and ranking components, update engine parameters, and track experiments from a single platform |
| Fine-grained metrics | Customize experiment metrics and pull results by day or by hour |
| Offline/online feature consistency | A built-in tool compares online and offline features to catch consistency issues before they affect experiment results |
| Intelligent data diagnostics | Automatically analyze your data to help select features and time windows for feature engineering |
| Recommendation result debugging | Visualize recommendation results and recall data directly in the platform |
| FeatureStore integration | Manage features centrally with FeatureStore to accelerate experiment iteration |
| Technical support | Alibaba Cloud provides support throughout your implementation. For in-depth tuning and customization, contact Alibaba Cloud customer service |
Dependent cloud services
PAI-Rec integrates with the following Alibaba Cloud services:
| Service | Role in PAI-Rec |
|---|---|
| PAI (Platform for AI) | End-to-end AI development: data labeling, model building, model training, model deployment, and inference optimization. EasyRec (recall and ranking model training) and Elastic Algorithm Service (EAS) (scalable scoring) run on PAI. |
| EasyRec algorithm framework | Algorithm framework incorporating industry-leading deep learning models. Supports TensorFlow 1.12 and later, TensorFlow 2.4 and earlier, and PAI-TensorFlow. Covers recall, coarse ranking, fine ranking, re-ranking, multi-objective ranking, and cold start. |
| DataWorks and MaxCompute | Cloud-native big data services for feature processing, sample generation, profile management, model scheduling, and data updates. PAI-Rec supports only DataWorks and MaxCompute for big data processing. To integrate other big data services, you need to modify the engine code — coordinate with your architects before doing so. |
| Hologres | Unified real-time data warehousing service. Stores user features and supports item-to-item (I2I) queries and vector queries. Compatible with standard SQL (PostgreSQL syntax), with support for OLAP and ad hoc analysis up to petabyte scale. Provides high-concurrency and low-latency online data services, and supports fine-grained isolation of multiple workloads and enterprise-level security capabilities. Deeply integrated with MaxCompute, Realtime Compute for Apache Flink, and DataWorks. |
| Graph Compute | High-performance distributed graph computing service supporting trillions of data records. Used for recommended advertisements for searches, real-time risk control, knowledge graph, and social network scenarios. |