PAI-Rec is a platform for end-to-end, in-depth customization and development of recommendation systems. With this platform, enterprise developers can independently build, develop, iterate, and maintain their recommendation systems.
Overview
PAI-Rec is a recommendation system development platform developed by Alibaba Cloud. It is designed to provide enterprise developers with end-to-end, in-depth customization of recommendation systems. The platform encompasses multiple dimensions, including offline processing, online services, real-time data streaming, and engineering architecture. It includes functional modules such as recall, ranking, filtering, and re-ranking. PAI-Rec work based on the big data architecture of Alibaba Cloud Apsara and allows developers to select service types based on their enterprise technology stacks and development habits, and to customize the code of recommendation pipelines. In addition, PAI-Rec provides a variety of tools, such as the data diagnostics and analysis tool, recommendation result debugging tool, and engine release management tool. The A/B testing service and the experiment report platform help you greatly improve the iteration efficiency of your recommendation systems.
Developers can perform data analysis based on the logs collected by frontend event tracking tasks, and then customize the code of feature engineering algorithms, engine configuration files, experiment report metrics, and statistics-related code based on business requirements. This helps you easily build and optimize recommendation systems. PAI-Rec provides a white-box development mode that brings developers a more transparent and flexible development experience. In addition, if the recommendation algorithm team and engineering team of your enterprise are not mature, we recommend that you start the service by using the industry-specific algorithm models customized by the Alibaba Cloud algorithm team at the initial stage. This not only assists your enterprise in deploying a complete recommendation system within a short period of time but also helps developers quickly get started with independent model training and effect evaluation. If you need further cooperation with Alibaba Cloud, such as wanting Alibaba Cloud engineers to provide in-depth tuning customization and share tuning experience, you can contact Alibaba Cloud customer service.
PAI-Rec also provides a variety of capabilities such as cold start, recommendation control, and online learning to meet various business requirements.
Benefits
PAI-Rec provides the following benefits:
Highly transparent white-box development
PAI-Rec provides a large amount of source code, which allows users to understand the details of recommendation algorithms and customize code to meet various business requirements.
The source code includes the following types: source code for feature engineering and sample processing, script code for calling recall and ranking models, source code of EasyRec recall and ranking models, and service source code of the PAI-Rec engine.
Convenient recommendation solution customization process
Users need to only configure user tables, item tables, and behavior tables to generate recall and ranking scripts and configuration files. This simplifies recommendation solution customization.
Comprehensive engine and experiment management system
A comprehensive engine management and experiment management platform is provided. Users can use this platform to easily manage recall and ranking components and update engine parameters.
Fine-grained metric monitoring and reporting
A metric and report management platform is provided for users to customize metrics and obtain experiment results by day or by hour. This ensures accurate monitoring of recommendation results and timely feedback.
Guarantee of consistency between offline and online features
A tool is provided for comparing consistency between online and offline features. This prevents experiment errors caused by consistency issues.
Intelligent data diagnostics and analytics
The intelligent data diagnostics feature is provided to help developers quickly understand data and select features and time windows for feature engineering based on diagnostics results.
Tools for intuitive observation of recommendation results
A variety of diagnostic tools are provided to help users observe recommendation results and recall data in a visualized manner.
Powerful FeatureStore
PAI-Rec is integrated with FeatureStore to help users manage features in a better way, which improves experiment efficiency.
All-round technical support
Various technical support services are provided to answer questions of users, helping them make good use of solutions.
Dependent cloud services
EasyRec of Platform for AI (PAI) is used to train recall and ranking models, and the PAI-Rec engine in Go is used to build recommendation systems. DataWorks or Machine Learning Designer of PAI is used to edit and schedule the code for feature engineering as well as sample and model training. Graph Compute and Hologres are used to store user features and support item-to-item (I2I) queries and vector queries. Elastic Algorithm Service (EAS) of PAI is used to provide scalable scoring services. The following section describes the related services:
Cloud service | Description |
PAI is a machine learning and deep learning engineering platform intended for developers and enterprises. PAI provides end-to-end AI development services including data labeling, model building, model training, model deployment, and inference optimization. | |
EasyRec algorithm framework | EasyRec is an algorithm framework that incorporates industry-leading deep learning models. It supports TensorFlow 1.12 and later versions, TensorFlow 2.4 and earlier versions, and PAI-TensorFlow. It meets end-to-end recommendation requirements, including recall, coarse ranking, fine ranking, re-ranking, multi-objective ranking, and cold start. EasyRec helps developers accelerate feature iterations to meet end-to-end recommendation requirements. |
DataWorks and MaxCompute are cloud-native big data services. The two services can be used together. They provide easy-to-use development tools and stable data environments for feature processing, sample generation, profile management, model scheduling, data update, and other phases in recommendation systems. Note PAI-Rec supports only DataWorks and MaxCompute. To use other big data services, you may need to modify related engine code. In this case, you must communicate with architects in advance. | |
Hologres is a unified real-time data warehousing service developed by Alibaba Cloud. You can use Hologres to write, update, process, and analyze large amounts of data in real time. Hologres supports standard SQL (compatible with PostgreSQL) syntax, most PostgreSQL functions, and online analytical processing (OLAP) and ad hoc analysis for up to petabytes of data. Hologres provides high-concurrency and low-latency online data services and supports fine-grained isolation of multiple workloads and enterprise-level security capabilities. Hologres is deeply integrated with MaxCompute, Realtime Compute for Apache Flink, and DataWorks to provide full-stack online and offline data warehousing solutions for enterprises. | |
Graph Compute | Graph Compute is a high-performance distributed graph computing service developed by Alibaba Cloud. It provides an end-to-end graph computing service that supports trillions of data records. Graph Compute allows you to efficiently store, query, and compute complex graph relationship data by using graph algorithms and models. Graph Compute can be used in a wide range of scenarios such as recommended advertisements for searches, real-time risk control, knowledge graph, and social networks. |