Platform For AI:What is PAI?

Machine learning is an interdisciplinary subject that covers knowledge of probability theory, statistics, approximation theory, and complex algorithms. Machine learning uses computers as a tool to simulate how humans learn in an authentic and real-time manner, and classifies existing knowledge to improve learning efficiency. When you use machine learning to build a model, you can assume the structure or type, train the model to obtain parameters, and then use the parameter and the trained model to perform analysis and prediction. Machine learning is applied to the following scenarios:

• Content generation: generates text, image, video, and audio content related to the topic based on business requirements.

• Marketing: commodity recommendation, user profiling, and targeted advertising.

• Finance: credit risk prediction for loans, financial risk management, stock forecast, and gold price forecast.

• Social network: analytics of key opinion leaders and relational networks.

• Text processing: news classification, keyword extraction, text summarization, and text analytics.

• Unstructured data processing: image classification and text extraction based on optical character recognition (OCR).

• Other forecast scenarios: rainfall forecast and football match result forecast.

Machine learning includes traditional machine learning and deep learning, and is divided into the following learning modes:

• Supervised learning: Each sample has an expected value. You can create a model to map input feature vectors to target values. Supervised learning can be used to solve regression and classification issues.

• Unsupervised learning: Samples do not have target values. Unsupervised learning is used to discover potential regular patterns from the sample data. You can use unsupervised learning to solve clustering issues.

• Reinforcement learning: This learning mode is complex. A system constantly interacts with the external environment to obtain feedback and determines its own behavior to achieve a long-term optimization of targets. Examples of reinforcement learning are AlphaGo and autonomous driving.

PAI was designed to serve business within Alibaba Group, such as Taobao, Alipay, and Amap.com. It enables developers of Alibaba Group to use AI technologies in an efficient, concise, and standard manner. PAI was officially released in 2018. It has gained tens of thousands of enterprises and individual developers, and has become one of the leading machine learning platforms on the cloud in China.

PAI supports the following underlying computing frameworks:

• Flink, a stream computing framework.

• TensorFlow, PyTorch, Megatron, and DeepSpeed, optimized open-source deep learning frameworks.

• Parameter Server, a computing framework that can process hundreds of billions of samples in parallel.

• Spark, PySpark, MapReduce, and other mainstream open source computing frameworks.

PAI provides the following services:

• Machine Learning Designer: a service for visualized modeling and distributed training, For more information, see Overview of Designer.

• Data Science Workshop (DSW): a Notebook-based service for interactive AI research and development, For more information, see DSW overview.

• Deep Learning Containers (DLC): a basic cloud-native platform for AI, For more information, see Before you begin.

• Elastic Algorithm Service (EAS): a service that allows you to deploy models as online prediction services, For more information, see Overview of online model services EAS.

PAI provides the following benefits based on years of service for Alibaba Cloud and Alibaba Group:

• Full-lifecycle end-to-end service for AI research and development:

  • Supports data labeling, model development, model training, model optimization, model deployment, and AI O&M as a one-stop AI platform.

  • Provides over 140 types of optimized built-in algorithm components.

  • Supports multiple deep learning frameworks, such as TensorFlow and PyTorch.

  • Provides core capabilities such as multiple modes, deep integration with big data engines, multi-framework compatibility, and custom images.

  • Provides cloud-native AI development, training, and deployment services.

• Diverse service modes:

  • Supports fully managed and semi-managed services for public cloud.

  • Provides high-performance AI computing clusters and lightweight service modes.

• Industry-leading AI optimization:

  • Supports high-performance training framework, sparse training scenarios, billions to tens of billions of sparse features, tens to hundreds of billions of samples, and distributed incremental training of thousands of workers.

  • Supports acceleration of mainstream framework models such as RestNet50 and Transformer language model (LM) by using PAI Blade.

• Supports services to be used separately or in combination. PAI provides an all-in-one platform for machine learning. After training data is prepared in Object Storage Service (OSS) or MaxCompute, you can use PAI to streamline all workflows, including data uploading, data preprocessing, feature engineering, model training, model evaluation, and model publishing to both online and offline environments.

• Integrates with DataWorks and allows you to process data by using SQL, user-defined functions (UDFs), user-defined aggregation functions (UDAFs), and MapReduce. This ensures higher flexibility and efficiency.

• Supports using DataWorks to schedule experiments that are used to train and generate models. You can run scheduled tasks in the staging or production environment. This enables data isolation.