All Products
Search
Document Center

Platform For AI:Development workflows

Last Updated:Apr 02, 2024

You can use the modules on the details page of a workspace in the Platform for AI (PAI) console to perform AI development. This topic describes common AI development workflows.

AI development workflows

The modules of PAI are displayed in the left-side navigation pane of the workspace details page. You can use different modules for AI development based on your business scenarios. This section describes how to use the modules in common business scenarios.

  • Cloud-native development

    image

    Module

    Description

    Reference

    High-quality datasets are essential to high-precision models. To manage datasets in a centralized manner and prepare for data labeling and model training, you can use the dataset management module to register public datasets, datasets on on-premises machines, and datasets in Alibaba Cloud storage services. You can also create index datasets by scanning Object Storage Service (OSS) objects.

    Create and manage datasets

    Data Science Workshop (DSW) is a cloud-based, interactive integrated development environment (IDE) for machine learning. You can use DSW notebooks to read data, develop algorithms, and train and deploy models.

    DSW overview

    The image management module allows you to use the official PAI images or custom images and manage the images in a centralized manner.

    Custom images

    Deep Learning Containers (DLC) provides a flexible, stable, easy-to-use, and high-performance training environment for machine learning. DLC supports mainstream and custom algorithm frameworks and allows you to run ultra-large-scale distributed deep learning tasks.

    Before you begin

    You can use the datasets that are stored in Apsara File Storage NAS (NAS) or OSS and the code that is stored in Git repositories when you submit training jobs.

    Before you begin

    The model management module allows you to manage trained models in a centralized manner and deploy a model to Elastic Algorithm Service (EAS) as an online service.

    Register and manage models

    EAS allows you to load models to CPUs and GPUs at the same time, provides high throughput with low latency, and supports real-time auto scaling. You can deploy a large-scale and complex model with a few clicks.

    Note

    EAS does not support DSW images or CPFS datasets.

    EAS overview

  • AI and big data

    image

    Module

    Description

    Reference

    The source data is stored in MaxCompute tables, preprocessed in DataWorks, and then used for model training in PAI.

    Machine Learning Designer can combine streaming training and batch training and supports large-scale distributed training for traditional machine learning, deep learning, and reinforcement learning. Machine Learning Designer provides hundreds of algorithm components that support automatic parameter tuning. You can drag the components to create models with minimal coding.

    Overview of Machine Learning Designer

    DataWorks schedules tasks based on the scheduling parameters and time properties that you configured.

    The job management module allows you to store the pipeline data of Machine Learning Designer and the execution records of custom jobs. This helps you compare the results of different pipelines or jobs.

    Create and manage container training jobs

    The model management module allows you to manage trained models in a centralized manner and deploy a model to EAS as an online service.

    Register and manage models

    EAS allows you to load models to CPUs and GPUs at the same time, provides high throughput with low latency, and supports real-time auto scaling. You can deploy a large-scale and complex model with a few clicks.

    Note

    EAS does not support DSW images or CPFS datasets.

    EAS overview