Platform For AI:Procedure

Step 1: Get prepared

1. Grant permissions

   1. Use your Alibaba Cloud account to activate PAI and create a default workspace. Log on to the PAI console. Select a region in the upper left corner and click Authorize and Activate. For more information, see Activate PAI and create a default workspace.

   2. Authorize your operation account. If you use the Alibaba Cloud account to manage DSW, you can skip this step. But if you use a RAM account, it is necessary to authorize the operation account.

2. (Optional) Prepare a dedicated resource group

   After step 1, a public resource group is automatically prepared for you. If you need to use a dedicated resource group, see Create a dedicated resource group and purchase general computing resources and Create a resource group and purchase Lingjun resources.

3. (Optional) Mount a dataset

   Public and dedicated resource groups have limited storage capacity that does not support persistent storage. To expand storage and ensure data persistence, you can mount a NAS or OSS dataset or an OSS path. For more information, see Create and manage datasets.

   Important

   • DSW instances of a public resource group use a free disk with limited capacity, which is cleared 15 days after the instance is stopped or deleted.

   • DSW instances of a dedicated resource group store data on the system disk, with temporary storage being cleared when stopping or deleting the instance.

4. (Optional) Custom image

   DSW offers a variety of official images, including PyTorch, TensorFlow, and ModelScope. For custom images tailored to specific development requirements, see Custom images.

Step 2: Create and access a DSW instance

1. Create an instance

   To create a DSW instance, see Create a DSW instance.

2. Manage an instance

   • For instance lifecycle management and configuration changes, such as starting, stopping, deleting instances, or modifying configurations like instance type, image, and dataset, see Manage DSW instances.

   • To optimize instance costs, configure idle and scheduled shutdown policies. Idle shutdown is enabled by default for free trial instances. Administrators can set resource recycling policies in the workspace scheduling center for general scenarios. For more information, see Workspace scheduling center. Note: Data in temporary storage of dedicated resource groups is cleared after stopping the instance. Please ensure to export it promptly.

3. Access an instance

   • We recommend that you to open the DSW instance through the console for simplicity and to rich features of DSW. For more information, see Access a DSW instance.

   • To access a DSW instance using SSH, such as for running local Notebook code on a cloud-based DSW instance, see Connect to a DSW instance.

Step 3: Use a DSW instance

1. Model development and training

   You can develop models in DSW. A DSW instance provides an interactive development environment in which you can debug code and view running results. You can manage third-party libraries to meet specific environment requirements.

   If you want to perform distributed training, you can submit the code in DSW to DLC. For more information, see Submit training jobs.

2. Model deployment

   You can deploy a trained model as an online service in Elastic Algorithm Service (EAS) to implement model inference. For more information, see Model deployment.

3. Data transmission

   • Accessing data sources. DSW supports integration with various data sources, such as OSS and MaxCompute, allowing for the import of data files into the DSW instance and the export of processed data back to the data sources. For more information, see Read and write data and file transfer.

   • Upload and download. DSW instances allows you to supload and download data, and export and share notebooks in DSW. For more information, see File transfer and processing.

Step 4: Explore a DSW instance

1. Best practices

   Notebook Gallery provides a wide range of notebook cases that cover fields such as large language models (LLMs) and AI content generation, and models such as Llama 2, Qwen, and Stable Diffusion. You can directly run the notebook case or perform custom development based on the notebook cases in DSW. For more information, see Notebook Gallery.

2. Advanced features

   DSW supports the following advanced features:

   • You can use TensorBoard to visualize training in DSW. For more information, see TensorBoard: training visualization.

   • You can run R in DSW. For more information, see Install R Kernel in DSW.

   • You can execute SQL statements to query data from MaxCompute data sources. For more information, see Use SQL File to query MaxCompute tables.

   • You can connect a DSW instance to an E-MapReduce cluster and submit a Spark job. For more information, see Connect a DSW instance to an EMR cluster.

For more information, see DSW use cases.