After you submit a training task, you can view its basic information, configuration, events, resource usage, and logs to understand its running status.
View basic task information and configuration
-
Log on to the PAI console. At the top of the page, select the destination region. In the upper-right corner, select the target workspace, and then click Enter DLC.
-
Click the name of the target task to open the task overview page.
-
On the Overview page, you can view the task's basic information, environment context, and resource information.
View task events
Event logs record the progress of task scheduling and resource-related activities. You can view these events to locate and troubleshoot problems.
-
View event logs for the task:
Switch to the Events tab to view the event logs.
-
View event logs for a node:
On the Overview page, in the Instances section, click Logs in the Actions column for an instance. Then, on the Events tab, view the event logs for the node.
View the resource view
The resource usage view shows key metrics such as GPU usage, GPU memory usage, CPU usage, memory usage, and network I/O. You can use this view to monitor the task's resource usage in real time. This helps you understand the task's resource needs, track utilization, and plan for resource optimization.
Switch to the Monitoring tab to view the resource usage for the task.
Monitoring metrics are available for the Job, Pod, and GPU dimensions. You can use the monitoring and alerts feature to monitor the resource usage of DLC jobs in real time. For more information, see Training Monitoring and Alerts.
View task logs
If a task is abnormal or you need to view its execution history, you can use one of the following two methods to view its logs:
-
On the Overview page, in the Instances section, click Logs in the Actions column for an instance to view the output logs for that node.
-
Switch to the Logs tab to search for log events by keyword. For more information, see Query aggregated logs by keyword.
View behavioral event logs
PAI is integrated with ActionTrail. You can use ActionTrail to view and retrieve the last 90 days of DLC behavioral event logs for your Alibaba Cloud account. For more information, see ActionTrail.
References
You can manage a task based on its running status. For more information, see Manage training tasks.