MaxCompute offers job insight capabilities that enable users to view job summaries, intelligent diagnostic details, resource consumption, and computing quota resource allocation at specific times. It also supports comparative analysis of running metrics for similar jobs, identified by the same signature or ExtNodeId, over a selected time period. These insights can be used to optimize task execution and enhance efficiency.
Access job insights
Log on to the MaxCompute console and select a region in the upper left corner.
In the left-side navigation pane, select .
In the job list, click Insight in the Actions column of the target Instance ID to enter the Job Insights page.
Switch tabs to view the required information. The specific features are as follows:
Tab Name
Feature
Job Summary
View basic information, runtime parameters, and intelligent diagnostic details.
In addition, you can perform the following operations:
Diagnose: Click Diagnose in the upper right corner on the Job Summary tab to trigger the real-time intelligent diagnosis for SQL/SQLRT jobs.
LogView: Click LogView in the upper right corner on the Job Summary tab to view detailed job run information. For more information about LogView, see Use LogView V2.0 to view job information.
Resource Consumption
View job-level resource consumption and resource allocation of computing quota at a specific time.
Similar Jobs
View a comparison list of metrics such as the running duration, job scan volume, and resource consumption of similar jobs (identified by the same signature or ExtNodeId) over a period of time.
View basic job information and runtime parameters
On the Job Insights page, click the Job Summary tab, where you can view basic job information and runtime parameters. The specific parameter descriptions are as follows:
Basic information
Parameter | Description |
Job Type | Valid values:
|
Job Owner | The person responsible for the job. |
Job Priority | The priority of the job. For more information, see Job priority. |
Billing Method | The billing method of the job. Valid values:
|
Submission Time | The submission time of the job. |
Start Time | The time when the job obtains the first computing resource. For jobs with a short runtime or that do not consume computing resources such as DDL statements, the job submission time is used instead. |
Job Status | The latest status of the job. Valid values:
Note The status described here is the overall status of the entire job. However, the job itself may involve multiple concurrent processes, each with different sub-statuses. Detailed information must be viewed through LogView. For more information, see Use LogView V2.0 to view job information. |
Wait Time | The duration from when the job is submitted to when it starts to run. |
Execution Duration | The duration from the start time to the end time of the job. |
End Time | The end time of the job being run. |
Total Run Time | The total period for which the job runs from submission to end time. |
Runtime parameters
Parameter | Description |
Project | The name of the project that initiates the job. |
CU Quota | The name of the computing quota used for running the job. |
View intelligent diagnostic information
Upon entering the Job Analysis page, a real-time SQL/SQLRT job diagnosis is initiated. You can view the detailed intelligent diagnostic results and optimization recommendations on the Job Summary tab.
The intelligent diagnostic feature is only available for SQL/SQLRT jobs. For jobs of other types, diagnostic options and intelligent diagnostic results are not displayed.
You can also click Diagnose repeatedly in the upper right corner to retrieve the latest diagnostic results.
For more information about diagnostic results, see Intelligent diagnostics for jobs.
View job-level resource consumption
You can view the resource consumption throughout the job lifecycle on the Job Insights page under the Resource Consumption tab.
Chart | Description |
Resource Usage During Job Lifecycle | Displays a curve chart showing the usage and waiting volume of computing resources over time at the job level, and the trend of using compute unit (CU) and waiting CU over time at the quota level during job execution. If the CU usage at the job level is low while the CU usage at the quota level is high and even continuously reaches the upper limit, this indicates that quota resources are tight and other jobs are preempting computing resources from the current job. |
Resource Allocation of Computing Quota at a Specific Time | You can click the time point on the horizontal axis of the resource consumption chart to view the resource allocation of the computing quota at the corresponding time. Specifically, it shows the number and priority distribution of jobs with allocated resources that are running or waiting. You can click the color block corresponding to the target priority to view detailed information about the corresponding jobs, so you can identify the jobs that preempt computing resources from the current job. You can optimize task execution based on business conditions by adjusting job priorities or managing computing resources. For more information, see Job priority or Manage quotas for computing resources in the MaxCompute console. |
For best practices on job-level resource usage, see Best practices for job-level resource analysis.
Comparative analysis of similar jobs
You can view the comparative analysis of running metrics for jobs similar to the current one over a selected time period on the Similar Jobs tab of the Job Insights page.
MaxCompute provides two standards for determining job similarity:
Same Signature (default): Identifies SQL jobs with a signature matching the current job.
Same ExtNodeId: Matches jobs with an external node ID identical to that of the current job. For example, DataWorks node ID. For more information on DataWorks node IDs, see Configure basic properties.
You can select a time range for analysis (1d, 7d, or 14d), with the default being the range selected on the Jobs page.
The following table shows the visual comparison of running metric parameters:
Metric | Description |
Waiting Duration | The duration from when the job is submitted to when it starts to run. |
Execution Duration | The duration from the job's start time to end time. |
CU-hour | The CU-hours consumed by the job.
|
Scan Size | The amount of data scanned by the job. |
Additional fields are also provided to enhance your understanding of job information:
Field Information | Description |
Instance ID | The Instance ID that meets the current job similarity standard within the selected time range. Click the ID to jump to the Job Insights page. |
Submitted At | The submission time of the job. |
Job Type | The type of the job. |
Job Priority | The priority of the job. |
Initiating Project | The name of the project that initiated the job. |
Computing Quota | The name of the computing quota used for running the job. |
Job Owner | The person responsible for the job. |