MaxCompute:Resource observation

View computing resource usage

1. Log on to the MaxCompute console. In the upper-left corner, select a region.

2. In the navigation pane on the left, click Resource Observation.

3. On the Resource Observation page, click the Computing Resources tab.

4. Select a level-1 quota name, a time range, and a time interval. The time interval is the number of minutes between data points. Options: Adaptive, 1, 5, or 15 minutes. If the time range exceeds 72 hours, only Adaptive is available.

5. Click the expand icon to the left of a level-2 quota to view its resource consumption trend graph. You can expand multiple level-2 quota charts at the same time.

6. View the list of projects associated with each level-2 quota.

Computing resource metrics

View storage resource usage

1. Log on to the MaxCompute console. In the top navigation bar, select a region.

2. In the left-side navigation pane, click Resource Observation.

3. On the Resource Observation page, click the Storage Resource tab to view the total storage usage and storage usage distribution of different storage types on the current day.

4. (Optional) Select a time range and one or more projects to view the Storage Trend. By default, 7d (7 days) is selected as the time range, and all projects are selected. You can manually select up to 8 projects.

5. (Optional) On the Project Details tab in the Storage Details section, select a date to view the storage usage of each project on that date. The default date is the current day.

6. (Optional) On the Table/Partition Details tab in the Storage Details section, select a date and a project to view the storage usage of tables and partitions in the project on that date. The default date is the current day.

Storage resource metrics

View data transmission metrics

1. Log on to the MaxCompute console, and select a region in the top navigation bar.

2. In the left-side navigation pane, click Resource Observation.

3. On the Resource Observation page, click the Data Transmission Service tab.

4. Select a quota, a project, a time range, and an aggregation algorithm to query metric data.

Data aggregation

An adaptive interval mechanism automatically adjusts the data point density based on the selected time range:

Two aggregation algorithms are available:

When the interval exceeds the native granularity (1 minute), the system pre-processes data using the selected aggregation algorithm. Longer time ranges produce greater differences between Average Value and Maximum Value results. This is expected behavior.

Filter conditions and limits

• A single query supports a maximum time range of 7 days. Shorter time ranges yield more precise monitoring data.

• Select at least a quota or a project. Combined filtering for both a quota and a project is also supported.

  • Quota dimension only: Specify either an exclusive resource group or a shared resource group. Since shared resource groups are project-level quotas, viewing the shared resource group for a specific project also requires specifying the project.

  • Project dimension only: Leave the Select Quota field blank and select the desired project for Choose Project. This displays the total usage for that project.

• After changing filter conditions, click Query to refresh the monitoring data.

• For table-level monitoring dashboards, select a project before viewing data or filtering by table name.

Data transmission metrics

View job performance metrics

1. Log on to the MaxCompute console, and select a region in the top navigation bar.

2. In the left-side navigation pane, click Resource Observation.

3. On the Resource Observation page, click the Job Performance Observation tab.

4. Configure the following filter and grouping parameters, then click Search to view statistics.

Filter parameters

Job Type values:

Job End Status values: Success, Failed, Canceled.

Data summary and comparison

1. (Optional) Select Data Summary to view statistics at a different time granularity.

1. (Optional) Select a Comparison Period to compare current data against a historical baseline.

Default: No Comparison. Other options: Previous 30 Days, Previous 7 Days, Previous 1 Day. For example, if the current time is 14:00 on May 6, 2024, and you select Previous 30 Days, data from 14:00 on April 6, 2024 is shown alongside the current data for comparison.

Job performance metrics

Query metrics with Information Schema

Use the tenant-level Information Schema to collect the same statistics programmatically.

The Information Schema task_history table contains task instances generated by all operations. The Job Performance Observation tab displays data only from jobs that consume computing resources. As a result, statistics from Information Schema may differ from those shown in the console.

The following SQL example queries job metrics at hourly granularity for the previous 7 days:

SET odps.namespace.schema=TRUE;
SELECT to_char (end_time, 'yyyy-mm-dd hh'), -- The hour in which the job completes.
       -- to_char (end_time, 'yyyy-mm-dd'), -- The date on which the job completes. If daily granularity is used, use this line to replace the preceding line.
       sum(cast(cost_cpu/100/3600 as DECIMAL(18,5) )) cost_cpuh, -- The CPU-hours.
       sum(cast(cost_mem/1024/3600 as DECIMAL(18,5) )) cost_memh, -- The memory-hours.
       avg(datediff(end_time, start_time, 'ss')), -- The average runtime duration of jobs.
       min(datediff(end_time, start_time, 'ss')), -- The minimum runtime duration of jobs.
       max(datediff(end_time, start_time, 'ss'))  -- The maximum runtime duration of jobs.
       -- status, -- Group basis: status: job status; project: task_catalog; job type: task_type.
FROM SYSTEM_CATALOG.INFORMATION_SCHEMA.tasks_history
WHERE ds>=to_char(date_add(getdate(),-7),'yyyymmdd')  -- You can add other filter conditions based on your business requirements.
and task_type in ('SQL','SQLRT','LOT','CUPID','ALgoTask')
GROUP BY to_char (end_time, 'yyyy-mm-dd hh')
         -- to_char (end_time, 'yyyy-mm-dd'), -- The date on which the job completes. If daily granularity is used, use this line to replace the preceding line.
         -- status, -- Group basis: status: job status; project: task_catalog; job type: task_type.
order BY to_char (end_time, 'yyyy-mm-dd hh') ASC;
         -- to_char (end_time, 'yyyy-mm-dd'); -- The date on which the job completes. If daily granularity is used, use this line to replace the preceding line.

Troubleshooting

Projects or quotas are not displayed in grouped charts

No completed jobs exist in those projects or quotas during the selected time range.

Comparison period data is not available

The project or quota did not exist during the comparison period, or no jobs ran in the project or quota during that time.