This topic describes how to use MaxCompute Management to view the running status and operation records of jobs, terminate jobs, and configure quota groups. It also describes how to view the consumption of storage resources and compute units (CUs).

Prerequisites

MaxCompute CUs that use the subscription billing method are purchased.
Note
  • We recommend that you purchase sufficient CUs to meet your business needs and make full use of MaxCompute Management.
  • If you disable the AccessKey pair of your Alibaba Cloud account, you are unable to use MaxCompute Management as a RAM user of this account.

Go to MaxCompute Management

To go to MaxCompute Management, perform the following steps:
  1. Log on to the MaxCompute console and select a region in the upper-left corner. MaxCompute console
  2. Click the Housekeeper tab to go to MaxCompute Management.

Overview

On the Overview page, you can select a quota group and a time range in the Overview of Subscription Resources section to view the information about CUs and storage resources. The information includes Used CUs, Total CUs, Used Storage, CU Usage Trend, and Storage Usage Trend.
Note The Overview of Pay-As-You-Go Resources section has no data displayed.
Overview
Parameter Description
Quota Group The quota group that you want to query. By default, this parameter is empty, which indicates that all quota groups are queried.

You can select a time range from the date and time picker next to Quota Group. By default, data from the past 24 hours is queried.

Used CUs The number of CUs that are used by all projects in the specified quota group within the specified time range. You can click a time point in the CU Usage Trend chart to view the job snapshots at that time point.
Note If a custom quota group exists and it is a sharing group, the number of used CUs may be greater than that of the purchased CUs in each quota group at a specific time point. The purpose is to ensure the minimum number of exclusive CUs. A sharing quota group refers to a group for which the maximum number of exclusive CUs is not equal to the minimum number.
Total CUs
  • If you do not specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Number of exclusive CUs in the current order within the specified time range + Number of shared CUs in the current order within the specified time range.
  • If you specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Maximum quota of exclusive CUs in the specified quota group within the specified time range + Maximum quota of shared CUs in the specified quota group within the specified time range.
Used Storage The storage resources that are used by all the projects in the specified quota group within the specified time range.
Requested CUs The trend in the number of exclusive and shared CUs that are requested by all the projects in the specified quota group within the specified time range.
Used CUs The trend in the number of exclusive and shared CUs that are used by all the projects in the specified quota group within the specified time range.
Total CUs The trend in the total number of exclusive and shared CUs in the specified quota group within the specified time range.
Storage Size The trend in the number of storage resources that are used by all the projects in the specified quota group within the specified time range.

View the running status of jobs

MaxCompute Management takes job snapshots every 2 minutes. Job snapshots allow you to view the running status of jobs in a specific quota group at a specific time point. Fixed parameters are displayed in job snapshots at each time point. This enables you to trace the resource usage of a specific quota group at any time point.

When a job is running, multiple snapshots may be taken for the job in different states. For example, for a job that is running at a specific time point, you can view the snapshots that are taken later to check the subsequent job status.
  1. In the left-side navigation pane, click Jobs.
  2. On the Job Snapshots tab, select a quota group or a project that uses the subscription or pay-as-you-go billing method and select a time point to view job snapshots.
    Note On the Overview page, you can click a time point in the CU Usage Trend chart to view job snapshots at that time point.
    88
    Parameter Description
    InstanceID The instance ID of the job. Each MaxCompute job runs as an instance. You can click the instance ID to go to the Logview page and view the progress of the job. For more information, see Use Logview to view job information.
    Submitted By The Alibaba Cloud account that is used to run the job. You can find the job owner based on the account information. If a job occupies excessive resources and affects the running of other jobs, you can request the job owner to terminate the job. For more information about how to terminate a job, see Instance operations.
    Project The project to which the job belongs.
    Quota Group Only subscription quota groups can be selected. This parameter is available only for the jobs that use subscription resources.
    DataWorks Node ID The ID of the DataWorks node on which the job is running. If this parameter is empty, the job is not submitted in DataWorks.
    Priority The priority of the job. This parameter is available only for the jobs that use subscription resources.

    Valid values: 0 to 9. A small value indicates a high priority.

    Billing Method The billing method that is used by the project to which the job belongs. Valid values: Subscription and Pay-as-you-go.
    CPU Utilization (%) The percentage of CPU resources used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.
    Memory Usage (%) The percentage of memory resources used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.
    Running Status The running status of the job.
    Job type The type of the job.
    Waiting Time The time spent waiting for available resources to run the instance.
    Running Duration The running duration of the job.
    Submitted At The time when the job was submitted.
  3. In the upper-right corner of the Job Snapshots tab, click the 1 icon to enable auto-refresh. In the dialog box that appears, specify a refresh interval. Enable auto-refresh

    To disable auto-refresh, click the Suspend icon icon in the upper-right corner of the Job Snapshots tab.

Terminate jobs

As a job owner, you can terminate the jobs that you no longer require. You can terminate a maximum of 10 jobs at a time.
  1. On the Job Snapshots tab, click Stop Jobs above the job list.
  2. In the Terminate Job_Subscription dialog box, specify Instance IDs and Description. In this example, a project that uses the subscription billing method is used. Terminate jobs
  3. Click Run.

View the operation records of jobs

MaxCompute Management allows you to view the operation records of jobs from the last seven days.
  1. In the left-side navigation pane, click Jobs.
  2. On the page that appears, click the Operation Records of Job Snapshots tab to view the operation records of jobs. Operation Records of Job Snapshots
  3. Find the required operation record and click View Details in the Actions column. In the panel that appears, view the operation details.

View the consumption of storage resources

On the Projects page, you can view the consumption of storage resources. MaxCompute Management collects statistics on storage resources on an hourly basis.
  1. In the left-side navigation pane, click Projects. On the page that appears, you can separately view the projects that use the subscription and pay-as-you-go billing methods.
    In a project list, you can view the storage resources that are used by each project. You can also select a quota group or a project in the upper-right corner of a section to filter projects. The following figure shows the Subscription Projects section. View the used storage resources
  2. In the Subscription Projects section, click the name of a project. The Storage tab appears. View the used storage resources
  3. Select a time range from the date and time picker and click OK.

View the consumption of CUs

On the Quotas page, you can view the consumption of CUs in projects that use the subscription billing method. MaxCompute Management collects statistics on CUs every 2 minutes.
Note The Pay-as-you-go Quota Groups section has no data displayed.
  1. In the left-side navigation pane, click Quotas. Quotas
  2. In the Subscription Quota Groups section, click the name of a quota group. The Resource Consumption tab appears. Resource Consumption
  3. Select a time range from the date and time picker and click OK.
    Note The interval at which data is displayed depends on the selected time range.

Configure quota groups

On the Quotas page, you can create, modify, or delete quota groups and specify multiple periods for resource scheduling. You can configure quota groups only for projects that use the subscription billing method. Quota group settings
The following table describes the operations in detail.
Operation Description Step
Create a quota group You can create a quota group. After a quota group is created, click Projects in the left-side navigation pane. Find the project that you want to modify and click Modify in the Actions column. In the Modify Quota Group Info panel, change the quota group of the project to the created quota group. Then, the jobs in the project use the CUs of the new quota group by default.
  1. Click Create Quota Group.
  2. In the Create Quota Group panel, specify Quota Group Name, Minimum Reserved CU Quota, Maximum Reserved CU Quota, Maximum Non-reserved CU Quota, and Tag.
    Note
    • The Maximum Non-reserved CU Quota parameter specifies the number of subscription-based shared CUs. For more information, see Computing pricing.
    • A tag is used to specify the quota group for a job. If the tag that you specify for a job is the same as that of an existing quota group, the job is preferentially scheduled to this quota group. For more information, see SET operations.
    • Make sure that each tag name is unique. Otherwise, jobs are randomly scheduled to one of the quota groups that have the same tag instead of being evenly scheduled to the quota groups.
  3. Click Run.
Modify a quota group You can modify an existing quota group.
  1. Find the quota group that you want to modify and click Modify in the Actions column.
  2. In the Modify Quota Group panel, modify the values of Minimum Reserved CU Quota, Maximum Reserved CU Quota, Maximum Non-reserved CU Quota, and Tag.
  3. Click Run.
Delete a quota group You can delete an existing quota group. You cannot delete a quota group that contains projects. Before you delete such a quota group, you must migrate the projects from the quota group to another quota group.
  1. Find the quota group that you want to delete and click Delete in the Actions column.
  2. In the Delete message, click Run.
Specify multiple periods for resource scheduling You can specify periods for resource scheduling. This way, different projects can use exclusive CUs in different periods based on business needs. For example, production projects require more CUs at night and fewer CUs in the daytime. Development or analysis projects require more CUs in the daytime and fewer CUs at night. You can specify different periods for these projects to schedule resources. This improves the CU usage. Take note of the following configuration rules:
  • For quota groups, the default period for resource scheduling is from 00:00:00 to 23:59:59. You can specify a maximum of three periods for resource scheduling. The specified periods for resource scheduling apply to all quota groups.
  • You must set each period on the hour, such as 00:00:00 to 07:00:00. The end time of the last period must be 23:59:59.
  • You can specify multiple periods for resource scheduling only for exclusive CUs in quota groups.
  • For the default quota group, you cannot change the minimum or maximum quota of exclusive CUs for the specified periods.
  1. Click Schedule Resources.
  2. In the Schedule Resources dialog box, add periods for resource scheduling and click Enable.
  3. Configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups. If you have created custom quota groups, find the required quota group and click Modify in the Actions column. In the Modify Quota Group panel, click the Edit icon in the Actions column of each period and configure the minimum and maximum quotas of exclusive CUs. If you have not created custom quota groups, click Create Quota Group to create one.
    Note If the minimum or maximum quota of exclusive CUs must remain the same for a quota group, configure the same quota for all the specified periods. Within a specified period, the sum of the minimum quotas of exclusive CUs specified for all quota groups equals the total number of exclusive CUs that you have purchased.
  4. Click Run.
  5. Optional. Disable multiple periods for resource scheduling. If the specified periods for resource scheduling are inappropriate or need to be modified, click Schedule Resources. In the Schedule Resources dialog box, click Disable. Then, the original settings of Reserved CU Quota take effect for all quota groups.
  6. Optional. Modify the specified periods for resource scheduling. You cannot directly modify the specified periods for resource scheduling. To do so, you must disable the existing periods, specify new periods, and then enable the new periods. After that, you must configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups.
Note
  • If you increase or decrease the number of available CUs, the minimum and maximum quotas of CUs vary accordingly for the default quota group. However, in this case, the quotas remain the same for custom quota groups.
  • You cannot reduce the number of available exclusive CUs to a value that is less than the minimum quota of exclusive CUs in the default quota group.
  • The minimum quota of CUs indicates the minimum number of CUs that a quota group can provide. The maximum quota of CUs indicates the maximum number of CUs that a quota group can provide.

Quota group configuration examples

In the following examples, 60 exclusive CUs and no shared CUs are available for quota groups A and B. The resources are allocated in different ways with or without multiple periods for resource scheduling.
  • Without multiple periods for resource scheduling
    • Resources are exclusively allocated to each quota group.

      [Maximum Reserved CU Quota,Minimum Reserved CU Quota,Maximum Non-reserved CU Quota]: [40,40,0] for quota group A and [20,20,0] for quota group B

    • Resources are allocated to quota groups on a first come, first served basis.

      [Maximum Reserved CU Quota,Minimum Reserved CU Quota,Maximum Non-reserved CU Quota]: [60,40,0] for quota group A and [40,20,0] for quota group B

  • With multiple periods for resource scheduling

    For example, a production project, a development project, and an analysis project exist. The peak hours of the production project are from 00:00:00 to 08:00:00, and the peak hours of the development and analysis projects are from 08:00:00 to 23:59:59. The resources are allocated in the following way:

    • Two periods for resource scheduling: 00:00:00 to 08:00:00 as Period 1 and 08:00:00 to 23:59:59 as Period 2.
    • [Maximum Reserved CU Quota,Minimum Reserved CU Quota,Maximum Non-reserved CU Quota] in Period 1: [60,50,0] for a custom quota group and [60,10,0] for the default quota group.
    • [Maximum Reserved CU Quota,Minimum Reserved CU Quota,Maximum Non-reserved CU Quota] in Period 2: [60,20,0] for a custom quota group and [60,40,0] for the default quota group.
    • The production project uses the resources in the custom quota group. The development and analysis projects use the resources in the default quota group.

You cannot specify the scheduling sequence for quota groups. Resources are used on a first come, first served basis and cannot be preempted. For example, 60 exclusive CUs and no shared CUs are available for quota groups A and B. You set [Maximum Reserved CU Quota,Minimum Reserved CU Quota,Maximum Non-reserved CU Quota] to [40,20,0] for quota group A and [30,10,0] for quota group B. If quota group A uses 40 CUs first, quota group B can use only the remaining 20 CUs and cannot preempt resources that are being used by quota group A. If quota group A releases 10 CUs after a period of use, quota group B can then use 30 CUs.

Change the quota group of a project

You can create a quota group and change the quota group of a project to the created quota group. This way, you can use the created quota group to isolate CUs for the project.
  1. In the left-side navigation pane, click Projects.
  2. In the Subscription Projects or Pay-As-You-Go Projects section, find the project that you want to modify and click Modify in the Actions column.
  3. In the Modify Quota Group Info panel, select another quota group from the Quota Group drop-down list. Modify the quota group
  4. Click Run.