MaxCompute Management is a management platform that allows you to perform routine O&M operations. For example, you can view the information of a job, view the consumption of resources (including storage resources and CUs), view the information of resources in a project, modify a project, and create, modify, query, or delete a quota group. This topic describes the features that are provided by MaxCompute Management in detail.

Prerequisites

MaxCompute CUs that use the subscription billing method are purchased. For more information, see Computing pricing.
Note
  • We recommend that you purchase sufficient CUs to meet your business requirements and make full use of MaxCompute Management.
  • If you disable the AccessKey pair of your Alibaba Cloud account, you cannot use MaxCompute Management as a RAM user of this account.

Go to MaxCompute Management

To go to MaxCompute Management, perform the following steps:
  1. Log on to the MaxCompute console and select a region in the top navigation bar. MaxCompute console
  2. Click the Housekeeper tab.

Overview

On the Overview page, you can select a quota group and a time range in the Overview of Subscription Resources section to view the information about CUs and storage resources. The information includes Used CUs, Total CUs, Used Storage, CU Usage Trend, and Storage Usage Trend.
Note The Overview of Pay-As-You-Go Resources section has no data.
Overview
Parameter Description
Quota Group The quota group that you want to query. This parameter is empty by default. This indicates that all quota groups are queried.

You can select a time range from the date and time picker next to Quota Group. By default, data from the last 24 hours is queried.

Used CUs The number of CUs that are used by all projects in the specified quota group within the specified time range. You can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.
Note If a custom quota group exists and it is a sharing quota group, the number of used CUs may be greater than the number of purchased CUs in each quota group at a specific point in time. This ensures that a minimum number of exclusive CUs are used in each quota group. A sharing quota group is a quota group in which the maximum number of exclusive CUs is greater than the minimum number.
Total CUs
  • If you do not specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Number of exclusive CUs in the current order within the specified time range + Number of shared CUs in the current order within the specified time range.
  • If you specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Maximum quota of exclusive CUs in the specified quota group within the specified time range + Maximum quota of shared CUs in the specified quota group within the specified time range.
Used Storage The storage resources that are used by all the projects in the specified quota group within the specified time range.
Requested CUs The trend in the number of exclusive and shared CUs that are requested by all the projects in the specified quota group within the specified time range.
Used CUs The trend in the number of exclusive and shared CUs that are used by all the projects in the specified quota group within the specified time range.
Total CUs The trend in the total number of exclusive and shared CUs in the specified quota group within the specified time range.
Storage Size The trend in the number of storage resources that are used by all the projects in the specified quota group within the specified time range.

View the status of jobs

MaxCompute Management takes job snapshots at an interval of 2 minutes. Job snapshots allow you to view the status of jobs in a specific quota group at a specific point in time. Fixed parameters are displayed in job snapshots at each point in time. This allows you to track the resource usage of a specific quota group at a specific point in time.

When a job is running, multiple snapshots may be taken for the job in different states. For example, for a job that is running at a specific point in time, you can view the snapshots that are taken later to check the subsequent job status.
  1. In the left-side navigation pane, click Jobs.
  2. On the Job Snapshots tab, select a quota group or a project that uses the subscription or pay-as-you-go billing method and select a point in time to view job snapshots.
    Note On the Overview page, you can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.
    Job snapshot
    Parameter Description
    InstanceID The instance ID of the job. Each MaxCompute job runs as an instance. You can click the instance ID to go to the Logview page and view the progress of the job. For more information about how to view job progress in Logview, see Use Logview to view job information.
    Submitted By The Alibaba Cloud account that is used to run the job. You can find the job owner based on the account information. If a job occupies an excessive amount of resources and affects the execution of other jobs, you can request the job owner to terminate the job. For more information about how to terminate a job, see Instance operations.
    Project The project to which the job belongs.
    Quota group You can select only subscription quota groups. This parameter is available only for the jobs that use subscription resources.
    DataWorks Node ID The ID of the DataWorks node on which the job is running. If this parameter is left empty, the job is not submitted in DataWorks.
    Priority The priority of the job. This parameter is available only for the jobs that use subscription resources.

    Valid values: 0 to 9. A small value indicates a high priority.

    Billing Method The billing method that is used by the project to which the job belongs. Valid values: Subscription and Pay-as-you-go.
    CPU Utilization (%) The percentage of CPU resources that are used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.
    Memory Usage (%) The percentage of memory resources used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.
    Running Status The status of the job.
    Job Type The type of the job.
    Waiting Time The time spent waiting for available resources to run the job.
    Running Duration The duration for which the job runs.
    Submitted At The time at which the job was submitted.
  3. On the Job Snapshots tab, click the 1 icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section to enable auto-refresh. In the dialog box that appears, specify a refresh interval. Enable auto-refresh

    To disable auto-refresh, on the Job Snapshots page, click the Turn off auto refresh icon icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section.

Terminate jobs

As a job owner, you can terminate the jobs that you no longer require. You can terminate a maximum of 10 jobs at a time.
  1. On the Job Snapshots tab, click Stop Jobs above the job list.
  2. In the Stop Jobs panel, specify Instance IDs and Remarks. In this example, a project that uses the subscription billing method is used. Stop Jobs panel
  3. Click Run.

View the operation records of jobs

MaxCompute Management allows you to view the operation records of jobs from the last seven days.
  1. In the left-side navigation pane, click Jobs.
  2. On the page that appears, click the Job Operation Records tab to view the operation records of jobs.
  3. Find the operation record that you want to view and click View Details in the Actions column. In the panel that appears, view the operation details.

View the consumption of storage resources

On the Projects page, you can view the consumption of storage resources. MaxCompute Management collects statistics on storage resources on an hourly basis.
  1. In the left-side navigation pane, click Projects. On the page that appears, you can separately view the projects that use the subscription and pay-as-you-go billing methods.
    In a project list, you can view the storage resources that are used by each project. You can also select a quota group or a project in the upper-right corner to filter projects. View the used storage resources
  2. In the Subscription Projects section, click the name of a project. The Storage tab appears. View the used storage resources
  3. Select a time range from the date and time picker and click OK.

View the consumption of CUs

On the Quotas page, you can view the consumption of CUs in projects that use the subscription billing method. MaxCompute Management collects statistics on CUs at an interval of 2 minutes.
Note The Pay-as-you-go Quota Groups section has no data.
  1. In the left-side navigation pane, click Quotas. Quota Group
  2. In the Subscription Quota Groups section, click the name of a quota group. The Resource Consumption tab appears. Resource Consumption
  3. Select a time range from the date and time picker and click OK.
    Note The interval at which data is displayed depends on the selected time range.

Configure quota groups

On the Quotas page, you can create, modify, or delete quota groups and specify multiple periods for resource scheduling. You can configure quota groups only for projects that use the subscription billing method. Quota group settings
The following table describes the operations.
Operation Description Step
Create a quota group You can create a quota group. After a quota group is created, click Projects in the left-side navigation pane. Find the project that you want to modify and click Modify in the Actions column. In the Modify Quota Group Information panel, change the quota group of the project to the quota group you created. Then, the jobs in the project use the CUs of the new quota group by default.
  1. Click Create Quota Group.
  2. In the Create Quota Group panel, specify Quota Group Name, Minimum Reserved CUs, Maximum Reserved CUs, Maximum Non-reserved CUs, and Tag.
    Note
    • The Maximum Non-reserved CUs parameter specifies the number of subscription-based shared CUs. For more information, see Computing pricing.
    • A tag is used to specify the quota group for a job. If the tag that you specify for a job is the same as that of an existing quota group, the job is preferentially scheduled to this quota group. For more information, see SET operations.
    • Make sure that each tag name is unique. Otherwise, jobs are randomly scheduled to one of the quota groups that have the same tag instead of being evenly scheduled to the quota groups.
  3. Click Run.
Modify a quota group You can modify an existing quota group.
  1. Find the quota group that you want to modify and click Modify in the Actions column.
  2. In the Modify Quota Group panel, change the values of Minimum Reserved CUs, Maximum Reserved CUs, Maximum Non-reserved CUs, and Tag.
  3. Click Run.
Delete a quota group You can delete an existing quota group. You cannot delete a quota group that contains projects. If you want to delete a quota group that contains projects, you must migrate the projects from the quota group to another quota group.
  1. Find the quota group that you want to delete and click Delete in the Actions column.
  2. In the Delete message, click Run.
Specify periods for resource scheduling You can specify periods for resource scheduling. This way, different projects can use exclusive CUs for different periods based on your business requirements. For example, production projects require more CUs at night and fewer CUs in the daytime. Development or analysis projects require more CUs in the daytime and fewer CUs at night. You can specify different periods for these projects to schedule resources. This optimizes CU usage. When you specify multiple periods for resource scheduling, take note of the following rules:
  • For quota groups, the default resource scheduling period is from 00:00:00 to 23:59:59. You can specify a maximum of three periods for resource scheduling. The specified periods for resource scheduling apply to all quota groups.
  • You must set each period accurate to the hour, such as 00:00:00 to 07:00:00. The end time of the last period must be 23:59:59.
  • You can specify multiple periods for resource scheduling only for exclusive CUs in quota groups.
  • For the default quota group, you cannot change the minimum or maximum quota of exclusive CUs for the specified periods.
  1. Click Set Periods.
  2. In the Set Periods dialog box, add periods for resource scheduling and click Enable.
  3. Configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups. If you have created custom quota groups, find the required quota group and click Modify in the Actions column. In the Modify Quota Group panel, click the Edit icon in the Actions column of each period and configure the minimum and maximum quotas of exclusive CUs. If you have not created custom quota groups, click Create Quota Group to create a quota group.
    Note If you require the minimum or maximum quota of exclusive CUs to remain the same for all periods of a quota group, configure the same quota for all the specified periods. Within a specified period, the sum of the minimum quotas of exclusive CUs that are specified for all quota groups equals the total number of exclusive CUs that you purchased.
  4. Click Run.
  5. Optional. Disable multiple periods for resource scheduling. If the specified periods for resource scheduling are inappropriate or need to be modified, click Set Periods. In the Set Periods dialog box, click Disable. Then, the original settings of Reserved CU Quota take effect for all quota groups.
  6. Optional. Modify the specified periods for resource scheduling. You cannot directly modify the specified periods for resource scheduling. To do so, you must disable the existing periods, specify new periods, and then enable the new periods. After that, you must configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups.
Note
  • If you increase or decrease the number of available CUs, the minimum and maximum quotas of CUs vary accordingly for the default quota group. However, in this case, the quotas remain the same for custom quota groups.
  • You cannot reduce the number of available exclusive CUs to a value that is less than the minimum quota of exclusive CUs in the default quota group.
  • The minimum quota of CUs indicates the minimum number of CUs that a quota group can provide. The maximum quota of CUs indicates the maximum number of CUs that a quota group can provide.

Quota group configuration examples

In the following examples, 60 exclusive CUs and no shared CUs are available for Quota Groups A and B. The resources are allocated in different ways based on whether multiple periods for resource scheduling are enabled.
  • Multiple periods for resource scheduling disabled
    • Resources are exclusively allocated to each quota group.

      [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [40,40,0] for Quota Group A and [20,20,0] for Quota Group B

    • Resources are allocated to quota groups on a first come, first served basis.

      [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [60,40,0] for Quota Group A and [40,20,0] for Quota Group B

  • Multiple periods for resource scheduling enabled

    For example, a production project, a development project, and an analysis project exist. The peak hours of the production project are from 00:00:00 to 08:00:00, and the peak hours of the development and analysis projects are from 08:00:00 to 23:59:59. The resources are allocated in the following way:

    • Two periods for resource scheduling are specified: 00:00:00 to 08:00:00 as Period 1 and 08:00:00 to 23:59:59 as Period 2.
    • [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 1: [60,50,0] for a custom quota group and [60,10,0] for the default quota group.
    • [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 2: [60,20,0] for a custom quota group and [60,40,0] for the default quota group.
    • The production project uses the resources in the custom quota group. The development and analysis projects use the resources in the default quota group.

You cannot specify the scheduling sequence for quota groups. Resources are used on a first come, first served basis and cannot be preempted. For example, 60 exclusive CUs and no shared CUs are available for quota groups A and B. You set [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] to [40,20,0] for Quota Group A and [30,10,0] for Quota Group B. If Quota Group A uses 40 CUs first, Quota Group B can use only the remaining 20 CUs and cannot use the resources that are being used by Quota Group A. If Quota Group A releases 10 CUs after a period of use, Quota Group B can then use 30 CUs.

Change the quota group of a project

You can create a quota group and change the quota group of a subscription project to the created quota group. This way, you can use the created quota group to isolate CUs for the project.
  1. In the left-side navigation pane, click Projects.
  2. In the Subscription Projects section, find the project that you want to modify and click Modify in the Actions column.
  3. In the Modify Quota Group Information panel, select a quota group from the Quota Group drop-down list. Modify the quota group
  4. Click Run.