All Products
Search
Document Center

MaxCompute:Use MaxCompute Management

Last Updated:Jan 05, 2024

MaxCompute Management is a management platform that allows you to perform routine O&M operations. For example, you can view the information of a job, view the consumption of resources (including storage resources and CUs), view the information of resources in a project, modify a project, and create, modify, query, or delete a quota group. This topic describes the features that are provided by MaxCompute Management in detail.

Important

MaxCompute Management will start to be phased out on January 10, 2024 (UTC+8). If you still want to use the features of MaxCompute Management, log on to the MaxCompute console. For more information, see December 4, 2023 (UTC+8): Notice on discontinuation of MaxCompute Management.

Prerequisites

MaxCompute CUs that use the subscription billing method are purchased. For more information, see Computing pricing.

Note
  • We recommend that you purchase sufficient CUs to meet your business requirements and make full use of MaxCompute Management.

  • If you disable the AccessKey pair of your Alibaba Cloud account, you cannot use MaxCompute Management as a RAM user of this account.

Go to MaxCompute Management

To go to MaxCompute Management, perform the following steps:

  1. Log on to the MaxCompute console, and select a region in the top navigation bar.

  2. In the left-side navigation pane, click MaxCompute Management.

Overview

On the Overview page, you can select a quota group and a time range in the Overview of Subscription Resources section to view the information about CUs and storage resources. The information includes Used CUs, Total CUs, Used Storage, CU Usage Trend, and Storage Resource Usage Trend.

Note

The Overview of Pay-As-You-Go Resources section has no data.

Parameter

Description

Quota Group

The quota group that you want to query. This parameter is empty by default. This indicates that all quota groups are queried.

You can select a time range from the date and time picker next to Quota Group. By default, data from the last 24 hours is queried.

Used CUs

The number of CUs that are used by all projects in the specified quota group within the specified time range. You can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.

Note

If a custom quota group exists and it is a sharing quota group, the number of used CUs may be greater than the number of purchased CUs in each quota group at a specific point in time. This ensures that a minimum number of exclusive CUs are used in each quota group. A sharing quota group is a quota group in which the maximum number of exclusive CUs is greater than the minimum number.

Total CUs

  • If you do not specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Number of exclusive CUs in the current order within the specified time range + Number of shared CUs in the current order within the specified time range.

  • If you specify a quota group, the total number of CUs is calculated by using the following formula: Total number of CUs = Maximum quota of exclusive CUs in the specified quota group within the specified time range + Maximum quota of shared CUs in the specified quota group within the specified time range.

Used Storage

The storage resources that are used by all the projects in the specified quota group within the specified time range.

Requested CUs

The trend in the number of exclusive and shared CUs that are requested by all the projects in the specified quota group within the specified time range.

Used CUs

The trend in the number of exclusive and shared CUs that are used by all the projects in the specified quota group within the specified time range.

Total CUs

The trend in the total number of exclusive and shared CUs in the specified quota group within the specified time range.

Storage Size

The trend in the number of storage resources that are used by all the projects in the specified quota group within the specified time range.

View the status of jobs

MaxCompute Management takes job snapshots at an interval of 2 minutes. Job snapshots allow you to view the status of jobs in a specific quota group at a specific point in time. Fixed parameters are displayed in job snapshots at each point in time. This allows you to track the resource usage of a specific quota group at a specific point in time.

When a job is running, multiple snapshots may be taken for the job in different states. For example, for a job that is running at a specific point in time, you can view the snapshots that are taken later to check the subsequent job status.

  1. In the left-side navigation pane, click Jobs.

  2. On the Job Snapshots tab, select a quota group or a project that uses the subscription or pay-as-you-go billing method and select a point in time to view job snapshots.

    Note

    On the Overview page, you can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.

    Parameter

    Description

    InstanceID

    The instance ID of the job. Each MaxCompute job runs as an instance. You can click the instance ID to go to the LogView page and view the progress of the job. For more information about how to view job progress in LogView, see Use LogView to view job information.

    Submitted By

    The Alibaba Cloud account that is used to run the job. You can find the job owner based on the account information. If a job occupies an excessive amount of resources and affects the execution of other jobs, you can request the job owner to terminate the job. For more information about how to terminate a job, see Instance operations.

    Project

    The project to which the job belongs.

    Quota Group

    You can select only subscription quota groups. This parameter is available only for the jobs that use subscription resources.

    DataWorks Node ID

    The ID of the DataWorks node on which the job is running. If this parameter is left empty, the job is not submitted in DataWorks.

    Priority

    The priority of the job. This parameter is available only for the jobs that use subscription resources.

    Valid values: 0 to 9. A small value indicates a high priority.

    Billing Method

    The billing method that is used by the project to which the job belongs. Valid values: Subscription and Pay-as-you-go.

    CPU Utilization (%)

    The percentage of CPU resources that are used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.

    Memory Usage (%)

    The percentage of memory resources used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources.

    Running Status

    The status of the job.

    Job Type

    The type of the job.

    Waiting Time

    The time spent waiting for available resources to run the job.

    Running Duration

    The duration for which the job runs.

    Submitted At

    The time at which the job was submitted.

  3. On the Job Snapshots tab, click the 1 icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section to enable auto-refresh. In the dialog box that appears, specify the Refreshing every parameter.image..png

    If you want to disable auto-refresh, click the 暂停 icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section on the Job Snapshots tab.

Terminate jobs

As a job owner, you can terminate the jobs that you no longer require. You can terminate a maximum of 10 jobs at a time.

  1. On the Job Snapshots tab, click Stop Jobs above the job list.

  2. In the Stop Jobs panel, specify Instance IDs and Remarks. In this example, a project that uses the subscription billing method is used.终止作业

  3. Click Run.

View the operation records of jobs

MaxCompute Management allows you to view the operation records of jobs from the last seven days.

  1. In the left-side navigation pane, click Jobs.

  2. On the page that appears, click the Job Operation Records tab to view the operation records of jobs.

  3. Find the operation record that you want to view and click View Details in the Actions column. In the panel that appears, view the operation details.

View the consumption of storage resources

On the Projects page, you can view the consumption of storage resources. MaxCompute Management collects statistics on storage resources on an hourly basis.

  1. In the left-side navigation pane, click Projects. On the page that appears, you can separately view the projects that use the subscription and pay-as-you-go billing methods.

    In a project list, you can view the storage resources that are used by each project. You can also select a quota group or a project in the upper-right corner to filter projects.

  2. In the Subscription Projects section, click the name of a project. The Storage tab appears.

  3. Select a time range from the date and time picker and click OK.

View the consumption of CUs

On the Quotas page, you can view the consumption of CUs in projects that use the subscription billing method. MaxCompute Management collects statistics on CUs at an interval of 2 minutes.

Note

The Pay-as-you-go Quota Groups section has no data.

  1. In the left-side navigation pane, click Quotas.

  2. In the Subscription Quota Groups section, click the name of a quota group. The Resource Consumption tab appears.资源消耗

  3. Select a time range from the date and time picker and click OK.

    Note

    The interval at which data is displayed depends on the selected time range.

Configure quota groups

On the Quotas page, you can create, modify, or delete quota groups and specify multiple periods for resource scheduling. You can configure quota groups only for projects that use the subscription billing method.

Note

If you are prompted to manage quota groups in the new MaxCompute console, go to the new console.

The following table describes the operations.

Operation

Description

Procedure

Create a quota group

You can create a quota group. After a quota group is created, click Projects > Modify in the Actions column. In the Modify Quota Group Information panel, change the quota group of the project to the quota group you created. Then, the jobs in the project use the CUs of the new quota group by default.

  1. Click Create Quota Group.

  2. In the Create Quota Group panel, specify Quota Group Name, Minimum Reserved CUs, Maximum Reserved CUs, Maximum Non-reserved CUs, and Tag.

    Note
    • The Maximum Non-reserved CUs parameter specifies the number of subscription-based shared CUs. For more information, see Computing pricing.

    • A tag is used to specify the quota group for a job. If the tag that you specify for a job is the same as that of an existing quota group, the job is preferentially scheduled to this quota group. For more information, see SET operations.

    • Make sure that each tag name is unique. Otherwise, jobs are randomly scheduled to one of the quota groups that have the same tag instead of being evenly scheduled to the quota groups.

  3. Click Run.

Modify a quota group

You can modify an existing quota group.

  1. Find the quota group that you want to modify and click Modify in the Actions column.

  2. In the Modify Quota Group panel, change the values of Minimum Reserved CUs, Maximum Reserved CUs, Maximum Non-reserved CUs, and Tag.

  3. Click Run.

Delete a quota group

You can delete an existing quota group. You cannot delete a quota group that contains projects. If you want to delete a quota group that contains projects, you must migrate the projects from the quota group to another quota group.

  1. Find the quota group that you want to delete and click Delete in the Actions column.

  2. In the Delete message, click Run.

Specify periods for resource scheduling

You can specify periods for resource scheduling. This way, different projects can use exclusive CUs for different periods based on your business requirements. For example, production projects require more CUs at night and fewer CUs in the daytime. Development or analysis projects require more CUs in the daytime and fewer CUs at night. You can specify different periods for these projects to schedule resources. This optimizes CU usage. When you specify multiple periods for resource scheduling, take note of the following rules:

  • For quota groups, the default resource scheduling period is from 00:00:00 to 23:59:59. You can specify a maximum of three periods for resource scheduling. The specified periods for resource scheduling apply to all quota groups.

  • You must set each period accurate to the hour, such as 00:00:00 to 07:00:00. The end time of the last period must be 23:59:59.

  • You can specify multiple periods for resource scheduling only for exclusive CUs in quota groups.

  • For the default quota group, you cannot change the minimum or maximum quota of exclusive CUs for the specified periods.

  1. Click Set Periods.

  2. In the Set Periods dialog box, add periods for resource scheduling and click Enable.

  3. Configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups. If you have created custom quota groups, find the required quota group and click Modify in the Actions column. In the Modify Quota Group panel, click the 编辑 icon in the Actions column of each period and configure the minimum and maximum quotas of exclusive CUs. If you have not created custom quota groups, click Create Quota Group to create a quota group.

    Note

    If you require the minimum or maximum quota of exclusive CUs to remain the same for all periods of a quota group, configure the same quota for all the specified periods. Within a specified period, the sum of the minimum quotas of exclusive CUs that are specified for all quota groups equals the total number of exclusive CUs that you purchased.

  4. Click Run.

  5. Optional. Disable multiple periods for resource scheduling. If the specified periods for resource scheduling are inappropriate or need to be modified, click Set Periods. In the Set Periods dialog box, click Disable. Then, the original settings of Reserved CU Quota take effect for all quota groups.

  6. Optional. Modify the specified periods for resource scheduling. You cannot directly modify the specified periods for resource scheduling. To do so, you must disable the existing periods, specify new periods, and then enable the new periods. After that, you must configure the minimum and maximum quotas of exclusive CUs for the specified periods for custom quota groups.

Note
  • If you increase or decrease the number of available CUs, the minimum and maximum quotas of CUs vary accordingly for the default quota group. However, in this case, the quotas remain the same for custom quota groups.

  • You cannot reduce the number of available exclusive CUs to a value that is less than the minimum quota of exclusive CUs in the default quota group.

  • The minimum quota of CUs indicates the minimum number of CUs that a quota group can provide. The maximum quota of CUs indicates the maximum number of CUs that a quota group can provide.

Quota group configuration examples

In the following examples, 60 exclusive CUs and no shared CUs are available for Quota Groups A and B. The resources are allocated in different ways based on whether multiple periods for resource scheduling are enabled.

  • Multiple periods for resource scheduling disabled

    • Resources are exclusively allocated to each quota group.

      [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [40,40,0] for Quota Group A and [20,20,0] for Quota Group B.

    • Resources are allocated to quota groups on a first come, first served basis.

      [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [60,40,0] for Quota Group A and [40,20,0] for Quota Group B.

  • Multiple periods for resource scheduling enabled

    For example, a production project, a development project, and an analysis project exist. The peak hours of the production project are from 00:00:00 to 08:00:00, and the peak hours of the development and analysis projects are from 08:00:00 to 23:59:59. The resources are allocated in the following way:

    • Two periods for resource scheduling are specified: 00:00:00 to 08:00:00 as Period 1 and 08:00:00 to 23:59:59 as Period 2.

    • [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 1: [60,50,0] for a custom quota group and [60,10,0] for the default quota group.

    • [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 2: [60,20,0] for a custom quota group and [60,40,0] for the default quota group.

    • The production project uses the resources in the custom quota group. The development and analysis projects use the resources in the default quota group.

You cannot specify the scheduling sequence for quota groups. Resources are used on a first come, first served basis and cannot be preempted. For example, 60 exclusive CUs and no shared CUs are available for quota groups A and B. You set [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] to [40,20,0] for Quota Group A and [30,10,0] for Quota Group B. If Quota Group A uses 40 CUs first, Quota Group B can use only the remaining 20 CUs and cannot use the resources that are being used by Quota Group A. If Quota Group A releases 10 CUs after a period of use, Quota Group B can then use 30 CUs.

Change the quota group of a project

You can create a quota group and change the quota group of a subscription project to the created quota group. This way, you can use the created quota group to isolate CUs for the project.

  1. In the left-side navigation pane, click Projects.

  2. In the Subscription Projects section, find the project that you want to modify and click Modify in the Actions column.

  3. In the Modify Quota Group Information panel, select a quota group from the Quota Group drop-down list.修改配额组

  4. Click Run.