MaxCompute Management is a management platform that allows you to perform routine O&M operations. For example, you can view the information of a job, view the consumption of resources (including storage resources and CUs), view the information of resources in a project, modify a project, and create, modify, query, or delete a quota group. This topic describes the features that are provided by MaxCompute Management in detail.
Prerequisites
- We recommend that you purchase sufficient CUs to meet your business requirements and make full use of MaxCompute Management.
- If you disable the AccessKey pair of your Alibaba Cloud account, you cannot use MaxCompute Management as a RAM user of this account.
Go to MaxCompute Management
- Log on to the MaxCompute console and select a region in the top navigation bar.
- Click the Housekeeper tab.
Overview
Parameter | Description |
---|---|
Quota Group | The quota group that you want to query. This parameter is empty by default. This indicates
that all quota groups are queried.
You can select a time range from the date and time picker next to Quota Group. By default, data from the last 24 hours is queried. |
Used CUs | The number of CUs that are used by all projects in the specified quota group within
the specified time range. You can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.
Note If a custom quota group exists and it is a sharing quota group, the number of used
CUs may be greater than the number of purchased CUs in each quota group at a specific
point in time. This ensures that a minimum number of exclusive CUs are used in each
quota group. A sharing quota group is a quota group in which the maximum number of
exclusive CUs is greater than the minimum number.
|
Total CUs |
|
Used Storage | The storage resources that are used by all the projects in the specified quota group within the specified time range. |
Requested CUs | The trend in the number of exclusive and shared CUs that are requested by all the projects in the specified quota group within the specified time range. |
Used CUs | The trend in the number of exclusive and shared CUs that are used by all the projects in the specified quota group within the specified time range. |
Total CUs | The trend in the total number of exclusive and shared CUs in the specified quota group within the specified time range. |
Storage Size | The trend in the number of storage resources that are used by all the projects in the specified quota group within the specified time range. |
View the status of jobs
MaxCompute Management takes job snapshots at an interval of 2 minutes. Job snapshots allow you to view the status of jobs in a specific quota group at a specific point in time. Fixed parameters are displayed in job snapshots at each point in time. This allows you to track the resource usage of a specific quota group at a specific point in time.
- In the left-side navigation pane, click Jobs.
- On the Job Snapshots tab, select a quota group or a project that uses the subscription or pay-as-you-go
billing method and select a point in time to view job snapshots.
Note On the Overview page, you can click a point in time in the CU Usage Trend chart to view the job snapshots at that point in time.
Parameter Description InstanceID The instance ID of the job. Each MaxCompute job runs as an instance. You can click the instance ID to go to the Logview page and view the progress of the job. For more information about how to view job progress in Logview, see Use Logview to view job information. Submitted By The Alibaba Cloud account that is used to run the job. You can find the job owner based on the account information. If a job occupies an excessive amount of resources and affects the execution of other jobs, you can request the job owner to terminate the job. For more information about how to terminate a job, see Instance operations. Project The project to which the job belongs. Quota Group You can select only subscription quota groups. This parameter is available only for the jobs that use subscription resources. DataWorks Node ID The ID of the DataWorks node on which the job is running. If this parameter is left empty, the job is not submitted in DataWorks. Priority The priority of the job. This parameter is available only for the jobs that use subscription resources. Valid values: 0 to 9. A small value indicates a high priority.
Billing Method The billing method that is used by the project to which the job belongs. Valid values: Subscription and Pay-as-you-go. CPU Utilization (%) The percentage of CPU resources that are used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources. Memory Usage (%) The percentage of memory resources used by the MaxCompute job to the maximum value in the quota group. This parameter is available only for the jobs that use subscription resources. Running Status The status of the job. Job Type The type of the job. Waiting Time The time spent waiting for available resources to run the job. Running Duration The duration for which the job runs. Submitted At The time at which the job was submitted. - On the Job Snapshots tab, click the icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section to enable auto-refresh. In the dialog box that appears, specify a refresh
interval.
To disable auto-refresh, on the Job Snapshots page, click the icon in the upper-right corner of the Job Snapshots of Subscription Projects or Job Snapshots of Pay-As-You-Go Projects section.
Terminate jobs
- On the Job Snapshots tab, click Stop Jobs above the job list.
- In the Stop Jobs panel, specify Instance IDs and Remarks. In this example, a project that uses the subscription billing method is used.
- Click Run.
View the operation records of jobs
- In the left-side navigation pane, click Jobs.
- On the page that appears, click the Job Operation Records tab to view the operation records of jobs.
- Find the operation record that you want to view and click View Details in the Actions column. In the panel that appears, view the operation details.
View the consumption of storage resources
- In the left-side navigation pane, click Projects. On the page that appears, you can separately view the projects that use the subscription
and pay-as-you-go billing methods.
In a project list, you can view the storage resources that are used by each project. You can also select a quota group or a project in the upper-right corner to filter projects.
- In the Subscription Projects section, click the name of a project. The Storage tab appears.
- Select a time range from the date and time picker and click OK.
View the consumption of CUs
- In the left-side navigation pane, click Quotas.
- In the Subscription Quota Groups section, click the name of a quota group. The Resource Consumption tab appears.
- Select a time range from the date and time picker and click OK.
Note The interval at which data is displayed depends on the selected time range.
Configure quota groups
Operation | Description | Step |
---|---|---|
Create a quota group | You can create a quota group. After a quota group is created, click Projects in the left-side navigation pane. Find the project that you want to modify and click Modify in the Actions column. In the Modify Quota Group Information panel, change the quota group of the project to the quota group you created. Then, the jobs in the project use the CUs of the new quota group by default. |
|
Modify a quota group | You can modify an existing quota group. |
|
Delete a quota group | You can delete an existing quota group. You cannot delete a quota group that contains projects. If you want to delete a quota group that contains projects, you must migrate the projects from the quota group to another quota group. |
|
Specify periods for resource scheduling | You can specify periods for resource scheduling. This way, different projects can
use exclusive CUs for different periods based on your business requirements. For example,
production projects require more CUs at night and fewer CUs in the daytime. Development
or analysis projects require more CUs in the daytime and fewer CUs at night. You can
specify different periods for these projects to schedule resources. This optimizes
CU usage. When you specify multiple periods for resource scheduling, take note of
the following rules:
|
|
- If you increase or decrease the number of available CUs, the minimum and maximum quotas of CUs vary accordingly for the default quota group. However, in this case, the quotas remain the same for custom quota groups.
- You cannot reduce the number of available exclusive CUs to a value that is less than the minimum quota of exclusive CUs in the default quota group.
- The minimum quota of CUs indicates the minimum number of CUs that a quota group can provide. The maximum quota of CUs indicates the maximum number of CUs that a quota group can provide.
Quota group configuration examples
- Multiple periods for resource scheduling disabled
- Resources are exclusively allocated to each quota group.
[Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [40,40,0] for Quota Group A and [20,20,0] for Quota Group B.
- Resources are allocated to quota groups on a first come, first served basis.
[Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs]: [60,40,0] for Quota Group A and [40,20,0] for Quota Group B.
- Resources are exclusively allocated to each quota group.
- Multiple periods for resource scheduling enabled
For example, a production project, a development project, and an analysis project exist. The peak hours of the production project are from 00:00:00 to 08:00:00, and the peak hours of the development and analysis projects are from 08:00:00 to 23:59:59. The resources are allocated in the following way:
- Two periods for resource scheduling are specified: 00:00:00 to 08:00:00 as Period 1 and 08:00:00 to 23:59:59 as Period 2.
- [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 1: [60,50,0] for a custom quota group and [60,10,0] for the default quota group.
- [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] in Period 2: [60,20,0] for a custom quota group and [60,40,0] for the default quota group.
- The production project uses the resources in the custom quota group. The development and analysis projects use the resources in the default quota group.
You cannot specify the scheduling sequence for quota groups. Resources are used on a first come, first served basis and cannot be preempted. For example, 60 exclusive CUs and no shared CUs are available for quota groups A and B. You set [Maximum Reserved CUs,Minimum Reserved CUs,Maximum Non-reserved CUs] to [40,20,0] for Quota Group A and [30,10,0] for Quota Group B. If Quota Group A uses 40 CUs first, Quota Group B can use only the remaining 20 CUs and cannot use the resources that are being used by Quota Group A. If Quota Group A releases 10 CUs after a period of use, Quota Group B can then use 30 CUs.
Change the quota group of a project
- In the left-side navigation pane, click Projects.
- In the Subscription Projects section, find the project that you want to modify and click Modify in the Actions column.
- In the Modify Quota Group Information panel, select a quota group from the Quota Group drop-down list.
- Click Run.