This topic describes how to impose limits on pay-as-you-go resources consumed by computing jobs.
Pay-as-you-go resources in MaxCompute are scalable. Computing jobs can use these resources without limits. To avoid unexpected consumption, we recommend that you monitor the resources consumed by computing jobs.
MaxCompute allows you to set an upper limit for resources consumed by an SQL statement. Before you execute an SQL statement, MaxCompute estimates the resources to be consumed by this statement. If the estimated consumption exceeds the specified upper limit, the SQL statement cannot be executed, and a failure message is returned. You can use this feature to avoid unexpected charges caused by an SQL statement.
Configure an upper limit for resources consumed by an SQL statement
- To configure an upper limit for a project, run the following command as the project
owner or a user assigned the Super_Administrator role:
m_value indicates an upper limit for resources consumed by an SQL statement. m_value is calculated by using the following formula:
Number of gigabytes scanned by an SQL statement x Complexity of the SQL statement. m_value does not indicate your consumption fees.
If the estimated consumption value of an SQL statement exceeds m_value, the SQL statement cannot be executed.
- To configure an upper limit for a session, run the following command along with the
SQL statement you want to execute. This configuration takes effect for the specified
SQL statement only once.
If you configure an upper limit for both a project and its session, the configuration for the session takes precedence over that for the project. If you set m_value to 100 for a project and to 200 for a session in the project, the statement cannot be executed if the estimated resource consumption exceeds 200. SQL statements that do not have a session-level upper limit are subject to the project-level upper limit. These SQL statements cannot be executed if the estimated resource consumption exceeds 100.
- Set m_value for a project based on the historical consumption records and your expected consumption
of resources that are used by an SQL statement.
You can download the daily consumption information and calculate the consumption of an SQL job whose data category is computation in a project by using the following formula:
Number of gigabytes scanned by an SQL statement × Complexity of the SQL statement. After the calculation is complete, you can determine an appropriate m_value based on the maximum daily consumption and business fluctuation.
You can also convert the calculated consumption to consumption fees,determine appropriate fees based on business conditions, and then convert the fees to an m_value.
- We recommend that you set m_value for a project and then for sessions for some SQL statements.
Note Before you set a session-level upper limit, make sure that this upper limit is required for the SQL statement you want to execute. This avoids abuse of this feature.
- The execution of a job is affected no matter whether you set an upper limit for a project or session. We recommend that you use this feature with the features that provide alerts related to job execution failures, for example, the intelligent monitoring feature of DataWorks. This way, failed jobs are immediately detected and processed to minimize the impact of these jobs on your business.