MaxCompute:Use cost optimization for cost reduction and efficiency

Notes

• The prices in this topic are for reference only. For actual pricing, refer to the product purchase page.

• We recommend gradually transitioning to the recommended plan based on your business needs and monitoring performance after applying the changes.

Scenario 1: High costs from idle subscription resources

A company in the early stages of building its data warehouse purchased 200 reserved CUs of subscription computing resources to ensure a critical batch of early-morning jobs completed by 08:00. Every day, 520 jobs run on these resources and finish on time or ahead of schedule, but the monthly computing cost reaches 4,400 USD.

Recently, the big data department received a cost-reduction OKR. The department head wanted to cut costs without impacting critical job completion times but was unsure how to develop the best plan. A data engineer suggested using the cost optimization feature in MaxCompute and demonstrated the following steps:

1. Go to the cost optimization page.

   1. Log on to the MaxCompute console. In the top-left corner, select a region. In the left-side navigation pane, choose Intelligent Optimization > Computing Resource Config Optimization.

   2. On the Computing Resource Config Optimization page, select a subscription level-1 quota to view the daily CU request forecast.

   The CU request forecast accurately reflects the current workload: smaller job batches run hourly, with a peak between 05:00 and 08:00.

2. Set estimation time points.

   In the Set Estimation Time Point section, the engineer sets two estimation time points based on the business workflow: 05:00 and 08:00. This ensures that pre-critical jobs finish by 05:00 and the critical batch itself finishes by 08:00.

3. View the evaluation result of the current plan.

   Click Current Plan Estimation to check the job completion status under the current resource configuration.

   

   The CU Consumption Simulation chart shows that with the current configuration (200 reserved CUs), no jobs are delayed but significant resource idle time exists, indicating room for cost optimization.

4. Set the optimization goal.

   Below the CU Consumption Simulation chart, the Set Optimization Goal section displays the current delay status for each estimation time point, consistent with the simulation chart.

   The engineer notices that the optimization goals default to the estimation time points, which align with the required job completion times. They then click Generate Recommended Plan

5. View the recommended plan.

   The recommended plan suggests purchasing 50 reserved CUs and adding 50 elastically reserved CUs for the time windows 04:00–05:00 and 06:00–08:00. Critical jobs are still estimated to finish before 08:00, while the monthly computing cost drops to 1,319.6 USD — a saving of approximately 70% compared to the current plan.

   

   Since a 30-minute delay would not significantly impact the business, the engineer decides to adjust the optimization goal.

6. Adjust the optimization goal.

   Return to the Set Optimization Goal section and change the optimization goal for the 08:00 estimation time point to 08:30.

7. View the new recommended plan.

   Click Generate Recommended Plan again. The new plan suggests purchasing 50 reserved CUs and adding 50 elastically reserved CUs for the time windows 04:00–05:00 and 06:00–07:00. Critical jobs are estimated to finish before 08:30, and the computing cost drops further to 1,246.4 USD — a saving of about 71.7% compared to the current plan.

8. Gradually implement the recommended plan.

   The department opts for a phased implementation, starting by reducing reserved CUs to 100 and re-evaluating after a trial period. Since the job volume did not increase significantly, the system still recommended downsizing and adding elastically reserved CUs. The team then implemented the initial recommended plan, which had no delays. After a trial run, critical jobs continued to complete on time almost every day, and the team successfully reduced their computing cost.

Scenario 2: Job delays from insufficient subscription resources

A company purchased 60 reserved CUs of subscription computing resources for its data warehouse. Each day, 520 jobs run on these resources, and a batch of critical jobs starting around 05:00 is expected to finish by 08:00. As the business expands and data volumes grow, an engineer notices frequent job delays caused by insufficient reserved CUs. To resolve the delays without a significant cost increase, the engineer uses the cost optimization feature in the MaxCompute console:

1. Go to the cost optimization page.

   1. Log on to the MaxCompute console. In the top-left corner, select a region. In the left-side navigation pane, choose Intelligent Optimization > Computing Resource Config Optimization.

   2. On the Computing Resource Config Optimization page, select a subscription level-1 quota to view the daily CU request forecast.

      The CU request forecast matches the current workload: smaller jobs run hourly, with a peak between 05:00 and 08:00.

2. Set estimation time points.

   In the Set Estimation Time Point section, the engineer sets two estimation time points based on the business workflow: 05:00 and 08:00. This ensures that pre-critical jobs finish by 05:00 and the critical batch finishes by 08:00.

3. View the evaluation result of the current plan.

   Click Current Plan Estimation to check the job completion status under the current resource configuration.

   

   The CU Consumption Simulation chart shows that with the current configuration (60 reserved CUs), jobs submitted before 05:00 are estimated to be delayed by 3 minutes. The critical jobs submitted between 05:00 and 08:00 experience estimated delays of up to 48 minutes. This simulation aligns with the actual delays observed.

4. Set the optimization goal.

   Below the CU Consumption Simulation chart, the Set Optimization Goal section displays the current delay status for each estimation time point, consistent with the simulation chart.

   The engineer sees that the optimization goals default to the desired completion times. They click Generate Recommended Plan

5. View the recommended plan.

   The CU Consumption Simulation (Recommended Plan) chart shows the recommended plan: purchase 50 reserved CUs and add 50 elastically reserved CUs for the time windows 04:00–05:00 and 06:00–08:00. This plan eliminates delays for both the 05:00 and 08:00 deadlines while reducing cost by 0.4 USD/month compared to the current plan.

   

6. Configure the recommended plan.

   The estimated cost is acceptable. The engineer begins by configuring only the recommended elastically reserved CUs and monitoring performance, without immediately adjusting the reserved CUs.

   1. In the left-side navigation pane of the MaxCompute console, choose Workspace > Quotas.

   2. On the Quotas page, find the level-1 quota you evaluated and click Quota Configuration in the Actions column.

   3. On the Quota Configuration page, go to the Quota Plans tab and click Add Plan.

   4. In the Add Quota Plan dialog box, set Elastically Reserved CUs to 50 and click OK.

   5. Configure the Quota time plan based on the recommended time windows. For more information, see Configure a Quota.

      Start time	Enabled quota plan
      00:00	Default
      04:00	The newly added Quota plan.
      05:00	Default
      06:00	The newly added Quota plan.
      08:00	Default

      Note

      The Default plan has 0 elastically reserved CUs.

   After a trial period, the engineer observes that the critical jobs complete on time almost every day, improving development efficiency without a significant increase in computing cost.