All Products
Search

This Product

    E-MapReduce:Create a managed auto scaling policy

    Document Center

    E-MapReduce:Create a managed auto scaling policy

    Last Updated:Sep 16, 2026

    Managed auto scaling lets you set the minimum and maximum number of task nodes. E-MapReduce (EMR) then automatically adjusts the number of task nodes based on your workload and optimizes resource allocation based on historical job runs. This ensures your jobs complete while controlling costs.

    Prerequisites

    • You have created a DataLake, Dataflow, OLAP, DataServing, or Custom cluster. For more information, see Create a cluster.

    • You have created a task node group that uses pay-as-you-go or preemptible instances in the cluster. For more information, see Create a node group.

    Limitations

    • To reduce scale-out failures caused by insufficient ECS inventory, you can specify multiple instance types when you create a node group. You can select up to 10 ECS instance types as alternatives. The system attempts to provision instances starting from the first instance type in your list. If an instance type is unavailable, the system automatically proceeds to the next one until the request is fulfilled. The actual instance types purchased may vary based on inventory changes.

    • Managed auto scaling rules are supported only on clusters where the YARN service is deployed.

    • If Trino, Presto, StarRocks, Impala, or ClickHouse services are deployed in the cluster, the effectiveness of managed auto scaling is not guaranteed.

    Notes

    Switching the auto scaling mode invalidates all existing rules. Proceed with caution.

    Procedure

    Method 1: For an existing cluster

    After you enable managed auto scaling, the system continuously monitors the cluster's YARN load and analyzes historical peak load changes to automatically adjust the number of task nodes.

    1. Go to the Auto Scaling page.

      1. Log on to the EMR on ECS console.

      2. In the top navigation bar, select a region and resource group based on your requirements.

      3. Click the name of the target cluster.

      4. On the page that appears, click the Auto Scaling tab.

    2. On the Configure Auto Scaling tab, in the Configure Auto Scaling Rule section, click Managed Auto Scaling Rule.

    3. Configure the scaling rule.

      1. In the dialog box that appears, click Reconfigure and configure the following parameters.

        • Minimum Number of Task Nodes: The minimum number of task nodes to keep in the cluster during a scale-in.

        • Maximum Number of Task Nodes: The maximum number of task nodes in the cluster during a scale-out.

        • Maximum Number of Pay-as-you-go Task Nodes: The maximum number of pay-as-you-go nodes that can be added when a scale-out is triggered. This parameter defines the proportion of pay-as-you-go nodes to preemptible instances. By default, it matches the value of Maximum number of task nodes.

          Note

          If the cluster has a task node group that consists of preemptible instances, the value of Maximum Number of Pay-as-you-go Task Nodes can be smaller than the value of Maximum Number of Task Nodes. For example, if Minimum number of task nodes is set to 0, Maximum number of task nodes is set to 20, and Maximum number of pay-as-you-go task nodes is set to 15, the system first attempts to add 15 pay-as-you-go nodes during a scale-out. If more capacity is needed, the system provisions the remaining nodes using preemptible instances.

      2. After you complete the configuration, click Save and Apply.

    Method 2: During cluster creation

    1. Log on to EMR on ECS.

    2. In the top navigation bar, select the required region and resource group.

    3. Click CREATE_CLUSTER. For more information about the parameters, see Create a cluster.

      Note

      To configure the rules, you must first add a task node group that uses pay-as-you-go or preemptible instances.

    4. Configure Cluster Scaling.

      1. Select Managed Auto Scaling Rule.

      2. Click Managed Auto Scaling Rule next to Edit, configure the parameters, and then click Save and Apply.

        • Minimum Number of Task Nodes: The minimum number of task nodes to keep in the cluster during a scale-in.

        • Maximum Number of Task Nodes: The maximum number of task nodes in the cluster during a scale-out.

        • Maximum Number of Pay-as-you-go Task Nodes: The maximum number of pay-as-you-go nodes that can be added when a scale-out is triggered. This parameter defines the proportion of pay-as-you-go nodes to preemptible instances. By default, it matches the value of Maximum number of task nodes.

          Note

          If the cluster has a task node group that consists of preemptible instances, the value of Maximum Number of Pay-as-you-go Task Nodes can be smaller than the value of Maximum Number of Task Nodes. For example, if Minimum number of task nodes is set to 0, Maximum number of task nodes is set to 20, and Maximum number of pay-as-you-go task nodes is set to 15, the system first attempts to add 15 pay-as-you-go nodes during a scale-out. If more capacity is needed, the system provisions the remaining nodes using preemptible instances.

    5. Confirm the order and create the cluster.