After auto scaling runs, you can review cluster node changes and the execution history of each scaling activity.
Prerequisites
-
You have created a DataLake, Dataflow, OLAP, DataServing, or Custom cluster. For more information, see Create a cluster.
-
An auto scaling policy is configured for the cluster. For more information, see Create a custom auto scaling policy and Create a managed auto scaling policy.
Procedure
-
Go to the Auto Scaling page.
-
Log on to the EMR on ECS console.
-
In the top navigation bar, select a region and resource group based on your requirements.
-
Click the name of the target cluster.
-
On the page that appears, click the Auto Scaling tab.
-
-
Click the Auto Scaling Activity Analysis tab to view the following information.
-
Auto scaling activity data charts
Chart
Description
Cluster overview
Displays the hourly number of running nodes and cores in the cluster over the selected time range.
Cluster instance distribution by billing method
Displays the hourly number of nodes and cores broken down by pay-as-you-go and preemptible instances over the selected time range.
Cluster core hour statistics
Displays the hourly core count for pay-as-you-go and preemptible instances over the selected time range.
Cluster auto scaling overview
Displays the hourly number of nodes added and removed by auto scaling over the selected time range.
Successful/failed auto scaling activities
Displays the hourly count of successful and failed auto scaling activities over the selected time range.
Distribution of scaling failure reasons
Displays the breakdown of failure reasons for auto scaling activities over the selected time range.
-
Auto scaling records
You can filter auto scaling records by triggering rule name, node group ID, status, date, and node name. The following table describes each activity status.
Status
Description
Running
The activity is in progress.
Successful Execution
All requested nodes were successfully added or removed as specified by the auto scaling rule.
Execution failed
No nodes were added or removed. The auto scaling rule failed entirely.
Partially Successful
Some nodes were added or removed as specified by the auto scaling rule, but others failed due to issues like disk quota limits or insufficient ECS resources.
Request Denied
The system rejects a scaling request if it would cause the instance count to exceed the configured maximum or drop below the minimum, or if another auto scaling activity is already in progress.
NoteThe system periodically clears expired data, so the console displays auto scaling records for the last 60 days only.
-