When ECS capacity is partially unavailable during a scale-out operation, Best-effort Delivery keeps the operation moving instead of failing entirely. The system delivers all successfully created nodes and marks the operation as Partially Successful, so your cluster grows as much as current capacity allows.
Prerequisites
Before you begin, ensure that you have:
-
A DataLake, OLAP (online analytical processing), Dataflow, DataServing, or custom cluster. For more information, see Create a cluster.
Limits
| Scale-out type | Constraint |
|---|---|
| Manual scale-out | Pay-as-you-go node groups only |
| Auto scale-out | Task node groups that contain preemptible instances or pay-as-you-go instances only |
Enable Best-effort Delivery for manual scale-out
Best-effort Delivery for manual scale-out applies to pay-as-you-go node groups only.
-
Go to the Nodes tab.
-
Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.
-
In the top navigation bar, select a region and a resource group.
-
Find the cluster and click Nodes in the Actions column.
-
-
On the Nodes tab, find the node group and click Scale Out in the Actions column.
-
In the Scale Out dialog box, turn on Best-effort Delivery.
-
In the confirmation message, click OK.
Best-effort Delivery applies only to the current scale-out activity. If ECS resources are insufficient or some ECS instances fail to deploy, the system delivers all successfully created nodes rather than aborting the entire operation.
-
Click OK.
For more information about scaling out a cluster, see Scale out an EMR cluster.
Enable Best-effort Delivery for auto scale-out
Best-effort Delivery for auto scale-out applies to task node groups that contain preemptible instances or pay-as-you-go instances only.
-
Go to the Auto Scaling tab.
-
Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.
-
In the top navigation bar, select a region and a resource group.
-
Find the cluster and click the cluster name in the Cluster ID/Name column.
-
Click the Auto Scaling tab.
-
-
On the Configure Auto Scaling subtab, find the node group and click Edit in the Actions column.
-
In the Configure Auto Scaling panel, click Add Rule in the Scale Out section.
-
In the Configure Scale Out Rule dialog box, turn on Best-effort Delivery.
-
In the confirmation message, click OK.
Best-effort Delivery applies only to the current scale-out activity triggered by this rule.
-
Click OK.
For more information about configuring auto scaling rules, see Configure custom auto scaling rules.
View scale-out results
When Best-effort Delivery is on and the number of delivered nodes falls between 1 and the requested count, the operation status is Partially Successful. This means some nodes were created successfully while others could not be provisioned due to insufficient ECS capacity.
To view the details of a Partially Successful operation:
-
Operation History: In the upper-right corner of the cluster details page, click Operation History. Find the operation and move the pointer over Partially Successful.
-
Events: In the left-side navigation pane, click Events. On the Operation Events tab, find the operation and click Details in the Actions column.
Subscribe to Partially Successful alerts
To receive notifications when a scale-out operation results in Partially Successful, create an event-triggered alert rule in CloudMonitor. For more information, see Create and test a system event-triggered alert rule.
Use the following parameters:
| Parameter | Value |
|---|---|
| Product type | E-MapReduce |
| Event type | Cluster |
| Event name | Cluster:NodeGroup.Resize:PartialCompleted |