E-MapReduce:Partitioning Nodes by Using Node Labels

Partition types

When you create a partition, you can select Exclusive as the partition type. With an Exclusive partition, containers are allocated to nodes in the partition only when the partition is an exact match. This ensures that jobs are physically isolated to their designated nodes.

For workload isolation—the primary use case of this topic—use Exclusive partitions.

Step 1: Edit resource queues

1. Log on to the EMR console. On the EMR on ECS page, find your cluster and click Services in the Actions column. On the Services tab, click YARN.

2. On the YARN service page, click the Edit Resource Queue tab, then click Manage Queues.

3. In the Manage Child Queues dialog box, click Add Queue to create the warehouse and analysis queues. Set the Queue Capacity for each queue, and set the Status to Started:

   Queue	Queue capacity
   warehouse	60
   analysis	30
   default	10

   

   

4. Click Not Effective Yet. In the Configurations to Take Effect dialog box, click refresh_queues in the Actions column.

   

Step 2: Add partitions and associate nodes

1. Click the Partitions tab, then click Add Partition to create the streaming partition. In the Add Partition dialog box, configure the following parameters:

   • Partition Name: streaming

   • Partition Type: Exclusive — containers are allocated to nodes in this partition only when the partition is an exact match.

   • Associated Node Groups: Select the node group to associate with this partition. In this example, emr-task-3 is selected. The emr-task-3 node group contains memory-optimized instances suited for real-time jobs that hold large amounts of intermediate data and require low latency. Choose a node group based on your instance type and workload requirements.

   

2. Repeat the preceding steps to add the batch partition.

   

3. Click Not Effective Yet. In the Configurations to Take Effect dialog box, click refresh_labels in the Actions column.

   

4. After the configuration takes effect, confirm the total resources for each partition in the Total Resources column.

   

Step 3: Enable partition-queue association management

1. On the Edit Resource Queue tab, select the batch partition from the Select Partition drop-down list, then turn on Enable Partition-queue Association Management.

2. In the Enable Partition-queue Association Management dialog box, set the Queue Capacity for each queue, then click OK. Repeat this step for the streaming partition.

   In this example, all queues have access to both partitions. To restrict a partition to jobs from specific queues only, set the capacity of other queues to 0.

   Queue	Capacity in batch partition	Capacity in streaming partition
   warehouse	60	70
   analysis	30	20
   default	10	10

   

3. Click Not Effective Yet. In the Configurations to Take Effect dialog box, click refresh_queues in the Actions column.

4. Verify that the node labels are active on the YARN web UI.

   • Node Labels page: confirms the partitions and their types.

   • Scheduler page: confirms the queue-partition capacity assignments.

Step 4: Submit jobs to specific partitions

With queues and node labels configured, submit jobs to a specific queue and partition using the spark.yarn.am.nodeLabelExpression and spark.yarn.executor.nodeLabelExpression parameters.

1. Log on to the master node and go to the Spark installation directory:

   cd /opt/apps/SPARK3/spark-3.3.1-hadoop3.2-1.1.1

2. Submit a Spark job targeting the warehouse queue and batch partition:

   If you specify only a queue without a node label expression, the job runs on the queue's default partition. To change a queue's default partition, click Edit in the Actions column of the queue on the Edit Resource Queue tab.

   Parameter	Description	Value in this example
   --class	Main class of the application	org.apache.spark.examples.SparkLR
   --master	Resource manager	yarn
   --deploy-mode	Location where the driver starts	cluster
   --driver-memory	Driver memory	1g
   --executor-memory	Memory per executor	2g
   --conf spark.yarn.am.nodeLabelExpression	Partition for the AM	batch
   --conf spark.yarn.executor.nodeLabelExpression	Partition for executors	batch
   --queue	Target queue	warehouse

   ./bin/spark-submit \
     --class org.apache.spark.examples.SparkLR \
     --master yarn \
     --deploy-mode cluster \
     --driver-memory 1g \
     --executor-memory 2g \
     --conf spark.yarn.am.nodeLabelExpression=batch \
     --conf spark.yarn.executor.nodeLabelExpression=batch \
     --queue=warehouse \
     examples/jars/spark-examples_2.12-3.3.1.jar

   Both spark.yarn.am.nodeLabelExpression and spark.yarn.executor.nodeLabelExpression must be set to route the ApplicationMaster (AM) and executors to the target partition.

3. On the YARN web UI, confirm that the SparkLR job is running in the warehouse queue on the batch partition.