E-MapReduce:Node labels

Version compatibility

This topic covers centralized mode. If your cluster runs EMR V5.11.1 or EMR V3.45.1 or later, manage YARN partitions directly in the EMR console. For more information, see Manage YARN partitions in the EMR console.

Key concepts

Prerequisites

Before you begin, ensure that you have:

• An EMR cluster running a version earlier than EMR V5.11.1 or EMR V3.45.1

• Sufficient permissions to modify YARN configurations and restart ResourceManager

• Access to the EMR console

Step 1: Enable node labels in yarn-site.xml

1. In the EMR console, go to the YARN service page and click the Configure tab.

2. Click the yarn-site.xml tab and add the following configuration items:

   Key	Value	Description
   yarn.node-labels.enabled	true	Enables the node label feature
   yarn.node-labels.fs-store.root-dir	/tmp/node-labels	Storage directory for node label data

3. Click Save, then restart ResourceManager for the changes to take effect.

If yarn.node-labels.fs-store.root-dir is set to a directory path rather than a full URL, YARN uses the file system specified by fs.defaultFS. This is equivalent to ${fs.defaultFS}/tmp/node-labels. The default file system in EMR is Hadoop Distributed File System (HDFS).

Step 2: Create partitions and assign nodes

Run the following commands as the hadoop user:

# Switch to the hadoop user
sudo su - hadoop

# Create partitions
yarn rmadmin -addToClusterNodeLabels "DEMO"
yarn rmadmin -addToClusterNodeLabels "CORE"

# List nodes to get their hostnames
yarn node -list

# Map nodes to partitions
# In EMR clusters, each node runs exactly one NodeManager, so no port is needed.
yarn rmadmin -replaceLabelsOnNode "core-1-1.c-XXX.cn-hangzhou.emr.aliyuncs.com=DEMO"
yarn rmadmin -replaceLabelsOnNode "core-1-2.c-XXX.cn-hangzhou.emr.aliyuncs.com=DEMO"

For node groups that scale out automatically: Add the yarn rmadmin -replaceLabelsOnNode command to the bootstrap script of the node group. This ensures partitions are assigned after NodeManager starts on each new node.

After running the commands, check the configuration results on the web UI of ResourceManager.

• Node configurations:

• Node label configurations:

Step 3: Configure Capacity Scheduler queues

Add the following configuration items to capacity-scheduler.xml. Replace <queue-path> with the queue path and <label> with the partition name.

Root queue requirement: The capacity value for all queues defaults to 0. For child queues to get any resources from a partition, first set yarn.scheduler.capacity.root.accessible-node-labels.<label>.capacity to 100 on the root queue. Then assign capacity values across child queues so they sum to 100.

The following example configures the default queue to use the DEMO partition:

<configuration>
  <!-- Grant the default queue access to the DEMO partition -->
  <property>
    <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
    <value>DEMO</value>
  </property>
  <!-- Set root queue capacity in the DEMO partition to 100 (required) -->
  <property>
    <name>yarn.scheduler.capacity.root.accessible-node-labels.DEMO.capacity</name>
    <value>100</value>
  </property>
  <!-- Set default queue capacity in the DEMO partition -->
  <property>
    <name>yarn.scheduler.capacity.root.default.accessible-node-labels.DEMO.capacity</name>
    <value>100</value>
  </property>
  <!-- Optional: set maximum capacity for the default queue in the DEMO partition -->
  <property>
    <name>yarn.scheduler.capacity.root.default.accessible-node-labels.DEMO.maximum-capacity</name>
    <value>100</value>
  </property>
  <!-- Optional: send jobs from the default queue to DEMO by default -->
  <property>
    <name>yarn.scheduler.capacity.root.default.default-node-label-expression</name>
    <value>DEMO</value>
  </property>
</configuration>

To apply the changes without restarting YARN, use the refreshQueues feature on the Status tab of the YARN service page.

After applying the Capacity Scheduler configuration and refreshing queues:

Step 4: Set node label expressions in compute jobs

In addition to queue-level defaults, you can specify partitions per job for MapReduce, Spark, and Flink. Tez does not support node label expressions.

MapReduce

Spark

Flink

Verify the configuration

After completing the steps above, confirm the configuration through the ResourceManager web UI.

Open the ResourceManager web UI and verify that node partition assignments and queue-level partition settings appear as expected. You can also run yarn node -list -showDetails to query the partition to which each node belongs.

FAQ

What's next

• Manage YARN partitions in the EMR console (EMR V5.11.1 / V3.45.1 and later)

• YARN Node Labels (Apache Hadoop documentation)