EMR (E-MapReduce) cluster resource management applies to scenarios that involve large-scale clusters with many tenants.

With EMR cluster resource management, the following features are supported.
  • Different resource queues are used based on the users. This helps to isolate resources.
  • Resource queues are elastic, which improves the efficiency of the cluster.
Note
  • Currently, you can choose Capacity Scheduler or Fair Scheduler for managing EMR cluster resources.
  • Currently, you can only manage resources of EMR Hadoop clusters.

Enable resource queues

  1. Log on to the Alibaba Cloud E-MapReduce console. Click Cluster Management to go to the Cluster Management page.
  2. Click Manage in the Actions column.
  3. In the left-side navigation pane, click Cluster Resources.
  4. Turn on Enable Resource Queue.
    Notice After turning on Enable Resource Queue, on the Configure tab page of YARN, capacity-scheduler and fair-scheduler sections are frozen. On the Cluster Resources page, the configurations are synchronized. You need to turn off Enable Resource Queue on the Cluster Resources page to configure clusters by using XML again on the Configure tab page of YARN.

Configure resource queues

  • After turning on Enable Resource Queue, choose fair scheduler or capacity scheduler as the scheduler type.
  • Click Queue Settings to go to the Queue Settings page.
  • Click Set Scheduler Defaults to set the default configurations of the scheduler.

Resource queues taking effect

After configuring resources queues, click Check Configuration to preview the configured XML file and click Deploy to deploy the configurations.

Note
  • After switching the scheduler type (for example, from Fair Scheduler to Capacity Scheduler) or modifying parameters related to resource preemption in Fair Scheduler, you need to restart all YARN components on the Clusters and Services page before clicking Deploy.
  • After creating resource queues or setting related parameters, you only need to click Deploy.
  • You can view the deployment progress of configurations on the Clusters and Services page.