E-MapReduce:Upgrade node configurations

Prerequisites

You must have an E-MapReduce (EMR) cluster. For more information, see Create a cluster.

Limitations

• You cannot upgrade the configuration of ECS instances from the big data instance family or those that use local SSDs.

• You can only upgrade the configuration of ECS instances in a node group. Downgrades are not supported.

Procedure

1. Go to the Nodes page.

   1. Log on to the E-MapReduce console.

   2. In the top navigation bar, select a region and a resource group.

   3. In the Actions column of the target cluster, click Nodes.

2. On the Nodes page, in the Actions column of the target node group, choose > Upgrade Configuration.

3. Configure the upgrade settings.

   1. In the Upgrade Configuration dialog box, configure the settings.

      Parameter	Description
      Instance Type	Select an instance type with higher specifications. Note The available instance types for an upgrade are displayed in the console.
      Rolling Restart	Off (Default): Restarts all ECS instances in the node group simultaneously. On: Restarts instances sequentially. The system waits for an ECS instance and its big data services to fully recover before restarting the next one.

   2. Click OK.

      Generating the order may take some time.

   3. Pay for the order.

      After payment, the new instance type is displayed in the console but does not take effect until the upgrade process is complete.

      Important

      The upgrade process begins after you pay for the order. During this process, nodes in the current node group are automatically restarted, which can impact your production workloads. Proceed with caution.

4. Optional: Modify the YARN configuration.

   If YARN is installed on your cluster, follow these steps to manually modify the YARN configuration to use the new cluster resources.

   1. On the Services tab in the E-MapReduce (EMR) console, click Configure in the YARN section.

   2. From the Default Cluster Configuration drop-down list, select Node Group Configuration.

      Click the Configure tab.

   3. From the Select a node group. drop-down list, select the upgraded node group.

      Available node groups include emr-master (MASTER), emr-core (CORE), and task (TASK).

   4. Modify the CPU configuration.

      In the search box, search for the yarn.nodemanager.resource.cpu-vcores parameter and modify its value based on your requirements.

      For compute-intensive workloads, a parameter-to-vCPU ratio of 1:1 is recommended. For mixed workloads, you can set a ratio of up to 2:1.

      For example, if a compute node has 32 vCores for a compute-intensive workload, set yarn.nodemanager.resource.cpu-vcores to 32. If a compute node has 32 vCores for a mixed workload, you can set the value of yarn.nodemanager.resource.cpu-vcores to a value between 32 and 64.

   5. Modify the memory configuration.

      In the search box, search for the yarn.nodemanager.resource.memory-mb parameter and set its value to node memory*0.8, in MB.

      For example, if the node has 32 GB of memory, set yarn.nodemanager.resource.memory-mb to 26214.

   6. Save the configuration.

      1. At the bottom of the page, click Save.

      2. In the dialog box that appears, enter an Execution Reason and click Save.

   7. Deploy the configuration.

      1. Choose More > Configure.

      2. In the dialog box that appears, enter an Execution Reason and click OK.

      3. In the confirmation dialog box, click OK.

         You can click Operation History at the top of the page to check the task status. When the task status changes to Complete, restart the service.

   8. Restart the service.

      1. Choose More > Restart.

      2. In the dialog box that appears, enter an Execution Reason and click OK.

      3. In the confirmation dialog box, click OK.

         You can click Operation History at the top of the page to check the status. When the status changes to Complete, the service has restarted successfully.

      Important

      Restart ResourceManager during off-peak hours to prevent running jobs from failing.