E-MapReduce:Customize software configurations

Configure during cluster creation

1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

2. In the top menu bar, select a region and resource group.

3. On the EMR on ECS page, click Create Cluster.

4. In the Software Configuration step, go to the Advanced Settings section and turn on the Custom Software Configuration switch.

   

5. Enter a JSON array that specifies the configuration items to override or add. Each object in the array targets one configuration item. The following example sets a YARN CPU vcore count and adds a custom key-value pair:

   [
       {
           "ApplicationName": "YARN",
           "ConfigFileName": "yarn-site.xml",
           "ConfigItemKey": "yarn.nodemanager.resource.cpu-vcores",
           "ConfigItemValue": "8"
       },
       {
           "ApplicationName": "YARN",
           "ConfigFileName": "yarn-site.xml",
           "ConfigItemKey": "aaa",
           "ConfigItemValue": "bbb"
       }
   ]

   The following table describes each parameter.

   Parameter	Description
   ApplicationName	The service name. Must be all uppercase letters—for example, YARN.
   ConfigFileName	The service configuration file that contains the setting—for example, yarn-site.xml maps to YARN's site configuration. See Supported configuration files for the full list. The file name format depends on your cluster type—see the note below.
   ConfigItemKey	The name of the configuration item to set.
   ConfigItemValue	The value to assign to the configuration item.

   The ConfigFileName format depends on your cluster type: - DataLake, DataFlow, OLAP, DataServing, or Custom clusters: Include the file extension. Example: yarn-site.xml. - Legacy data lake (Hadoop) clusters: Omit the file extension. Example: yarn-site.

6. Continue with the remaining cluster creation steps. For details, see Create a cluster.

Configure when adding a node group

When you scale out a running cluster by adding a new node group, you can also adjust service configuration items at that time. For details, see Manage node groups.

Supported configuration files

The following tables list the configuration files available for each service.

YARN

Hive

What's next

After a cluster is created, you can adjust configuration items directly on the service configuration page. See Manage configuration items.