E-MapReduce:Configuring custom software

Configure during cluster creation

1. Log on to the E-MapReduce console.

2. In the top menu bar, select a region and resource group as needed.

3. On the EMR on ECS page, click CREATE_CLUSTER.

4. In the Software Configuration section, under Advanced Settings, turn on the Custom Software Configuration switch.

   You can add a JSON-formatted configuration file to override default service parameters or add new ones during cluster creation. The following code is an example.

   [
       {
           "ApplicationName":"YARN",
           "ConfigFileName":"yarn-site.xml",
           "ConfigItemKey":"yarn.nodemanager.resource.cpu-vcores",
           "ConfigItemValue":"8"
       },
       {
           "ApplicationName":"YARN",
           "ConfigFileName":"yarn-site.xml",
           "ConfigItemKey":"aaa",
           "ConfigItemValue":"bbb"
       }
   ]

   • The following table describes each parameter.

     Parameter	Description
     ApplicationName	The service name. The value must be in uppercase.
     ConfigFileName	The name of the configuration file to be modified. Note To ensure the configuration file is applied correctly, follow the specified file naming conventions. For Data Lake, DataFlow, OLAP, DataServing, or Custom clusters, the file extension is required. For example: yarn-site.xml. For legacy data lake (Hadoop) clusters, the file extension is not required. For example: yarn-site.
     ConfigItemKey	The name of the configuration item.
     ConfigItemValue	The value to be set for the configuration item.

   • The following table lists the configuration files for each service.

     Service	Configuration file
     YARN	core-site.xml log4j.properties hdfs-site.xml mapred-site.xml yarn-site.xml httpfs-site.xml capacity-scheduler.xml hadoop-env.sh httpfs-env.sh mapred-env.sh yarn-env.sh
     Hive	hive-env.sh hive-site.xml hive-exec-log4j.properties hive-log4j.properties

   After you configure the component parameters, continue creating the cluster. For details, see Create Cluster.

Configure when adding a node group

You can scale out a running cluster by adding node groups. When adding a node group, you can also adjust service configurations. For details, see Manage Node Groups.

Related documents

After a cluster is created, you can also modify configurations on the page for each service. For details, see Manage Configuration Items.