E-MapReduce:Configuring custom software

Configure during cluster creation

1. Log on to the E-MapReduce console.

2. In the top menu bar, select a region and resource group as needed.

3. On the EMR on ECS page, click CREATE_CLUSTER.

4. In the Software Configuration section, under Advanced Settings, turn on the Custom Software Configuration switch.

   Add a JSON-formatted configuration file to override default service parameters or add new ones during cluster creation. Example:

   [
       {
           "ApplicationName":"YARN",
           "ConfigFileName":"yarn-site.xml",
           "ConfigItemKey":"yarn.nodemanager.resource.cpu-vcores",
           "ConfigItemValue":"8"
       },
       {
           "ApplicationName":"YARN",
           "ConfigFileName":"yarn-site.xml",
           "ConfigItemKey":"aaa",
           "ConfigItemValue":"bbb"
       }
   ]

   • The following table describes the parameters.

     Parameter	Description
     ApplicationName	The service name, in uppercase.
     ConfigFileName	The name of the configuration file to modify. Note Follow the file naming conventions below. For Data Lake, DataFlow, OLAP, DataServing, or Custom clusters, the file extension is required. For example: yarn-site.xml. For legacy data lake (Hadoop) clusters, the file extension is not required. For example: yarn-site.
     ConfigItemKey	The name of the configuration item.
     ConfigItemValue	The value of the configuration item.

   • The following table lists the configuration files for each service.

     Service	Configuration file
     YARN	core-site.xml log4j.properties hdfs-site.xml mapred-site.xml yarn-site.xml httpfs-site.xml capacity-scheduler.xml hadoop-env.sh httpfs-env.sh mapred-env.sh yarn-env.sh
     Hive	hive-env.sh hive-site.xml hive-exec-log4j.properties hive-log4j.properties

   After you configure the parameters, continue creating the cluster. For more information, see Create Cluster.

Configure when adding a node group

When you add a node group to scale out a running cluster, you can also adjust service configurations. For more information, see Manage Node Groups.

Related documents

After a cluster is created, you can modify configurations on the service-specific page. For more information, see Manage Configuration Items.