E-MapReduce (EMR) provides the Custom Software Settings feature for you to customize the configurations of software, such as Hadoop, Hive, and Pig, when you create a cluster.

Prerequisites

You can use this feature only when you create a cluster.

Procedure

  1. Log on to the Alibaba Cloud EMR console.
  2. In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.
  3. Click the Cluster Management tab.
  4. Click Cluster Wizard in the upper-right corner.
  5. In the Software Settings step, turn on Custom Software Settings in the Advanced Settings section.
    You can select a configuration file in the JSON format to overwrite or add default cluster parameters. The following example shows the content of a configuration file in the JSON format:
    [
        {
            "ServiceName":"YARN",
            "FileName":"yarn-site",
            "ConfigKey":"yarn.nodemanager.resource.cpu-vcores",
            "ConfigValue":"8"
        },
        {
            "ServiceName":"YARN",
            "FileName":"yarn-site",
            "ConfigKey":"aaa",
            "ConfigValue":"bbb"
        }
    ]
    where:
    • FileName: the name of the configuration file. You must remove the file name extension.
    • ServiceName : the name of the service. Only uppercase letters are allowed.
    • ConfigKey: the name of the parameter.
    • ConfigValue: the value of the parameter.
    The following table lists the configuration files of each service.
    Service Configuration file
    Hadoop
    • core-site.xml
    • log4j.properties
    • hdfs-site.xml
    • mapred-site.xml
    • yarn-site.xml
    • httpsfs-site.xml
    • capacity-scheduler.xml
    • hadoop-env.sh
    • httpfs-env.sh
    • mapred-env.sh
    • yarn-env.sh
    Pig
    • pig.properties
    • log4j.properties
    Hive
    • hive-env.sh
    • hive-site.xml
    • hive-exec-log4j.properties
    • hive-log4j.properties
  6. After software configuration is complete, click Next: Hardware Settings to continue the cluster creation.
    For more information about how to create a cluster, see Create a cluster.