E-MapReduce is an open-source big data platform in Alibaba Cloud. You can log on to the primary node of a cluster to view relevant installation paths. You can also run the env |grep xxx command to view the paths.

Big data components

The big data software is installed in the /usr/lib/xxx directory. The following directories are used as examples:
  • Hadoop: /usr/lib/hadoop-current
  • Spark: /usr/lib/spark-current
  • Hive: /usr/lib/hive-current
  • Flink: /usr/lib/flink-current

Component logs

Component logs are stored in the /mnt/disk1/log/xxx directory. The following directories are used as examples:
  • YARN ResourceManager logs: /mnt/disk1/log/hadoop-yarn on the primary node
  • YARN NodeManager logs: /mnt/disk1/log/hadoop-yarn on a secondary node
  • HDFS NameNode logs: /mnt/disk1/log/hadoop-hdfs on the primary node
  • HDFS DataNode logs: /mnt/disk1/log/hadoop-yarn on a secondary node
  • Hive logs: /mnt/disk1/log/hive on the primary node

Configuration files

Configuration files are stored in the /etc/ecm/xxx directory. If you log on to a node to modify a configuration file of the cluster, the modified configuration file does not take effect. The following directories are used as examples:
Notice You can only view the parameter settings in configuration files. If you want to modify parameters, go to the EMR console.
  • Hadoop: /etc/ecm/hadoop-conf/
  • Spark: /etc/ecm/spark-conf/
  • Hive: /etc/ecm/hive-conf/