All Products
Search

This Product

    E-MapReduce:Common file paths

    Document Center

    E-MapReduce:Common file paths

    Last Updated:Mar 26, 2026

    E-MapReduce (EMR) clusters use a consistent directory layout for service binaries, logs, and configuration files. Log on to the master node of your cluster to access these paths directly.

    The paths differ by cluster type: DataLake cluster and Hadoop cluster.

    DataLake cluster

    Service directories

    Services are installed under /opt/apps/<SERVICE>/<service>-current.

    Service Directory
    HDFS /opt/apps/HDFS/hdfs-current
    Hive /opt/apps/HIVE/hive-current
    Hudi /opt/apps/HUDI/hudi-current
    YARN /opt/apps/YARN/yarn-current
    Presto /opt/apps/PRESTO/presto-current
    Ranger /opt/apps/RANGER/ranger-current

    To look up the installation directory of any service, run:

    env |grep <service-name>

    For example, run env |grep hive to locate the Hive installation. The output is similar to:

    JINDOTABLE_EXTRA_CLASSPATH=/opt/apps/METASTORE/metastore-current/hive2
HIVE_HOME=/opt/apps/HIVE/hive-current
HIVE_LOG_DIR=/var/log/taihao-apps/hive
HIVE_CONF_DIR=/etc/taihao-apps/hive-conf
PATH=/opt/apps/JINDOSDK/jindosdk-current/bin:/opt/apps/HADOOP-COMMON/hadoop-common-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/apps/HIVE/hive-current/bin:/opt/apps/JINDODATA/jindodata-current/bin:/opt/apps/JINDODATA/jindodata-current/sbin:/opt/apps/SPARK-EXTENSION/spark-extension-current/bin:/opt/apps/SPARK3/spark-current/bin:/root/bin
OLDPWD=/var/log/emr/hive

    Log directories

    Logs are stored under /var/log/emr/<service>/.

    Service Log directory
    Spark /var/log/emr/spark/
    Hive /var/log/emr/hive/
    YARN /var/log/emr/yarn/
    JindoSDK /var/log/emr/jindosdk/

    Configuration file directories

    Configuration files are stored under /etc/emr/<service>-conf/.

    Service Configuration directory
    HDFS /etc/emr/hdfs-conf/
    Spark /etc/emr/spark-conf/
    Hive /etc/emr/hive-conf/
    Hudi /etc/emr/hudi-conf/
    Knox /etc/emr/knox-conf/
    YARN /etc/emr/hadoop-conf/
    ZooKeeper /etc/emr/zookeeper-conf/

    Hadoop cluster

    Service directories

    Services are installed under /usr/lib/<service>-current.

    Service Directory
    Hadoop /usr/lib/hadoop-current
    Spark /usr/lib/spark-current
    Hive /usr/lib/hive-current
    Flink /usr/lib/flink-current
    Flume /usr/lib/flume-current

    To look up the installation directory of any service, run:

    env |grep <service-name>

    For example, run env |grep spark to locate the Spark installation. The output is similar to:

    SPARK_HOME=/usr/lib/spark-current
SPARK_CONF_DIR=/etc/ecm/spark-conf
SPARK_LOG_DIR=/mnt/disk1/log/spark
PATH=/usr/lib/sqoop-current/bin:/usr/lib/jindosdk-current/bin:/usr/lib/hudi-current/bin:/usr/lib/hive-current/hcatalog/bin:/usr/lib/hive-current/bin:/usr/lib/datafactory-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib/flow-agent-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/usr/lib/jindodata-current//bin:/usr/lib/jindodata-current//sbin:/usr/lib/spark-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/root/bin
HADOOP_CLASSPATH=/opt/apps/extra-jars/*:/usr/lib/spark-current/yarn/spark-3.2.1-yarn-shuffle.jar
SPARK_PID_DIR=/usr/lib/spark-current/pids

    Log directories

    Logs are stored under /mnt/disk1/log/<service>/. Some services write to different directories depending on the node type.

    Service Log directory Node
    YARN ResourceManager /mnt/disk1/log/hadoop-yarn Master node
    YARN NodeManager /mnt/disk1/log/hadoop-yarn Core or task node
    HDFS NameNode /mnt/disk1/log/hadoop-hdfs Master node
    HDFS DataNode /mnt/disk1/log/hadoop-hdfs Core or task node
    Hive /mnt/disk1/log/hive Master node
    ESS /mnt/disk1/log/ess/ Master node, core node, or task node

    Configuration file directories

    Configuration files are stored under /etc/ecm/<service>-conf/.

    Service Configuration directory
    Hadoop /etc/ecm/hadoop-conf/
    Spark /etc/ecm/spark-conf/
    Hive /etc/ecm/hive-conf/
    Flink /etc/ecm/flink-conf/
    Flume /etc/ecm/flume-conf/
    SSH access is read-only for configuration files. To modify configuration parameters, log on to the EMR console.

    Data directories

    Data type Directory
    JindoFS cached data /mnt/disk1/jindodata/