All Products
Search

This Product

    E-MapReduce:Jalur file umum

    Document Center

    E-MapReduce:Jalur file umum

    Last Updated:Mar 27, 2026

    Kluster E-MapReduce (EMR) menggunakan tata letak direktori yang konsisten untuk binari layanan, log, dan file konfigurasi. Login ke node master kluster Anda untuk mengakses jalur-jalur ini secara langsung.

    Jalur tersebut berbeda berdasarkan jenis kluster: DataLake cluster dan Hadoop cluster.

    Kluster DataLake

    Direktori layanan

    Layanan diinstal di bawah /opt/apps/<SERVICE>/<service>-current.

    Layanan Direktori
    HDFS /opt/apps/HDFS/hdfs-current
    Hive /opt/apps/HIVE/hive-current
    Hudi /opt/apps/HUDI/hudi-current
    YARN /opt/apps/YARN/yarn-current
    Presto /opt/apps/PRESTO/presto-current
    Ranger /opt/apps/RANGER/ranger-current

    Untuk mencari direktori instalasi layanan apa pun, jalankan:

    env |grep <service-name>

    Misalnya, jalankan env | grep hive untuk menemukan lokasi instalasi Hive. Output-nya mirip dengan:

    JINDOTABLE_EXTRA_CLASSPATH=/opt/apps/METASTORE/metastore-current/hive2
HIVE_HOME=/opt/apps/HIVE/hive-current
HIVE_LOG_DIR=/var/log/taihao-apps/hive
HIVE_CONF_DIR=/etc/taihao-apps/hive-conf
PATH=/opt/apps/JINDOSDK/jindosdk-current/bin:/opt/apps/HADOOP-COMMON/hadoop-common-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/apps/HIVE/hive-current/bin:/opt/apps/JINDODATA/jindodata-current/bin:/opt/apps/JINDODATA/jindodata-current/sbin:/opt/apps/SPARK-EXTENSION/spark-extension-current/bin:/opt/apps/SPARK3/spark-current/bin:/root/bin
OLDPWD=/var/log/emr/hive

    Direktori log

    Log disimpan di bawah /var/log/emr/<service>/.

    Layanan Direktori log
    Spark /var/log/emr/spark/
    Hive /var/log/emr/hive/
    YARN /var/log/emr/yarn/
    JindoSDK /var/log/emr/jindosdk/

    Direktori file konfigurasi

    File konfigurasi disimpan di bawah /etc/emr/<service>-conf/.

    Layanan Direktori konfigurasi
    HDFS /etc/emr/hdfs-conf/
    Spark /etc/emr/spark-conf/
    Hive /etc/emr/hive-conf/
    Hudi /etc/emr/hudi-conf/
    Knox /etc/emr/knox-conf/
    YARN /etc/emr/hadoop-conf/
    ZooKeeper /etc/emr/zookeeper-conf/

    Kluster Hadoop

    Direktori layanan

    Layanan diinstal di bawah /usr/lib/<service>-current.

    Layanan Direktori
    Hadoop /usr/lib/hadoop-current
    Spark /usr/lib/spark-current
    Hive /usr/lib/hive-current
    Flink /usr/lib/flink-current
    Flume /usr/lib/flume-current

    Untuk mencari direktori instalasi layanan apa pun, jalankan:

    env |grep <service-name>

    Misalnya, jalankan env | grep spark untuk menemukan lokasi instalasi Spark. Output-nya mirip dengan:

    SPARK_HOME=/usr/lib/spark-current
SPARK_CONF_DIR=/etc/ecm/spark-conf
SPARK_LOG_DIR=/mnt/disk1/log/spark
PATH=/usr/lib/sqoop-current/bin:/usr/lib/jindosdk-current/bin:/usr/lib/hudi-current/bin:/usr/lib/hive-current/hcatalog/bin:/usr/lib/hive-current/bin:/usr/lib/datafactory-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib/flow-agent-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/usr/lib/jindodata-current//bin:/usr/lib/jindodata-current//sbin:/usr/lib/spark-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/root/bin
HADOOP_CLASSPATH=/opt/apps/extra-jars/*:/usr/lib/spark-current/yarn/spark-3.2.1-yarn-shuffle.jar
SPARK_PID_DIR=/usr/lib/spark-current/pids

    Direktori log

    Log disimpan di bawah /mnt/disk1/log/<service>/. Beberapa layanan menulis ke direktori berbeda tergantung pada jenis node.

    Layanan Direktori log Node
    YARN ResourceManager /mnt/disk1/log/hadoop-yarn Node master
    YARN NodeManager /mnt/disk1/log/hadoop-yarn Node core atau node task
    HDFS NameNode /mnt/disk1/log/hadoop-hdfs Node master
    HDFS DataNode /mnt/disk1/log/hadoop-hdfs Node core atau node task
    Hive /mnt/disk1/log/hive Master node
    ESS /mnt/disk1/log/ess/ Node master, node core, atau node task

    Direktori file konfigurasi

    File konfigurasi disimpan di bawah /etc/ecm/<service>-conf/.

    Layanan Direktori konfigurasi
    Hadoop /etc/ecm/hadoop-conf/
    Spark /etc/ecm/spark-conf/
    Hive /etc/ecm/hive-conf/
    Flink /etc/ecm/flink-conf/
    Flume /etc/ecm/flume-conf/
    Akses SSH bersifat read-only untuk file konfigurasi. Untuk mengubah parameter konfigurasi, login ke Konsol EMR.

    Direktori data

    Jenis data Direktori
    Data cache JindoFS /mnt/disk1/jindodata/