Hadoop Distributed File System (HDFS) service logs are stored in /mnt/disk1/log/hadoop-hdfs. Use this reference to identify which log file to check when troubleshooting a specific HDFS component.
Each HDFS component produces up to four types of logs:
Process startup logs (
.out): capture startup exceptions for the component process.Process run logs (
.log): record internal runtime activity for the component process.Audit logs: record file system access operations. Available on NameNode only.
JVM-GC logs (
*-gc.log.0.current): record Java Virtual Machine (JVM) Garbage Collection (GC) events. Available in EMR V3.43.0, EMR V5.8.0, and any minor version later than those releases.
| Component | Log file | Type |
|---|---|---|
| ZKFailoverController (ZKFC) | hadoop-hdfs-zkfc-*.out | Process startup logs |
| ZKFailoverController (ZKFC) | hadoop-hdfs-zkfc-*.log | Process run logs |
| ZKFailoverController (ZKFC) | zkfc-gc.log.0.current | JVM-GC logs |
| NameNode | hadoop-hdfs-namenode-*.out | Process startup logs |
| NameNode | hadoop-hdfs-namenode-*.log | Process run logs |
| NameNode | hdfs-audit.log | Audit logs |
| NameNode | namenode-gc.log.0.current | JVM-GC logs |
| JournalNode | hadoop-hdfs-journalnode-*.out | Process startup logs |
| JournalNode | hadoop-hdfs-journalnode-*.log | Process run logs |
| JournalNode | journalnode-gc.log.0.current | JVM-GC logs |
| DataNode | hadoop-hdfs-datanode-*.out | Process startup logs |
| DataNode | hadoop-hdfs-datanode-*.log | Process run logs |
| DataNode | datanode-gc.log.0.current | JVM-GC logs |